Sep 15 22:25:30.706812 ip-10-0-128-208 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 15 22:25:30.706823 ip-10-0-128-208 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 15 22:25:30.706832 ip-10-0-128-208 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 15 22:25:30.707049 ip-10-0-128-208 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 15 22:25:40.912840 ip-10-0-128-208 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 15 22:25:40.912857 ip-10-0-128-208 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 561f15db3704419995ce1c1118e06ecb --
Sep 15 22:27:28.667181 ip-10-0-128-208 systemd[1]: Starting Kubernetes Kubelet...
Sep 15 22:27:29.067662 ip-10-0-128-208 kubenswrapper[2565]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 15 22:27:29.067662 ip-10-0-128-208 kubenswrapper[2565]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 15 22:27:29.067662 ip-10-0-128-208 kubenswrapper[2565]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 15 22:27:29.067662 ip-10-0-128-208 kubenswrapper[2565]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 15 22:27:29.067662 ip-10-0-128-208 kubenswrapper[2565]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 15 22:27:29.068327 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.067675    2565 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 15 22:27:29.071974 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.071957    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 22:27:29.071974 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.071973    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 22:27:29.071974 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.071976    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 22:27:29.072065 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.071980    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 22:27:29.072065 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.071984    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 22:27:29.072065 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.071987    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 22:27:29.072065 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.071991    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 22:27:29.072065 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.071993    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 22:27:29.072065 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.071996    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 22:27:29.072065 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.071999    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 22:27:29.072065 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072002    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 22:27:29.072065 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072005    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 22:27:29.072065 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072008    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 22:27:29.072065 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072011    2565 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 22:27:29.072065 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072021    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 22:27:29.072065 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072024    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 22:27:29.072065 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072027    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 22:27:29.072065 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072030    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 22:27:29.072065 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072032    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 22:27:29.072065 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072035    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 22:27:29.072065 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072037    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 22:27:29.072065 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072040    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 22:27:29.072065 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072043    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 22:27:29.072539 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072046    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 22:27:29.072539 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072048    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 22:27:29.072539 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072051    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 22:27:29.072539 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072054    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 22:27:29.072539 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072056    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 22:27:29.072539 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072059    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 22:27:29.072539 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072062    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 22:27:29.072539 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072064    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 22:27:29.072539 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072067    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 22:27:29.072539 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072070    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 22:27:29.072539 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072072    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 22:27:29.072539 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072075    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 22:27:29.072539 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072078    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 22:27:29.072539 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072081    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 22:27:29.072539 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072083    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 22:27:29.072539 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072086    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 22:27:29.072539 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072089    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 22:27:29.072539 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072093    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 22:27:29.072539 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072096    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 22:27:29.072539 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072098    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 22:27:29.073048 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072101    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 22:27:29.073048 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072104    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 22:27:29.073048 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072108    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 22:27:29.073048 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072112    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 22:27:29.073048 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072115    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 22:27:29.073048 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072118    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 22:27:29.073048 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072121    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 22:27:29.073048 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072124    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 22:27:29.073048 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072126    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 22:27:29.073048 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072129    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 22:27:29.073048 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072132    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 22:27:29.073048 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072134    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 22:27:29.073048 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072137    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 22:27:29.073048 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072141    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 22:27:29.073048 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072145    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 22:27:29.073048 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072149    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 22:27:29.073048 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072152    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 22:27:29.073048 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072156    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 22:27:29.073048 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072160    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 22:27:29.073501 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072163    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 22:27:29.073501 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072166    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 22:27:29.073501 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072170    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 22:27:29.073501 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072172    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 22:27:29.073501 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072175    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 22:27:29.073501 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072178    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 22:27:29.073501 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072180    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 22:27:29.073501 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072183    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 22:27:29.073501 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072186    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 22:27:29.073501 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072189    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 22:27:29.073501 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072192    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 22:27:29.073501 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072194    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 22:27:29.073501 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072197    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 22:27:29.073501 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072201    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 22:27:29.073501 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072203    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 22:27:29.073501 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072206    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 22:27:29.073501 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072208    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 22:27:29.073501 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072211    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 22:27:29.073501 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072214    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 22:27:29.073501 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072216    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 22:27:29.073988 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072219    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 22:27:29.073988 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072221    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 22:27:29.073988 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072223    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 22:27:29.073988 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072226    2565 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 22:27:29.073988 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072228    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 22:27:29.073988 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072232    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 22:27:29.073988 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072235    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 22:27:29.073988 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072238    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 22:27:29.073988 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072241    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 22:27:29.073988 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072243    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 22:27:29.073988 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072246    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 22:27:29.073988 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072249    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 22:27:29.073988 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072251    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 22:27:29.073988 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072254    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 22:27:29.073988 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072257    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 22:27:29.073988 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072259    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 22:27:29.073988 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072262    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 22:27:29.073988 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072961    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 22:27:29.073988 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072968    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 22:27:29.073988 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072971    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 22:27:29.074475 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072974    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 22:27:29.074475 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072977    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 22:27:29.074475 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072980    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 22:27:29.074475 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072983    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 22:27:29.074475 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072986    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 22:27:29.074475 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072988    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 22:27:29.074475 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072991    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 22:27:29.074475 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072994    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 22:27:29.074475 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072997    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 22:27:29.074475 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.072999    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 22:27:29.074475 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073002    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 22:27:29.074475 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073005    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 22:27:29.074475 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073007    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 22:27:29.074475 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073010    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 22:27:29.074475 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073013    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 22:27:29.074475 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073015    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 22:27:29.074475 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073018    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 22:27:29.074475 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073022    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 22:27:29.074475 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073025    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 22:27:29.074475 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073029    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 22:27:29.075008 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073031    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 22:27:29.075008 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073034    2565 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 22:27:29.075008 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073037    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 22:27:29.075008 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073040    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 22:27:29.075008 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073042    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 22:27:29.075008 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073045    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 22:27:29.075008 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073048    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 22:27:29.075008 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073050    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 22:27:29.075008 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073053    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 22:27:29.075008 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073058    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 22:27:29.075008 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073062    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 22:27:29.075008 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073065    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 22:27:29.075008 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073069    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 22:27:29.075008 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073072    2565 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 22:27:29.075008 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073075    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 22:27:29.075008 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073079    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 22:27:29.075008 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073083    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 22:27:29.075008 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073085    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 22:27:29.075008 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073088    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 22:27:29.075483 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073090    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 22:27:29.075483 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073093    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 22:27:29.075483 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073096    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 22:27:29.075483 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073098    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 22:27:29.075483 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073101    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 22:27:29.075483 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073103    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 22:27:29.075483 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073106    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 22:27:29.075483 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073109    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 22:27:29.075483 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073111    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 22:27:29.075483 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073114    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 22:27:29.075483 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073117    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 22:27:29.075483 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073121    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 22:27:29.075483 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073124    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 22:27:29.075483 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073127    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 22:27:29.075483 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073130    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 22:27:29.075483 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073132    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 22:27:29.075483 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073135    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 22:27:29.075483 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073139    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 22:27:29.075483 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073141    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 22:27:29.075483 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073144    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 22:27:29.075984 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073147    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 22:27:29.075984 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073149    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 22:27:29.075984 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073152    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 22:27:29.075984 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073155    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 22:27:29.075984 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073157    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 22:27:29.075984 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073160    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 22:27:29.075984 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073162    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 22:27:29.075984 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073164    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 22:27:29.075984 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073167    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 22:27:29.075984 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073170    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 22:27:29.075984 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073172    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 22:27:29.075984 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073175    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 22:27:29.075984 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073177    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 22:27:29.075984 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073180    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 22:27:29.075984 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073182    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 22:27:29.075984 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073185    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 22:27:29.075984 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073188    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 22:27:29.075984 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073190    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 22:27:29.075984 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073193    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 22:27:29.075984 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073195    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 22:27:29.076471 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073198    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 22:27:29.076471 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073201    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 22:27:29.076471 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073203    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 22:27:29.076471 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073206    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 22:27:29.076471 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073209    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 22:27:29.076471 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073212    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 22:27:29.076471 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073214    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 22:27:29.076471 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073217    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 22:27:29.076471 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073220    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 22:27:29.076471 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073222    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 22:27:29.076471 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073226    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 22:27:29.076471 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073230    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 22:27:29.076471 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073232    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 22:27:29.076471 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073235    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 22:27:29.076471 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073238    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 22:27:29.076471 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073240    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 22:27:29.076471 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073244    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 22:27:29.076471 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073610    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 22:27:29.076471 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073617    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 22:27:29.076956 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073620    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 22:27:29.076956 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073623    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 22:27:29.076956 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073627    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 22:27:29.076956 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073630    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 22:27:29.076956 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073633    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 22:27:29.076956 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073636    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 22:27:29.076956 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073638    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 22:27:29.076956 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073641    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 22:27:29.076956 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073644    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 22:27:29.076956 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073646    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 22:27:29.076956 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073649    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 22:27:29.076956 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073651    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 22:27:29.076956 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073654    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 22:27:29.076956 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073657    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 22:27:29.076956 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073660    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 22:27:29.076956 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073662    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 22:27:29.076956 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073665    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 22:27:29.076956 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073668    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 22:27:29.076956 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073671    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 22:27:29.076956 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073674    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 22:27:29.077452 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073676    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 22:27:29.077452 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073679    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 22:27:29.077452 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073681    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 22:27:29.077452 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073684    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 22:27:29.077452 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073687    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 22:27:29.077452 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073689    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 22:27:29.077452 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073692    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 22:27:29.077452 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073695    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 22:27:29.077452 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073697    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 22:27:29.077452 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073700    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 22:27:29.077452 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073702    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 22:27:29.077452 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073705    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 22:27:29.077452 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073708    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 22:27:29.077452 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073710    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 22:27:29.077452 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073713    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 22:27:29.077452 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073717    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 22:27:29.077452 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073719    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 22:27:29.077452 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073722    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 22:27:29.077452 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073725    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 22:27:29.077452 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073728    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 22:27:29.077942 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073730    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 22:27:29.077942 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073733    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 22:27:29.077942 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073735    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 22:27:29.077942 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073738    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 22:27:29.077942 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073740    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 22:27:29.077942 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073743    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 22:27:29.077942 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073746    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 22:27:29.077942 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073749    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 22:27:29.077942 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073752    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 22:27:29.077942 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073755    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 22:27:29.077942 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073757    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 22:27:29.077942 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073760    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 22:27:29.077942 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073762    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 22:27:29.077942 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073765    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 22:27:29.077942 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073767    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 22:27:29.077942 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073770    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 22:27:29.077942 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073773    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 22:27:29.077942 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073775    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 22:27:29.077942 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073778    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 22:27:29.077942 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073780    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 22:27:29.078510 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073783    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 22:27:29.078510 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073785    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 22:27:29.078510 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073788    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 22:27:29.078510 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073790    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 22:27:29.078510 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073793    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 22:27:29.078510 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073796    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 22:27:29.078510 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073799    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 22:27:29.078510 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073801    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 22:27:29.078510 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073804    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 22:27:29.078510 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073806    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 22:27:29.078510 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073809    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 22:27:29.078510 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073811    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 22:27:29.078510 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073816    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 22:27:29.078510 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073820    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 22:27:29.078510 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073823    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 22:27:29.078510 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073825    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 22:27:29.078510 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073828    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 22:27:29.078510 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073831    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 22:27:29.078510 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073833    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 22:27:29.078983 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073836    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 22:27:29.078983 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073839    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 22:27:29.078983 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073843    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 22:27:29.078983 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073846    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 22:27:29.078983 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073849    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 22:27:29.078983 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073853    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 22:27:29.078983 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073856    2565 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 22:27:29.078983 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073859    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 22:27:29.078983 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073862    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 22:27:29.078983 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073865    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 22:27:29.078983 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073867    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 22:27:29.078983 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073870    2565 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 22:27:29.078983 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073873    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 22:27:29.078983 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073875    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 22:27:29.078983 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073878    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 22:27:29.078983 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073880    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 22:27:29.078983 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073883    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 22:27:29.078983 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.073886    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 22:27:29.078983 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.073965    2565 flags.go:64] FLAG: --address="0.0.0.0"
Sep 15 22:27:29.078983 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074004    2565 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 15 22:27:29.079505 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074010    2565 flags.go:64] FLAG: --anonymous-auth="true"
Sep 15 22:27:29.079505 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074015    2565 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 15 22:27:29.079505 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074021    2565 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 15 22:27:29.079505 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074025    2565 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 15 22:27:29.079505 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074029    2565 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 15 22:27:29.079505 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074033    2565 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 15 22:27:29.079505 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074036    2565 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 15 22:27:29.079505 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074039    2565 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 15 22:27:29.079505 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074042    2565 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 15 22:27:29.079505 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074046    2565 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 15 22:27:29.079505 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074049    2565 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 15 22:27:29.079505 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074052    2565 flags.go:64] FLAG: --cgroup-root=""
Sep 15 22:27:29.079505 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074055    2565 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 15 22:27:29.079505 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074060    2565 flags.go:64] FLAG: --client-ca-file=""
Sep 15 22:27:29.079505 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074063    2565 flags.go:64] FLAG: --cloud-provider="external"
Sep 15 22:27:29.079505 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074066    2565 flags.go:64] FLAG: --cluster-dns="[]"
Sep 15 22:27:29.079505 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074070    2565 flags.go:64] FLAG: --cluster-domain=""
Sep 15 22:27:29.079505 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074073    2565 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 15 22:27:29.079505 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074076    2565 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 15 22:27:29.079505 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074079    2565 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 15 22:27:29.079505 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074082    2565 flags.go:64] FLAG: --container-log-max-files="5"
Sep 15 22:27:29.079505 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074086    2565 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 15 22:27:29.079505 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074089    2565 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 15 22:27:29.080078 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074092    2565 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 15 22:27:29.080078 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074095    2565 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 15 22:27:29.080078 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074099    2565 flags.go:64] FLAG: --contention-profiling="false"
Sep 15 22:27:29.080078 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074102    2565 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 15 22:27:29.080078 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074104    2565 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 15 22:27:29.080078 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074108    2565 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 15 22:27:29.080078 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074111    2565 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 15 22:27:29.080078 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074115    2565 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 15 22:27:29.080078 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074118    2565 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 15 22:27:29.080078 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074121    2565 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 15 22:27:29.080078 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074124    2565 flags.go:64] FLAG: --enable-load-reader="false"
Sep 15 22:27:29.080078 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074127    2565 flags.go:64] FLAG: --enable-server="true"
Sep 15 22:27:29.080078 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074131    2565 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 15 22:27:29.080078 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074136    2565 flags.go:64] FLAG: --event-burst="100"
Sep 15 22:27:29.080078 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074139    2565 flags.go:64] FLAG: --event-qps="50"
Sep 15 22:27:29.080078 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074142    2565 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 15 22:27:29.080078 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074145    2565 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 15 22:27:29.080078 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074148    2565 flags.go:64] FLAG: --eviction-hard=""
Sep 15 22:27:29.080078 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074154    2565 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 15 22:27:29.080078 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074157    2565 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 15 22:27:29.080078 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074160    2565 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 15 22:27:29.080078 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074163    2565 flags.go:64] FLAG: --eviction-soft=""
Sep 15 22:27:29.080078 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074169    2565 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 15 22:27:29.080078 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074172    2565 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 15 22:27:29.080078 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074175    2565 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 15 22:27:29.080690 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074177    2565 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 15 22:27:29.080690 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074180    2565 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 15 22:27:29.080690 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074183    2565 flags.go:64] FLAG: --fail-swap-on="true"
Sep 15 22:27:29.080690 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074186    2565 flags.go:64] FLAG: --feature-gates=""
Sep 15 22:27:29.080690 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074190    2565 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 15 22:27:29.080690 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074193    2565 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 15 22:27:29.080690 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074196    2565 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 15 22:27:29.080690 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074200    2565 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 15 22:27:29.080690 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074203    2565 flags.go:64] FLAG: --healthz-port="10248"
Sep 15 22:27:29.080690 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074206    2565 flags.go:64] FLAG: --help="false"
Sep 15 22:27:29.080690 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074209    2565 flags.go:64] FLAG: --hostname-override="ip-10-0-128-208.ec2.internal"
Sep 15 22:27:29.080690 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074212    2565 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 15 22:27:29.080690 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074215    2565 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 15 22:27:29.080690 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074218    2565 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 15 22:27:29.080690 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074222    2565 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 15 22:27:29.080690 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074225    2565 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 15 22:27:29.080690 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074228    2565 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 15 22:27:29.080690 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074231    2565 flags.go:64] FLAG: --image-service-endpoint=""
Sep 15 22:27:29.080690 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074234    2565 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 15 22:27:29.080690 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074237    2565 flags.go:64] FLAG: --kube-api-burst="100"
Sep 15 22:27:29.080690 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074240    2565 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 15 22:27:29.080690 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074243    2565 flags.go:64] FLAG: --kube-api-qps="50"
Sep 15 22:27:29.080690 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074246    2565 flags.go:64] FLAG: --kube-reserved=""
Sep 15 22:27:29.080690 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074249    2565 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 15 22:27:29.081572 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074252    2565 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 15 22:27:29.081572 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074255    2565 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 15 22:27:29.081572 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074258    2565 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 15 22:27:29.081572 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074260    2565 flags.go:64] FLAG: --lock-file=""
Sep 15 22:27:29.081572 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074263    2565 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 15 22:27:29.081572 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074267    2565 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 15 22:27:29.081572 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074270    2565 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 15 22:27:29.081572 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074276    2565 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 15 22:27:29.081572 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074278    2565 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 15 22:27:29.081572 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074281    2565 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 15 22:27:29.081572 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074284    2565 flags.go:64] FLAG: --logging-format="text"
Sep 15 22:27:29.081572 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074287    2565 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 15 22:27:29.081572 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074291    2565 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 15 22:27:29.081572 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074294    2565 flags.go:64] FLAG: --manifest-url=""
Sep 15 22:27:29.081572 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074297    2565 flags.go:64] FLAG: --manifest-url-header=""
Sep 15 22:27:29.081572 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074301    2565 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 15 22:27:29.081572 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074304    2565 flags.go:64] FLAG: --max-open-files="1000000"
Sep 15 22:27:29.081572 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074308    2565 flags.go:64] FLAG: --max-pods="110"
Sep 15 22:27:29.081572 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074311    2565 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 15 22:27:29.081572 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074315    2565 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 15 22:27:29.081572 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074318    2565 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 15 22:27:29.081572 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074321    2565 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 15 22:27:29.081572 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074326    2565 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 15 22:27:29.081572 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074329    2565 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 15 22:27:29.081572 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074332    2565 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 15 22:27:29.082716 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074339    2565 flags.go:64] FLAG: --node-status-max-images="50"
Sep 15 22:27:29.082716 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074342    2565 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 15 22:27:29.082716 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074345    2565 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 15 22:27:29.082716 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074348    2565 flags.go:64] FLAG: --pod-cidr=""
Sep 15 22:27:29.082716 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074351    2565 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 15 22:27:29.082716 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074357    2565 flags.go:64] FLAG: --pod-manifest-path=""
Sep 15 22:27:29.082716 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074359    2565 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 15 22:27:29.082716 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074363    2565 flags.go:64] FLAG: --pods-per-core="0"
Sep 15 22:27:29.082716 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074365    2565 flags.go:64] FLAG: --port="10250"
Sep 15 22:27:29.082716 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074369    2565 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 15 22:27:29.082716 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074372    2565 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0eb1558dcfb3570f9"
Sep 15 22:27:29.082716 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074375    2565 flags.go:64] FLAG: --qos-reserved=""
Sep 15 22:27:29.082716 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074378    2565 flags.go:64] FLAG: --read-only-port="10255"
Sep 15 22:27:29.082716 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074382    2565 flags.go:64] FLAG: --register-node="true"
Sep 15 22:27:29.082716 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074385    2565 flags.go:64] FLAG: --register-with-taints=""
Sep 15 22:27:29.082716 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074389    2565 flags.go:64] FLAG: --registry-burst="10"
Sep 15 22:27:29.082716 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074391    2565 flags.go:64] FLAG: --registry-qps="5"
Sep 15 22:27:29.082716 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074394    2565 flags.go:64] FLAG: --reserved-cpus=""
Sep 15 22:27:29.082716 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074397    2565 flags.go:64] FLAG: --reserved-memory=""
Sep 15 22:27:29.082716 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074401    2565 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 15 22:27:29.082716 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074404    2565 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 15 22:27:29.082716 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074407    2565 flags.go:64] FLAG: --rotate-certificates="false"
Sep 15 22:27:29.082716 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074410    2565 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 15 22:27:29.082716 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074413    2565 flags.go:64] FLAG: --runonce="false"
Sep 15 22:27:29.082716 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074416    2565 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 15 22:27:29.083588 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074419    2565 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 15 22:27:29.083588 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074422    2565 flags.go:64] FLAG: --seccomp-default="false"
Sep 15 22:27:29.083588 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074425    2565 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 15 22:27:29.083588 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074427    2565 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 15 22:27:29.083588 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074432    2565 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 15 22:27:29.083588 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074435    2565 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 15 22:27:29.083588 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074438    2565 flags.go:64] FLAG: --storage-driver-password="root"
Sep 15 22:27:29.083588 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074441    2565 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 15 22:27:29.083588 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074444    2565 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 15 22:27:29.083588 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074447    2565 flags.go:64] FLAG: --storage-driver-user="root"
Sep 15 22:27:29.083588 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074449    2565 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 15 22:27:29.083588 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074452    2565 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 15 22:27:29.083588 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074456    2565 flags.go:64] FLAG: --system-cgroups=""
Sep 15 22:27:29.083588 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074459    2565 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 15 22:27:29.083588 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074464    2565 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 15 22:27:29.083588 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074467    2565 flags.go:64] FLAG: --tls-cert-file=""
Sep 15 22:27:29.083588 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074470    2565 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 15 22:27:29.083588 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074475    2565 flags.go:64] FLAG: --tls-min-version=""
Sep 15 22:27:29.083588 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074477    2565 flags.go:64] FLAG: --tls-private-key-file=""
Sep 15 22:27:29.083588 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074481    2565 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 15 22:27:29.083588 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074485    2565 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 15 22:27:29.083588 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074488    2565 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 15 22:27:29.083588 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074491    2565 flags.go:64] FLAG: --v="2"
Sep 15 22:27:29.083588 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074495    2565 flags.go:64] FLAG: --version="false"
Sep 15 22:27:29.083588 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074499    2565 flags.go:64] FLAG: --vmodule=""
Sep 15 22:27:29.084367 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074502    2565 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 15 22:27:29.084367 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.074506    2565 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 15 22:27:29.084367 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074616    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 22:27:29.084367 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074619    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 22:27:29.084367 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074623    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 22:27:29.084367 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074626    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 22:27:29.084367 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074630    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 22:27:29.084367 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074634    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 22:27:29.084367 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074637    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 22:27:29.084367 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074640    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 22:27:29.084367 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074643    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 22:27:29.084367 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074648    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 22:27:29.084367 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074650    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 22:27:29.084367 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074653    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 22:27:29.084367 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074656    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 22:27:29.084367 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074659    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 22:27:29.084367 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074661    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 22:27:29.084367 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074664    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 22:27:29.084367 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074667    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 22:27:29.085316 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074669    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 22:27:29.085316 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074673    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 22:27:29.085316 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074676    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 22:27:29.085316 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074679    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 22:27:29.085316 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074681    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 22:27:29.085316 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074685    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 22:27:29.085316 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074688    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 22:27:29.085316 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074691    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 22:27:29.085316 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074696    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 22:27:29.085316 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074699    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 22:27:29.085316 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074701    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 22:27:29.085316 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074704    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 22:27:29.085316 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074707    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 22:27:29.085316 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074709    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 22:27:29.085316 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074712    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 22:27:29.085316 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074714    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 22:27:29.085316 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074717    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 22:27:29.085316 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074720    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 22:27:29.085316 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074722    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 22:27:29.085923 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074725    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 22:27:29.085923 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074727    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 22:27:29.085923 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074730    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 22:27:29.085923 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074733    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 22:27:29.085923 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074736    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 22:27:29.085923 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074740    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 22:27:29.085923 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074743    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 22:27:29.085923 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074747    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 22:27:29.085923 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074751    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 22:27:29.085923 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074754    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 22:27:29.085923 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074757    2565 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 22:27:29.085923 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074759    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 22:27:29.085923 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074762    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 22:27:29.085923 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074765    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 22:27:29.085923 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074767    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 22:27:29.085923 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074771    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 22:27:29.085923 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074773    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 22:27:29.085923 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074776    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 22:27:29.085923 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074778    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 22:27:29.086665 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074781    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 22:27:29.086665 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074784    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 22:27:29.086665 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074787    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 22:27:29.086665 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074790    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 22:27:29.086665 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074793    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 22:27:29.086665 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074795    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 22:27:29.086665 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074798    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 22:27:29.086665 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074801    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 22:27:29.086665 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074803    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 22:27:29.086665 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074806    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 22:27:29.086665 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074808    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 22:27:29.086665 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074811    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 22:27:29.086665 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074814    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 22:27:29.086665 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074817    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 22:27:29.086665 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074819    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 22:27:29.086665 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074822    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 22:27:29.086665 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074824    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 22:27:29.086665 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074827    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 22:27:29.086665 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074832    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 22:27:29.086665 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074834    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 22:27:29.087546 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074837    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 22:27:29.087546 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074840    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 22:27:29.087546 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074842    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 22:27:29.087546 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074845    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 22:27:29.087546 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074847    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 22:27:29.087546 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074850    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 22:27:29.087546 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074853    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 22:27:29.087546 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074855    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 22:27:29.087546 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074858    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 22:27:29.087546 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074861    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 22:27:29.087546 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074864    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 22:27:29.087546 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074866    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 22:27:29.087546 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074869    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 22:27:29.087546 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074871    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 22:27:29.087546 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074875    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 22:27:29.087546 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074878    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 22:27:29.087546 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074881    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 22:27:29.087546 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074884    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 22:27:29.087546 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074886    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 22:27:29.087546 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074902    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 22:27:29.088067 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074905    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 22:27:29.088067 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074907    2565 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 22:27:29.088067 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074910    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 22:27:29.088067 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.074913    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 22:27:29.088067 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.075720    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 15 22:27:29.088067 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.083261    2565 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 15 22:27:29.088067 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.083279    2565 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 15 22:27:29.088067 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.084121    2565 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 15 22:27:29.088067 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.084135    2565 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 15 22:27:29.088067 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084224    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 22:27:29.088067 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084232    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 22:27:29.088067 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084237    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 22:27:29.088067 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084244    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 22:27:29.088067 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084249    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 22:27:29.088067 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084253    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 22:27:29.088578 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084258    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 22:27:29.088578 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084262    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 22:27:29.088578 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084267    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 22:27:29.088578 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084271    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 22:27:29.088578 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084275    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 22:27:29.088578 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084279    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 22:27:29.088578 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084285    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 22:27:29.088578 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084289    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 22:27:29.088578 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084294    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 22:27:29.088578 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084298    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 22:27:29.088578 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084302    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 22:27:29.088578 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084307    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 22:27:29.088578 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084311    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 22:27:29.088578 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084316    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 22:27:29.088578 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084320    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 22:27:29.088578 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084324    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 22:27:29.088578 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084328    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 22:27:29.088578 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084332    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 22:27:29.088578 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084337    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 22:27:29.088578 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084341    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 22:27:29.089169 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084345    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 22:27:29.089169 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084349    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 22:27:29.089169 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084354    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 22:27:29.089169 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084361    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 22:27:29.089169 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084368    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 22:27:29.089169 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084372    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 22:27:29.089169 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084379    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 22:27:29.089169 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084384    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 22:27:29.089169 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084389    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 22:27:29.089169 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084394    2565 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 22:27:29.089169 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084399    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 22:27:29.089169 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084404    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 22:27:29.089169 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084409    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 22:27:29.089169 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084413    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 22:27:29.089169 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084418    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 22:27:29.089169 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084422    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 22:27:29.089169 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084427    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 22:27:29.089169 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084431    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 22:27:29.089169 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084435    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 22:27:29.089686 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084440    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 22:27:29.089686 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084445    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 22:27:29.089686 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084449    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 22:27:29.089686 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084453    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 22:27:29.089686 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084457    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 22:27:29.089686 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084461    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 22:27:29.089686 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084465    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 22:27:29.089686 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084470    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 22:27:29.089686 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084474    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 22:27:29.089686 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084478    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 22:27:29.089686 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084482    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 22:27:29.089686 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084486    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 22:27:29.089686 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084490    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 22:27:29.089686 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084494    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 22:27:29.089686 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084498    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 22:27:29.089686 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084503    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 22:27:29.089686 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084508    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 22:27:29.089686 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084512    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 22:27:29.089686 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084516    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 22:27:29.089686 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084520    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 22:27:29.090244 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084525    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 22:27:29.090244 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084529    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 22:27:29.090244 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084533    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 22:27:29.090244 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084538    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 22:27:29.090244 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084542    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 22:27:29.090244 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084547    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 22:27:29.090244 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084551    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 22:27:29.090244 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084555    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 22:27:29.090244 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084559    2565 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 22:27:29.090244 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084563    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 22:27:29.090244 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084567    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 22:27:29.090244 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084571    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 22:27:29.090244 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084575    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 22:27:29.090244 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084581    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 22:27:29.090244 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084585    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 22:27:29.090244 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084589    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 22:27:29.090244 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084593    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 22:27:29.090244 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084597    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 22:27:29.090244 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084601    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 22:27:29.090244 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084606    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 22:27:29.090244 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084611    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 22:27:29.090741 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084615    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 22:27:29.090741 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084622    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 22:27:29.090741 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084627    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 22:27:29.090741 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084632    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 22:27:29.090741 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084637    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 22:27:29.090741 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084641    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 22:27:29.090741 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084646    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 22:27:29.090741 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084651    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 22:27:29.090741 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084656    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 22:27:29.090741 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084660    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 22:27:29.090741 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084664    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 22:27:29.090741 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084668    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 22:27:29.090741 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084674    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 22:27:29.090741 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.084681    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 15 22:27:29.091145 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084852    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 22:27:29.091145 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084861    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 22:27:29.091145 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084866    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 22:27:29.091145 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084871    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 22:27:29.091145 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084875    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 22:27:29.091145 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084879    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 22:27:29.091145 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084884    2565 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 22:27:29.091145 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084888    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 22:27:29.091145 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084911    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 22:27:29.091145 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084915    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 22:27:29.091145 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084919    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 22:27:29.091145 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084924    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 22:27:29.091145 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084928    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 22:27:29.091145 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084932    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 22:27:29.091145 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084936    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 22:27:29.091145 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084939    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 22:27:29.091145 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084943    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 22:27:29.091145 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084947    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 22:27:29.091145 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084951    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 22:27:29.091145 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084955    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 22:27:29.091654 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084958    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 22:27:29.091654 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084962    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 22:27:29.091654 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084966    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 22:27:29.091654 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084971    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 22:27:29.091654 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084978    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 22:27:29.091654 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084982    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 22:27:29.091654 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084987    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 22:27:29.091654 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084991    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 22:27:29.091654 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.084996    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 22:27:29.091654 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085000    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 22:27:29.091654 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085004    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 22:27:29.091654 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085009    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 22:27:29.091654 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085014    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 22:27:29.091654 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085018    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 22:27:29.091654 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085023    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 22:27:29.091654 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085026    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 22:27:29.091654 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085031    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 22:27:29.091654 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085035    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 22:27:29.091654 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085039    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 22:27:29.091654 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085043    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 22:27:29.092186 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085047    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 22:27:29.092186 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085051    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 22:27:29.092186 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085056    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 22:27:29.092186 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085061    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 22:27:29.092186 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085066    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 22:27:29.092186 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085070    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 22:27:29.092186 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085074    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 22:27:29.092186 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085079    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 22:27:29.092186 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085083    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 22:27:29.092186 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085087    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 22:27:29.092186 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085091    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 22:27:29.092186 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085095    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 22:27:29.092186 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085100    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 22:27:29.092186 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085104    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 22:27:29.092186 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085108    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 22:27:29.092186 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085113    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 22:27:29.092186 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085117    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 22:27:29.092186 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085122    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 22:27:29.092186 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085126    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 22:27:29.092186 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085131    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 22:27:29.092677 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085134    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 22:27:29.092677 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085139    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 22:27:29.092677 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085143    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 22:27:29.092677 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085148    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 22:27:29.092677 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085152    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 22:27:29.092677 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085156    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 22:27:29.092677 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085160    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 22:27:29.092677 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085164    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 22:27:29.092677 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085168    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 22:27:29.092677 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085172    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 22:27:29.092677 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085176    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 22:27:29.092677 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085181    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 22:27:29.092677 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085185    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 22:27:29.092677 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085189    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 22:27:29.092677 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085193    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 22:27:29.092677 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085197    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 22:27:29.092677 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085201    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 22:27:29.092677 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085206    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 22:27:29.092677 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085210    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 22:27:29.092677 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085215    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 22:27:29.093183 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085219    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 22:27:29.093183 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085223    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 22:27:29.093183 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085227    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 22:27:29.093183 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085231    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 22:27:29.093183 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085235    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 22:27:29.093183 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085239    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 22:27:29.093183 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085243    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 22:27:29.093183 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085247    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 22:27:29.093183 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085252    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 22:27:29.093183 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085257    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 22:27:29.093183 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085261    2565 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 22:27:29.093183 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085265    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 22:27:29.093183 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085269    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 22:27:29.093183 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085273    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 22:27:29.093183 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085277    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 22:27:29.093183 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085282    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 22:27:29.093183 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085288    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 22:27:29.093183 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085294    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 22:27:29.093183 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:29.085300    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 22:27:29.093640 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.085309    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 15 22:27:29.093640 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.085452    2565 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 15 22:27:29.093640 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.088252    2565 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 15 22:27:29.093640 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.089264    2565 server.go:1019] "Starting client certificate rotation"
Sep 15 22:27:29.093640 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.089375    2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 15 22:27:29.093640 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.089446    2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 15 22:27:29.093640 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.091954    2565 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 15 22:27:29.099099 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.099066    2565 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 15 22:27:29.108432 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.108415    2565 log.go:25] "Validated CRI v1 runtime API"
Sep 15 22:27:29.114062 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.114046    2565 log.go:25] "Validated CRI v1 image API"
Sep 15 22:27:29.114932 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.114915    2565 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 15 22:27:29.118548 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.118524    2565 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 dbd169b7-647b-48b5-8951-6076df421461:/dev/nvme0n1p4 fc794f51-3231-45d2-a824-843845d8e1d8:/dev/nvme0n1p3]
Sep 15 22:27:29.118607 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.118548    2565 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 15 22:27:29.123615 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.123495    2565 manager.go:216] Machine: {Timestamp:2026-09-15 22:27:29.122156554 +0000 UTC m=+0.341999550 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3092110 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2247e115778e3347a97aa8d3640d31 SystemUUID:ec2247e1-1577-8e33-47a9-7aa8d3640d31 BootID:561f15db-3704-4199-95ce-1c1118e06ecb Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:ca:25:92:19 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:ca:25:92:19 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e2:93:d3:27:e6:c4 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 15 22:27:29.123615 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.123604    2565 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 15 22:27:29.123744 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.123677    2565 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 15 22:27:29.125954 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.125937    2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 15 22:27:29.126710 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.126684    2565 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 15 22:27:29.126837 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.126712    2565 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-208.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 15 22:27:29.127945 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.127935    2565 topology_manager.go:138] "Creating topology manager with none policy"
Sep 15 22:27:29.127978 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.127947    2565 container_manager_linux.go:309] "Creating device plugin manager"
Sep 15 22:27:29.127978 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.127955    2565 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 15 22:27:29.130590 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.130578    2565 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 15 22:27:29.131315 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.131304    2565 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 15 22:27:29.134976 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.134964    2565 state_mem.go:36] "Initialized new in-memory state store"
Sep 15 22:27:29.135081 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.135071    2565 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 15 22:27:29.135129 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.135120    2565 kubelet.go:486] "Attempting to sync node with API server"
Sep 15 22:27:29.135160 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.135132    2565 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 15 22:27:29.135160 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.135143    2565 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 15 22:27:29.135160 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.135153    2565 kubelet.go:398] "Adding apiserver pod source"
Sep 15 22:27:29.135244 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.135162    2565 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 15 22:27:29.135490 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.135479    2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 15 22:27:29.135520 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.135500    2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 15 22:27:29.137456 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.137439    2565 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 15 22:27:29.139083 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.139070    2565 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 15 22:27:29.139182 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.139107    2565 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 15 22:27:29.140926 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.140913    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 15 22:27:29.140965 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.140931    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 15 22:27:29.140965 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.140938    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 15 22:27:29.140965 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.140944    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 15 22:27:29.140965 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.140954    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 15 22:27:29.140965 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.140959    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 15 22:27:29.141102 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.140973    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 15 22:27:29.141102 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.140978    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 15 22:27:29.141102 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.140984    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 15 22:27:29.141102 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.140990    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 15 22:27:29.141102 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.141004    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 15 22:27:29.141102 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.141009    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 15 22:27:29.142581 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.142566    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 15 22:27:29.142654 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.142585    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 15 22:27:29.146253 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.146231    2565 server.go:1268] "Started kubelet"
Sep 15 22:27:29.146399 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.146301    2565 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 15 22:27:29.146500 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.146381    2565 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 15 22:27:29.150953 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.150932    2565 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-208.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 15 22:27:29.151156 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:29.151130    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 15 22:27:29.151153 ip-10-0-128-208 systemd[1]: Started Kubernetes Kubelet.
Sep 15 22:27:29.152266 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:29.151210    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-208.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 15 22:27:29.152266 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.151312    2565 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 15 22:27:29.152627 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.152589    2565 server.go:310] "Adding debug handlers to kubelet server"
Sep 15 22:27:29.153869 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.153845    2565 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 15 22:27:29.155429 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:29.155302    2565 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 15 22:27:29.156323 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.156305    2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-x6s4x"
Sep 15 22:27:29.159194 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:29.158254    2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-208.ec2.internal.18d59efff6ecca14  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-208.ec2.internal,UID:ip-10-0-128-208.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-208.ec2.internal,},FirstTimestamp:2026-09-15 22:27:29.146202644 +0000 UTC m=+0.366045688,LastTimestamp:2026-09-15 22:27:29.146202644 +0000 UTC m=+0.366045688,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-208.ec2.internal,}"
Sep 15 22:27:29.159747 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.159725    2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 15 22:27:29.160273 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.160254    2565 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 15 22:27:29.160696 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:29.160677    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-208.ec2.internal\" not found"
Sep 15 22:27:29.160819 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.160799    2565 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 15 22:27:29.160819 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.160820    2565 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 15 22:27:29.160968 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.160912    2565 reconstruct.go:97] "Volume reconstruction finished"
Sep 15 22:27:29.160968 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.160923    2565 reconciler.go:29] "Reconciler: start to sync state"
Sep 15 22:27:29.160968 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.160922    2565 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 15 22:27:29.161077 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.161007    2565 factory.go:55] Registering systemd factory
Sep 15 22:27:29.161077 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.161020    2565 factory.go:223] Registration of the systemd container factory successfully
Sep 15 22:27:29.161236 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.161221    2565 factory.go:153] Registering CRI-O factory
Sep 15 22:27:29.161286 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.161240    2565 factory.go:223] Registration of the crio container factory successfully
Sep 15 22:27:29.161336 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.161289    2565 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 15 22:27:29.161336 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.161307    2565 factory.go:103] Registering Raw factory
Sep 15 22:27:29.161336 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.161321    2565 manager.go:1196] Started watching for new ooms in manager
Sep 15 22:27:29.161718 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.161704    2565 manager.go:349] Starting recovery of all containers
Sep 15 22:27:29.170009 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:29.166108    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 15 22:27:29.170009 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:29.166208    2565 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-208.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 15 22:27:29.170009 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.166646    2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-x6s4x"
Sep 15 22:27:29.173035 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.172821    2565 manager.go:354] Recovery completed
Sep 15 22:27:29.177417 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.177405    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 22:27:29.179487 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.179467    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-208.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 22:27:29.179577 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.179503    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-208.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 22:27:29.179577 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.179517    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-208.ec2.internal" event="NodeHasSufficientPID"
Sep 15 22:27:29.180219 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.180204    2565 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 15 22:27:29.180219 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.180218    2565 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 15 22:27:29.180325 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.180235    2565 state_mem.go:36] "Initialized new in-memory state store"
Sep 15 22:27:29.180325 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.180243    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 22:27:29.180993 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.180975    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-208.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 22:27:29.181044 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.181009    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-208.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 22:27:29.181044 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.181023    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-208.ec2.internal" event="NodeHasSufficientPID"
Sep 15 22:27:29.182142 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:29.182046    2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-208.ec2.internal.18d59efff8e8a909  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-208.ec2.internal,UID:ip-10-0-128-208.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-208.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-208.ec2.internal,},FirstTimestamp:2026-09-15 22:27:29.179486473 +0000 UTC m=+0.399329475,LastTimestamp:2026-09-15 22:27:29.179486473 +0000 UTC m=+0.399329475,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-208.ec2.internal,}"
Sep 15 22:27:29.182292 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.182280    2565 policy_none.go:49] "None policy: Start"
Sep 15 22:27:29.182333 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.182298    2565 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 15 22:27:29.182333 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.182310    2565 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 15 22:27:29.184254 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.184241    2565 policy_none.go:47] "Start"
Sep 15 22:27:29.215601 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.215582    2565 manager.go:337] "Starting Device Plugin manager"
Sep 15 22:27:29.234939 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:29.215628    2565 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 15 22:27:29.234939 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.215642    2565 server.go:87] "Starting device plugin registration server"
Sep 15 22:27:29.234939 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.216021    2565 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 15 22:27:29.234939 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.216033    2565 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 15 22:27:29.234939 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.216208    2565 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 15 22:27:29.234939 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.216279    2565 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 15 22:27:29.234939 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.216286    2565 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 15 22:27:29.234939 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:29.216879    2565 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 15 22:27:29.234939 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:29.216938    2565 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-208.ec2.internal\" not found"
Sep 15 22:27:29.234939 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.222321    2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 15 22:27:29.234939 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.223432    2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 15 22:27:29.234939 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.223481    2565 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 15 22:27:29.234939 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.223507    2565 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 15 22:27:29.234939 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:29.223585    2565 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 15 22:27:29.234939 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.225933    2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 22:27:29.316645 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.316618    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 22:27:29.317648 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.317631    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-208.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 22:27:29.317779 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.317660    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-208.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 22:27:29.317779 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.317671    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-208.ec2.internal" event="NodeHasSufficientPID"
Sep 15 22:27:29.317779 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.317695    2565 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-208.ec2.internal"
Sep 15 22:27:29.324098 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.324080    2565 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-208.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-208.ec2.internal"]
Sep 15 22:27:29.324155 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.324144    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 22:27:29.324933 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.324915    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-208.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 22:27:29.325020 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.324944    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-208.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 22:27:29.325020 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.324956    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-208.ec2.internal" event="NodeHasSufficientPID"
Sep 15 22:27:29.327179 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.327164    2565 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-208.ec2.internal"
Sep 15 22:27:29.327223 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.327185    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 22:27:29.327254 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:29.327185    2565 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-208.ec2.internal\": node \"ip-10-0-128-208.ec2.internal\" not found"
Sep 15 22:27:29.327704 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.327690    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-208.ec2.internal"
Sep 15 22:27:29.327761 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.327719    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 22:27:29.327945 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.327925    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-208.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 22:27:29.328037 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.327961    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-208.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 22:27:29.328037 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.327975    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-208.ec2.internal" event="NodeHasSufficientPID"
Sep 15 22:27:29.328965 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.328951    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-208.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 22:27:29.329015 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.328987    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-208.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 22:27:29.329015 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.328996    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-208.ec2.internal" event="NodeHasSufficientPID"
Sep 15 22:27:29.330241 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.330227    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-208.ec2.internal"
Sep 15 22:27:29.330308 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.330255    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 22:27:29.330926 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.330914    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-208.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 22:27:29.330976 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.330938    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-208.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 22:27:29.330976 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.330947    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-208.ec2.internal" event="NodeHasSufficientPID"
Sep 15 22:27:29.351991 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:29.351975    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-208.ec2.internal\" not found"
Sep 15 22:27:29.365492 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:29.365474    2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-208.ec2.internal\" not found" node="ip-10-0-128-208.ec2.internal"
Sep 15 22:27:29.369992 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:29.369976    2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-208.ec2.internal\" not found" node="ip-10-0-128-208.ec2.internal"
Sep 15 22:27:29.453054 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:29.453028    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-208.ec2.internal\" not found"
Sep 15 22:27:29.462332 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.462314    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b4901f8e95f7812e155a9ad303f1521a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-208.ec2.internal\" (UID: \"b4901f8e95f7812e155a9ad303f1521a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-208.ec2.internal"
Sep 15 22:27:29.462394 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.462338    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b4901f8e95f7812e155a9ad303f1521a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-208.ec2.internal\" (UID: \"b4901f8e95f7812e155a9ad303f1521a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-208.ec2.internal"
Sep 15 22:27:29.462394 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.462357    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d23b474b380f11cca08f39b02958fed1-config\") pod \"kube-apiserver-proxy-ip-10-0-128-208.ec2.internal\" (UID: \"d23b474b380f11cca08f39b02958fed1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-208.ec2.internal"
Sep 15 22:27:29.553841 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:29.553811    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-208.ec2.internal\" not found"
Sep 15 22:27:29.563164 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.563143    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b4901f8e95f7812e155a9ad303f1521a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-208.ec2.internal\" (UID: \"b4901f8e95f7812e155a9ad303f1521a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-208.ec2.internal"
Sep 15 22:27:29.563252 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.563168    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b4901f8e95f7812e155a9ad303f1521a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-208.ec2.internal\" (UID: \"b4901f8e95f7812e155a9ad303f1521a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-208.ec2.internal"
Sep 15 22:27:29.563252 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.563187    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d23b474b380f11cca08f39b02958fed1-config\") pod \"kube-apiserver-proxy-ip-10-0-128-208.ec2.internal\" (UID: \"d23b474b380f11cca08f39b02958fed1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-208.ec2.internal"
Sep 15 22:27:29.563252 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.563229    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d23b474b380f11cca08f39b02958fed1-config\") pod \"kube-apiserver-proxy-ip-10-0-128-208.ec2.internal\" (UID: \"d23b474b380f11cca08f39b02958fed1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-208.ec2.internal"
Sep 15 22:27:29.563371 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.563254    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b4901f8e95f7812e155a9ad303f1521a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-208.ec2.internal\" (UID: \"b4901f8e95f7812e155a9ad303f1521a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-208.ec2.internal"
Sep 15 22:27:29.563371 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.563256    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b4901f8e95f7812e155a9ad303f1521a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-208.ec2.internal\" (UID: \"b4901f8e95f7812e155a9ad303f1521a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-208.ec2.internal"
Sep 15 22:27:29.654641 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:29.654564    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-208.ec2.internal\" not found"
Sep 15 22:27:29.668087 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.668066    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-208.ec2.internal"
Sep 15 22:27:29.672236 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:29.672213    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-208.ec2.internal"
Sep 15 22:27:29.754717 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:29.754677    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-208.ec2.internal\" not found"
Sep 15 22:27:29.855210 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:29.855168    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-208.ec2.internal\" not found"
Sep 15 22:27:29.955720 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:29.955641    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-208.ec2.internal\" not found"
Sep 15 22:27:31.416495 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:31.416450    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-208.ec2.internal\" not found"
Sep 15 22:27:31.448933 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:31.448907    2565 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 15 22:27:31.449363 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:31.449086    2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 15 22:27:31.485502 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:31.485481    2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 22:27:31.516773 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:31.516746    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-208.ec2.internal\" not found"
Sep 15 22:27:31.531988 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:31.531952    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-14 22:22:30 +0000 UTC" deadline="2028-06-24 13:36:06.651811462 +0000 UTC"
Sep 15 22:27:31.531988 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:31.531981    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15543h8m35.119834335s"
Sep 15 22:27:31.601926 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:31.601894    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd23b474b380f11cca08f39b02958fed1.slice/crio-510c3aa09a8aa50fa0a86b17ab80fd95cdfb01aaecfc9f57d51d831944ad5106 WatchSource:0}: Error finding container 510c3aa09a8aa50fa0a86b17ab80fd95cdfb01aaecfc9f57d51d831944ad5106: Status 404 returned error can't find the container with id 510c3aa09a8aa50fa0a86b17ab80fd95cdfb01aaecfc9f57d51d831944ad5106
Sep 15 22:27:31.607592 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:31.607571    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 22:27:31.613109 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:31.613088    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4901f8e95f7812e155a9ad303f1521a.slice/crio-1b0b446caac799936f9680f46db8277661bd226dde7ad62b4b98902e7ad7c598 WatchSource:0}: Error finding container 1b0b446caac799936f9680f46db8277661bd226dde7ad62b4b98902e7ad7c598: Status 404 returned error can't find the container with id 1b0b446caac799936f9680f46db8277661bd226dde7ad62b4b98902e7ad7c598
Sep 15 22:27:31.617035 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:31.617005    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-208.ec2.internal\" not found"
Sep 15 22:27:31.646592 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:31.646571    2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 22:27:31.717132 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:31.717094    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-208.ec2.internal\" not found"
Sep 15 22:27:31.759988 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:31.759958    2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 22:27:31.821446 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:31.821418    2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-208.ec2.internal"
Sep 15 22:27:31.832792 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:31.832771    2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 15 22:27:31.834069 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:31.834053    2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-208.ec2.internal"
Sep 15 22:27:31.842544 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:31.842476    2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 15 22:27:32.496872 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.496840    2565 apiserver.go:52] "Watching apiserver"
Sep 15 22:27:32.503127 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.503099    2565 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 15 22:27:32.503493 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.503462    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-4d5lp","kube-system/konnectivity-agent-l9m9m","kube-system/kube-apiserver-proxy-ip-10-0-128-208.ec2.internal","openshift-cluster-node-tuning-operator/tuned-q689h","openshift-dns/node-resolver-m4w7h","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-208.ec2.internal","openshift-multus/multus-nmmxx","openshift-network-operator/iptables-alerter-d2c7v","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsbbm","openshift-image-registry/node-ca-8tjs6","openshift-multus/multus-additional-cni-plugins-5xcfb","openshift-multus/network-metrics-daemon-j4tsb","openshift-network-diagnostics/network-check-target-vqrqt","openshift-ovn-kubernetes/ovnkube-node-74hmh"]
Sep 15 22:27:32.505951 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.505928    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.508220 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.508198    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 15 22:27:32.508447 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.508414    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 15 22:27:32.508599 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.508576    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:27:32.508678 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.508661    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 15 22:27:32.508725 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:32.508702    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqrqt" podUID="e5ff0db4-42a1-4d9a-8ba1-1c6068724788"
Sep 15 22:27:32.509195 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.508873    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 15 22:27:32.509195 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.508991    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-vxqpm\""
Sep 15 22:27:32.510903 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.510536    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.512797 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.512756    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.515278 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.514986    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-m4w7h"
Sep 15 22:27:32.515737 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.515441    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 15 22:27:32.515737 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.515555    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 15 22:27:32.515737 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.515712    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 15 22:27:32.515895 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.515797    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 15 22:27:32.516017 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.516001    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 15 22:27:32.516449 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.516428    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-pgtcl\""
Sep 15 22:27:32.516542 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.516456    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 15 22:27:32.516542 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.516471    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ldz54\""
Sep 15 22:27:32.516643 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.516581    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 15 22:27:32.516693 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.516683    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 15 22:27:32.517316 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.517297    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-d2c7v"
Sep 15 22:27:32.517398 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.517379    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mzx62\""
Sep 15 22:27:32.517590 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.517575    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 15 22:27:32.517887 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.517869    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 15 22:27:32.519478 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.519463    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 15 22:27:32.519644 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.519629    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 15 22:27:32.519907 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.519890    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7m7v6\""
Sep 15 22:27:32.522311 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.520110    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsbbm"
Sep 15 22:27:32.522311 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.520187    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 15 22:27:32.522311 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.520856    2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 15 22:27:32.522519 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.522392    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:27:32.522519 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:32.522454    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4d5lp" podUID="b22214d5-9c42-4710-bfff-08651d7942a3"
Sep 15 22:27:32.522617 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.522565    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 15 22:27:32.522769 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.522750    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 15 22:27:32.522864 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.522768    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 15 22:27:32.522864 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.522841    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7ssbg\""
Sep 15 22:27:32.524524 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.524503    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-l9m9m"
Sep 15 22:27:32.526424 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.526407    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 15 22:27:32.526884 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.526833    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 15 22:27:32.527081 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.527066    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-7bpdd\""
Sep 15 22:27:32.527160 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.527068    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8tjs6"
Sep 15 22:27:32.529519 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.529174    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 15 22:27:32.529519 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.529191    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 15 22:27:32.529519 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.529487    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-xqbx5\""
Sep 15 22:27:32.529713 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.529623    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5xcfb"
Sep 15 22:27:32.529768 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.529714    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 15 22:27:32.533388 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.531964    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 15 22:27:32.533388 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.532256    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 15 22:27:32.533388 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.532314    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hmjfv\""
Sep 15 22:27:32.533388 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.532606    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-host-run-k8s-cni-cncf-io\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.533388 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.532636    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-multus-conf-dir\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.533388 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.532667    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f132ba8d-e9aa-4a73-8713-82fe09532325-multus-daemon-config\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.533388 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.532720    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbjzq\" (UniqueName: \"kubernetes.io/projected/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-kube-api-access-qbjzq\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.533388 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.532808    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvmnm\" (UniqueName: \"kubernetes.io/projected/2c8a934f-80db-473e-957d-fd3047d27df9-kube-api-access-cvmnm\") pod \"aws-ebs-csi-driver-node-zsbbm\" (UID: \"2c8a934f-80db-473e-957d-fd3047d27df9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsbbm"
Sep 15 22:27:32.533388 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.532842    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-host-run-multus-certs\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.533388 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.532874    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-etc-kubernetes\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.533388 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.532932    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-etc-openvswitch\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.533388 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.532984    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2c8a934f-80db-473e-957d-fd3047d27df9-sys-fs\") pod \"aws-ebs-csi-driver-node-zsbbm\" (UID: \"2c8a934f-80db-473e-957d-fd3047d27df9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsbbm"
Sep 15 22:27:32.533388 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.533043    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b22214d5-9c42-4710-bfff-08651d7942a3-kubelet-config\") pod \"global-pull-secret-syncer-4d5lp\" (UID: \"b22214d5-9c42-4710-bfff-08651d7942a3\") " pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:27:32.533388 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.533110    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b22214d5-9c42-4710-bfff-08651d7942a3-original-pull-secret\") pod \"global-pull-secret-syncer-4d5lp\" (UID: \"b22214d5-9c42-4710-bfff-08651d7942a3\") " pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:27:32.533388 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.533181    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-host-run-netns\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.533388 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.533218    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-systemd-units\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.533388 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.533250    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-run-openvswitch\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.533388 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.533279    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-hostroot\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.534322 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.533308    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-run-systemd\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.534322 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.533332    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-log-socket\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.534322 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.533362    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-ovnkube-script-lib\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.534322 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.533392    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e706f82-e509-4ba2-9f07-c4f5508fa685-etc-kubernetes\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.534322 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.533432    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6stw\" (UniqueName: \"kubernetes.io/projected/4e706f82-e509-4ba2-9f07-c4f5508fa685-kube-api-access-s6stw\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.534322 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.533461    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2c8a934f-80db-473e-957d-fd3047d27df9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zsbbm\" (UID: \"2c8a934f-80db-473e-957d-fd3047d27df9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsbbm"
Sep 15 22:27:32.534322 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.533488    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-multus-socket-dir-parent\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.534322 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.533602    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-run-ovn\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.534322 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.533632    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4e706f82-e509-4ba2-9f07-c4f5508fa685-etc-modprobe-d\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.534322 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.533660    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4e706f82-e509-4ba2-9f07-c4f5508fa685-sys\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.534322 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.533703    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4e706f82-e509-4ba2-9f07-c4f5508fa685-lib-modules\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.534322 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.533728    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b22214d5-9c42-4710-bfff-08651d7942a3-dbus\") pod \"global-pull-secret-syncer-4d5lp\" (UID: \"b22214d5-9c42-4710-bfff-08651d7942a3\") " pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:27:32.534322 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.533746    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:27:32.534322 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.533754    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-host-run-netns\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.534322 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.533816    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-host-run-ovn-kubernetes\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.534322 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:32.533850    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j4tsb" podUID="909a094b-77e5-4f6e-9d69-6f4214156810"
Sep 15 22:27:32.534322 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.533938    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.534885 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.534066    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4e706f82-e509-4ba2-9f07-c4f5508fa685-etc-systemd\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.534885 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.534108    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2c8a934f-80db-473e-957d-fd3047d27df9-etc-selinux\") pod \"aws-ebs-csi-driver-node-zsbbm\" (UID: \"2c8a934f-80db-473e-957d-fd3047d27df9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsbbm"
Sep 15 22:27:32.534885 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.534213    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-host-slash\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.534885 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.534258    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-node-log\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.534885 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.534313    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-host-cni-bin\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.534885 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.534362    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/caf80c9a-3157-42ba-86bb-4be3a7932714-host-slash\") pod \"iptables-alerter-d2c7v\" (UID: \"caf80c9a-3157-42ba-86bb-4be3a7932714\") " pod="openshift-network-operator/iptables-alerter-d2c7v"
Sep 15 22:27:32.534885 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.534395    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c47nf\" (UniqueName: \"kubernetes.io/projected/caf80c9a-3157-42ba-86bb-4be3a7932714-kube-api-access-c47nf\") pod \"iptables-alerter-d2c7v\" (UID: \"caf80c9a-3157-42ba-86bb-4be3a7932714\") " pod="openshift-network-operator/iptables-alerter-d2c7v"
Sep 15 22:27:32.534885 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.534431    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-cnibin\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.534885 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.534458    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4e706f82-e509-4ba2-9f07-c4f5508fa685-run\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.534885 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.534492    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4e706f82-e509-4ba2-9f07-c4f5508fa685-etc-tuned\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.534885 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.534520    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1bd5f596-e129-4139-97d6-b8f7696c26e4-hosts-file\") pod \"node-resolver-m4w7h\" (UID: \"1bd5f596-e129-4139-97d6-b8f7696c26e4\") " pod="openshift-dns/node-resolver-m4w7h"
Sep 15 22:27:32.534885 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.534599    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxp9c\" (UniqueName: \"kubernetes.io/projected/1bd5f596-e129-4139-97d6-b8f7696c26e4-kube-api-access-vxp9c\") pod \"node-resolver-m4w7h\" (UID: \"1bd5f596-e129-4139-97d6-b8f7696c26e4\") " pod="openshift-dns/node-resolver-m4w7h"
Sep 15 22:27:32.534885 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.534630    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-host-kubelet\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.534885 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.534659    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-ovnkube-config\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.534885 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.534695    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-ovn-node-metrics-cert\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.534885 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.534721    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4e706f82-e509-4ba2-9f07-c4f5508fa685-tmp\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.534885 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.534750    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2c8a934f-80db-473e-957d-fd3047d27df9-socket-dir\") pod \"aws-ebs-csi-driver-node-zsbbm\" (UID: \"2c8a934f-80db-473e-957d-fd3047d27df9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsbbm"
Sep 15 22:27:32.535713 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.534783    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2c8a934f-80db-473e-957d-fd3047d27df9-registration-dir\") pod \"aws-ebs-csi-driver-node-zsbbm\" (UID: \"2c8a934f-80db-473e-957d-fd3047d27df9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsbbm"
Sep 15 22:27:32.535713 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.534815    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f132ba8d-e9aa-4a73-8713-82fe09532325-cni-binary-copy\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.535713 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.534850    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-var-lib-openvswitch\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.535713 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.534888    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-host-cni-netd\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.535713 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.534929    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4e706f82-e509-4ba2-9f07-c4f5508fa685-etc-sysctl-d\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.535713 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.534954    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e706f82-e509-4ba2-9f07-c4f5508fa685-var-lib-kubelet\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.535713 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.534986    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-system-cni-dir\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.535713 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.535231    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-host-var-lib-cni-bin\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.535713 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.535289    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-host-var-lib-cni-multus\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.535713 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.535357    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4e706f82-e509-4ba2-9f07-c4f5508fa685-etc-sysconfig\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.535713 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.535565    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-host-var-lib-kubelet\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.535713 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.535632    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cp95\" (UniqueName: \"kubernetes.io/projected/f132ba8d-e9aa-4a73-8713-82fe09532325-kube-api-access-4cp95\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.535713 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.535662    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-env-overrides\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.535713 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.535717    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4e706f82-e509-4ba2-9f07-c4f5508fa685-etc-sysctl-conf\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.536243 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.535773    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vxxv\" (UniqueName: \"kubernetes.io/projected/e5ff0db4-42a1-4d9a-8ba1-1c6068724788-kube-api-access-9vxxv\") pod \"network-check-target-vqrqt\" (UID: \"e5ff0db4-42a1-4d9a-8ba1-1c6068724788\") " pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:27:32.536243 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.535818    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/caf80c9a-3157-42ba-86bb-4be3a7932714-iptables-alerter-script\") pod \"iptables-alerter-d2c7v\" (UID: \"caf80c9a-3157-42ba-86bb-4be3a7932714\") " pod="openshift-network-operator/iptables-alerter-d2c7v"
Sep 15 22:27:32.536243 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.535849    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-multus-cni-dir\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.536243 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.535911    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bd5f596-e129-4139-97d6-b8f7696c26e4-tmp-dir\") pod \"node-resolver-m4w7h\" (UID: \"1bd5f596-e129-4139-97d6-b8f7696c26e4\") " pod="openshift-dns/node-resolver-m4w7h"
Sep 15 22:27:32.536243 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.535944    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4e706f82-e509-4ba2-9f07-c4f5508fa685-host\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.536243 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.535972    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2c8a934f-80db-473e-957d-fd3047d27df9-device-dir\") pod \"aws-ebs-csi-driver-node-zsbbm\" (UID: \"2c8a934f-80db-473e-957d-fd3047d27df9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsbbm"
Sep 15 22:27:32.543339 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.543303    2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 15 22:27:32.564593 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.564336    2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-zp77f"
Sep 15 22:27:32.571335 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.571308    2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-zp77f"
Sep 15 22:27:32.585925 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.585904    2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 22:27:32.590741 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.590693    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-208.ec2.internal" event={"ID":"b4901f8e95f7812e155a9ad303f1521a","Type":"ContainerStarted","Data":"1b0b446caac799936f9680f46db8277661bd226dde7ad62b4b98902e7ad7c598"}
Sep 15 22:27:32.592392 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.592366    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-208.ec2.internal" event={"ID":"d23b474b380f11cca08f39b02958fed1","Type":"ContainerStarted","Data":"510c3aa09a8aa50fa0a86b17ab80fd95cdfb01aaecfc9f57d51d831944ad5106"}
Sep 15 22:27:32.621956 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.621928    2565 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 15 22:27:32.637002 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.636977    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4e706f82-e509-4ba2-9f07-c4f5508fa685-run\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.637002 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637007    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4e706f82-e509-4ba2-9f07-c4f5508fa685-etc-tuned\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.637234 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637041    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1bd5f596-e129-4139-97d6-b8f7696c26e4-hosts-file\") pod \"node-resolver-m4w7h\" (UID: \"1bd5f596-e129-4139-97d6-b8f7696c26e4\") " pod="openshift-dns/node-resolver-m4w7h"
Sep 15 22:27:32.637234 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637109    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1bd5f596-e129-4139-97d6-b8f7696c26e4-hosts-file\") pod \"node-resolver-m4w7h\" (UID: \"1bd5f596-e129-4139-97d6-b8f7696c26e4\") " pod="openshift-dns/node-resolver-m4w7h"
Sep 15 22:27:32.637234 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637122    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4e706f82-e509-4ba2-9f07-c4f5508fa685-run\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.637234 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637170    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxp9c\" (UniqueName: \"kubernetes.io/projected/1bd5f596-e129-4139-97d6-b8f7696c26e4-kube-api-access-vxp9c\") pod \"node-resolver-m4w7h\" (UID: \"1bd5f596-e129-4139-97d6-b8f7696c26e4\") " pod="openshift-dns/node-resolver-m4w7h"
Sep 15 22:27:32.637234 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637199    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-host-kubelet\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.637234 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637227    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-ovnkube-config\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.637509 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637243    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-ovn-node-metrics-cert\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.637509 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637276    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4e706f82-e509-4ba2-9f07-c4f5508fa685-tmp\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.637509 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637299    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-host-kubelet\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.637509 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637302    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2c8a934f-80db-473e-957d-fd3047d27df9-socket-dir\") pod \"aws-ebs-csi-driver-node-zsbbm\" (UID: \"2c8a934f-80db-473e-957d-fd3047d27df9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsbbm"
Sep 15 22:27:32.637509 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637340    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2c8a934f-80db-473e-957d-fd3047d27df9-registration-dir\") pod \"aws-ebs-csi-driver-node-zsbbm\" (UID: \"2c8a934f-80db-473e-957d-fd3047d27df9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsbbm"
Sep 15 22:27:32.637509 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637367    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e096afa4-418d-41a6-95ee-a181ccd16731-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5xcfb\" (UID: \"e096afa4-418d-41a6-95ee-a181ccd16731\") " pod="openshift-multus/multus-additional-cni-plugins-5xcfb"
Sep 15 22:27:32.637509 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637389    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f132ba8d-e9aa-4a73-8713-82fe09532325-cni-binary-copy\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.637509 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637419    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-var-lib-openvswitch\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.637509 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637439    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-host-cni-netd\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.637509 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637457    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4e706f82-e509-4ba2-9f07-c4f5508fa685-etc-sysctl-d\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.637509 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637476    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e706f82-e509-4ba2-9f07-c4f5508fa685-var-lib-kubelet\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.637509 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637497    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-system-cni-dir\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.638069 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637510    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2c8a934f-80db-473e-957d-fd3047d27df9-registration-dir\") pod \"aws-ebs-csi-driver-node-zsbbm\" (UID: \"2c8a934f-80db-473e-957d-fd3047d27df9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsbbm"
Sep 15 22:27:32.638069 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637517    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-host-var-lib-cni-bin\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.638069 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637536    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-host-cni-netd\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.638069 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637565    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-host-var-lib-cni-multus\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.638069 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637571    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-host-var-lib-cni-bin\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.638069 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637592    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4e706f82-e509-4ba2-9f07-c4f5508fa685-etc-sysconfig\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.638069 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637598    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-var-lib-openvswitch\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.638069 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637625    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpg2l\" (UniqueName: \"kubernetes.io/projected/e096afa4-418d-41a6-95ee-a181ccd16731-kube-api-access-mpg2l\") pod \"multus-additional-cni-plugins-5xcfb\" (UID: \"e096afa4-418d-41a6-95ee-a181ccd16731\") " pod="openshift-multus/multus-additional-cni-plugins-5xcfb"
Sep 15 22:27:32.638069 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637642    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e706f82-e509-4ba2-9f07-c4f5508fa685-var-lib-kubelet\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.638069 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637653    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-host-var-lib-kubelet\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.638069 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637671    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4e706f82-e509-4ba2-9f07-c4f5508fa685-etc-sysconfig\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.638069 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637669    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4e706f82-e509-4ba2-9f07-c4f5508fa685-etc-sysctl-d\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.638069 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637679    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cp95\" (UniqueName: \"kubernetes.io/projected/f132ba8d-e9aa-4a73-8713-82fe09532325-kube-api-access-4cp95\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.638069 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637696    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-host-var-lib-cni-multus\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.638069 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637744    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e096afa4-418d-41a6-95ee-a181ccd16731-cni-binary-copy\") pod \"multus-additional-cni-plugins-5xcfb\" (UID: \"e096afa4-418d-41a6-95ee-a181ccd16731\") " pod="openshift-multus/multus-additional-cni-plugins-5xcfb"
Sep 15 22:27:32.638069 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637750    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-host-var-lib-kubelet\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.638069 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637751    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-system-cni-dir\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.638069 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637749    2565 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 15 22:27:32.638942 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637782    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e096afa4-418d-41a6-95ee-a181ccd16731-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5xcfb\" (UID: \"e096afa4-418d-41a6-95ee-a181ccd16731\") " pod="openshift-multus/multus-additional-cni-plugins-5xcfb"
Sep 15 22:27:32.638942 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637819    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-env-overrides\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.638942 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637843    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4e706f82-e509-4ba2-9f07-c4f5508fa685-etc-sysctl-conf\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.638942 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637866    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e096afa4-418d-41a6-95ee-a181ccd16731-cnibin\") pod \"multus-additional-cni-plugins-5xcfb\" (UID: \"e096afa4-418d-41a6-95ee-a181ccd16731\") " pod="openshift-multus/multus-additional-cni-plugins-5xcfb"
Sep 15 22:27:32.638942 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637886    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-ovnkube-config\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.638942 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637890    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vxxv\" (UniqueName: \"kubernetes.io/projected/e5ff0db4-42a1-4d9a-8ba1-1c6068724788-kube-api-access-9vxxv\") pod \"network-check-target-vqrqt\" (UID: \"e5ff0db4-42a1-4d9a-8ba1-1c6068724788\") " pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:27:32.638942 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637965    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/caf80c9a-3157-42ba-86bb-4be3a7932714-iptables-alerter-script\") pod \"iptables-alerter-d2c7v\" (UID: \"caf80c9a-3157-42ba-86bb-4be3a7932714\") " pod="openshift-network-operator/iptables-alerter-d2c7v"
Sep 15 22:27:32.638942 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.637995    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-multus-cni-dir\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.638942 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638020    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bd5f596-e129-4139-97d6-b8f7696c26e4-tmp-dir\") pod \"node-resolver-m4w7h\" (UID: \"1bd5f596-e129-4139-97d6-b8f7696c26e4\") " pod="openshift-dns/node-resolver-m4w7h"
Sep 15 22:27:32.638942 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638085    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/93274f69-efc2-48a0-8223-f3a37aca3947-agent-certs\") pod \"konnectivity-agent-l9m9m\" (UID: \"93274f69-efc2-48a0-8223-f3a37aca3947\") " pod="kube-system/konnectivity-agent-l9m9m"
Sep 15 22:27:32.638942 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638107    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f132ba8d-e9aa-4a73-8713-82fe09532325-cni-binary-copy\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.638942 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638127    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4e706f82-e509-4ba2-9f07-c4f5508fa685-host\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.638942 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638152    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2c8a934f-80db-473e-957d-fd3047d27df9-device-dir\") pod \"aws-ebs-csi-driver-node-zsbbm\" (UID: \"2c8a934f-80db-473e-957d-fd3047d27df9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsbbm"
Sep 15 22:27:32.638942 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638179    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e096afa4-418d-41a6-95ee-a181ccd16731-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5xcfb\" (UID: \"e096afa4-418d-41a6-95ee-a181ccd16731\") " pod="openshift-multus/multus-additional-cni-plugins-5xcfb"
Sep 15 22:27:32.638942 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638189    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-multus-cni-dir\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.638942 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638203    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7n27\" (UniqueName: \"kubernetes.io/projected/d6f6b883-5742-4ebd-bde1-5c7295563940-kube-api-access-s7n27\") pod \"node-ca-8tjs6\" (UID: \"d6f6b883-5742-4ebd-bde1-5c7295563940\") " pod="openshift-image-registry/node-ca-8tjs6"
Sep 15 22:27:32.638942 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638230    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-host-run-k8s-cni-cncf-io\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.639774 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638244    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2c8a934f-80db-473e-957d-fd3047d27df9-socket-dir\") pod \"aws-ebs-csi-driver-node-zsbbm\" (UID: \"2c8a934f-80db-473e-957d-fd3047d27df9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsbbm"
Sep 15 22:27:32.639774 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638253    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-multus-conf-dir\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.639774 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638277    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f132ba8d-e9aa-4a73-8713-82fe09532325-multus-daemon-config\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.639774 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638297    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4e706f82-e509-4ba2-9f07-c4f5508fa685-etc-sysctl-conf\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.639774 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638301    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbjzq\" (UniqueName: \"kubernetes.io/projected/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-kube-api-access-qbjzq\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.639774 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638315    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-env-overrides\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.639774 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638358    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvmnm\" (UniqueName: \"kubernetes.io/projected/2c8a934f-80db-473e-957d-fd3047d27df9-kube-api-access-cvmnm\") pod \"aws-ebs-csi-driver-node-zsbbm\" (UID: \"2c8a934f-80db-473e-957d-fd3047d27df9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsbbm"
Sep 15 22:27:32.639774 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638383    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-host-run-k8s-cni-cncf-io\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.639774 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638385    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/909a094b-77e5-4f6e-9d69-6f4214156810-metrics-certs\") pod \"network-metrics-daemon-j4tsb\" (UID: \"909a094b-77e5-4f6e-9d69-6f4214156810\") " pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:27:32.639774 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638431    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-host-run-multus-certs\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.639774 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638459    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-etc-kubernetes\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.639774 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638469    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bd5f596-e129-4139-97d6-b8f7696c26e4-tmp-dir\") pod \"node-resolver-m4w7h\" (UID: \"1bd5f596-e129-4139-97d6-b8f7696c26e4\") " pod="openshift-dns/node-resolver-m4w7h"
Sep 15 22:27:32.639774 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638467    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2c8a934f-80db-473e-957d-fd3047d27df9-device-dir\") pod \"aws-ebs-csi-driver-node-zsbbm\" (UID: \"2c8a934f-80db-473e-957d-fd3047d27df9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsbbm"
Sep 15 22:27:32.639774 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638486    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-etc-openvswitch\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.639774 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638432    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-multus-conf-dir\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.639774 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638536    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-host-run-multus-certs\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.639774 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638575    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-etc-kubernetes\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.639774 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638579    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4e706f82-e509-4ba2-9f07-c4f5508fa685-host\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.640637 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638657    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2c8a934f-80db-473e-957d-fd3047d27df9-sys-fs\") pod \"aws-ebs-csi-driver-node-zsbbm\" (UID: \"2c8a934f-80db-473e-957d-fd3047d27df9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsbbm"
Sep 15 22:27:32.640637 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638691    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b22214d5-9c42-4710-bfff-08651d7942a3-kubelet-config\") pod \"global-pull-secret-syncer-4d5lp\" (UID: \"b22214d5-9c42-4710-bfff-08651d7942a3\") " pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:27:32.640637 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638753    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b22214d5-9c42-4710-bfff-08651d7942a3-original-pull-secret\") pod \"global-pull-secret-syncer-4d5lp\" (UID: \"b22214d5-9c42-4710-bfff-08651d7942a3\") " pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:27:32.640637 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638781    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-host-run-netns\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.640637 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638805    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-systemd-units\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.640637 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638832    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-run-openvswitch\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.640637 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638803    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2c8a934f-80db-473e-957d-fd3047d27df9-sys-fs\") pod \"aws-ebs-csi-driver-node-zsbbm\" (UID: \"2c8a934f-80db-473e-957d-fd3047d27df9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsbbm"
Sep 15 22:27:32.640637 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638841    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b22214d5-9c42-4710-bfff-08651d7942a3-kubelet-config\") pod \"global-pull-secret-syncer-4d5lp\" (UID: \"b22214d5-9c42-4710-bfff-08651d7942a3\") " pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:27:32.640637 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638885    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/caf80c9a-3157-42ba-86bb-4be3a7932714-iptables-alerter-script\") pod \"iptables-alerter-d2c7v\" (UID: \"caf80c9a-3157-42ba-86bb-4be3a7932714\") " pod="openshift-network-operator/iptables-alerter-d2c7v"
Sep 15 22:27:32.640637 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638893    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-systemd-units\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.640637 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638887    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-run-openvswitch\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.640637 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:32.638903    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 22:27:32.640637 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.638916    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-etc-openvswitch\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.640637 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:32.638971    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b22214d5-9c42-4710-bfff-08651d7942a3-original-pull-secret podName:b22214d5-9c42-4710-bfff-08651d7942a3 nodeName:}" failed. No retries permitted until 2026-09-15 22:27:33.138954212 +0000 UTC m=+2.998665494 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b22214d5-9c42-4710-bfff-08651d7942a3-original-pull-secret") pod "global-pull-secret-syncer-4d5lp" (UID: "b22214d5-9c42-4710-bfff-08651d7942a3") : object "kube-system"/"original-pull-secret" not registered
Sep 15 22:27:32.640637 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639035    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-host-run-netns\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.640637 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639068    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-hostroot\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.640637 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639111    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-run-systemd\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.641234 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639134    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-hostroot\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.641234 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639174    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-run-systemd\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.641234 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639170    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-log-socket\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.641234 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639220    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-ovnkube-script-lib\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.641234 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639246    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e706f82-e509-4ba2-9f07-c4f5508fa685-etc-kubernetes\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.641234 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639273    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6stw\" (UniqueName: \"kubernetes.io/projected/4e706f82-e509-4ba2-9f07-c4f5508fa685-kube-api-access-s6stw\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.641234 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639295    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2c8a934f-80db-473e-957d-fd3047d27df9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zsbbm\" (UID: \"2c8a934f-80db-473e-957d-fd3047d27df9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsbbm"
Sep 15 22:27:32.641234 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639318    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-multus-socket-dir-parent\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.641234 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639345    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-run-ovn\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.641234 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639367    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4e706f82-e509-4ba2-9f07-c4f5508fa685-etc-modprobe-d\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.641234 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639370    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f132ba8d-e9aa-4a73-8713-82fe09532325-multus-daemon-config\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.641234 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639388    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4e706f82-e509-4ba2-9f07-c4f5508fa685-sys\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.641234 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639423    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4e706f82-e509-4ba2-9f07-c4f5508fa685-lib-modules\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.641234 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639437    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4e706f82-e509-4ba2-9f07-c4f5508fa685-sys\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.641234 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639451    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e096afa4-418d-41a6-95ee-a181ccd16731-system-cni-dir\") pod \"multus-additional-cni-plugins-5xcfb\" (UID: \"e096afa4-418d-41a6-95ee-a181ccd16731\") " pod="openshift-multus/multus-additional-cni-plugins-5xcfb"
Sep 15 22:27:32.641234 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639480    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/93274f69-efc2-48a0-8223-f3a37aca3947-konnectivity-ca\") pod \"konnectivity-agent-l9m9m\" (UID: \"93274f69-efc2-48a0-8223-f3a37aca3947\") " pod="kube-system/konnectivity-agent-l9m9m"
Sep 15 22:27:32.641234 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639483    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e706f82-e509-4ba2-9f07-c4f5508fa685-etc-kubernetes\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.641915 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639504    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b22214d5-9c42-4710-bfff-08651d7942a3-dbus\") pod \"global-pull-secret-syncer-4d5lp\" (UID: \"b22214d5-9c42-4710-bfff-08651d7942a3\") " pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:27:32.641915 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639544    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-host-run-netns\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.641915 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639568    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-host-run-ovn-kubernetes\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.641915 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639593    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.641915 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639618    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4e706f82-e509-4ba2-9f07-c4f5508fa685-etc-systemd\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.641915 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639641    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2c8a934f-80db-473e-957d-fd3047d27df9-etc-selinux\") pod \"aws-ebs-csi-driver-node-zsbbm\" (UID: \"2c8a934f-80db-473e-957d-fd3047d27df9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsbbm"
Sep 15 22:27:32.641915 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639665    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d6f6b883-5742-4ebd-bde1-5c7295563940-serviceca\") pod \"node-ca-8tjs6\" (UID: \"d6f6b883-5742-4ebd-bde1-5c7295563940\") " pod="openshift-image-registry/node-ca-8tjs6"
Sep 15 22:27:32.641915 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639687    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2c8a934f-80db-473e-957d-fd3047d27df9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zsbbm\" (UID: \"2c8a934f-80db-473e-957d-fd3047d27df9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsbbm"
Sep 15 22:27:32.641915 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639690    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-host-slash\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.641915 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639732    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-host-slash\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.641915 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639731    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-node-log\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.641915 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639769    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-ovnkube-script-lib\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.641915 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639775    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-host-cni-bin\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.641915 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639804    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-host-cni-bin\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.641915 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639815    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4e706f82-e509-4ba2-9f07-c4f5508fa685-lib-modules\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.641915 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639830    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2k6ws\" (UniqueName: \"kubernetes.io/projected/909a094b-77e5-4f6e-9d69-6f4214156810-kube-api-access-2k6ws\") pod \"network-metrics-daemon-j4tsb\" (UID: \"909a094b-77e5-4f6e-9d69-6f4214156810\") " pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:27:32.641915 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639850    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-multus-socket-dir-parent\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.642707 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639859    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d6f6b883-5742-4ebd-bde1-5c7295563940-host\") pod \"node-ca-8tjs6\" (UID: \"d6f6b883-5742-4ebd-bde1-5c7295563940\") " pod="openshift-image-registry/node-ca-8tjs6"
Sep 15 22:27:32.642707 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639890    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b22214d5-9c42-4710-bfff-08651d7942a3-dbus\") pod \"global-pull-secret-syncer-4d5lp\" (UID: \"b22214d5-9c42-4710-bfff-08651d7942a3\") " pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:27:32.642707 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639892    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-run-ovn\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.642707 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639930    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/caf80c9a-3157-42ba-86bb-4be3a7932714-host-slash\") pod \"iptables-alerter-d2c7v\" (UID: \"caf80c9a-3157-42ba-86bb-4be3a7932714\") " pod="openshift-network-operator/iptables-alerter-d2c7v"
Sep 15 22:27:32.642707 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639931    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-log-socket\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.642707 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639932    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-node-log\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.642707 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639966    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.642707 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.640000    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4e706f82-e509-4ba2-9f07-c4f5508fa685-etc-systemd\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.642707 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.639890    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/caf80c9a-3157-42ba-86bb-4be3a7932714-host-slash\") pod \"iptables-alerter-d2c7v\" (UID: \"caf80c9a-3157-42ba-86bb-4be3a7932714\") " pod="openshift-network-operator/iptables-alerter-d2c7v"
Sep 15 22:27:32.642707 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.640074    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c47nf\" (UniqueName: \"kubernetes.io/projected/caf80c9a-3157-42ba-86bb-4be3a7932714-kube-api-access-c47nf\") pod \"iptables-alerter-d2c7v\" (UID: \"caf80c9a-3157-42ba-86bb-4be3a7932714\") " pod="openshift-network-operator/iptables-alerter-d2c7v"
Sep 15 22:27:32.642707 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.640103    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2c8a934f-80db-473e-957d-fd3047d27df9-etc-selinux\") pod \"aws-ebs-csi-driver-node-zsbbm\" (UID: \"2c8a934f-80db-473e-957d-fd3047d27df9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsbbm"
Sep 15 22:27:32.642707 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.640108    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-cnibin\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.642707 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.640150    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f132ba8d-e9aa-4a73-8713-82fe09532325-cnibin\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.642707 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.640039    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-host-run-ovn-kubernetes\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.642707 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.640002    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-host-run-netns\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.642707 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.640209    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4e706f82-e509-4ba2-9f07-c4f5508fa685-etc-modprobe-d\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.642707 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.641890    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4e706f82-e509-4ba2-9f07-c4f5508fa685-etc-tuned\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.642707 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.641940    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4e706f82-e509-4ba2-9f07-c4f5508fa685-tmp\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.643377 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.642186    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-ovn-node-metrics-cert\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.649633 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:32.649610    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 22:27:32.649633 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:32.649636    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 22:27:32.649633 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:32.649648    2565 projected.go:196] Error preparing data for projected volume kube-api-access-9vxxv for pod openshift-network-diagnostics/network-check-target-vqrqt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 22:27:32.649885 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:32.649740    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e5ff0db4-42a1-4d9a-8ba1-1c6068724788-kube-api-access-9vxxv podName:e5ff0db4-42a1-4d9a-8ba1-1c6068724788 nodeName:}" failed. No retries permitted until 2026-09-15 22:27:33.149709585 +0000 UTC m=+3.009420864 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-9vxxv" (UniqueName: "kubernetes.io/projected/e5ff0db4-42a1-4d9a-8ba1-1c6068724788-kube-api-access-9vxxv") pod "network-check-target-vqrqt" (UID: "e5ff0db4-42a1-4d9a-8ba1-1c6068724788") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 22:27:32.652665 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.652638    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvmnm\" (UniqueName: \"kubernetes.io/projected/2c8a934f-80db-473e-957d-fd3047d27df9-kube-api-access-cvmnm\") pod \"aws-ebs-csi-driver-node-zsbbm\" (UID: \"2c8a934f-80db-473e-957d-fd3047d27df9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsbbm"
Sep 15 22:27:32.652776 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.652683    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbjzq\" (UniqueName: \"kubernetes.io/projected/bcb145e6-76ea-43fd-98c0-2c30c59dc2d0-kube-api-access-qbjzq\") pod \"ovnkube-node-74hmh\" (UID: \"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.653915 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.653882    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxp9c\" (UniqueName: \"kubernetes.io/projected/1bd5f596-e129-4139-97d6-b8f7696c26e4-kube-api-access-vxp9c\") pod \"node-resolver-m4w7h\" (UID: \"1bd5f596-e129-4139-97d6-b8f7696c26e4\") " pod="openshift-dns/node-resolver-m4w7h"
Sep 15 22:27:32.654791 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.654767    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cp95\" (UniqueName: \"kubernetes.io/projected/f132ba8d-e9aa-4a73-8713-82fe09532325-kube-api-access-4cp95\") pod \"multus-nmmxx\" (UID: \"f132ba8d-e9aa-4a73-8713-82fe09532325\") " pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.654881 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.654844    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c47nf\" (UniqueName: \"kubernetes.io/projected/caf80c9a-3157-42ba-86bb-4be3a7932714-kube-api-access-c47nf\") pod \"iptables-alerter-d2c7v\" (UID: \"caf80c9a-3157-42ba-86bb-4be3a7932714\") " pod="openshift-network-operator/iptables-alerter-d2c7v"
Sep 15 22:27:32.655129 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.655110    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6stw\" (UniqueName: \"kubernetes.io/projected/4e706f82-e509-4ba2-9f07-c4f5508fa685-kube-api-access-s6stw\") pod \"tuned-q689h\" (UID: \"4e706f82-e509-4ba2-9f07-c4f5508fa685\") " pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.740691 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.740654    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpg2l\" (UniqueName: \"kubernetes.io/projected/e096afa4-418d-41a6-95ee-a181ccd16731-kube-api-access-mpg2l\") pod \"multus-additional-cni-plugins-5xcfb\" (UID: \"e096afa4-418d-41a6-95ee-a181ccd16731\") " pod="openshift-multus/multus-additional-cni-plugins-5xcfb"
Sep 15 22:27:32.740691 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.740700    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e096afa4-418d-41a6-95ee-a181ccd16731-cni-binary-copy\") pod \"multus-additional-cni-plugins-5xcfb\" (UID: \"e096afa4-418d-41a6-95ee-a181ccd16731\") " pod="openshift-multus/multus-additional-cni-plugins-5xcfb"
Sep 15 22:27:32.740937 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.740725    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e096afa4-418d-41a6-95ee-a181ccd16731-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5xcfb\" (UID: \"e096afa4-418d-41a6-95ee-a181ccd16731\") " pod="openshift-multus/multus-additional-cni-plugins-5xcfb"
Sep 15 22:27:32.740937 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.740755    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e096afa4-418d-41a6-95ee-a181ccd16731-cnibin\") pod \"multus-additional-cni-plugins-5xcfb\" (UID: \"e096afa4-418d-41a6-95ee-a181ccd16731\") " pod="openshift-multus/multus-additional-cni-plugins-5xcfb"
Sep 15 22:27:32.740937 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.740790    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/93274f69-efc2-48a0-8223-f3a37aca3947-agent-certs\") pod \"konnectivity-agent-l9m9m\" (UID: \"93274f69-efc2-48a0-8223-f3a37aca3947\") " pod="kube-system/konnectivity-agent-l9m9m"
Sep 15 22:27:32.740937 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.740810    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e096afa4-418d-41a6-95ee-a181ccd16731-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5xcfb\" (UID: \"e096afa4-418d-41a6-95ee-a181ccd16731\") " pod="openshift-multus/multus-additional-cni-plugins-5xcfb"
Sep 15 22:27:32.740937 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.740826    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7n27\" (UniqueName: \"kubernetes.io/projected/d6f6b883-5742-4ebd-bde1-5c7295563940-kube-api-access-s7n27\") pod \"node-ca-8tjs6\" (UID: \"d6f6b883-5742-4ebd-bde1-5c7295563940\") " pod="openshift-image-registry/node-ca-8tjs6"
Sep 15 22:27:32.740937 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.740863    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/909a094b-77e5-4f6e-9d69-6f4214156810-metrics-certs\") pod \"network-metrics-daemon-j4tsb\" (UID: \"909a094b-77e5-4f6e-9d69-6f4214156810\") " pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:27:32.740937 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.740901    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e096afa4-418d-41a6-95ee-a181ccd16731-system-cni-dir\") pod \"multus-additional-cni-plugins-5xcfb\" (UID: \"e096afa4-418d-41a6-95ee-a181ccd16731\") " pod="openshift-multus/multus-additional-cni-plugins-5xcfb"
Sep 15 22:27:32.740937 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.740924    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/93274f69-efc2-48a0-8223-f3a37aca3947-konnectivity-ca\") pod \"konnectivity-agent-l9m9m\" (UID: \"93274f69-efc2-48a0-8223-f3a37aca3947\") " pod="kube-system/konnectivity-agent-l9m9m"
Sep 15 22:27:32.741338 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.740952    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d6f6b883-5742-4ebd-bde1-5c7295563940-serviceca\") pod \"node-ca-8tjs6\" (UID: \"d6f6b883-5742-4ebd-bde1-5c7295563940\") " pod="openshift-image-registry/node-ca-8tjs6"
Sep 15 22:27:32.741338 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.740970    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e096afa4-418d-41a6-95ee-a181ccd16731-cnibin\") pod \"multus-additional-cni-plugins-5xcfb\" (UID: \"e096afa4-418d-41a6-95ee-a181ccd16731\") " pod="openshift-multus/multus-additional-cni-plugins-5xcfb"
Sep 15 22:27:32.741338 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.740978    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2k6ws\" (UniqueName: \"kubernetes.io/projected/909a094b-77e5-4f6e-9d69-6f4214156810-kube-api-access-2k6ws\") pod \"network-metrics-daemon-j4tsb\" (UID: \"909a094b-77e5-4f6e-9d69-6f4214156810\") " pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:27:32.741338 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.741044    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d6f6b883-5742-4ebd-bde1-5c7295563940-host\") pod \"node-ca-8tjs6\" (UID: \"d6f6b883-5742-4ebd-bde1-5c7295563940\") " pod="openshift-image-registry/node-ca-8tjs6"
Sep 15 22:27:32.741338 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.741060    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e096afa4-418d-41a6-95ee-a181ccd16731-system-cni-dir\") pod \"multus-additional-cni-plugins-5xcfb\" (UID: \"e096afa4-418d-41a6-95ee-a181ccd16731\") " pod="openshift-multus/multus-additional-cni-plugins-5xcfb"
Sep 15 22:27:32.741338 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.741084    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e096afa4-418d-41a6-95ee-a181ccd16731-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5xcfb\" (UID: \"e096afa4-418d-41a6-95ee-a181ccd16731\") " pod="openshift-multus/multus-additional-cni-plugins-5xcfb"
Sep 15 22:27:32.741338 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.741116    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e096afa4-418d-41a6-95ee-a181ccd16731-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5xcfb\" (UID: \"e096afa4-418d-41a6-95ee-a181ccd16731\") " pod="openshift-multus/multus-additional-cni-plugins-5xcfb"
Sep 15 22:27:32.741338 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:32.741227    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 22:27:32.741338 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:32.741275    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/909a094b-77e5-4f6e-9d69-6f4214156810-metrics-certs podName:909a094b-77e5-4f6e-9d69-6f4214156810 nodeName:}" failed. No retries permitted until 2026-09-15 22:27:33.241259312 +0000 UTC m=+3.100970593 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/909a094b-77e5-4f6e-9d69-6f4214156810-metrics-certs") pod "network-metrics-daemon-j4tsb" (UID: "909a094b-77e5-4f6e-9d69-6f4214156810") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 22:27:32.741673 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.741350    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d6f6b883-5742-4ebd-bde1-5c7295563940-host\") pod \"node-ca-8tjs6\" (UID: \"d6f6b883-5742-4ebd-bde1-5c7295563940\") " pod="openshift-image-registry/node-ca-8tjs6"
Sep 15 22:27:32.741673 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.741360    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e096afa4-418d-41a6-95ee-a181ccd16731-cni-binary-copy\") pod \"multus-additional-cni-plugins-5xcfb\" (UID: \"e096afa4-418d-41a6-95ee-a181ccd16731\") " pod="openshift-multus/multus-additional-cni-plugins-5xcfb"
Sep 15 22:27:32.741673 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.741479    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e096afa4-418d-41a6-95ee-a181ccd16731-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5xcfb\" (UID: \"e096afa4-418d-41a6-95ee-a181ccd16731\") " pod="openshift-multus/multus-additional-cni-plugins-5xcfb"
Sep 15 22:27:32.741673 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.741583    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d6f6b883-5742-4ebd-bde1-5c7295563940-serviceca\") pod \"node-ca-8tjs6\" (UID: \"d6f6b883-5742-4ebd-bde1-5c7295563940\") " pod="openshift-image-registry/node-ca-8tjs6"
Sep 15 22:27:32.741863 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.741679    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e096afa4-418d-41a6-95ee-a181ccd16731-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5xcfb\" (UID: \"e096afa4-418d-41a6-95ee-a181ccd16731\") " pod="openshift-multus/multus-additional-cni-plugins-5xcfb"
Sep 15 22:27:32.741863 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.741751    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/93274f69-efc2-48a0-8223-f3a37aca3947-konnectivity-ca\") pod \"konnectivity-agent-l9m9m\" (UID: \"93274f69-efc2-48a0-8223-f3a37aca3947\") " pod="kube-system/konnectivity-agent-l9m9m"
Sep 15 22:27:32.743667 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.743639    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/93274f69-efc2-48a0-8223-f3a37aca3947-agent-certs\") pod \"konnectivity-agent-l9m9m\" (UID: \"93274f69-efc2-48a0-8223-f3a37aca3947\") " pod="kube-system/konnectivity-agent-l9m9m"
Sep 15 22:27:32.751580 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.751339    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2k6ws\" (UniqueName: \"kubernetes.io/projected/909a094b-77e5-4f6e-9d69-6f4214156810-kube-api-access-2k6ws\") pod \"network-metrics-daemon-j4tsb\" (UID: \"909a094b-77e5-4f6e-9d69-6f4214156810\") " pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:27:32.755402 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.755371    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7n27\" (UniqueName: \"kubernetes.io/projected/d6f6b883-5742-4ebd-bde1-5c7295563940-kube-api-access-s7n27\") pod \"node-ca-8tjs6\" (UID: \"d6f6b883-5742-4ebd-bde1-5c7295563940\") " pod="openshift-image-registry/node-ca-8tjs6"
Sep 15 22:27:32.755689 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.755667    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpg2l\" (UniqueName: \"kubernetes.io/projected/e096afa4-418d-41a6-95ee-a181ccd16731-kube-api-access-mpg2l\") pod \"multus-additional-cni-plugins-5xcfb\" (UID: \"e096afa4-418d-41a6-95ee-a181ccd16731\") " pod="openshift-multus/multus-additional-cni-plugins-5xcfb"
Sep 15 22:27:32.818491 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.818456    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nmmxx"
Sep 15 22:27:32.827194 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.827172    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:32.842867 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.842845    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-q689h"
Sep 15 22:27:32.843904 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.843885    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-m4w7h"
Sep 15 22:27:32.852749 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.852730    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-d2c7v"
Sep 15 22:27:32.860420 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.860399    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsbbm"
Sep 15 22:27:32.867963 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.867939    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-l9m9m"
Sep 15 22:27:32.874679 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.874662    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8tjs6"
Sep 15 22:27:32.881240 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:32.881223    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5xcfb"
Sep 15 22:27:33.143195 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:33.143124    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b22214d5-9c42-4710-bfff-08651d7942a3-original-pull-secret\") pod \"global-pull-secret-syncer-4d5lp\" (UID: \"b22214d5-9c42-4710-bfff-08651d7942a3\") " pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:27:33.143345 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:33.143276    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 22:27:33.143345 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:33.143331    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b22214d5-9c42-4710-bfff-08651d7942a3-original-pull-secret podName:b22214d5-9c42-4710-bfff-08651d7942a3 nodeName:}" failed. No retries permitted until 2026-09-15 22:27:34.143316618 +0000 UTC m=+4.003027893 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b22214d5-9c42-4710-bfff-08651d7942a3-original-pull-secret") pod "global-pull-secret-syncer-4d5lp" (UID: "b22214d5-9c42-4710-bfff-08651d7942a3") : object "kube-system"/"original-pull-secret" not registered
Sep 15 22:27:33.243809 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:33.243771    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vxxv\" (UniqueName: \"kubernetes.io/projected/e5ff0db4-42a1-4d9a-8ba1-1c6068724788-kube-api-access-9vxxv\") pod \"network-check-target-vqrqt\" (UID: \"e5ff0db4-42a1-4d9a-8ba1-1c6068724788\") " pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:27:33.243983 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:33.243829    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/909a094b-77e5-4f6e-9d69-6f4214156810-metrics-certs\") pod \"network-metrics-daemon-j4tsb\" (UID: \"909a094b-77e5-4f6e-9d69-6f4214156810\") " pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:27:33.243983 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:33.243951    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 22:27:33.243983 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:33.243961    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 22:27:33.243983 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:33.243984    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 22:27:33.244235 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:33.243999    2565 projected.go:196] Error preparing data for projected volume kube-api-access-9vxxv for pod openshift-network-diagnostics/network-check-target-vqrqt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 22:27:33.244235 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:33.244015    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/909a094b-77e5-4f6e-9d69-6f4214156810-metrics-certs podName:909a094b-77e5-4f6e-9d69-6f4214156810 nodeName:}" failed. No retries permitted until 2026-09-15 22:27:34.243997479 +0000 UTC m=+4.103708742 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/909a094b-77e5-4f6e-9d69-6f4214156810-metrics-certs") pod "network-metrics-daemon-j4tsb" (UID: "909a094b-77e5-4f6e-9d69-6f4214156810") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 22:27:33.244235 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:33.244066    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e5ff0db4-42a1-4d9a-8ba1-1c6068724788-kube-api-access-9vxxv podName:e5ff0db4-42a1-4d9a-8ba1-1c6068724788 nodeName:}" failed. No retries permitted until 2026-09-15 22:27:34.244047923 +0000 UTC m=+4.103759199 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-9vxxv" (UniqueName: "kubernetes.io/projected/e5ff0db4-42a1-4d9a-8ba1-1c6068724788-kube-api-access-9vxxv") pod "network-check-target-vqrqt" (UID: "e5ff0db4-42a1-4d9a-8ba1-1c6068724788") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 22:27:33.563776 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:33.563748    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode096afa4_418d_41a6_95ee_a181ccd16731.slice/crio-24901b58be8896f9aa60db8eeeb95156b702ec6838824e756aceddefb5ab76f4 WatchSource:0}: Error finding container 24901b58be8896f9aa60db8eeeb95156b702ec6838824e756aceddefb5ab76f4: Status 404 returned error can't find the container with id 24901b58be8896f9aa60db8eeeb95156b702ec6838824e756aceddefb5ab76f4
Sep 15 22:27:33.564506 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:33.564483    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bd5f596_e129_4139_97d6_b8f7696c26e4.slice/crio-49ce5d1aa13425fb16c5bdb69d8ea798c0767b0352f52782259b2ca052e59e10 WatchSource:0}: Error finding container 49ce5d1aa13425fb16c5bdb69d8ea798c0767b0352f52782259b2ca052e59e10: Status 404 returned error can't find the container with id 49ce5d1aa13425fb16c5bdb69d8ea798c0767b0352f52782259b2ca052e59e10
Sep 15 22:27:33.565633 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:33.565607    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf132ba8d_e9aa_4a73_8713_82fe09532325.slice/crio-4262e03474ecb7dc482f9ee244088b5cb71e5105b2f5dadccc4641aa8395d953 WatchSource:0}: Error finding container 4262e03474ecb7dc482f9ee244088b5cb71e5105b2f5dadccc4641aa8395d953: Status 404 returned error can't find the container with id 4262e03474ecb7dc482f9ee244088b5cb71e5105b2f5dadccc4641aa8395d953
Sep 15 22:27:33.569155 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:33.569135    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93274f69_efc2_48a0_8223_f3a37aca3947.slice/crio-2da0565f726b15707322e4efe86e070e2458d9c056770688dcb831861449df6d WatchSource:0}: Error finding container 2da0565f726b15707322e4efe86e070e2458d9c056770688dcb831861449df6d: Status 404 returned error can't find the container with id 2da0565f726b15707322e4efe86e070e2458d9c056770688dcb831861449df6d
Sep 15 22:27:33.570076 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:33.570055    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e706f82_e509_4ba2_9f07_c4f5508fa685.slice/crio-93d9304d1815039ff6ba79d0fd612751e386bf0f8b74c1e4c52ce9d33ee3c19b WatchSource:0}: Error finding container 93d9304d1815039ff6ba79d0fd612751e386bf0f8b74c1e4c52ce9d33ee3c19b: Status 404 returned error can't find the container with id 93d9304d1815039ff6ba79d0fd612751e386bf0f8b74c1e4c52ce9d33ee3c19b
Sep 15 22:27:33.571269 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:33.571224    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcb145e6_76ea_43fd_98c0_2c30c59dc2d0.slice/crio-6ab1879088302e2a1d0ada378b226b56850e606e7b7932cee37ea091ddbf3dbf WatchSource:0}: Error finding container 6ab1879088302e2a1d0ada378b226b56850e606e7b7932cee37ea091ddbf3dbf: Status 404 returned error can't find the container with id 6ab1879088302e2a1d0ada378b226b56850e606e7b7932cee37ea091ddbf3dbf
Sep 15 22:27:33.572890 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:33.572772    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6f6b883_5742_4ebd_bde1_5c7295563940.slice/crio-db3f978a6febd461acf14a1023b0917efde3a7d643384d395ecd039ba00b23a6 WatchSource:0}: Error finding container db3f978a6febd461acf14a1023b0917efde3a7d643384d395ecd039ba00b23a6: Status 404 returned error can't find the container with id db3f978a6febd461acf14a1023b0917efde3a7d643384d395ecd039ba00b23a6
Sep 15 22:27:33.573877 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:33.573759    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c8a934f_80db_473e_957d_fd3047d27df9.slice/crio-84be41bed8110d4a1b98bdcf2e8e7b2c33a1a388564f499e269575df9874387d WatchSource:0}: Error finding container 84be41bed8110d4a1b98bdcf2e8e7b2c33a1a388564f499e269575df9874387d: Status 404 returned error can't find the container with id 84be41bed8110d4a1b98bdcf2e8e7b2c33a1a388564f499e269575df9874387d
Sep 15 22:27:33.574744 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:33.574359    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-14 22:22:32 +0000 UTC" deadline="2028-03-19 10:39:16.164173837 +0000 UTC"
Sep 15 22:27:33.574744 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:33.574383    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13212h11m42.589794983s"
Sep 15 22:27:33.574744 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:27:33.574696    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaf80c9a_3157_42ba_86bb_4be3a7932714.slice/crio-50111c5f8c5e8d266a1235c908ac254fb2671bed3fa156d654cf62604301dd39 WatchSource:0}: Error finding container 50111c5f8c5e8d266a1235c908ac254fb2671bed3fa156d654cf62604301dd39: Status 404 returned error can't find the container with id 50111c5f8c5e8d266a1235c908ac254fb2671bed3fa156d654cf62604301dd39
Sep 15 22:27:33.584223 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:33.584202    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:27:33.584337 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:33.584317    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4d5lp" podUID="b22214d5-9c42-4710-bfff-08651d7942a3"
Sep 15 22:27:33.597564 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:33.597537    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8tjs6" event={"ID":"d6f6b883-5742-4ebd-bde1-5c7295563940","Type":"ContainerStarted","Data":"db3f978a6febd461acf14a1023b0917efde3a7d643384d395ecd039ba00b23a6"}
Sep 15 22:27:33.598623 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:33.598602    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-74hmh" event={"ID":"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0","Type":"ContainerStarted","Data":"6ab1879088302e2a1d0ada378b226b56850e606e7b7932cee37ea091ddbf3dbf"}
Sep 15 22:27:33.599788 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:33.599766    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-q689h" event={"ID":"4e706f82-e509-4ba2-9f07-c4f5508fa685","Type":"ContainerStarted","Data":"93d9304d1815039ff6ba79d0fd612751e386bf0f8b74c1e4c52ce9d33ee3c19b"}
Sep 15 22:27:33.600913 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:33.600891    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-l9m9m" event={"ID":"93274f69-efc2-48a0-8223-f3a37aca3947","Type":"ContainerStarted","Data":"2da0565f726b15707322e4efe86e070e2458d9c056770688dcb831861449df6d"}
Sep 15 22:27:33.602074 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:33.602054    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nmmxx" event={"ID":"f132ba8d-e9aa-4a73-8713-82fe09532325","Type":"ContainerStarted","Data":"4262e03474ecb7dc482f9ee244088b5cb71e5105b2f5dadccc4641aa8395d953"}
Sep 15 22:27:33.603706 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:33.603686    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-m4w7h" event={"ID":"1bd5f596-e129-4139-97d6-b8f7696c26e4","Type":"ContainerStarted","Data":"49ce5d1aa13425fb16c5bdb69d8ea798c0767b0352f52782259b2ca052e59e10"}
Sep 15 22:27:33.605888 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:33.605861    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-d2c7v" event={"ID":"caf80c9a-3157-42ba-86bb-4be3a7932714","Type":"ContainerStarted","Data":"50111c5f8c5e8d266a1235c908ac254fb2671bed3fa156d654cf62604301dd39"}
Sep 15 22:27:33.607757 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:33.607733    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsbbm" event={"ID":"2c8a934f-80db-473e-957d-fd3047d27df9","Type":"ContainerStarted","Data":"84be41bed8110d4a1b98bdcf2e8e7b2c33a1a388564f499e269575df9874387d"}
Sep 15 22:27:33.612851 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:33.612829    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5xcfb" event={"ID":"e096afa4-418d-41a6-95ee-a181ccd16731","Type":"ContainerStarted","Data":"24901b58be8896f9aa60db8eeeb95156b702ec6838824e756aceddefb5ab76f4"}
Sep 15 22:27:34.149610 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:34.149531    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b22214d5-9c42-4710-bfff-08651d7942a3-original-pull-secret\") pod \"global-pull-secret-syncer-4d5lp\" (UID: \"b22214d5-9c42-4710-bfff-08651d7942a3\") " pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:27:34.149772 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:34.149675    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 22:27:34.149772 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:34.149739    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b22214d5-9c42-4710-bfff-08651d7942a3-original-pull-secret podName:b22214d5-9c42-4710-bfff-08651d7942a3 nodeName:}" failed. No retries permitted until 2026-09-15 22:27:36.149720508 +0000 UTC m=+6.009431774 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b22214d5-9c42-4710-bfff-08651d7942a3-original-pull-secret") pod "global-pull-secret-syncer-4d5lp" (UID: "b22214d5-9c42-4710-bfff-08651d7942a3") : object "kube-system"/"original-pull-secret" not registered
Sep 15 22:27:34.250481 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:34.250365    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vxxv\" (UniqueName: \"kubernetes.io/projected/e5ff0db4-42a1-4d9a-8ba1-1c6068724788-kube-api-access-9vxxv\") pod \"network-check-target-vqrqt\" (UID: \"e5ff0db4-42a1-4d9a-8ba1-1c6068724788\") " pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:27:34.250481 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:34.250423    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/909a094b-77e5-4f6e-9d69-6f4214156810-metrics-certs\") pod \"network-metrics-daemon-j4tsb\" (UID: \"909a094b-77e5-4f6e-9d69-6f4214156810\") " pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:27:34.250711 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:34.250577    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 22:27:34.250711 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:34.250648    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/909a094b-77e5-4f6e-9d69-6f4214156810-metrics-certs podName:909a094b-77e5-4f6e-9d69-6f4214156810 nodeName:}" failed. No retries permitted until 2026-09-15 22:27:36.250618841 +0000 UTC m=+6.110330117 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/909a094b-77e5-4f6e-9d69-6f4214156810-metrics-certs") pod "network-metrics-daemon-j4tsb" (UID: "909a094b-77e5-4f6e-9d69-6f4214156810") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 22:27:34.251607 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:34.251081    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 22:27:34.251607 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:34.251108    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 22:27:34.251607 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:34.251122    2565 projected.go:196] Error preparing data for projected volume kube-api-access-9vxxv for pod openshift-network-diagnostics/network-check-target-vqrqt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 22:27:34.251607 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:34.251168    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e5ff0db4-42a1-4d9a-8ba1-1c6068724788-kube-api-access-9vxxv podName:e5ff0db4-42a1-4d9a-8ba1-1c6068724788 nodeName:}" failed. No retries permitted until 2026-09-15 22:27:36.251152744 +0000 UTC m=+6.110864013 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-9vxxv" (UniqueName: "kubernetes.io/projected/e5ff0db4-42a1-4d9a-8ba1-1c6068724788-kube-api-access-9vxxv") pod "network-check-target-vqrqt" (UID: "e5ff0db4-42a1-4d9a-8ba1-1c6068724788") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 22:27:34.575510 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:34.575366    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-14 22:22:32 +0000 UTC" deadline="2028-05-17 11:37:55.48332643 +0000 UTC"
Sep 15 22:27:34.575510 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:34.575434    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14629h10m20.907895834s"
Sep 15 22:27:34.587096 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:34.584880    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:27:34.587096 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:34.585007    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j4tsb" podUID="909a094b-77e5-4f6e-9d69-6f4214156810"
Sep 15 22:27:34.587096 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:34.585458    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:27:34.587096 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:34.585541    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqrqt" podUID="e5ff0db4-42a1-4d9a-8ba1-1c6068724788"
Sep 15 22:27:34.628232 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:34.627418    2565 generic.go:356] "Generic (PLEG): container finished" podID="b4901f8e95f7812e155a9ad303f1521a" containerID="0395d65ab7ae6e77e6377507a6ce3652ff57ade83534238fe0765f52691df4ae" exitCode=0
Sep 15 22:27:34.628232 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:34.627994    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-208.ec2.internal" event={"ID":"b4901f8e95f7812e155a9ad303f1521a","Type":"ContainerDied","Data":"0395d65ab7ae6e77e6377507a6ce3652ff57ade83534238fe0765f52691df4ae"}
Sep 15 22:27:34.639920 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:34.639857    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-208.ec2.internal" event={"ID":"d23b474b380f11cca08f39b02958fed1","Type":"ContainerStarted","Data":"448b35f470d53c0f2377f5afe9dae72cf1139e8c9facaf29fbb359473dc083aa"}
Sep 15 22:27:34.655474 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:34.655421    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-208.ec2.internal" podStartSLOduration=3.655401457 podStartE2EDuration="3.655401457s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 22:27:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:27:34.655202798 +0000 UTC m=+4.514914086" watchObservedRunningTime="2026-09-15 22:27:34.655401457 +0000 UTC m=+4.515112746"
Sep 15 22:27:35.584129 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:35.584052    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:27:35.584677 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:35.584192    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4d5lp" podUID="b22214d5-9c42-4710-bfff-08651d7942a3"
Sep 15 22:27:35.661480 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:35.661402    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-208.ec2.internal" event={"ID":"b4901f8e95f7812e155a9ad303f1521a","Type":"ContainerStarted","Data":"35e0f11a5fbe8e119cf3da16008f646d0c1f36954eafcdf9e397c03ed9774fff"}
Sep 15 22:27:36.177894 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:36.177801    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b22214d5-9c42-4710-bfff-08651d7942a3-original-pull-secret\") pod \"global-pull-secret-syncer-4d5lp\" (UID: \"b22214d5-9c42-4710-bfff-08651d7942a3\") " pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:27:36.178085 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:36.177967    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 22:27:36.178709 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:36.178543    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b22214d5-9c42-4710-bfff-08651d7942a3-original-pull-secret podName:b22214d5-9c42-4710-bfff-08651d7942a3 nodeName:}" failed. No retries permitted until 2026-09-15 22:27:40.178004061 +0000 UTC m=+10.037715326 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b22214d5-9c42-4710-bfff-08651d7942a3-original-pull-secret") pod "global-pull-secret-syncer-4d5lp" (UID: "b22214d5-9c42-4710-bfff-08651d7942a3") : object "kube-system"/"original-pull-secret" not registered
Sep 15 22:27:36.278660 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:36.278623    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vxxv\" (UniqueName: \"kubernetes.io/projected/e5ff0db4-42a1-4d9a-8ba1-1c6068724788-kube-api-access-9vxxv\") pod \"network-check-target-vqrqt\" (UID: \"e5ff0db4-42a1-4d9a-8ba1-1c6068724788\") " pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:27:36.278907 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:36.278681    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/909a094b-77e5-4f6e-9d69-6f4214156810-metrics-certs\") pod \"network-metrics-daemon-j4tsb\" (UID: \"909a094b-77e5-4f6e-9d69-6f4214156810\") " pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:27:36.278907 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:36.278849    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 22:27:36.279040 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:36.278910    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/909a094b-77e5-4f6e-9d69-6f4214156810-metrics-certs podName:909a094b-77e5-4f6e-9d69-6f4214156810 nodeName:}" failed. No retries permitted until 2026-09-15 22:27:40.278891189 +0000 UTC m=+10.138602454 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/909a094b-77e5-4f6e-9d69-6f4214156810-metrics-certs") pod "network-metrics-daemon-j4tsb" (UID: "909a094b-77e5-4f6e-9d69-6f4214156810") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 22:27:36.279373 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:36.279343    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 22:27:36.279373 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:36.279373    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 22:27:36.279536 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:36.279386    2565 projected.go:196] Error preparing data for projected volume kube-api-access-9vxxv for pod openshift-network-diagnostics/network-check-target-vqrqt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 22:27:36.279536 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:36.279432    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e5ff0db4-42a1-4d9a-8ba1-1c6068724788-kube-api-access-9vxxv podName:e5ff0db4-42a1-4d9a-8ba1-1c6068724788 nodeName:}" failed. No retries permitted until 2026-09-15 22:27:40.279415291 +0000 UTC m=+10.139126568 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-9vxxv" (UniqueName: "kubernetes.io/projected/e5ff0db4-42a1-4d9a-8ba1-1c6068724788-kube-api-access-9vxxv") pod "network-check-target-vqrqt" (UID: "e5ff0db4-42a1-4d9a-8ba1-1c6068724788") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 22:27:36.590243 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:36.590171    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:27:36.590660 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:36.590290    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqrqt" podUID="e5ff0db4-42a1-4d9a-8ba1-1c6068724788"
Sep 15 22:27:36.590718 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:36.590657    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:27:36.590795 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:36.590760    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j4tsb" podUID="909a094b-77e5-4f6e-9d69-6f4214156810"
Sep 15 22:27:37.584598 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:37.584060    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:27:37.584598 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:37.584213    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4d5lp" podUID="b22214d5-9c42-4710-bfff-08651d7942a3"
Sep 15 22:27:38.584167 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:38.584134    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:27:38.584620 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:38.584266    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j4tsb" podUID="909a094b-77e5-4f6e-9d69-6f4214156810"
Sep 15 22:27:38.584757 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:38.584736    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:27:38.584861 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:38.584841    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqrqt" podUID="e5ff0db4-42a1-4d9a-8ba1-1c6068724788"
Sep 15 22:27:39.584316 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:39.584230    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:27:39.584755 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:39.584373    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4d5lp" podUID="b22214d5-9c42-4710-bfff-08651d7942a3"
Sep 15 22:27:40.211551 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:40.211511    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b22214d5-9c42-4710-bfff-08651d7942a3-original-pull-secret\") pod \"global-pull-secret-syncer-4d5lp\" (UID: \"b22214d5-9c42-4710-bfff-08651d7942a3\") " pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:27:40.211742 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:40.211687    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 22:27:40.211803 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:40.211765    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b22214d5-9c42-4710-bfff-08651d7942a3-original-pull-secret podName:b22214d5-9c42-4710-bfff-08651d7942a3 nodeName:}" failed. No retries permitted until 2026-09-15 22:27:48.211744363 +0000 UTC m=+18.071455650 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b22214d5-9c42-4710-bfff-08651d7942a3-original-pull-secret") pod "global-pull-secret-syncer-4d5lp" (UID: "b22214d5-9c42-4710-bfff-08651d7942a3") : object "kube-system"/"original-pull-secret" not registered
Sep 15 22:27:40.312581 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:40.312537    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/909a094b-77e5-4f6e-9d69-6f4214156810-metrics-certs\") pod \"network-metrics-daemon-j4tsb\" (UID: \"909a094b-77e5-4f6e-9d69-6f4214156810\") " pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:27:40.312738 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:40.312635    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vxxv\" (UniqueName: \"kubernetes.io/projected/e5ff0db4-42a1-4d9a-8ba1-1c6068724788-kube-api-access-9vxxv\") pod \"network-check-target-vqrqt\" (UID: \"e5ff0db4-42a1-4d9a-8ba1-1c6068724788\") " pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:27:40.312876 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:40.312763    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 22:27:40.312876 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:40.312779    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 22:27:40.312876 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:40.312797    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 22:27:40.312876 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:40.312810    2565 projected.go:196] Error preparing data for projected volume kube-api-access-9vxxv for pod openshift-network-diagnostics/network-check-target-vqrqt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 22:27:40.312876 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:40.312845    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/909a094b-77e5-4f6e-9d69-6f4214156810-metrics-certs podName:909a094b-77e5-4f6e-9d69-6f4214156810 nodeName:}" failed. No retries permitted until 2026-09-15 22:27:48.312823524 +0000 UTC m=+18.172534800 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/909a094b-77e5-4f6e-9d69-6f4214156810-metrics-certs") pod "network-metrics-daemon-j4tsb" (UID: "909a094b-77e5-4f6e-9d69-6f4214156810") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 22:27:40.312876 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:40.312867    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e5ff0db4-42a1-4d9a-8ba1-1c6068724788-kube-api-access-9vxxv podName:e5ff0db4-42a1-4d9a-8ba1-1c6068724788 nodeName:}" failed. No retries permitted until 2026-09-15 22:27:48.312856753 +0000 UTC m=+18.172568015 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-9vxxv" (UniqueName: "kubernetes.io/projected/e5ff0db4-42a1-4d9a-8ba1-1c6068724788-kube-api-access-9vxxv") pod "network-check-target-vqrqt" (UID: "e5ff0db4-42a1-4d9a-8ba1-1c6068724788") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 22:27:40.588359 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:40.587642    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:27:40.588359 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:40.587962    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqrqt" podUID="e5ff0db4-42a1-4d9a-8ba1-1c6068724788"
Sep 15 22:27:40.588880 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:40.587587    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:27:40.589102 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:40.589080    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j4tsb" podUID="909a094b-77e5-4f6e-9d69-6f4214156810"
Sep 15 22:27:41.584301 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:41.584266    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:27:41.584465 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:41.584403    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4d5lp" podUID="b22214d5-9c42-4710-bfff-08651d7942a3"
Sep 15 22:27:42.584076 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:42.584034    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:27:42.584518 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:42.584171    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j4tsb" podUID="909a094b-77e5-4f6e-9d69-6f4214156810"
Sep 15 22:27:42.584518 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:42.584209    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:27:42.584518 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:42.584315    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqrqt" podUID="e5ff0db4-42a1-4d9a-8ba1-1c6068724788"
Sep 15 22:27:43.584350 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:43.584309    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:27:43.584779 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:43.584504    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4d5lp" podUID="b22214d5-9c42-4710-bfff-08651d7942a3"
Sep 15 22:27:44.584470 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:44.584431    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:27:44.584914 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:44.584559    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j4tsb" podUID="909a094b-77e5-4f6e-9d69-6f4214156810"
Sep 15 22:27:44.584914 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:44.584617    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:27:44.584914 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:44.584712    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqrqt" podUID="e5ff0db4-42a1-4d9a-8ba1-1c6068724788"
Sep 15 22:27:45.584470 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:45.584437    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:27:45.584628 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:45.584549    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4d5lp" podUID="b22214d5-9c42-4710-bfff-08651d7942a3"
Sep 15 22:27:46.583900 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:46.583865    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:27:46.583900 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:46.583903    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:27:46.584130 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:46.584004    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j4tsb" podUID="909a094b-77e5-4f6e-9d69-6f4214156810"
Sep 15 22:27:46.584130 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:46.584114    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqrqt" podUID="e5ff0db4-42a1-4d9a-8ba1-1c6068724788"
Sep 15 22:27:47.584760 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:47.584725    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:27:47.585212 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:47.584842    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4d5lp" podUID="b22214d5-9c42-4710-bfff-08651d7942a3"
Sep 15 22:27:48.271013 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:48.270927    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b22214d5-9c42-4710-bfff-08651d7942a3-original-pull-secret\") pod \"global-pull-secret-syncer-4d5lp\" (UID: \"b22214d5-9c42-4710-bfff-08651d7942a3\") " pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:27:48.271189 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:48.271074    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 22:27:48.271189 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:48.271140    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b22214d5-9c42-4710-bfff-08651d7942a3-original-pull-secret podName:b22214d5-9c42-4710-bfff-08651d7942a3 nodeName:}" failed. No retries permitted until 2026-09-15 22:28:04.271122367 +0000 UTC m=+34.130833633 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b22214d5-9c42-4710-bfff-08651d7942a3-original-pull-secret") pod "global-pull-secret-syncer-4d5lp" (UID: "b22214d5-9c42-4710-bfff-08651d7942a3") : object "kube-system"/"original-pull-secret" not registered
Sep 15 22:27:48.371793 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:48.371750    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vxxv\" (UniqueName: \"kubernetes.io/projected/e5ff0db4-42a1-4d9a-8ba1-1c6068724788-kube-api-access-9vxxv\") pod \"network-check-target-vqrqt\" (UID: \"e5ff0db4-42a1-4d9a-8ba1-1c6068724788\") " pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:27:48.371949 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:48.371809    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/909a094b-77e5-4f6e-9d69-6f4214156810-metrics-certs\") pod \"network-metrics-daemon-j4tsb\" (UID: \"909a094b-77e5-4f6e-9d69-6f4214156810\") " pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:27:48.371949 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:48.371922    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 22:27:48.371949 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:48.371936    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 22:27:48.372123 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:48.372006    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/909a094b-77e5-4f6e-9d69-6f4214156810-metrics-certs podName:909a094b-77e5-4f6e-9d69-6f4214156810 nodeName:}" failed. No retries permitted until 2026-09-15 22:28:04.371989718 +0000 UTC m=+34.231700980 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/909a094b-77e5-4f6e-9d69-6f4214156810-metrics-certs") pod "network-metrics-daemon-j4tsb" (UID: "909a094b-77e5-4f6e-9d69-6f4214156810") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 22:27:48.372123 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:48.371943    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 22:27:48.372123 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:48.372051    2565 projected.go:196] Error preparing data for projected volume kube-api-access-9vxxv for pod openshift-network-diagnostics/network-check-target-vqrqt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 22:27:48.372123 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:48.372100    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e5ff0db4-42a1-4d9a-8ba1-1c6068724788-kube-api-access-9vxxv podName:e5ff0db4-42a1-4d9a-8ba1-1c6068724788 nodeName:}" failed. No retries permitted until 2026-09-15 22:28:04.372088591 +0000 UTC m=+34.231799868 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-9vxxv" (UniqueName: "kubernetes.io/projected/e5ff0db4-42a1-4d9a-8ba1-1c6068724788-kube-api-access-9vxxv") pod "network-check-target-vqrqt" (UID: "e5ff0db4-42a1-4d9a-8ba1-1c6068724788") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 22:27:48.584782 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:48.584697    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:27:48.585242 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:48.584855    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j4tsb" podUID="909a094b-77e5-4f6e-9d69-6f4214156810"
Sep 15 22:27:48.585242 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:48.584905    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:27:48.585242 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:48.585045    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqrqt" podUID="e5ff0db4-42a1-4d9a-8ba1-1c6068724788"
Sep 15 22:27:49.583905 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:49.583873    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:27:49.584151 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:49.583982    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4d5lp" podUID="b22214d5-9c42-4710-bfff-08651d7942a3"
Sep 15 22:27:50.585442 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:50.585408    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:27:50.585892 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:50.585523    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqrqt" podUID="e5ff0db4-42a1-4d9a-8ba1-1c6068724788"
Sep 15 22:27:50.585892 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:50.585582    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:27:50.585892 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:50.585717    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j4tsb" podUID="909a094b-77e5-4f6e-9d69-6f4214156810"
Sep 15 22:27:51.584749 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:51.584719    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:27:51.584873 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:51.584848    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4d5lp" podUID="b22214d5-9c42-4710-bfff-08651d7942a3"
Sep 15 22:27:51.697810 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:51.697322    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-q689h" event={"ID":"4e706f82-e509-4ba2-9f07-c4f5508fa685","Type":"ContainerStarted","Data":"3c5abf3d3dda6df4725f3d58df4716c23e4d542ab0c0069a245cc7c0d813c961"}
Sep 15 22:27:51.701344 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:51.701227    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-l9m9m" event={"ID":"93274f69-efc2-48a0-8223-f3a37aca3947","Type":"ContainerStarted","Data":"15efea0cf202d0f30e1789f2305f9a15f1eb0765081c49a62c46ce7f86075d90"}
Sep 15 22:27:51.704652 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:51.704616    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nmmxx" event={"ID":"f132ba8d-e9aa-4a73-8713-82fe09532325","Type":"ContainerStarted","Data":"8b854b1d50b900aee402d66e47ebaca2426e169d84246a551975ae8d1f01b00c"}
Sep 15 22:27:51.707088 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:51.706525    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5xcfb" event={"ID":"e096afa4-418d-41a6-95ee-a181ccd16731","Type":"ContainerStarted","Data":"562a5097a927ae557c6cf18b8996c5afe5a85570193acc53cc4d426cc5624f67"}
Sep 15 22:27:51.711860 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:51.711741    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-74hmh" event={"ID":"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0","Type":"ContainerStarted","Data":"cb3c9311f25c2a333d555d2c3cd550640a84d05e0da138e079b8103ff8945d18"}
Sep 15 22:27:51.714152 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:51.714093    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-q689h" podStartSLOduration=3.786186207 podStartE2EDuration="21.714073903s" totalImagesPullingTime="17.927887696s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 22:27:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:27:51.713685871 +0000 UTC m=+21.573397158" watchObservedRunningTime="2026-09-15 22:27:51.714073903 +0000 UTC m=+21.573785181"
Sep 15 22:27:51.714403 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:51.714302    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-208.ec2.internal" podStartSLOduration=20.714292349 podStartE2EDuration="20.714292349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 22:27:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:27:35.68986564 +0000 UTC m=+5.549576925" watchObservedRunningTime="2026-09-15 22:27:51.714292349 +0000 UTC m=+21.574003708"
Sep 15 22:27:51.750686 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:51.749662    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-nmmxx" podStartSLOduration=3.788773861 podStartE2EDuration="21.749640013s" totalImagesPullingTime="17.960866152s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 22:27:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:27:51.749285293 +0000 UTC m=+21.608996589" watchObservedRunningTime="2026-09-15 22:27:51.749640013 +0000 UTC m=+21.609351298"
Sep 15 22:27:51.764234 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:51.764183    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-l9m9m" podStartSLOduration=7.381051889 podStartE2EDuration="21.764165526s" totalImagesPullingTime="14.383113637s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:27:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:27:51.763834753 +0000 UTC m=+21.623546037" watchObservedRunningTime="2026-09-15 22:27:51.764165526 +0000 UTC m=+21.623876810"
Sep 15 22:27:52.584285 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:52.584243    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:27:52.584461 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:52.584342    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqrqt" podUID="e5ff0db4-42a1-4d9a-8ba1-1c6068724788"
Sep 15 22:27:52.584461 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:52.584402    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:27:52.584578 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:52.584520    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j4tsb" podUID="909a094b-77e5-4f6e-9d69-6f4214156810"
Sep 15 22:27:52.715848 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:52.715809    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8tjs6" event={"ID":"d6f6b883-5742-4ebd-bde1-5c7295563940","Type":"ContainerStarted","Data":"0c67586bfa7365143edd2e832b520ab511b32268d9aad2549bfe2a7ed8ed4c13"}
Sep 15 22:27:52.719505 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:52.719477    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-74hmh" event={"ID":"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0","Type":"ContainerStarted","Data":"7a879e0bed7ea047f6ac546d9c09ff2ad632e0febe4a2f7487cd64732f257df5"}
Sep 15 22:27:52.719642 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:52.719560    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-74hmh" event={"ID":"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0","Type":"ContainerStarted","Data":"ef5676630e015fefba768c3d7b7cfeed769eeb5fa21c203d7dd8a9f74b566a6c"}
Sep 15 22:27:52.719642 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:52.719612    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-74hmh" event={"ID":"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0","Type":"ContainerStarted","Data":"e213c22de75aee4dba2f8d472f0d2c0761ba4805e95335d2f6283ab2af728be4"}
Sep 15 22:27:52.719764 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:52.719658    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-74hmh" event={"ID":"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0","Type":"ContainerStarted","Data":"444d607a58265cbbc66bc0339671a9bdd81c9d65cd70f67017adf11074549a39"}
Sep 15 22:27:52.719764 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:52.719700    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-74hmh" event={"ID":"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0","Type":"ContainerStarted","Data":"8ccc137175f46bf4dabdcdd3173ac9e674732477fa8f14c4bc91a7f4737d8474"}
Sep 15 22:27:52.721145 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:52.721096    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-m4w7h" event={"ID":"1bd5f596-e129-4139-97d6-b8f7696c26e4","Type":"ContainerStarted","Data":"dcaad116f241b5287b43b62ede43febbe2929ffdb431b6e60b08671fd1a4e2b2"}
Sep 15 22:27:52.722815 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:52.722785    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsbbm" event={"ID":"2c8a934f-80db-473e-957d-fd3047d27df9","Type":"ContainerStarted","Data":"099ca695ed46a2cec71714dd9064c1d59fd3208f93598c02f2da91438a32af45"}
Sep 15 22:27:52.724452 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:52.724410    2565 generic.go:356] "Generic (PLEG): container finished" podID="e096afa4-418d-41a6-95ee-a181ccd16731" containerID="562a5097a927ae557c6cf18b8996c5afe5a85570193acc53cc4d426cc5624f67" exitCode=0
Sep 15 22:27:52.724452 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:52.724440    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5xcfb" event={"ID":"e096afa4-418d-41a6-95ee-a181ccd16731","Type":"ContainerDied","Data":"562a5097a927ae557c6cf18b8996c5afe5a85570193acc53cc4d426cc5624f67"}
Sep 15 22:27:52.730545 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:52.730502    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-8tjs6" podStartSLOduration=4.807301156 podStartE2EDuration="22.730489988s" totalImagesPullingTime="17.923188832s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 22:27:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:27:52.730338646 +0000 UTC m=+22.590049931" watchObservedRunningTime="2026-09-15 22:27:52.730489988 +0000 UTC m=+22.590201272"
Sep 15 22:27:52.769753 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:52.769704    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-m4w7h" podStartSLOduration=4.84684617 podStartE2EDuration="22.769686037s" totalImagesPullingTime="17.922839867s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 22:27:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:27:52.769045463 +0000 UTC m=+22.628756745" watchObservedRunningTime="2026-09-15 22:27:52.769686037 +0000 UTC m=+22.629397321"
Sep 15 22:27:53.176345 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:53.176323    2565 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 15 22:27:53.584293 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:53.584208    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:27:53.584451 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:53.584340    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4d5lp" podUID="b22214d5-9c42-4710-bfff-08651d7942a3"
Sep 15 22:27:53.586305 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:53.586226    2565 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-15T22:27:53.176340639Z","UUID":"eeafffc9-d6bb-4da9-a6c1-c260e806a626","Handler":null,"Name":"","Endpoint":""}
Sep 15 22:27:53.588917 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:53.588890    2565 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 15 22:27:53.589048 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:53.588937    2565 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 15 22:27:53.728762 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:53.728725    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-d2c7v" event={"ID":"caf80c9a-3157-42ba-86bb-4be3a7932714","Type":"ContainerStarted","Data":"5158f0db861bde380cdd3e0a34c31d2598e3fcda01397fb9225e9c4f1adde33a"}
Sep 15 22:27:53.731649 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:53.731385    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsbbm" event={"ID":"2c8a934f-80db-473e-957d-fd3047d27df9","Type":"ContainerStarted","Data":"5856cfe7f414276d3e86c11203916c68ebbf98d6de7de091d17ebb5604b8c22d"}
Sep 15 22:27:53.743260 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:53.743207    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-d2c7v" podStartSLOduration=5.822689126 podStartE2EDuration="23.743182689s" totalImagesPullingTime="17.920493563s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 22:27:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:27:53.742761945 +0000 UTC m=+23.602473231" watchObservedRunningTime="2026-09-15 22:27:53.743182689 +0000 UTC m=+23.602893974"
Sep 15 22:27:54.584317 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:54.584284    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:27:54.584477 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:54.584285    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:27:54.584477 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:54.584410    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j4tsb" podUID="909a094b-77e5-4f6e-9d69-6f4214156810"
Sep 15 22:27:54.584593 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:54.584492    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqrqt" podUID="e5ff0db4-42a1-4d9a-8ba1-1c6068724788"
Sep 15 22:27:54.735594 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:54.735557    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsbbm" event={"ID":"2c8a934f-80db-473e-957d-fd3047d27df9","Type":"ContainerStarted","Data":"6b6ffcf5e5c55c19e36bb6716e4ff74bf55658d5edb90cce503554798e9223d7"}
Sep 15 22:27:54.739424 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:54.739397    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-74hmh" event={"ID":"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0","Type":"ContainerStarted","Data":"f03504ac7571dd4193063f792f34f956e96354d4fc55ca0600355a515e940410"}
Sep 15 22:27:54.752664 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:54.752582    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zsbbm" podStartSLOduration=4.149066264 podStartE2EDuration="24.752569329s" totalImagesPullingTime="20.603503065s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 22:27:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:27:54.752209049 +0000 UTC m=+24.611920335" watchObservedRunningTime="2026-09-15 22:27:54.752569329 +0000 UTC m=+24.612280613"
Sep 15 22:27:55.584184 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:55.584155    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:27:55.584360 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:55.584256    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4d5lp" podUID="b22214d5-9c42-4710-bfff-08651d7942a3"
Sep 15 22:27:55.638338 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:55.638302    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-l9m9m"
Sep 15 22:27:55.638986 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:55.638965    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-l9m9m"
Sep 15 22:27:56.290465 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:56.290431    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-m4w7h_1bd5f596-e129-4139-97d6-b8f7696c26e4/dns-node-resolver/0.log"
Sep 15 22:27:56.584796 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:56.584663    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:27:56.584796 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:56.584711    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:27:56.584796 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:56.584790    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqrqt" podUID="e5ff0db4-42a1-4d9a-8ba1-1c6068724788"
Sep 15 22:27:56.585089 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:56.584938    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j4tsb" podUID="909a094b-77e5-4f6e-9d69-6f4214156810"
Sep 15 22:27:57.274171 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:57.273983    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8tjs6_d6f6b883-5742-4ebd-bde1-5c7295563940/node-ca/0.log"
Sep 15 22:27:57.584098 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:57.584071    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:27:57.584630 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:57.584178    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4d5lp" podUID="b22214d5-9c42-4710-bfff-08651d7942a3"
Sep 15 22:27:57.753009 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:57.752925    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-74hmh" event={"ID":"bcb145e6-76ea-43fd-98c0-2c30c59dc2d0","Type":"ContainerStarted","Data":"3679cc8d8b0a56960ad7fb65950b42d38c420262707f76262c3a58c3b907c767"}
Sep 15 22:27:57.753171 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:57.753135    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:57.754798 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:57.754768    2565 generic.go:356] "Generic (PLEG): container finished" podID="e096afa4-418d-41a6-95ee-a181ccd16731" containerID="986fb19a863fda4840a231d9f6056f3022e3a91b939ecf78d809fbce1ad60214" exitCode=0
Sep 15 22:27:57.754884 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:57.754801    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5xcfb" event={"ID":"e096afa4-418d-41a6-95ee-a181ccd16731","Type":"ContainerDied","Data":"986fb19a863fda4840a231d9f6056f3022e3a91b939ecf78d809fbce1ad60214"}
Sep 15 22:27:57.769226 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:57.769169    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:57.782647 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:57.782607    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-74hmh" podStartSLOduration=9.809638961 podStartE2EDuration="27.782594171s" totalImagesPullingTime="17.97295521s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 22:27:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:27:57.782019874 +0000 UTC m=+27.641731160" watchObservedRunningTime="2026-09-15 22:27:57.782594171 +0000 UTC m=+27.642305455"
Sep 15 22:27:58.584272 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:58.584247    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:27:58.584674 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:58.584366    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqrqt" podUID="e5ff0db4-42a1-4d9a-8ba1-1c6068724788"
Sep 15 22:27:58.584674 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:58.584421    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:27:58.584674 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:58.584547    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j4tsb" podUID="909a094b-77e5-4f6e-9d69-6f4214156810"
Sep 15 22:27:58.760214 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:58.760135    2565 generic.go:356] "Generic (PLEG): container finished" podID="e096afa4-418d-41a6-95ee-a181ccd16731" containerID="2edaf7ddb9b0217261f3b01ed1c8b457de99103c9f70f667c0349815324e5fe4" exitCode=0
Sep 15 22:27:58.760357 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:58.760207    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5xcfb" event={"ID":"e096afa4-418d-41a6-95ee-a181ccd16731","Type":"ContainerDied","Data":"2edaf7ddb9b0217261f3b01ed1c8b457de99103c9f70f667c0349815324e5fe4"}
Sep 15 22:27:58.760622 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:58.760591    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:58.760758 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:58.760679    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:58.775485 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:58.775461    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:27:59.098472 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:59.098272    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4d5lp"]
Sep 15 22:27:59.098648 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:59.098588    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:27:59.098722 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:59.098686    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4d5lp" podUID="b22214d5-9c42-4710-bfff-08651d7942a3"
Sep 15 22:27:59.101998 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:59.101971    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-j4tsb"]
Sep 15 22:27:59.102126 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:59.102113    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:27:59.102217 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:59.102202    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j4tsb" podUID="909a094b-77e5-4f6e-9d69-6f4214156810"
Sep 15 22:27:59.102569 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:59.102550    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vqrqt"]
Sep 15 22:27:59.102671 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:59.102631    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:27:59.102743 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:27:59.102701    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqrqt" podUID="e5ff0db4-42a1-4d9a-8ba1-1c6068724788"
Sep 15 22:27:59.764650 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:59.764556    2565 generic.go:356] "Generic (PLEG): container finished" podID="e096afa4-418d-41a6-95ee-a181ccd16731" containerID="b3d0474976bbfb3f9489e75f10c9147278a0b425d92b5994b104132d429da7cf" exitCode=0
Sep 15 22:27:59.765048 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:27:59.764641    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5xcfb" event={"ID":"e096afa4-418d-41a6-95ee-a181ccd16731","Type":"ContainerDied","Data":"b3d0474976bbfb3f9489e75f10c9147278a0b425d92b5994b104132d429da7cf"}
Sep 15 22:28:00.584660 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:00.584632    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:28:00.584798 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:00.584759    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:28:00.584798 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:00.584762    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j4tsb" podUID="909a094b-77e5-4f6e-9d69-6f4214156810"
Sep 15 22:28:00.584902 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:00.584825    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqrqt" podUID="e5ff0db4-42a1-4d9a-8ba1-1c6068724788"
Sep 15 22:28:00.584902 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:00.584869    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:28:00.585002 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:00.584981    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4d5lp" podUID="b22214d5-9c42-4710-bfff-08651d7942a3"
Sep 15 22:28:01.053192 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:01.053162    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-l9m9m"
Sep 15 22:28:01.053601 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:01.053299    2565 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 15 22:28:01.053926 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:01.053879    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-l9m9m"
Sep 15 22:28:02.584601 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:02.584572    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:28:02.585005 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:02.584571    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:28:02.585005 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:02.584681    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j4tsb" podUID="909a094b-77e5-4f6e-9d69-6f4214156810"
Sep 15 22:28:02.585005 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:02.584571    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:28:02.585005 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:02.584788    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4d5lp" podUID="b22214d5-9c42-4710-bfff-08651d7942a3"
Sep 15 22:28:02.585005 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:02.584816    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqrqt" podUID="e5ff0db4-42a1-4d9a-8ba1-1c6068724788"
Sep 15 22:28:04.286267 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:04.286224    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b22214d5-9c42-4710-bfff-08651d7942a3-original-pull-secret\") pod \"global-pull-secret-syncer-4d5lp\" (UID: \"b22214d5-9c42-4710-bfff-08651d7942a3\") " pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:28:04.286799 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:04.286377    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 22:28:04.286799 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:04.286450    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b22214d5-9c42-4710-bfff-08651d7942a3-original-pull-secret podName:b22214d5-9c42-4710-bfff-08651d7942a3 nodeName:}" failed. No retries permitted until 2026-09-15 22:28:36.28643142 +0000 UTC m=+66.146142682 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b22214d5-9c42-4710-bfff-08651d7942a3-original-pull-secret") pod "global-pull-secret-syncer-4d5lp" (UID: "b22214d5-9c42-4710-bfff-08651d7942a3") : object "kube-system"/"original-pull-secret" not registered
Sep 15 22:28:04.386967 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:04.386936    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vxxv\" (UniqueName: \"kubernetes.io/projected/e5ff0db4-42a1-4d9a-8ba1-1c6068724788-kube-api-access-9vxxv\") pod \"network-check-target-vqrqt\" (UID: \"e5ff0db4-42a1-4d9a-8ba1-1c6068724788\") " pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:28:04.387136 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:04.386981    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/909a094b-77e5-4f6e-9d69-6f4214156810-metrics-certs\") pod \"network-metrics-daemon-j4tsb\" (UID: \"909a094b-77e5-4f6e-9d69-6f4214156810\") " pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:28:04.387136 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:04.387104    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 22:28:04.387136 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:04.387111    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 22:28:04.387136 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:04.387131    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 22:28:04.387136 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:04.387141    2565 projected.go:196] Error preparing data for projected volume kube-api-access-9vxxv for pod openshift-network-diagnostics/network-check-target-vqrqt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 22:28:04.387413 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:04.387166    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/909a094b-77e5-4f6e-9d69-6f4214156810-metrics-certs podName:909a094b-77e5-4f6e-9d69-6f4214156810 nodeName:}" failed. No retries permitted until 2026-09-15 22:28:36.387147319 +0000 UTC m=+66.246858581 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/909a094b-77e5-4f6e-9d69-6f4214156810-metrics-certs") pod "network-metrics-daemon-j4tsb" (UID: "909a094b-77e5-4f6e-9d69-6f4214156810") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 22:28:04.387413 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:04.387183    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e5ff0db4-42a1-4d9a-8ba1-1c6068724788-kube-api-access-9vxxv podName:e5ff0db4-42a1-4d9a-8ba1-1c6068724788 nodeName:}" failed. No retries permitted until 2026-09-15 22:28:36.387174964 +0000 UTC m=+66.246886234 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-9vxxv" (UniqueName: "kubernetes.io/projected/e5ff0db4-42a1-4d9a-8ba1-1c6068724788-kube-api-access-9vxxv") pod "network-check-target-vqrqt" (UID: "e5ff0db4-42a1-4d9a-8ba1-1c6068724788") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 22:28:04.583952 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:04.583885    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:28:04.583952 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:04.583899    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:28:04.584155 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:04.583895    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:28:04.584155 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:04.583987    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j4tsb" podUID="909a094b-77e5-4f6e-9d69-6f4214156810"
Sep 15 22:28:04.584155 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:04.584073    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqrqt" podUID="e5ff0db4-42a1-4d9a-8ba1-1c6068724788"
Sep 15 22:28:04.584155 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:04.584138    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4d5lp" podUID="b22214d5-9c42-4710-bfff-08651d7942a3"
Sep 15 22:28:05.781856 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:05.781670    2565 generic.go:356] "Generic (PLEG): container finished" podID="e096afa4-418d-41a6-95ee-a181ccd16731" containerID="0aeb4c8a529afcd0b9a1318680adc93e7b5688837db5a29d40f68371545ccac5" exitCode=0
Sep 15 22:28:05.782232 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:05.781751    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5xcfb" event={"ID":"e096afa4-418d-41a6-95ee-a181ccd16731","Type":"ContainerDied","Data":"0aeb4c8a529afcd0b9a1318680adc93e7b5688837db5a29d40f68371545ccac5"}
Sep 15 22:28:06.584016 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:06.583983    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:28:06.584016 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:06.583983    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:28:06.584282 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:06.583997    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:28:06.584282 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:06.584130    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j4tsb" podUID="909a094b-77e5-4f6e-9d69-6f4214156810"
Sep 15 22:28:06.584282 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:06.584202    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqrqt" podUID="e5ff0db4-42a1-4d9a-8ba1-1c6068724788"
Sep 15 22:28:06.584282 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:06.584265    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4d5lp" podUID="b22214d5-9c42-4710-bfff-08651d7942a3"
Sep 15 22:28:06.787045 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:06.786993    2565 generic.go:356] "Generic (PLEG): container finished" podID="e096afa4-418d-41a6-95ee-a181ccd16731" containerID="1bd61d60384978405ee5f43ae7b9ba6a51d854d766e40ae73d15a0f9624ef920" exitCode=0
Sep 15 22:28:06.787401 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:06.787051    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5xcfb" event={"ID":"e096afa4-418d-41a6-95ee-a181ccd16731","Type":"ContainerDied","Data":"1bd61d60384978405ee5f43ae7b9ba6a51d854d766e40ae73d15a0f9624ef920"}
Sep 15 22:28:07.794366 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:07.793634    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5xcfb" event={"ID":"e096afa4-418d-41a6-95ee-a181ccd16731","Type":"ContainerStarted","Data":"5ca5e490570e2ff31561d722db7ce8539c053a547702c9ed6a00844a1fadb486"}
Sep 15 22:28:07.814733 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:07.814687    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-5xcfb" podStartSLOduration=9.108621227 podStartE2EDuration="37.814670383s" totalImagesPullingTime="28.706049156s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 22:27:30 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:28:07.813741502 +0000 UTC m=+37.673452789" watchObservedRunningTime="2026-09-15 22:28:07.814670383 +0000 UTC m=+37.674381667"
Sep 15 22:28:08.584907 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:08.584873    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:28:08.585089 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:08.584976    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j4tsb" podUID="909a094b-77e5-4f6e-9d69-6f4214156810"
Sep 15 22:28:08.585089 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:08.584873    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:28:08.585089 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:08.585021    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:28:08.585207 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:08.585100    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqrqt" podUID="e5ff0db4-42a1-4d9a-8ba1-1c6068724788"
Sep 15 22:28:08.585207 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:08.585166    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4d5lp" podUID="b22214d5-9c42-4710-bfff-08651d7942a3"
Sep 15 22:28:10.585124 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:10.585091    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:28:10.585595 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:10.585200    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:28:10.585595 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:10.585231    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j4tsb" podUID="909a094b-77e5-4f6e-9d69-6f4214156810"
Sep 15 22:28:10.585595 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:10.585271    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqrqt" podUID="e5ff0db4-42a1-4d9a-8ba1-1c6068724788"
Sep 15 22:28:10.585595 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:10.585297    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:28:10.585595 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:10.585363    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4d5lp" podUID="b22214d5-9c42-4710-bfff-08651d7942a3"
Sep 15 22:28:12.584682 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:12.584649    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:28:12.585183 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:12.584658    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:28:12.585183 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:12.584772    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j4tsb" podUID="909a094b-77e5-4f6e-9d69-6f4214156810"
Sep 15 22:28:12.585183 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:12.584827    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqrqt" podUID="e5ff0db4-42a1-4d9a-8ba1-1c6068724788"
Sep 15 22:28:12.585183 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:12.584856    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:28:12.585183 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:12.584932    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4d5lp" podUID="b22214d5-9c42-4710-bfff-08651d7942a3"
Sep 15 22:28:14.584584 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:14.584550    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:28:14.584584 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:14.584575    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:28:14.584990 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:14.584558    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:28:14.584990 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:14.584654    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4d5lp" podUID="b22214d5-9c42-4710-bfff-08651d7942a3"
Sep 15 22:28:14.584990 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:14.584725    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j4tsb" podUID="909a094b-77e5-4f6e-9d69-6f4214156810"
Sep 15 22:28:14.584990 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:14.584812    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqrqt" podUID="e5ff0db4-42a1-4d9a-8ba1-1c6068724788"
Sep 15 22:28:16.584451 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:16.584418    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:28:16.584790 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:16.584417    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:28:16.584790 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:16.584542    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j4tsb" podUID="909a094b-77e5-4f6e-9d69-6f4214156810"
Sep 15 22:28:16.584790 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:16.584563    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:28:16.584790 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:16.584638    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqrqt" podUID="e5ff0db4-42a1-4d9a-8ba1-1c6068724788"
Sep 15 22:28:16.584790 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:16.584696    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4d5lp" podUID="b22214d5-9c42-4710-bfff-08651d7942a3"
Sep 15 22:28:18.584460 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:18.584427    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:28:18.584942 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:18.584436    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:28:18.584942 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:18.584531    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j4tsb" podUID="909a094b-77e5-4f6e-9d69-6f4214156810"
Sep 15 22:28:18.584942 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:18.584436    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:28:18.584942 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:18.584593    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqrqt" podUID="e5ff0db4-42a1-4d9a-8ba1-1c6068724788"
Sep 15 22:28:18.584942 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:18.584673    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4d5lp" podUID="b22214d5-9c42-4710-bfff-08651d7942a3"
Sep 15 22:28:20.584773 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:20.584735    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:28:20.585350 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:20.584851    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j4tsb" podUID="909a094b-77e5-4f6e-9d69-6f4214156810"
Sep 15 22:28:20.585350 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:20.584909    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:28:20.585350 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:20.584974    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqrqt" podUID="e5ff0db4-42a1-4d9a-8ba1-1c6068724788"
Sep 15 22:28:20.585350 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:20.585012    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:28:20.585350 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:20.585104    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4d5lp" podUID="b22214d5-9c42-4710-bfff-08651d7942a3"
Sep 15 22:28:22.584585 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:22.584549    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:28:22.584963 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:22.584557    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:28:22.584963 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:22.584656    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:28:22.584963 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:22.584652    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vqrqt" podUID="e5ff0db4-42a1-4d9a-8ba1-1c6068724788"
Sep 15 22:28:22.584963 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:22.584736    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4d5lp" podUID="b22214d5-9c42-4710-bfff-08651d7942a3"
Sep 15 22:28:22.584963 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:22.584854    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j4tsb" podUID="909a094b-77e5-4f6e-9d69-6f4214156810"
Sep 15 22:28:23.428365 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.428338    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-208.ec2.internal" event="NodeReady"
Sep 15 22:28:23.428524 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.428456    2565 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 15 22:28:23.519984 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.519951    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5cddfccf5c-5zq8q"]
Sep 15 22:28:23.521790 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.521775    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cddfccf5c-5zq8q"
Sep 15 22:28:23.525370 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:23.525330    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"image-registry-private-configuration\" is forbidden: User \"system:node:ip-10-0-128-208.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-image-registry\": no relationship found between node 'ip-10-0-128-208.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" type="*v1.Secret"
Sep 15 22:28:23.525885 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:23.525860    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"image-registry-tls\" is forbidden: User \"system:node:ip-10-0-128-208.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-image-registry\": no relationship found between node 'ip-10-0-128-208.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" type="*v1.Secret"
Sep 15 22:28:23.526159 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:23.526137    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"registry-dockercfg-6tvss\" is forbidden: User \"system:node:ip-10-0-128-208.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-image-registry\": no relationship found between node 'ip-10-0-128-208.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6tvss\"" type="*v1.Secret"
Sep 15 22:28:23.526211 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:23.526139    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"image-registry-5cddfccf5c-5zq8q\" is forbidden: User \"system:node:ip-10-0-128-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-image-registry\": no relationship found between node 'ip-10-0-128-208.ec2.internal' and this object" podUID="4f8a5789-3f4c-4671-b9b7-91374ea81f86" pod="openshift-image-registry/image-registry-5cddfccf5c-5zq8q"
Sep 15 22:28:23.527679 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:23.527592    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"installation-pull-secrets\" is forbidden: User \"system:node:ip-10-0-128-208.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-image-registry\": no relationship found between node 'ip-10-0-128-208.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" type="*v1.Secret"
Sep 15 22:28:23.537145 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.537122    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 15 22:28:23.566974 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.566948    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5cddfccf5c-5zq8q"]
Sep 15 22:28:23.567448 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.567436    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-gk8zw"]
Sep 15 22:28:23.569103 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.569088    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-q8hxs"]
Sep 15 22:28:23.569218 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.569201    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gk8zw"
Sep 15 22:28:23.570942 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.570924    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-q8hxs"
Sep 15 22:28:23.587179 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.587155    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gk8zw"]
Sep 15 22:28:23.593068 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.593010    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 15 22:28:23.593185 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.593045    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 15 22:28:23.593252 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.593192    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 15 22:28:23.593252 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.593216    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 15 22:28:23.593336 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.593286    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-44p4v\""
Sep 15 22:28:23.594098 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.594071    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8pmw9\""
Sep 15 22:28:23.594294 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.594111    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 15 22:28:23.594294 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.594184    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 15 22:28:23.601760 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.601736    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-q8hxs"]
Sep 15 22:28:23.631943 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.631913    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8rxv\" (UniqueName: \"kubernetes.io/projected/f618f110-7684-44c9-85ec-cb5e0802bed0-kube-api-access-k8rxv\") pod \"insights-runtime-extractor-q8hxs\" (UID: \"f618f110-7684-44c9-85ec-cb5e0802bed0\") " pod="openshift-insights/insights-runtime-extractor-q8hxs"
Sep 15 22:28:23.631943 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.631944    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d08ce97f-50da-4ee1-9e09-b8dfa23cefb3-tmp-dir\") pod \"dns-default-gk8zw\" (UID: \"d08ce97f-50da-4ee1-9e09-b8dfa23cefb3\") " pod="openshift-dns/dns-default-gk8zw"
Sep 15 22:28:23.632133 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.631967    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f618f110-7684-44c9-85ec-cb5e0802bed0-data-volume\") pod \"insights-runtime-extractor-q8hxs\" (UID: \"f618f110-7684-44c9-85ec-cb5e0802bed0\") " pod="openshift-insights/insights-runtime-extractor-q8hxs"
Sep 15 22:28:23.632133 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.632020    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4f8a5789-3f4c-4671-b9b7-91374ea81f86-installation-pull-secrets\") pod \"image-registry-5cddfccf5c-5zq8q\" (UID: \"4f8a5789-3f4c-4671-b9b7-91374ea81f86\") " pod="openshift-image-registry/image-registry-5cddfccf5c-5zq8q"
Sep 15 22:28:23.632133 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.632059    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f618f110-7684-44c9-85ec-cb5e0802bed0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q8hxs\" (UID: \"f618f110-7684-44c9-85ec-cb5e0802bed0\") " pod="openshift-insights/insights-runtime-extractor-q8hxs"
Sep 15 22:28:23.632133 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.632094    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4f8a5789-3f4c-4671-b9b7-91374ea81f86-trusted-ca\") pod \"image-registry-5cddfccf5c-5zq8q\" (UID: \"4f8a5789-3f4c-4671-b9b7-91374ea81f86\") " pod="openshift-image-registry/image-registry-5cddfccf5c-5zq8q"
Sep 15 22:28:23.632133 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.632114    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4f8a5789-3f4c-4671-b9b7-91374ea81f86-bound-sa-token\") pod \"image-registry-5cddfccf5c-5zq8q\" (UID: \"4f8a5789-3f4c-4671-b9b7-91374ea81f86\") " pod="openshift-image-registry/image-registry-5cddfccf5c-5zq8q"
Sep 15 22:28:23.632133 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.632129    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d08ce97f-50da-4ee1-9e09-b8dfa23cefb3-config-volume\") pod \"dns-default-gk8zw\" (UID: \"d08ce97f-50da-4ee1-9e09-b8dfa23cefb3\") " pod="openshift-dns/dns-default-gk8zw"
Sep 15 22:28:23.632307 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.632150    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4f8a5789-3f4c-4671-b9b7-91374ea81f86-image-registry-private-configuration\") pod \"image-registry-5cddfccf5c-5zq8q\" (UID: \"4f8a5789-3f4c-4671-b9b7-91374ea81f86\") " pod="openshift-image-registry/image-registry-5cddfccf5c-5zq8q"
Sep 15 22:28:23.632307 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.632173    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4f8a5789-3f4c-4671-b9b7-91374ea81f86-ca-trust-extracted\") pod \"image-registry-5cddfccf5c-5zq8q\" (UID: \"4f8a5789-3f4c-4671-b9b7-91374ea81f86\") " pod="openshift-image-registry/image-registry-5cddfccf5c-5zq8q"
Sep 15 22:28:23.632307 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.632202    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rgqh\" (UniqueName: \"kubernetes.io/projected/4f8a5789-3f4c-4671-b9b7-91374ea81f86-kube-api-access-8rgqh\") pod \"image-registry-5cddfccf5c-5zq8q\" (UID: \"4f8a5789-3f4c-4671-b9b7-91374ea81f86\") " pod="openshift-image-registry/image-registry-5cddfccf5c-5zq8q"
Sep 15 22:28:23.632307 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.632270    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d08ce97f-50da-4ee1-9e09-b8dfa23cefb3-metrics-tls\") pod \"dns-default-gk8zw\" (UID: \"d08ce97f-50da-4ee1-9e09-b8dfa23cefb3\") " pod="openshift-dns/dns-default-gk8zw"
Sep 15 22:28:23.632307 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.632301    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rjvb\" (UniqueName: \"kubernetes.io/projected/d08ce97f-50da-4ee1-9e09-b8dfa23cefb3-kube-api-access-4rjvb\") pod \"dns-default-gk8zw\" (UID: \"d08ce97f-50da-4ee1-9e09-b8dfa23cefb3\") " pod="openshift-dns/dns-default-gk8zw"
Sep 15 22:28:23.632451 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.632328    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f8a5789-3f4c-4671-b9b7-91374ea81f86-registry-tls\") pod \"image-registry-5cddfccf5c-5zq8q\" (UID: \"4f8a5789-3f4c-4671-b9b7-91374ea81f86\") " pod="openshift-image-registry/image-registry-5cddfccf5c-5zq8q"
Sep 15 22:28:23.632451 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.632343    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4f8a5789-3f4c-4671-b9b7-91374ea81f86-registry-certificates\") pod \"image-registry-5cddfccf5c-5zq8q\" (UID: \"4f8a5789-3f4c-4671-b9b7-91374ea81f86\") " pod="openshift-image-registry/image-registry-5cddfccf5c-5zq8q"
Sep 15 22:28:23.632451 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.632357    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f618f110-7684-44c9-85ec-cb5e0802bed0-crio-socket\") pod \"insights-runtime-extractor-q8hxs\" (UID: \"f618f110-7684-44c9-85ec-cb5e0802bed0\") " pod="openshift-insights/insights-runtime-extractor-q8hxs"
Sep 15 22:28:23.632451 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.632379    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f618f110-7684-44c9-85ec-cb5e0802bed0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q8hxs\" (UID: \"f618f110-7684-44c9-85ec-cb5e0802bed0\") " pod="openshift-insights/insights-runtime-extractor-q8hxs"
Sep 15 22:28:23.665905 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.665870    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-m6xfw"]
Sep 15 22:28:23.667639 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.667619    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-m6xfw"
Sep 15 22:28:23.673077 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.673055    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 15 22:28:23.673207 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.673113    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 15 22:28:23.673372 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.673302    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 15 22:28:23.673732 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.673710    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-vrlmr\""
Sep 15 22:28:23.693450 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.693392    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-m6xfw"]
Sep 15 22:28:23.733634 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.733605    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rgqh\" (UniqueName: \"kubernetes.io/projected/4f8a5789-3f4c-4671-b9b7-91374ea81f86-kube-api-access-8rgqh\") pod \"image-registry-5cddfccf5c-5zq8q\" (UID: \"4f8a5789-3f4c-4671-b9b7-91374ea81f86\") " pod="openshift-image-registry/image-registry-5cddfccf5c-5zq8q"
Sep 15 22:28:23.733762 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.733645    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d08ce97f-50da-4ee1-9e09-b8dfa23cefb3-metrics-tls\") pod \"dns-default-gk8zw\" (UID: \"d08ce97f-50da-4ee1-9e09-b8dfa23cefb3\") " pod="openshift-dns/dns-default-gk8zw"
Sep 15 22:28:23.733762 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.733660    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rjvb\" (UniqueName: \"kubernetes.io/projected/d08ce97f-50da-4ee1-9e09-b8dfa23cefb3-kube-api-access-4rjvb\") pod \"dns-default-gk8zw\" (UID: \"d08ce97f-50da-4ee1-9e09-b8dfa23cefb3\") " pod="openshift-dns/dns-default-gk8zw"
Sep 15 22:28:23.733762 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.733693    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f8a5789-3f4c-4671-b9b7-91374ea81f86-registry-tls\") pod \"image-registry-5cddfccf5c-5zq8q\" (UID: \"4f8a5789-3f4c-4671-b9b7-91374ea81f86\") " pod="openshift-image-registry/image-registry-5cddfccf5c-5zq8q"
Sep 15 22:28:23.733762 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.733716    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4f8a5789-3f4c-4671-b9b7-91374ea81f86-registry-certificates\") pod \"image-registry-5cddfccf5c-5zq8q\" (UID: \"4f8a5789-3f4c-4671-b9b7-91374ea81f86\") " pod="openshift-image-registry/image-registry-5cddfccf5c-5zq8q"
Sep 15 22:28:23.733762 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.733739    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f618f110-7684-44c9-85ec-cb5e0802bed0-crio-socket\") pod \"insights-runtime-extractor-q8hxs\" (UID: \"f618f110-7684-44c9-85ec-cb5e0802bed0\") " pod="openshift-insights/insights-runtime-extractor-q8hxs"
Sep 15 22:28:23.733762 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.733759    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f618f110-7684-44c9-85ec-cb5e0802bed0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q8hxs\" (UID: \"f618f110-7684-44c9-85ec-cb5e0802bed0\") " pod="openshift-insights/insights-runtime-extractor-q8hxs"
Sep 15 22:28:23.734072 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.733784    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8rxv\" (UniqueName: \"kubernetes.io/projected/f618f110-7684-44c9-85ec-cb5e0802bed0-kube-api-access-k8rxv\") pod \"insights-runtime-extractor-q8hxs\" (UID: \"f618f110-7684-44c9-85ec-cb5e0802bed0\") " pod="openshift-insights/insights-runtime-extractor-q8hxs"
Sep 15 22:28:23.734072 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.733837    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d08ce97f-50da-4ee1-9e09-b8dfa23cefb3-tmp-dir\") pod \"dns-default-gk8zw\" (UID: \"d08ce97f-50da-4ee1-9e09-b8dfa23cefb3\") " pod="openshift-dns/dns-default-gk8zw"
Sep 15 22:28:23.734072 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.733884    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f618f110-7684-44c9-85ec-cb5e0802bed0-data-volume\") pod \"insights-runtime-extractor-q8hxs\" (UID: \"f618f110-7684-44c9-85ec-cb5e0802bed0\") " pod="openshift-insights/insights-runtime-extractor-q8hxs"
Sep 15 22:28:23.734072 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.733928    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4f8a5789-3f4c-4671-b9b7-91374ea81f86-installation-pull-secrets\") pod \"image-registry-5cddfccf5c-5zq8q\" (UID: \"4f8a5789-3f4c-4671-b9b7-91374ea81f86\") " pod="openshift-image-registry/image-registry-5cddfccf5c-5zq8q"
Sep 15 22:28:23.734072 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.733955    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f618f110-7684-44c9-85ec-cb5e0802bed0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q8hxs\" (UID: \"f618f110-7684-44c9-85ec-cb5e0802bed0\") " pod="openshift-insights/insights-runtime-extractor-q8hxs"
Sep 15 22:28:23.734072 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.733982    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bf3ece62-14b8-4d64-b799-3b9796eb2ea4-cert\") pod \"ingress-canary-m6xfw\" (UID: \"bf3ece62-14b8-4d64-b799-3b9796eb2ea4\") " pod="openshift-ingress-canary/ingress-canary-m6xfw"
Sep 15 22:28:23.734072 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.734020    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f618f110-7684-44c9-85ec-cb5e0802bed0-crio-socket\") pod \"insights-runtime-extractor-q8hxs\" (UID: \"f618f110-7684-44c9-85ec-cb5e0802bed0\") " pod="openshift-insights/insights-runtime-extractor-q8hxs"
Sep 15 22:28:23.734072 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.734043    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4f8a5789-3f4c-4671-b9b7-91374ea81f86-trusted-ca\") pod \"image-registry-5cddfccf5c-5zq8q\" (UID: \"4f8a5789-3f4c-4671-b9b7-91374ea81f86\") " pod="openshift-image-registry/image-registry-5cddfccf5c-5zq8q"
Sep 15 22:28:23.734469 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.734098    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52qhk\" (UniqueName: \"kubernetes.io/projected/bf3ece62-14b8-4d64-b799-3b9796eb2ea4-kube-api-access-52qhk\") pod \"ingress-canary-m6xfw\" (UID: \"bf3ece62-14b8-4d64-b799-3b9796eb2ea4\") " pod="openshift-ingress-canary/ingress-canary-m6xfw"
Sep 15 22:28:23.734469 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.734134    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4f8a5789-3f4c-4671-b9b7-91374ea81f86-bound-sa-token\") pod \"image-registry-5cddfccf5c-5zq8q\" (UID: \"4f8a5789-3f4c-4671-b9b7-91374ea81f86\") " pod="openshift-image-registry/image-registry-5cddfccf5c-5zq8q"
Sep 15 22:28:23.734469 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.734161    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d08ce97f-50da-4ee1-9e09-b8dfa23cefb3-config-volume\") pod \"dns-default-gk8zw\" (UID: \"d08ce97f-50da-4ee1-9e09-b8dfa23cefb3\") " pod="openshift-dns/dns-default-gk8zw"
Sep 15 22:28:23.734469 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.734165    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d08ce97f-50da-4ee1-9e09-b8dfa23cefb3-tmp-dir\") pod \"dns-default-gk8zw\" (UID: \"d08ce97f-50da-4ee1-9e09-b8dfa23cefb3\") " pod="openshift-dns/dns-default-gk8zw"
Sep 15 22:28:23.734469 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.734190    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4f8a5789-3f4c-4671-b9b7-91374ea81f86-image-registry-private-configuration\") pod \"image-registry-5cddfccf5c-5zq8q\" (UID: \"4f8a5789-3f4c-4671-b9b7-91374ea81f86\") " pod="openshift-image-registry/image-registry-5cddfccf5c-5zq8q"
Sep 15 22:28:23.734469 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.734221    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4f8a5789-3f4c-4671-b9b7-91374ea81f86-ca-trust-extracted\") pod \"image-registry-5cddfccf5c-5zq8q\" (UID: \"4f8a5789-3f4c-4671-b9b7-91374ea81f86\") " pod="openshift-image-registry/image-registry-5cddfccf5c-5zq8q"
Sep 15 22:28:23.734469 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.734402    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f618f110-7684-44c9-85ec-cb5e0802bed0-data-volume\") pod \"insights-runtime-extractor-q8hxs\" (UID: \"f618f110-7684-44c9-85ec-cb5e0802bed0\") " pod="openshift-insights/insights-runtime-extractor-q8hxs"
Sep 15 22:28:23.735727 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.735261    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f618f110-7684-44c9-85ec-cb5e0802bed0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q8hxs\" (UID: \"f618f110-7684-44c9-85ec-cb5e0802bed0\") " pod="openshift-insights/insights-runtime-extractor-q8hxs"
Sep 15 22:28:23.735727 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.735311    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4f8a5789-3f4c-4671-b9b7-91374ea81f86-registry-certificates\") pod \"image-registry-5cddfccf5c-5zq8q\" (UID: \"4f8a5789-3f4c-4671-b9b7-91374ea81f86\") " pod="openshift-image-registry/image-registry-5cddfccf5c-5zq8q"
Sep 15 22:28:23.735727 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.735521    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d08ce97f-50da-4ee1-9e09-b8dfa23cefb3-config-volume\") pod \"dns-default-gk8zw\" (UID: \"d08ce97f-50da-4ee1-9e09-b8dfa23cefb3\") " pod="openshift-dns/dns-default-gk8zw"
Sep 15 22:28:23.735727 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.735543    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4f8a5789-3f4c-4671-b9b7-91374ea81f86-ca-trust-extracted\") pod \"image-registry-5cddfccf5c-5zq8q\" (UID: \"4f8a5789-3f4c-4671-b9b7-91374ea81f86\") " pod="openshift-image-registry/image-registry-5cddfccf5c-5zq8q"
Sep 15 22:28:23.736051 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.735945    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4f8a5789-3f4c-4671-b9b7-91374ea81f86-trusted-ca\") pod \"image-registry-5cddfccf5c-5zq8q\" (UID: \"4f8a5789-3f4c-4671-b9b7-91374ea81f86\") " pod="openshift-image-registry/image-registry-5cddfccf5c-5zq8q"
Sep 15 22:28:23.738663 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.738641    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d08ce97f-50da-4ee1-9e09-b8dfa23cefb3-metrics-tls\") pod \"dns-default-gk8zw\" (UID: \"d08ce97f-50da-4ee1-9e09-b8dfa23cefb3\") " pod="openshift-dns/dns-default-gk8zw"
Sep 15 22:28:23.739209 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.738638    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f618f110-7684-44c9-85ec-cb5e0802bed0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q8hxs\" (UID: \"f618f110-7684-44c9-85ec-cb5e0802bed0\") " pod="openshift-insights/insights-runtime-extractor-q8hxs"
Sep 15 22:28:23.747003 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.746980    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rjvb\" (UniqueName: \"kubernetes.io/projected/d08ce97f-50da-4ee1-9e09-b8dfa23cefb3-kube-api-access-4rjvb\") pod \"dns-default-gk8zw\" (UID: \"d08ce97f-50da-4ee1-9e09-b8dfa23cefb3\") " pod="openshift-dns/dns-default-gk8zw"
Sep 15 22:28:23.748062 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.748015    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8rxv\" (UniqueName: \"kubernetes.io/projected/f618f110-7684-44c9-85ec-cb5e0802bed0-kube-api-access-k8rxv\") pod \"insights-runtime-extractor-q8hxs\" (UID: \"f618f110-7684-44c9-85ec-cb5e0802bed0\") " pod="openshift-insights/insights-runtime-extractor-q8hxs"
Sep 15 22:28:23.748884 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.748864    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rgqh\" (UniqueName: \"kubernetes.io/projected/4f8a5789-3f4c-4671-b9b7-91374ea81f86-kube-api-access-8rgqh\") pod \"image-registry-5cddfccf5c-5zq8q\" (UID: \"4f8a5789-3f4c-4671-b9b7-91374ea81f86\") " pod="openshift-image-registry/image-registry-5cddfccf5c-5zq8q"
Sep 15 22:28:23.765213 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.765188    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4f8a5789-3f4c-4671-b9b7-91374ea81f86-bound-sa-token\") pod \"image-registry-5cddfccf5c-5zq8q\" (UID: \"4f8a5789-3f4c-4671-b9b7-91374ea81f86\") " pod="openshift-image-registry/image-registry-5cddfccf5c-5zq8q"
Sep 15 22:28:23.834656 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.834614    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bf3ece62-14b8-4d64-b799-3b9796eb2ea4-cert\") pod \"ingress-canary-m6xfw\" (UID: \"bf3ece62-14b8-4d64-b799-3b9796eb2ea4\") " pod="openshift-ingress-canary/ingress-canary-m6xfw"
Sep 15 22:28:23.834816 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.834671    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52qhk\" (UniqueName: \"kubernetes.io/projected/bf3ece62-14b8-4d64-b799-3b9796eb2ea4-kube-api-access-52qhk\") pod \"ingress-canary-m6xfw\" (UID: \"bf3ece62-14b8-4d64-b799-3b9796eb2ea4\") " pod="openshift-ingress-canary/ingress-canary-m6xfw"
Sep 15 22:28:23.836845 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.836825    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bf3ece62-14b8-4d64-b799-3b9796eb2ea4-cert\") pod \"ingress-canary-m6xfw\" (UID: \"bf3ece62-14b8-4d64-b799-3b9796eb2ea4\") " pod="openshift-ingress-canary/ingress-canary-m6xfw"
Sep 15 22:28:23.849591 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.849561    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52qhk\" (UniqueName: \"kubernetes.io/projected/bf3ece62-14b8-4d64-b799-3b9796eb2ea4-kube-api-access-52qhk\") pod \"ingress-canary-m6xfw\" (UID: \"bf3ece62-14b8-4d64-b799-3b9796eb2ea4\") " pod="openshift-ingress-canary/ingress-canary-m6xfw"
Sep 15 22:28:23.878127 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.878107    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gk8zw"
Sep 15 22:28:23.882638 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.882623    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-q8hxs"
Sep 15 22:28:23.975975 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:23.975946    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-m6xfw"
Sep 15 22:28:24.194053 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:24.193894    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-q8hxs"]
Sep 15 22:28:24.198783 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:28:24.198754    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf618f110_7684_44c9_85ec_cb5e0802bed0.slice/crio-3ad5ceff115bb7d796b6e9c3d192154c11ab27524f48c8f8f534df91f34c433f WatchSource:0}: Error finding container 3ad5ceff115bb7d796b6e9c3d192154c11ab27524f48c8f8f534df91f34c433f: Status 404 returned error can't find the container with id 3ad5ceff115bb7d796b6e9c3d192154c11ab27524f48c8f8f534df91f34c433f
Sep 15 22:28:24.199562 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:24.199491    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gk8zw"]
Sep 15 22:28:24.201837 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:28:24.201811    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd08ce97f_50da_4ee1_9e09_b8dfa23cefb3.slice/crio-16cac926cdd7705424bd8339e14146c8f7f0d3985699db2058b72a4bc66555b2 WatchSource:0}: Error finding container 16cac926cdd7705424bd8339e14146c8f7f0d3985699db2058b72a4bc66555b2: Status 404 returned error can't find the container with id 16cac926cdd7705424bd8339e14146c8f7f0d3985699db2058b72a4bc66555b2
Sep 15 22:28:24.250210 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:24.250178    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-m6xfw"]
Sep 15 22:28:24.253520 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:28:24.253492    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf3ece62_14b8_4d64_b799_3b9796eb2ea4.slice/crio-54909fdd5fe71d3a36db941d7eee92a339e24e1103627ffbaee328dba8640a2a WatchSource:0}: Error finding container 54909fdd5fe71d3a36db941d7eee92a339e24e1103627ffbaee328dba8640a2a: Status 404 returned error can't find the container with id 54909fdd5fe71d3a36db941d7eee92a339e24e1103627ffbaee328dba8640a2a
Sep 15 22:28:24.537215 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:24.537188    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6tvss\""
Sep 15 22:28:24.584368 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:24.584340    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:28:24.584535 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:24.584464    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:28:24.584535 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:24.584340    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:28:24.591733 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:24.591713    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 15 22:28:24.592143 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:24.592127    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 15 22:28:24.592206 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:24.592140    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 15 22:28:24.592206 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:24.592163    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 15 22:28:24.592306 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:24.592231    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fznt9\""
Sep 15 22:28:24.592306 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:24.592253    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-29265\""
Sep 15 22:28:24.733995 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:24.733967    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: failed to sync secret cache: timed out waiting for the condition
Sep 15 22:28:24.733995 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:24.733994    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cddfccf5c-5zq8q: failed to sync secret cache: timed out waiting for the condition
Sep 15 22:28:24.734266 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:24.734077    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f8a5789-3f4c-4671-b9b7-91374ea81f86-registry-tls podName:4f8a5789-3f4c-4671-b9b7-91374ea81f86 nodeName:}" failed. No retries permitted until 2026-09-15 22:28:25.234058896 +0000 UTC m=+55.093770178 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4f8a5789-3f4c-4671-b9b7-91374ea81f86-registry-tls") pod "image-registry-5cddfccf5c-5zq8q" (UID: "4f8a5789-3f4c-4671-b9b7-91374ea81f86") : failed to sync secret cache: timed out waiting for the condition
Sep 15 22:28:24.734266 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:24.734164    2565 secret.go:189] Couldn't get secret openshift-image-registry/installation-pull-secrets: failed to sync secret cache: timed out waiting for the condition
Sep 15 22:28:24.734266 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:24.734226    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f8a5789-3f4c-4671-b9b7-91374ea81f86-installation-pull-secrets podName:4f8a5789-3f4c-4671-b9b7-91374ea81f86 nodeName:}" failed. No retries permitted until 2026-09-15 22:28:25.23420863 +0000 UTC m=+55.093919895 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "installation-pull-secrets" (UniqueName: "kubernetes.io/secret/4f8a5789-3f4c-4671-b9b7-91374ea81f86-installation-pull-secrets") pod "image-registry-5cddfccf5c-5zq8q" (UID: "4f8a5789-3f4c-4671-b9b7-91374ea81f86") : failed to sync secret cache: timed out waiting for the condition
Sep 15 22:28:24.734266 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:24.734229    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 15 22:28:24.740738 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:24.740711    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4f8a5789-3f4c-4671-b9b7-91374ea81f86-image-registry-private-configuration\") pod \"image-registry-5cddfccf5c-5zq8q\" (UID: \"4f8a5789-3f4c-4671-b9b7-91374ea81f86\") " pod="openshift-image-registry/image-registry-5cddfccf5c-5zq8q"
Sep 15 22:28:24.827520 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:24.827440    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 15 22:28:24.831040 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:24.830990    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-m6xfw" event={"ID":"bf3ece62-14b8-4d64-b799-3b9796eb2ea4","Type":"ContainerStarted","Data":"54909fdd5fe71d3a36db941d7eee92a339e24e1103627ffbaee328dba8640a2a"}
Sep 15 22:28:24.832724 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:24.832699    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q8hxs" event={"ID":"f618f110-7684-44c9-85ec-cb5e0802bed0","Type":"ContainerStarted","Data":"135a298556edbf8a71b257f49f0fa389456903aca9bb327f2c7cce7b99337adf"}
Sep 15 22:28:24.832887 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:24.832754    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q8hxs" event={"ID":"f618f110-7684-44c9-85ec-cb5e0802bed0","Type":"ContainerStarted","Data":"3ad5ceff115bb7d796b6e9c3d192154c11ab27524f48c8f8f534df91f34c433f"}
Sep 15 22:28:24.833900 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:24.833875    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gk8zw" event={"ID":"d08ce97f-50da-4ee1-9e09-b8dfa23cefb3","Type":"ContainerStarted","Data":"16cac926cdd7705424bd8339e14146c8f7f0d3985699db2058b72a4bc66555b2"}
Sep 15 22:28:24.982271 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:24.982237    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 15 22:28:25.245848 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:25.245751    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4f8a5789-3f4c-4671-b9b7-91374ea81f86-installation-pull-secrets\") pod \"image-registry-5cddfccf5c-5zq8q\" (UID: \"4f8a5789-3f4c-4671-b9b7-91374ea81f86\") " pod="openshift-image-registry/image-registry-5cddfccf5c-5zq8q"
Sep 15 22:28:25.246101 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:25.245867    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f8a5789-3f4c-4671-b9b7-91374ea81f86-registry-tls\") pod \"image-registry-5cddfccf5c-5zq8q\" (UID: \"4f8a5789-3f4c-4671-b9b7-91374ea81f86\") " pod="openshift-image-registry/image-registry-5cddfccf5c-5zq8q"
Sep 15 22:28:25.248450 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:25.248425    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f8a5789-3f4c-4671-b9b7-91374ea81f86-registry-tls\") pod \"image-registry-5cddfccf5c-5zq8q\" (UID: \"4f8a5789-3f4c-4671-b9b7-91374ea81f86\") " pod="openshift-image-registry/image-registry-5cddfccf5c-5zq8q"
Sep 15 22:28:25.248554 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:25.248456    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4f8a5789-3f4c-4671-b9b7-91374ea81f86-installation-pull-secrets\") pod \"image-registry-5cddfccf5c-5zq8q\" (UID: \"4f8a5789-3f4c-4671-b9b7-91374ea81f86\") " pod="openshift-image-registry/image-registry-5cddfccf5c-5zq8q"
Sep 15 22:28:25.330456 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:25.330421    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cddfccf5c-5zq8q"
Sep 15 22:28:25.626012 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:25.625764    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5cddfccf5c-5zq8q"]
Sep 15 22:28:25.839014 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:25.838984    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q8hxs" event={"ID":"f618f110-7684-44c9-85ec-cb5e0802bed0","Type":"ContainerStarted","Data":"6b7ab4cc69a40f9edecb5669854993341f21bfa491f9d3cc8026969cd37e7463"}
Sep 15 22:28:25.957396 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:28:25.957323    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f8a5789_3f4c_4671_b9b7_91374ea81f86.slice/crio-3404c841d71b15b04e7ee8c4a5bee8a71b831b61c485b35084bb6d84f575411c WatchSource:0}: Error finding container 3404c841d71b15b04e7ee8c4a5bee8a71b831b61c485b35084bb6d84f575411c: Status 404 returned error can't find the container with id 3404c841d71b15b04e7ee8c4a5bee8a71b831b61c485b35084bb6d84f575411c
Sep 15 22:28:26.837985 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:26.837953    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b576979d9-2vkj4"]
Sep 15 22:28:26.839889 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:26.839856    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b576979d9-2vkj4"
Sep 15 22:28:26.841912 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:26.841887    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 15 22:28:26.842703 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:26.842659    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 15 22:28:26.842703 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:26.842672    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 15 22:28:26.842877 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:26.842754    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 15 22:28:26.842877 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:26.842768    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 15 22:28:26.843127 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:26.843099    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4z6d6\""
Sep 15 22:28:26.843325 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:26.843306    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 15 22:28:26.843396 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:26.843310    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 15 22:28:26.843857 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:26.843746    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cddfccf5c-5zq8q" event={"ID":"4f8a5789-3f4c-4671-b9b7-91374ea81f86","Type":"ContainerStarted","Data":"03ad41520be5f769ab730d6f1dcd91a44e68aea75e1afcad42df0886c6e3b536"}
Sep 15 22:28:26.843857 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:26.843795    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cddfccf5c-5zq8q" event={"ID":"4f8a5789-3f4c-4671-b9b7-91374ea81f86","Type":"ContainerStarted","Data":"3404c841d71b15b04e7ee8c4a5bee8a71b831b61c485b35084bb6d84f575411c"}
Sep 15 22:28:26.843857 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:26.843824    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5cddfccf5c-5zq8q"
Sep 15 22:28:26.846006 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:26.845982    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gk8zw" event={"ID":"d08ce97f-50da-4ee1-9e09-b8dfa23cefb3","Type":"ContainerStarted","Data":"1dc4a2a22e2d710f4032a4ac5418047ec37cd0f1c45b0c22e49a2a74ecfb98ba"}
Sep 15 22:28:26.846105 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:26.846045    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gk8zw" event={"ID":"d08ce97f-50da-4ee1-9e09-b8dfa23cefb3","Type":"ContainerStarted","Data":"6f0cf904978faaafe9c556552e6dfb6c85ff736f76ab607f08619bd56a669ec1"}
Sep 15 22:28:26.846191 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:26.846158    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-gk8zw"
Sep 15 22:28:26.852874 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:26.852844    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b576979d9-2vkj4"]
Sep 15 22:28:26.860140 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:26.860115    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07af1527-72f1-49e8-ac8e-54ab1c1bccdf-console-serving-cert\") pod \"console-6b576979d9-2vkj4\" (UID: \"07af1527-72f1-49e8-ac8e-54ab1c1bccdf\") " pod="openshift-console/console-6b576979d9-2vkj4"
Sep 15 22:28:26.860238 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:26.860159    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07af1527-72f1-49e8-ac8e-54ab1c1bccdf-service-ca\") pod \"console-6b576979d9-2vkj4\" (UID: \"07af1527-72f1-49e8-ac8e-54ab1c1bccdf\") " pod="openshift-console/console-6b576979d9-2vkj4"
Sep 15 22:28:26.860280 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:26.860245    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07af1527-72f1-49e8-ac8e-54ab1c1bccdf-console-oauth-config\") pod \"console-6b576979d9-2vkj4\" (UID: \"07af1527-72f1-49e8-ac8e-54ab1c1bccdf\") " pod="openshift-console/console-6b576979d9-2vkj4"
Sep 15 22:28:26.860280 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:26.860272    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78mxd\" (UniqueName: \"kubernetes.io/projected/07af1527-72f1-49e8-ac8e-54ab1c1bccdf-kube-api-access-78mxd\") pod \"console-6b576979d9-2vkj4\" (UID: \"07af1527-72f1-49e8-ac8e-54ab1c1bccdf\") " pod="openshift-console/console-6b576979d9-2vkj4"
Sep 15 22:28:26.860372 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:26.860303    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07af1527-72f1-49e8-ac8e-54ab1c1bccdf-oauth-serving-cert\") pod \"console-6b576979d9-2vkj4\" (UID: \"07af1527-72f1-49e8-ac8e-54ab1c1bccdf\") " pod="openshift-console/console-6b576979d9-2vkj4"
Sep 15 22:28:26.860372 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:26.860331    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07af1527-72f1-49e8-ac8e-54ab1c1bccdf-console-config\") pod \"console-6b576979d9-2vkj4\" (UID: \"07af1527-72f1-49e8-ac8e-54ab1c1bccdf\") " pod="openshift-console/console-6b576979d9-2vkj4"
Sep 15 22:28:26.883707 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:26.883665    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-gk8zw" podStartSLOduration=2.101874082 podStartE2EDuration="3.883648097s" totalImagesPullingTime="1.781774015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:28:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:28:26.882672636 +0000 UTC m=+56.742383952" watchObservedRunningTime="2026-09-15 22:28:26.883648097 +0000 UTC m=+56.743359375"
Sep 15 22:28:26.903502 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:26.903454    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5cddfccf5c-5zq8q" podStartSLOduration=4.903437705 podStartE2EDuration="4.903437705s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:28:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:28:26.902386957 +0000 UTC m=+56.762098242" watchObservedRunningTime="2026-09-15 22:28:26.903437705 +0000 UTC m=+56.763148986"
Sep 15 22:28:26.961713 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:26.961673    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07af1527-72f1-49e8-ac8e-54ab1c1bccdf-console-oauth-config\") pod \"console-6b576979d9-2vkj4\" (UID: \"07af1527-72f1-49e8-ac8e-54ab1c1bccdf\") " pod="openshift-console/console-6b576979d9-2vkj4"
Sep 15 22:28:26.961875 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:26.961750    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78mxd\" (UniqueName: \"kubernetes.io/projected/07af1527-72f1-49e8-ac8e-54ab1c1bccdf-kube-api-access-78mxd\") pod \"console-6b576979d9-2vkj4\" (UID: \"07af1527-72f1-49e8-ac8e-54ab1c1bccdf\") " pod="openshift-console/console-6b576979d9-2vkj4"
Sep 15 22:28:26.961875 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:26.961838    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07af1527-72f1-49e8-ac8e-54ab1c1bccdf-oauth-serving-cert\") pod \"console-6b576979d9-2vkj4\" (UID: \"07af1527-72f1-49e8-ac8e-54ab1c1bccdf\") " pod="openshift-console/console-6b576979d9-2vkj4"
Sep 15 22:28:26.961993 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:26.961882    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07af1527-72f1-49e8-ac8e-54ab1c1bccdf-console-config\") pod \"console-6b576979d9-2vkj4\" (UID: \"07af1527-72f1-49e8-ac8e-54ab1c1bccdf\") " pod="openshift-console/console-6b576979d9-2vkj4"
Sep 15 22:28:26.961993 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:26.961953    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07af1527-72f1-49e8-ac8e-54ab1c1bccdf-console-serving-cert\") pod \"console-6b576979d9-2vkj4\" (UID: \"07af1527-72f1-49e8-ac8e-54ab1c1bccdf\") " pod="openshift-console/console-6b576979d9-2vkj4"
Sep 15 22:28:26.962778 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:26.962288    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07af1527-72f1-49e8-ac8e-54ab1c1bccdf-service-ca\") pod \"console-6b576979d9-2vkj4\" (UID: \"07af1527-72f1-49e8-ac8e-54ab1c1bccdf\") " pod="openshift-console/console-6b576979d9-2vkj4"
Sep 15 22:28:26.962778 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:26.962743    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07af1527-72f1-49e8-ac8e-54ab1c1bccdf-oauth-serving-cert\") pod \"console-6b576979d9-2vkj4\" (UID: \"07af1527-72f1-49e8-ac8e-54ab1c1bccdf\") " pod="openshift-console/console-6b576979d9-2vkj4"
Sep 15 22:28:26.962965 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:26.962939    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07af1527-72f1-49e8-ac8e-54ab1c1bccdf-console-config\") pod \"console-6b576979d9-2vkj4\" (UID: \"07af1527-72f1-49e8-ac8e-54ab1c1bccdf\") " pod="openshift-console/console-6b576979d9-2vkj4"
Sep 15 22:28:26.963142 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:26.963120    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07af1527-72f1-49e8-ac8e-54ab1c1bccdf-service-ca\") pod \"console-6b576979d9-2vkj4\" (UID: \"07af1527-72f1-49e8-ac8e-54ab1c1bccdf\") " pod="openshift-console/console-6b576979d9-2vkj4"
Sep 15 22:28:26.964427 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:26.964397    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07af1527-72f1-49e8-ac8e-54ab1c1bccdf-console-oauth-config\") pod \"console-6b576979d9-2vkj4\" (UID: \"07af1527-72f1-49e8-ac8e-54ab1c1bccdf\") " pod="openshift-console/console-6b576979d9-2vkj4"
Sep 15 22:28:26.964779 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:26.964758    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07af1527-72f1-49e8-ac8e-54ab1c1bccdf-console-serving-cert\") pod \"console-6b576979d9-2vkj4\" (UID: \"07af1527-72f1-49e8-ac8e-54ab1c1bccdf\") " pod="openshift-console/console-6b576979d9-2vkj4"
Sep 15 22:28:26.970579 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:26.970559    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78mxd\" (UniqueName: \"kubernetes.io/projected/07af1527-72f1-49e8-ac8e-54ab1c1bccdf-kube-api-access-78mxd\") pod \"console-6b576979d9-2vkj4\" (UID: \"07af1527-72f1-49e8-ac8e-54ab1c1bccdf\") " pod="openshift-console/console-6b576979d9-2vkj4"
Sep 15 22:28:27.150364 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:27.150280    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b576979d9-2vkj4"
Sep 15 22:28:27.718227 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:27.718150    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b576979d9-2vkj4"]
Sep 15 22:28:27.721384 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:28:27.721355    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07af1527_72f1_49e8_ac8e_54ab1c1bccdf.slice/crio-22e88e25b6d035d7e60ab16e57e275f3230a0aad3ae8e314858814b25b19f2f8 WatchSource:0}: Error finding container 22e88e25b6d035d7e60ab16e57e275f3230a0aad3ae8e314858814b25b19f2f8: Status 404 returned error can't find the container with id 22e88e25b6d035d7e60ab16e57e275f3230a0aad3ae8e314858814b25b19f2f8
Sep 15 22:28:27.853449 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:27.853412    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-m6xfw" event={"ID":"bf3ece62-14b8-4d64-b799-3b9796eb2ea4","Type":"ContainerStarted","Data":"db9ced573e131f169c638a8f80978139079ce828c2cd48c535d5481cb04f940f"}
Sep 15 22:28:27.855739 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:27.855706    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q8hxs" event={"ID":"f618f110-7684-44c9-85ec-cb5e0802bed0","Type":"ContainerStarted","Data":"2444389762ab1adb82a844d8f54c2cf51ef88f558b0aa883636c0d96fff2b614"}
Sep 15 22:28:27.856865 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:27.856838    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b576979d9-2vkj4" event={"ID":"07af1527-72f1-49e8-ac8e-54ab1c1bccdf","Type":"ContainerStarted","Data":"22e88e25b6d035d7e60ab16e57e275f3230a0aad3ae8e314858814b25b19f2f8"}
Sep 15 22:28:27.868922 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:27.868884    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-m6xfw" podStartSLOduration=1.672381035 podStartE2EDuration="4.868871227s" totalImagesPullingTime="3.196490192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:28:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:28:27.868056382 +0000 UTC m=+57.727767663" watchObservedRunningTime="2026-09-15 22:28:27.868871227 +0000 UTC m=+57.728582504"
Sep 15 22:28:27.886103 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:27.886062    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-q8hxs" podStartSLOduration=1.766738564 podStartE2EDuration="4.886044738s" totalImagesPullingTime="3.119306174s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 22:28:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:28:27.885366589 +0000 UTC m=+57.745077875" watchObservedRunningTime="2026-09-15 22:28:27.886044738 +0000 UTC m=+57.745756016"
Sep 15 22:28:30.777776 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:30.777704    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-74hmh"
Sep 15 22:28:30.868393 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:30.868356    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b576979d9-2vkj4" event={"ID":"07af1527-72f1-49e8-ac8e-54ab1c1bccdf","Type":"ContainerStarted","Data":"5380372c9aa4b838395025205dfa7652af680d8e229cee15399a84e9eb011bd5"}
Sep 15 22:28:30.887509 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:30.887462    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b576979d9-2vkj4" podStartSLOduration=2.09595245 podStartE2EDuration="4.887446002s" totalImagesPullingTime="2.791493552s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:28:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:28:30.886418491 +0000 UTC m=+60.746129774" watchObservedRunningTime="2026-09-15 22:28:30.887446002 +0000 UTC m=+60.747157364"
Sep 15 22:28:36.330220 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:36.330178    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b22214d5-9c42-4710-bfff-08651d7942a3-original-pull-secret\") pod \"global-pull-secret-syncer-4d5lp\" (UID: \"b22214d5-9c42-4710-bfff-08651d7942a3\") " pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:28:36.332444 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:36.332423    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 15 22:28:36.343047 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:36.343004    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b22214d5-9c42-4710-bfff-08651d7942a3-original-pull-secret\") pod \"global-pull-secret-syncer-4d5lp\" (UID: \"b22214d5-9c42-4710-bfff-08651d7942a3\") " pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:28:36.431422 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:36.431386    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/909a094b-77e5-4f6e-9d69-6f4214156810-metrics-certs\") pod \"network-metrics-daemon-j4tsb\" (UID: \"909a094b-77e5-4f6e-9d69-6f4214156810\") " pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:28:36.431561 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:36.431462    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vxxv\" (UniqueName: \"kubernetes.io/projected/e5ff0db4-42a1-4d9a-8ba1-1c6068724788-kube-api-access-9vxxv\") pod \"network-check-target-vqrqt\" (UID: \"e5ff0db4-42a1-4d9a-8ba1-1c6068724788\") " pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:28:36.433556 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:36.433536    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 15 22:28:36.433599 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:36.433536    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 15 22:28:36.443700 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:36.443680    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 15 22:28:36.444313 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:36.444286    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/909a094b-77e5-4f6e-9d69-6f4214156810-metrics-certs\") pod \"network-metrics-daemon-j4tsb\" (UID: \"909a094b-77e5-4f6e-9d69-6f4214156810\") " pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:28:36.454396 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:36.454369    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vxxv\" (UniqueName: \"kubernetes.io/projected/e5ff0db4-42a1-4d9a-8ba1-1c6068724788-kube-api-access-9vxxv\") pod \"network-check-target-vqrqt\" (UID: \"e5ff0db4-42a1-4d9a-8ba1-1c6068724788\") " pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:28:36.595399 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:36.595328    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-29265\""
Sep 15 22:28:36.599905 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:36.599890    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4d5lp"
Sep 15 22:28:36.604181 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:36.604162    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j4tsb"
Sep 15 22:28:36.606422 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:36.606404    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fznt9\""
Sep 15 22:28:36.614249 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:36.614230    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:28:36.859994 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:36.859927    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-gk8zw"
Sep 15 22:28:36.875376 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:36.875320    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4d5lp"]
Sep 15 22:28:36.876087 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:36.876054    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-j4tsb"]
Sep 15 22:28:36.879370 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:28:36.879345    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb22214d5_9c42_4710_bfff_08651d7942a3.slice/crio-3a857ec80592c02e5d0bfacfff6c422e1afce88fd5e4f87fbdf4174d4b097059 WatchSource:0}: Error finding container 3a857ec80592c02e5d0bfacfff6c422e1afce88fd5e4f87fbdf4174d4b097059: Status 404 returned error can't find the container with id 3a857ec80592c02e5d0bfacfff6c422e1afce88fd5e4f87fbdf4174d4b097059
Sep 15 22:28:36.879833 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:28:36.879813    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod909a094b_77e5_4f6e_9d69_6f4214156810.slice/crio-388dfb087ff73d576725b6b8efee099e5a6d7a73c0f1ca94e6f91a1e9dc87eed WatchSource:0}: Error finding container 388dfb087ff73d576725b6b8efee099e5a6d7a73c0f1ca94e6f91a1e9dc87eed: Status 404 returned error can't find the container with id 388dfb087ff73d576725b6b8efee099e5a6d7a73c0f1ca94e6f91a1e9dc87eed
Sep 15 22:28:36.888859 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:36.888826    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-j4tsb" event={"ID":"909a094b-77e5-4f6e-9d69-6f4214156810","Type":"ContainerStarted","Data":"388dfb087ff73d576725b6b8efee099e5a6d7a73c0f1ca94e6f91a1e9dc87eed"}
Sep 15 22:28:36.890272 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:36.890246    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4d5lp" event={"ID":"b22214d5-9c42-4710-bfff-08651d7942a3","Type":"ContainerStarted","Data":"3a857ec80592c02e5d0bfacfff6c422e1afce88fd5e4f87fbdf4174d4b097059"}
Sep 15 22:28:36.908076 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:36.908049    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vqrqt"]
Sep 15 22:28:36.910997 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:28:36.910974    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5ff0db4_42a1_4d9a_8ba1_1c6068724788.slice/crio-649bdd3268f8efcf76a1d90a9edabbcf22fbf0642cd8dde373f82498b28740ab WatchSource:0}: Error finding container 649bdd3268f8efcf76a1d90a9edabbcf22fbf0642cd8dde373f82498b28740ab: Status 404 returned error can't find the container with id 649bdd3268f8efcf76a1d90a9edabbcf22fbf0642cd8dde373f82498b28740ab
Sep 15 22:28:37.151084 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:37.150996    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b576979d9-2vkj4"
Sep 15 22:28:37.151224 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:37.151084    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b576979d9-2vkj4"
Sep 15 22:28:37.155844 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:37.155822    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b576979d9-2vkj4"
Sep 15 22:28:37.884548 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:37.883175    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-2bv8w"]
Sep 15 22:28:37.885776 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:37.885755    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2bv8w"
Sep 15 22:28:37.888191 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:37.888164    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 15 22:28:37.888320 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:37.888301    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 15 22:28:37.888493 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:37.888470    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 15 22:28:37.888616 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:37.888594    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 15 22:28:37.889113 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:37.889094    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 15 22:28:37.889195 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:37.889102    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-mhws8\""
Sep 15 22:28:37.904677 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:37.904650    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-2bv8w"]
Sep 15 22:28:37.913266 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:37.913236    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-wjrjh"]
Sep 15 22:28:37.917972 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:37.917955    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-4s6mz"]
Sep 15 22:28:37.920967 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:37.918200    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wjrjh"
Sep 15 22:28:37.924201 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:37.924179    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vqrqt" event={"ID":"e5ff0db4-42a1-4d9a-8ba1-1c6068724788","Type":"ContainerStarted","Data":"649bdd3268f8efcf76a1d90a9edabbcf22fbf0642cd8dde373f82498b28740ab"}
Sep 15 22:28:37.924492 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:37.924456    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b576979d9-2vkj4"
Sep 15 22:28:37.924636 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:37.924386    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-4s6mz"
Sep 15 22:28:37.927942 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:37.927924    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-99wnf\""
Sep 15 22:28:37.930391 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:37.930372    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 15 22:28:37.930703 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:37.930504    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 15 22:28:37.931865 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:37.931848    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 15 22:28:37.932201 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:37.932183    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 15 22:28:37.932680 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:37.932663    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-7nbnp\""
Sep 15 22:28:37.933146 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:37.933128    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 15 22:28:37.933709 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:37.933692    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 15 22:28:37.943158 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:37.943135    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1899c63e-fd24-4dae-8c3a-0059a53237ba-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-2bv8w\" (UID: \"1899c63e-fd24-4dae-8c3a-0059a53237ba\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2bv8w"
Sep 15 22:28:37.943375 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:37.943357    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1899c63e-fd24-4dae-8c3a-0059a53237ba-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-2bv8w\" (UID: \"1899c63e-fd24-4dae-8c3a-0059a53237ba\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2bv8w"
Sep 15 22:28:37.943575 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:37.943559    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2dm7\" (UniqueName: \"kubernetes.io/projected/1899c63e-fd24-4dae-8c3a-0059a53237ba-kube-api-access-n2dm7\") pod \"openshift-state-metrics-67dcf8c979-2bv8w\" (UID: \"1899c63e-fd24-4dae-8c3a-0059a53237ba\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2bv8w"
Sep 15 22:28:37.943745 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:37.943727    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1899c63e-fd24-4dae-8c3a-0059a53237ba-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-2bv8w\" (UID: \"1899c63e-fd24-4dae-8c3a-0059a53237ba\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2bv8w"
Sep 15 22:28:37.945338 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:37.945314    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-4s6mz"]
Sep 15 22:28:38.044633 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.044601    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee812c64-1a26-483b-b4c8-d7abc9bcbca7-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-4s6mz\" (UID: \"ee812c64-1a26-483b-b4c8-d7abc9bcbca7\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-4s6mz"
Sep 15 22:28:38.044801 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.044650    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/448978b1-d7a1-4152-81b9-37aaacf532d5-node-exporter-wtmp\") pod \"node-exporter-wjrjh\" (UID: \"448978b1-d7a1-4152-81b9-37aaacf532d5\") " pod="openshift-monitoring/node-exporter-wjrjh"
Sep 15 22:28:38.044801 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.044704    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1899c63e-fd24-4dae-8c3a-0059a53237ba-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-2bv8w\" (UID: \"1899c63e-fd24-4dae-8c3a-0059a53237ba\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2bv8w"
Sep 15 22:28:38.044801 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.044734    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/448978b1-d7a1-4152-81b9-37aaacf532d5-sys\") pod \"node-exporter-wjrjh\" (UID: \"448978b1-d7a1-4152-81b9-37aaacf532d5\") " pod="openshift-monitoring/node-exporter-wjrjh"
Sep 15 22:28:38.044801 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.044790    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1899c63e-fd24-4dae-8c3a-0059a53237ba-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-2bv8w\" (UID: \"1899c63e-fd24-4dae-8c3a-0059a53237ba\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2bv8w"
Sep 15 22:28:38.045020 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.044816    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ee812c64-1a26-483b-b4c8-d7abc9bcbca7-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-4s6mz\" (UID: \"ee812c64-1a26-483b-b4c8-d7abc9bcbca7\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-4s6mz"
Sep 15 22:28:38.045020 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.044846    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/448978b1-d7a1-4152-81b9-37aaacf532d5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wjrjh\" (UID: \"448978b1-d7a1-4152-81b9-37aaacf532d5\") " pod="openshift-monitoring/node-exporter-wjrjh"
Sep 15 22:28:38.045020 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.044899    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/448978b1-d7a1-4152-81b9-37aaacf532d5-node-exporter-tls\") pod \"node-exporter-wjrjh\" (UID: \"448978b1-d7a1-4152-81b9-37aaacf532d5\") " pod="openshift-monitoring/node-exporter-wjrjh"
Sep 15 22:28:38.045020 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.044926    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1899c63e-fd24-4dae-8c3a-0059a53237ba-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-2bv8w\" (UID: \"1899c63e-fd24-4dae-8c3a-0059a53237ba\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2bv8w"
Sep 15 22:28:38.045020 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.044956    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/448978b1-d7a1-4152-81b9-37aaacf532d5-metrics-client-ca\") pod \"node-exporter-wjrjh\" (UID: \"448978b1-d7a1-4152-81b9-37aaacf532d5\") " pod="openshift-monitoring/node-exporter-wjrjh"
Sep 15 22:28:38.045020 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.044984    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ee812c64-1a26-483b-b4c8-d7abc9bcbca7-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-4s6mz\" (UID: \"ee812c64-1a26-483b-b4c8-d7abc9bcbca7\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-4s6mz"
Sep 15 22:28:38.045020 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.045010    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmbfz\" (UniqueName: \"kubernetes.io/projected/ee812c64-1a26-483b-b4c8-d7abc9bcbca7-kube-api-access-cmbfz\") pod \"kube-state-metrics-59fbdd77d-4s6mz\" (UID: \"ee812c64-1a26-483b-b4c8-d7abc9bcbca7\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-4s6mz"
Sep 15 22:28:38.045312 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.045051    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/448978b1-d7a1-4152-81b9-37aaacf532d5-node-exporter-textfile\") pod \"node-exporter-wjrjh\" (UID: \"448978b1-d7a1-4152-81b9-37aaacf532d5\") " pod="openshift-monitoring/node-exporter-wjrjh"
Sep 15 22:28:38.045312 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.045094    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2m9p\" (UniqueName: \"kubernetes.io/projected/448978b1-d7a1-4152-81b9-37aaacf532d5-kube-api-access-r2m9p\") pod \"node-exporter-wjrjh\" (UID: \"448978b1-d7a1-4152-81b9-37aaacf532d5\") " pod="openshift-monitoring/node-exporter-wjrjh"
Sep 15 22:28:38.045312 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.045143    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee812c64-1a26-483b-b4c8-d7abc9bcbca7-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-4s6mz\" (UID: \"ee812c64-1a26-483b-b4c8-d7abc9bcbca7\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-4s6mz"
Sep 15 22:28:38.045312 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.045168    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/448978b1-d7a1-4152-81b9-37aaacf532d5-node-exporter-accelerators-collector-config\") pod \"node-exporter-wjrjh\" (UID: \"448978b1-d7a1-4152-81b9-37aaacf532d5\") " pod="openshift-monitoring/node-exporter-wjrjh"
Sep 15 22:28:38.045312 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.045194    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/448978b1-d7a1-4152-81b9-37aaacf532d5-root\") pod \"node-exporter-wjrjh\" (UID: \"448978b1-d7a1-4152-81b9-37aaacf532d5\") " pod="openshift-monitoring/node-exporter-wjrjh"
Sep 15 22:28:38.045312 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.045242    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2dm7\" (UniqueName: \"kubernetes.io/projected/1899c63e-fd24-4dae-8c3a-0059a53237ba-kube-api-access-n2dm7\") pod \"openshift-state-metrics-67dcf8c979-2bv8w\" (UID: \"1899c63e-fd24-4dae-8c3a-0059a53237ba\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2bv8w"
Sep 15 22:28:38.045312 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.045266    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee812c64-1a26-483b-b4c8-d7abc9bcbca7-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-4s6mz\" (UID: \"ee812c64-1a26-483b-b4c8-d7abc9bcbca7\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-4s6mz"
Sep 15 22:28:38.046862 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.046831    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1899c63e-fd24-4dae-8c3a-0059a53237ba-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-2bv8w\" (UID: \"1899c63e-fd24-4dae-8c3a-0059a53237ba\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2bv8w"
Sep 15 22:28:38.047223 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:38.047195    2565 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 15 22:28:38.047330 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:38.047270    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1899c63e-fd24-4dae-8c3a-0059a53237ba-openshift-state-metrics-tls podName:1899c63e-fd24-4dae-8c3a-0059a53237ba nodeName:}" failed. No retries permitted until 2026-09-15 22:28:38.54724963 +0000 UTC m=+68.406960896 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/1899c63e-fd24-4dae-8c3a-0059a53237ba-openshift-state-metrics-tls") pod "openshift-state-metrics-67dcf8c979-2bv8w" (UID: "1899c63e-fd24-4dae-8c3a-0059a53237ba") : secret "openshift-state-metrics-tls" not found
Sep 15 22:28:38.048991 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.048947    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1899c63e-fd24-4dae-8c3a-0059a53237ba-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-2bv8w\" (UID: \"1899c63e-fd24-4dae-8c3a-0059a53237ba\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2bv8w"
Sep 15 22:28:38.062824 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.062795    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2dm7\" (UniqueName: \"kubernetes.io/projected/1899c63e-fd24-4dae-8c3a-0059a53237ba-kube-api-access-n2dm7\") pod \"openshift-state-metrics-67dcf8c979-2bv8w\" (UID: \"1899c63e-fd24-4dae-8c3a-0059a53237ba\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2bv8w"
Sep 15 22:28:38.146713 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.146645    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee812c64-1a26-483b-b4c8-d7abc9bcbca7-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-4s6mz\" (UID: \"ee812c64-1a26-483b-b4c8-d7abc9bcbca7\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-4s6mz"
Sep 15 22:28:38.146713 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.146705    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/448978b1-d7a1-4152-81b9-37aaacf532d5-node-exporter-wtmp\") pod \"node-exporter-wjrjh\" (UID: \"448978b1-d7a1-4152-81b9-37aaacf532d5\") " pod="openshift-monitoring/node-exporter-wjrjh"
Sep 15 22:28:38.146881 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.146744    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/448978b1-d7a1-4152-81b9-37aaacf532d5-sys\") pod \"node-exporter-wjrjh\" (UID: \"448978b1-d7a1-4152-81b9-37aaacf532d5\") " pod="openshift-monitoring/node-exporter-wjrjh"
Sep 15 22:28:38.146881 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.146777    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ee812c64-1a26-483b-b4c8-d7abc9bcbca7-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-4s6mz\" (UID: \"ee812c64-1a26-483b-b4c8-d7abc9bcbca7\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-4s6mz"
Sep 15 22:28:38.146881 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.146806    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/448978b1-d7a1-4152-81b9-37aaacf532d5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wjrjh\" (UID: \"448978b1-d7a1-4152-81b9-37aaacf532d5\") " pod="openshift-monitoring/node-exporter-wjrjh"
Sep 15 22:28:38.146881 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.146841    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/448978b1-d7a1-4152-81b9-37aaacf532d5-node-exporter-tls\") pod \"node-exporter-wjrjh\" (UID: \"448978b1-d7a1-4152-81b9-37aaacf532d5\") " pod="openshift-monitoring/node-exporter-wjrjh"
Sep 15 22:28:38.147107 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.146896    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/448978b1-d7a1-4152-81b9-37aaacf532d5-metrics-client-ca\") pod \"node-exporter-wjrjh\" (UID: \"448978b1-d7a1-4152-81b9-37aaacf532d5\") " pod="openshift-monitoring/node-exporter-wjrjh"
Sep 15 22:28:38.147107 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.146923    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ee812c64-1a26-483b-b4c8-d7abc9bcbca7-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-4s6mz\" (UID: \"ee812c64-1a26-483b-b4c8-d7abc9bcbca7\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-4s6mz"
Sep 15 22:28:38.147107 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.146947    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmbfz\" (UniqueName: \"kubernetes.io/projected/ee812c64-1a26-483b-b4c8-d7abc9bcbca7-kube-api-access-cmbfz\") pod \"kube-state-metrics-59fbdd77d-4s6mz\" (UID: \"ee812c64-1a26-483b-b4c8-d7abc9bcbca7\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-4s6mz"
Sep 15 22:28:38.147107 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.146972    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/448978b1-d7a1-4152-81b9-37aaacf532d5-node-exporter-textfile\") pod \"node-exporter-wjrjh\" (UID: \"448978b1-d7a1-4152-81b9-37aaacf532d5\") " pod="openshift-monitoring/node-exporter-wjrjh"
Sep 15 22:28:38.147107 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.147000    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2m9p\" (UniqueName: \"kubernetes.io/projected/448978b1-d7a1-4152-81b9-37aaacf532d5-kube-api-access-r2m9p\") pod \"node-exporter-wjrjh\" (UID: \"448978b1-d7a1-4152-81b9-37aaacf532d5\") " pod="openshift-monitoring/node-exporter-wjrjh"
Sep 15 22:28:38.147107 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.147058    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee812c64-1a26-483b-b4c8-d7abc9bcbca7-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-4s6mz\" (UID: \"ee812c64-1a26-483b-b4c8-d7abc9bcbca7\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-4s6mz"
Sep 15 22:28:38.147107 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.147091    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/448978b1-d7a1-4152-81b9-37aaacf532d5-node-exporter-accelerators-collector-config\") pod \"node-exporter-wjrjh\" (UID: \"448978b1-d7a1-4152-81b9-37aaacf532d5\") " pod="openshift-monitoring/node-exporter-wjrjh"
Sep 15 22:28:38.147457 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.147118    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/448978b1-d7a1-4152-81b9-37aaacf532d5-root\") pod \"node-exporter-wjrjh\" (UID: \"448978b1-d7a1-4152-81b9-37aaacf532d5\") " pod="openshift-monitoring/node-exporter-wjrjh"
Sep 15 22:28:38.147457 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.147151    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee812c64-1a26-483b-b4c8-d7abc9bcbca7-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-4s6mz\" (UID: \"ee812c64-1a26-483b-b4c8-d7abc9bcbca7\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-4s6mz"
Sep 15 22:28:38.148597 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.147741    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/448978b1-d7a1-4152-81b9-37aaacf532d5-node-exporter-wtmp\") pod \"node-exporter-wjrjh\" (UID: \"448978b1-d7a1-4152-81b9-37aaacf532d5\") " pod="openshift-monitoring/node-exporter-wjrjh"
Sep 15 22:28:38.148597 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.148270    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/448978b1-d7a1-4152-81b9-37aaacf532d5-metrics-client-ca\") pod \"node-exporter-wjrjh\" (UID: \"448978b1-d7a1-4152-81b9-37aaacf532d5\") " pod="openshift-monitoring/node-exporter-wjrjh"
Sep 15 22:28:38.148597 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.148349    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/448978b1-d7a1-4152-81b9-37aaacf532d5-root\") pod \"node-exporter-wjrjh\" (UID: \"448978b1-d7a1-4152-81b9-37aaacf532d5\") " pod="openshift-monitoring/node-exporter-wjrjh"
Sep 15 22:28:38.148597 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.148362    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ee812c64-1a26-483b-b4c8-d7abc9bcbca7-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-4s6mz\" (UID: \"ee812c64-1a26-483b-b4c8-d7abc9bcbca7\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-4s6mz"
Sep 15 22:28:38.148597 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.148416    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/448978b1-d7a1-4152-81b9-37aaacf532d5-node-exporter-accelerators-collector-config\") pod \"node-exporter-wjrjh\" (UID: \"448978b1-d7a1-4152-81b9-37aaacf532d5\") " pod="openshift-monitoring/node-exporter-wjrjh"
Sep 15 22:28:38.148902 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.148610    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/448978b1-d7a1-4152-81b9-37aaacf532d5-node-exporter-textfile\") pod \"node-exporter-wjrjh\" (UID: \"448978b1-d7a1-4152-81b9-37aaacf532d5\") " pod="openshift-monitoring/node-exporter-wjrjh"
Sep 15 22:28:38.148902 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.148733    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/448978b1-d7a1-4152-81b9-37aaacf532d5-sys\") pod \"node-exporter-wjrjh\" (UID: \"448978b1-d7a1-4152-81b9-37aaacf532d5\") " pod="openshift-monitoring/node-exporter-wjrjh"
Sep 15 22:28:38.149618 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.149050    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee812c64-1a26-483b-b4c8-d7abc9bcbca7-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-4s6mz\" (UID: \"ee812c64-1a26-483b-b4c8-d7abc9bcbca7\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-4s6mz"
Sep 15 22:28:38.149618 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.149346    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ee812c64-1a26-483b-b4c8-d7abc9bcbca7-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-4s6mz\" (UID: \"ee812c64-1a26-483b-b4c8-d7abc9bcbca7\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-4s6mz"
Sep 15 22:28:38.153601 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.153571    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee812c64-1a26-483b-b4c8-d7abc9bcbca7-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-4s6mz\" (UID: \"ee812c64-1a26-483b-b4c8-d7abc9bcbca7\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-4s6mz"
Sep 15 22:28:38.155735 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.154695    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee812c64-1a26-483b-b4c8-d7abc9bcbca7-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-4s6mz\" (UID: \"ee812c64-1a26-483b-b4c8-d7abc9bcbca7\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-4s6mz"
Sep 15 22:28:38.155735 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.155043    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/448978b1-d7a1-4152-81b9-37aaacf532d5-node-exporter-tls\") pod \"node-exporter-wjrjh\" (UID: \"448978b1-d7a1-4152-81b9-37aaacf532d5\") " pod="openshift-monitoring/node-exporter-wjrjh"
Sep 15 22:28:38.155735 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.155330    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/448978b1-d7a1-4152-81b9-37aaacf532d5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wjrjh\" (UID: \"448978b1-d7a1-4152-81b9-37aaacf532d5\") " pod="openshift-monitoring/node-exporter-wjrjh"
Sep 15 22:28:38.157851 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.157823    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2m9p\" (UniqueName: \"kubernetes.io/projected/448978b1-d7a1-4152-81b9-37aaacf532d5-kube-api-access-r2m9p\") pod \"node-exporter-wjrjh\" (UID: \"448978b1-d7a1-4152-81b9-37aaacf532d5\") " pod="openshift-monitoring/node-exporter-wjrjh"
Sep 15 22:28:38.161217 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.161175    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmbfz\" (UniqueName: \"kubernetes.io/projected/ee812c64-1a26-483b-b4c8-d7abc9bcbca7-kube-api-access-cmbfz\") pod \"kube-state-metrics-59fbdd77d-4s6mz\" (UID: \"ee812c64-1a26-483b-b4c8-d7abc9bcbca7\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-4s6mz"
Sep 15 22:28:38.238657 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.238528    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wjrjh"
Sep 15 22:28:38.245237 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.245074    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-4s6mz"
Sep 15 22:28:38.252566 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:28:38.252535    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod448978b1_d7a1_4152_81b9_37aaacf532d5.slice/crio-c3d549aa77065e23f1744007a2648d7d99b572a5c9eb067103e05a0ac8e3154b WatchSource:0}: Error finding container c3d549aa77065e23f1744007a2648d7d99b572a5c9eb067103e05a0ac8e3154b: Status 404 returned error can't find the container with id c3d549aa77065e23f1744007a2648d7d99b572a5c9eb067103e05a0ac8e3154b
Sep 15 22:28:38.541128 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.541102    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-4s6mz"]
Sep 15 22:28:38.544890 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:28:38.544859    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee812c64_1a26_483b_b4c8_d7abc9bcbca7.slice/crio-3704c2da9a8335cc8d893838179352e471091d5b960519bb3256903170528cee WatchSource:0}: Error finding container 3704c2da9a8335cc8d893838179352e471091d5b960519bb3256903170528cee: Status 404 returned error can't find the container with id 3704c2da9a8335cc8d893838179352e471091d5b960519bb3256903170528cee
Sep 15 22:28:38.550141 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.550016    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1899c63e-fd24-4dae-8c3a-0059a53237ba-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-2bv8w\" (UID: \"1899c63e-fd24-4dae-8c3a-0059a53237ba\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2bv8w"
Sep 15 22:28:38.552839 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.552791    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1899c63e-fd24-4dae-8c3a-0059a53237ba-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-2bv8w\" (UID: \"1899c63e-fd24-4dae-8c3a-0059a53237ba\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2bv8w"
Sep 15 22:28:38.810401 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.810311    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2bv8w"
Sep 15 22:28:38.925490 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.925392    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-j4tsb" event={"ID":"909a094b-77e5-4f6e-9d69-6f4214156810","Type":"ContainerStarted","Data":"928b0736f166670d3df65552f87543159872f1db12d2b63748ef75715d94d236"}
Sep 15 22:28:38.925490 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.925450    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-j4tsb" event={"ID":"909a094b-77e5-4f6e-9d69-6f4214156810","Type":"ContainerStarted","Data":"8165b53a60bb7b676492410f2acc4a5d49020802ec379ad68698bed1f18ff2bf"}
Sep 15 22:28:38.927111 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.927005    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-4s6mz" event={"ID":"ee812c64-1a26-483b-b4c8-d7abc9bcbca7","Type":"ContainerStarted","Data":"3704c2da9a8335cc8d893838179352e471091d5b960519bb3256903170528cee"}
Sep 15 22:28:38.929238 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.929133    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wjrjh" event={"ID":"448978b1-d7a1-4152-81b9-37aaacf532d5","Type":"ContainerStarted","Data":"c3d549aa77065e23f1744007a2648d7d99b572a5c9eb067103e05a0ac8e3154b"}
Sep 15 22:28:38.944075 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.942808    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-j4tsb" podStartSLOduration=67.708920034 podStartE2EDuration="1m8.942791949s" totalImagesPullingTime="1.233871915s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:27:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:28:38.942099707 +0000 UTC m=+68.801810992" watchObservedRunningTime="2026-09-15 22:28:38.942791949 +0000 UTC m=+68.802503236"
Sep 15 22:28:38.979936 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.979019    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 15 22:28:38.982045 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.982001    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:38.985740 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.985711    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 15 22:28:38.985854 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.985710    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 15 22:28:38.985854 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.985794    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-7h9vd\""
Sep 15 22:28:38.985854 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.985712    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 15 22:28:38.986896 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.986490    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 15 22:28:38.986896 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.986626    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 15 22:28:38.986896 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.986741    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 15 22:28:38.986896 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.986804    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 15 22:28:38.986896 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.986858    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 15 22:28:38.987371 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:38.987353    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 15 22:28:39.003797 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.003719    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 15 22:28:39.056987 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.056253    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.056987 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.056348    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-config-out\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.056987 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.056377    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.056987 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.056414    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.056987 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.056442    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.056987 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.056470    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p79zv\" (UniqueName: \"kubernetes.io/projected/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-kube-api-access-p79zv\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.056987 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.056495    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-config-volume\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.056987 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.056558    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.056987 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.056597    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-web-config\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.056987 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.056641    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.056987 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.056667    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.056987 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.056791    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.056987 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.056817    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.103362 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.102999    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-2bv8w"]
Sep 15 22:28:39.108081 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:28:39.108049    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1899c63e_fd24_4dae_8c3a_0059a53237ba.slice/crio-f17c62e0b394ea3b094f19978070d4e64874e8e7f18dcddfeb46028e2a700b2f WatchSource:0}: Error finding container f17c62e0b394ea3b094f19978070d4e64874e8e7f18dcddfeb46028e2a700b2f: Status 404 returned error can't find the container with id f17c62e0b394ea3b094f19978070d4e64874e8e7f18dcddfeb46028e2a700b2f
Sep 15 22:28:39.158499 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.158002    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-web-config\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.161064 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.158678    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.161064 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.158905    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.161064 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.158959    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.161064 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.158984    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.161064 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.159041    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.161064 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.159094    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-config-out\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.161064 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.159118    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.161064 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.159142    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.161064 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.159171    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.161064 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.159201    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p79zv\" (UniqueName: \"kubernetes.io/projected/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-kube-api-access-p79zv\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.161064 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.159234    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-config-volume\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.161064 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.159274    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.161064 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.159623    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.161064 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:28:39.159741    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-alertmanager-trusted-ca-bundle podName:9fbcc674-9072-4d42-bbe1-3fc4289d9cbc nodeName:}" failed. No retries permitted until 2026-09-15 22:28:39.659721384 +0000 UTC m=+69.519432666 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "9fbcc674-9072-4d42-bbe1-3fc4289d9cbc") : configmap references non-existent config key: ca-bundle.crt
Sep 15 22:28:39.161766 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.161162    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.165399 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.165348    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-config-out\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.165619 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.165594    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.165755 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.165728    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.165927 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.165905    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.171422 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.171372    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.171592 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.171569    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-web-config\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.174491 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.174462    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-config-volume\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.175213 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.175178    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.176078 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.176055    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.180860 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.180834    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p79zv\" (UniqueName: \"kubernetes.io/projected/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-kube-api-access-p79zv\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.664881 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.664839    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.665778 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.665747    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.898123 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.898085    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:28:39.935037 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.934936    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2bv8w" event={"ID":"1899c63e-fd24-4dae-8c3a-0059a53237ba","Type":"ContainerStarted","Data":"7ff9d3035b13c7395527553fb7ce0b6d06afb482619a0b67377e6b27bb4f2999"}
Sep 15 22:28:39.935469 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.935062    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2bv8w" event={"ID":"1899c63e-fd24-4dae-8c3a-0059a53237ba","Type":"ContainerStarted","Data":"6c79d02c882a7e1615d3888524d8ff90677586a7729a02fc2fb07ac6839f75ab"}
Sep 15 22:28:39.935469 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:39.935085    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2bv8w" event={"ID":"1899c63e-fd24-4dae-8c3a-0059a53237ba","Type":"ContainerStarted","Data":"f17c62e0b394ea3b094f19978070d4e64874e8e7f18dcddfeb46028e2a700b2f"}
Sep 15 22:28:40.118516 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:40.118479    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b576979d9-2vkj4"]
Sep 15 22:28:42.810198 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:42.810163    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 15 22:28:42.816333 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:28:42.816294    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fbcc674_9072_4d42_bbe1_3fc4289d9cbc.slice/crio-68861e74e3c06686b0e43bbc8cf74d17b91bc4661f7c52898cb11879039f9ba2 WatchSource:0}: Error finding container 68861e74e3c06686b0e43bbc8cf74d17b91bc4661f7c52898cb11879039f9ba2: Status 404 returned error can't find the container with id 68861e74e3c06686b0e43bbc8cf74d17b91bc4661f7c52898cb11879039f9ba2
Sep 15 22:28:42.949343 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:42.949305    2565 generic.go:356] "Generic (PLEG): container finished" podID="448978b1-d7a1-4152-81b9-37aaacf532d5" containerID="47b362e41792574668057173080e5c4472967adc2de547174022f293a5978a12" exitCode=0
Sep 15 22:28:42.949508 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:42.949389    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wjrjh" event={"ID":"448978b1-d7a1-4152-81b9-37aaacf532d5","Type":"ContainerDied","Data":"47b362e41792574668057173080e5c4472967adc2de547174022f293a5978a12"}
Sep 15 22:28:42.952160 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:42.952102    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2bv8w" event={"ID":"1899c63e-fd24-4dae-8c3a-0059a53237ba","Type":"ContainerStarted","Data":"77b41d5687bf762caf34c0b7544796018f4310839fdcc63fed16cca6bdfa0b9c"}
Sep 15 22:28:42.953682 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:42.953656    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc","Type":"ContainerStarted","Data":"68861e74e3c06686b0e43bbc8cf74d17b91bc4661f7c52898cb11879039f9ba2"}
Sep 15 22:28:42.955413 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:42.955388    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4d5lp" event={"ID":"b22214d5-9c42-4710-bfff-08651d7942a3","Type":"ContainerStarted","Data":"69e6c935f59debbbc8e3c7fd67f99c34afed7aff056a9b803e599b06a81fc112"}
Sep 15 22:28:42.957143 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:42.957113    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vqrqt" event={"ID":"e5ff0db4-42a1-4d9a-8ba1-1c6068724788","Type":"ContainerStarted","Data":"9fc84c6406be64ffe96fabd1418f807a2d85bf9fdce935ae1186679788f31c90"}
Sep 15 22:28:42.958524 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:42.958498    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:28:42.959690 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:42.959663    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-4s6mz" event={"ID":"ee812c64-1a26-483b-b4c8-d7abc9bcbca7","Type":"ContainerStarted","Data":"79574be984d45ed48700764bea8fb45eecbf189a7000cd9babf240b55dc6dddb"}
Sep 15 22:28:42.959791 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:42.959715    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-4s6mz" event={"ID":"ee812c64-1a26-483b-b4c8-d7abc9bcbca7","Type":"ContainerStarted","Data":"f4aff938a5d8b8de2b0eded5dc08b7347f2553257f5b0418a1c8bc641fa36b37"}
Sep 15 22:28:42.959791 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:42.959744    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-4s6mz" event={"ID":"ee812c64-1a26-483b-b4c8-d7abc9bcbca7","Type":"ContainerStarted","Data":"292d6381fdafca4d6cbc4a4a5a12667f68f4633d39724a7720c2106508450622"}
Sep 15 22:28:42.981573 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:42.980967    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4d5lp" podStartSLOduration=67.335562893 podStartE2EDuration="1m12.980948025s" totalImagesPullingTime="5.645385132s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 22:27:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:28:42.979826123 +0000 UTC m=+72.839537407" watchObservedRunningTime="2026-09-15 22:28:42.980948025 +0000 UTC m=+72.840659310"
Sep 15 22:28:42.999722 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:42.999680    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-4s6mz" podStartSLOduration=2.021282581 podStartE2EDuration="5.999663895s" totalImagesPullingTime="3.978381314s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:28:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:28:42.99871397 +0000 UTC m=+72.858425256" watchObservedRunningTime="2026-09-15 22:28:42.999663895 +0000 UTC m=+72.859375178"
Sep 15 22:28:43.017206 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:43.017155    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-2bv8w" podStartSLOduration=2.705562359 podStartE2EDuration="6.017133617s" totalImagesPullingTime="3.311571258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:28:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:28:43.016693199 +0000 UTC m=+72.876404490" watchObservedRunningTime="2026-09-15 22:28:43.017133617 +0000 UTC m=+72.876844902"
Sep 15 22:28:43.038365 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:43.038323    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vqrqt" podStartSLOduration=67.426251341 podStartE2EDuration="1m13.038306124s" totalImagesPullingTime="5.612054783s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:27:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:28:43.036531033 +0000 UTC m=+72.896242317" watchObservedRunningTime="2026-09-15 22:28:43.038306124 +0000 UTC m=+72.898017408"
Sep 15 22:28:43.969085 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:43.969039    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wjrjh" event={"ID":"448978b1-d7a1-4152-81b9-37aaacf532d5","Type":"ContainerStarted","Data":"af22a0bcc7d14a0ea3cd579dc3399c1906d0d1a0e1bc579337865b5bff5e7d4b"}
Sep 15 22:28:43.969499 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:43.969117    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wjrjh" event={"ID":"448978b1-d7a1-4152-81b9-37aaacf532d5","Type":"ContainerStarted","Data":"d0989f34f5cc5869e104bcca61548b663d4a07372dc4929de73c2dbffa26a081"}
Sep 15 22:28:43.991123 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:43.991062    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-wjrjh" podStartSLOduration=2.637069681 podStartE2EDuration="6.991040065s" totalImagesPullingTime="4.269253153s" totalInitContainerRuntime="84.717231ms" isStatefulPod=true podCreationTimestamp="2026-09-15 22:28:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:28:43.989664893 +0000 UTC m=+73.849376177" watchObservedRunningTime="2026-09-15 22:28:43.991040065 +0000 UTC m=+73.850751344"
Sep 15 22:28:44.973924 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:44.973889    2565 generic.go:356] "Generic (PLEG): container finished" podID="9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" containerID="ddfa1b2b0d2ae63aa448b50a68c52f9ffa502533857aeb259d11ee2f90d9ff85" exitCode=0
Sep 15 22:28:44.974340 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:44.973980    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc","Type":"ContainerDied","Data":"ddfa1b2b0d2ae63aa448b50a68c52f9ffa502533857aeb259d11ee2f90d9ff85"}
Sep 15 22:28:46.984564 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:46.984490    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc","Type":"ContainerStarted","Data":"d40e6b5a0a6ce0ea253725ba7dab4e540542a02b930774178d7b11559261847e"}
Sep 15 22:28:46.984564 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:46.984552    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc","Type":"ContainerStarted","Data":"0e8f237ba6648c89f5d8f663d701b302eff7500b6c2adb38f37995e335aa0fca"}
Sep 15 22:28:46.984919 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:46.984578    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc","Type":"ContainerStarted","Data":"800069f83fc7bd344ff60ed541b996e2f12d640aec2ce23972f59745a452282f"}
Sep 15 22:28:46.984919 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:46.984603    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc","Type":"ContainerStarted","Data":"d65f66c31a26688b9d5e90698a36b89c7fa461decabf47a1004d3be8ea61298b"}
Sep 15 22:28:46.984919 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:46.984624    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc","Type":"ContainerStarted","Data":"6de639c5cc5c9efb7be92b8a53940a9d0f1cef31468453a7dce11ccd0d085f60"}
Sep 15 22:28:47.861035 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:47.861005    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5cddfccf5c-5zq8q"
Sep 15 22:28:47.991595 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:47.991565    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc","Type":"ContainerStarted","Data":"3a7a467bbfacaa7990ae23a441089ec5402f170d0b98a28cbb3118377ad505e3"}
Sep 15 22:28:48.027529 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:28:48.027386    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=6.352397629 podStartE2EDuration="10.027365124s" totalImagesPullingTime="3.660226714s" totalInitContainerRuntime="14.740781ms" isStatefulPod=false podCreationTimestamp="2026-09-15 22:28:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:28:48.025934792 +0000 UTC m=+77.885646075" watchObservedRunningTime="2026-09-15 22:28:48.027365124 +0000 UTC m=+77.887076409"
Sep 15 22:29:05.139388 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:05.139320    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6b576979d9-2vkj4" podUID="07af1527-72f1-49e8-ac8e-54ab1c1bccdf" containerName="console" containerID="cri-o://5380372c9aa4b838395025205dfa7652af680d8e229cee15399a84e9eb011bd5" gracePeriod=15
Sep 15 22:29:05.375136 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:05.375112    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b576979d9-2vkj4"
Sep 15 22:29:05.460084 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:05.459984    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07af1527-72f1-49e8-ac8e-54ab1c1bccdf-service-ca\") pod \"07af1527-72f1-49e8-ac8e-54ab1c1bccdf\" (UID: \"07af1527-72f1-49e8-ac8e-54ab1c1bccdf\") "
Sep 15 22:29:05.460084 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:05.460061    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07af1527-72f1-49e8-ac8e-54ab1c1bccdf-console-oauth-config\") pod \"07af1527-72f1-49e8-ac8e-54ab1c1bccdf\" (UID: \"07af1527-72f1-49e8-ac8e-54ab1c1bccdf\") "
Sep 15 22:29:05.460293 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:05.460107    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07af1527-72f1-49e8-ac8e-54ab1c1bccdf-oauth-serving-cert\") pod \"07af1527-72f1-49e8-ac8e-54ab1c1bccdf\" (UID: \"07af1527-72f1-49e8-ac8e-54ab1c1bccdf\") "
Sep 15 22:29:05.460293 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:05.460140    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07af1527-72f1-49e8-ac8e-54ab1c1bccdf-console-serving-cert\") pod \"07af1527-72f1-49e8-ac8e-54ab1c1bccdf\" (UID: \"07af1527-72f1-49e8-ac8e-54ab1c1bccdf\") "
Sep 15 22:29:05.460293 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:05.460178    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07af1527-72f1-49e8-ac8e-54ab1c1bccdf-console-config\") pod \"07af1527-72f1-49e8-ac8e-54ab1c1bccdf\" (UID: \"07af1527-72f1-49e8-ac8e-54ab1c1bccdf\") "
Sep 15 22:29:05.460293 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:05.460209    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-78mxd\" (UniqueName: \"kubernetes.io/projected/07af1527-72f1-49e8-ac8e-54ab1c1bccdf-kube-api-access-78mxd\") pod \"07af1527-72f1-49e8-ac8e-54ab1c1bccdf\" (UID: \"07af1527-72f1-49e8-ac8e-54ab1c1bccdf\") "
Sep 15 22:29:05.460513 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:05.460478    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07af1527-72f1-49e8-ac8e-54ab1c1bccdf-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "07af1527-72f1-49e8-ac8e-54ab1c1bccdf" (UID: "07af1527-72f1-49e8-ac8e-54ab1c1bccdf"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:29:05.460513 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:05.460499    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07af1527-72f1-49e8-ac8e-54ab1c1bccdf-service-ca" (OuterVolumeSpecName: "service-ca") pod "07af1527-72f1-49e8-ac8e-54ab1c1bccdf" (UID: "07af1527-72f1-49e8-ac8e-54ab1c1bccdf"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:29:05.460679 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:05.460571    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07af1527-72f1-49e8-ac8e-54ab1c1bccdf-console-config" (OuterVolumeSpecName: "console-config") pod "07af1527-72f1-49e8-ac8e-54ab1c1bccdf" (UID: "07af1527-72f1-49e8-ac8e-54ab1c1bccdf"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:29:05.462425 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:05.462393    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07af1527-72f1-49e8-ac8e-54ab1c1bccdf-kube-api-access-78mxd" (OuterVolumeSpecName: "kube-api-access-78mxd") pod "07af1527-72f1-49e8-ac8e-54ab1c1bccdf" (UID: "07af1527-72f1-49e8-ac8e-54ab1c1bccdf"). InnerVolumeSpecName "kube-api-access-78mxd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:29:05.462529 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:05.462419    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07af1527-72f1-49e8-ac8e-54ab1c1bccdf-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "07af1527-72f1-49e8-ac8e-54ab1c1bccdf" (UID: "07af1527-72f1-49e8-ac8e-54ab1c1bccdf"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:29:05.462529 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:05.462496    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07af1527-72f1-49e8-ac8e-54ab1c1bccdf-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "07af1527-72f1-49e8-ac8e-54ab1c1bccdf" (UID: "07af1527-72f1-49e8-ac8e-54ab1c1bccdf"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:29:05.561131 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:05.561085    2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07af1527-72f1-49e8-ac8e-54ab1c1bccdf-console-config\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:29:05.561131 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:05.561126    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-78mxd\" (UniqueName: \"kubernetes.io/projected/07af1527-72f1-49e8-ac8e-54ab1c1bccdf-kube-api-access-78mxd\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:29:05.561131 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:05.561138    2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07af1527-72f1-49e8-ac8e-54ab1c1bccdf-service-ca\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:29:05.561131 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:05.561148    2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07af1527-72f1-49e8-ac8e-54ab1c1bccdf-console-oauth-config\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:29:05.561370 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:05.561157    2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07af1527-72f1-49e8-ac8e-54ab1c1bccdf-oauth-serving-cert\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:29:05.561370 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:05.561165    2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07af1527-72f1-49e8-ac8e-54ab1c1bccdf-console-serving-cert\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:29:06.055833 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:06.055796    2565 generic.go:356] "Generic (PLEG): container finished" podID="07af1527-72f1-49e8-ac8e-54ab1c1bccdf" containerID="5380372c9aa4b838395025205dfa7652af680d8e229cee15399a84e9eb011bd5" exitCode=0
Sep 15 22:29:06.056004 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:06.055841    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b576979d9-2vkj4"
Sep 15 22:29:06.056004 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:06.055881    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b576979d9-2vkj4" event={"ID":"07af1527-72f1-49e8-ac8e-54ab1c1bccdf","Type":"ContainerDied","Data":"5380372c9aa4b838395025205dfa7652af680d8e229cee15399a84e9eb011bd5"}
Sep 15 22:29:06.056004 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:06.055934    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b576979d9-2vkj4" event={"ID":"07af1527-72f1-49e8-ac8e-54ab1c1bccdf","Type":"ContainerDied","Data":"22e88e25b6d035d7e60ab16e57e275f3230a0aad3ae8e314858814b25b19f2f8"}
Sep 15 22:29:06.056004 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:06.055967    2565 scope.go:117] "RemoveContainer" containerID="5380372c9aa4b838395025205dfa7652af680d8e229cee15399a84e9eb011bd5"
Sep 15 22:29:06.064096 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:06.064077    2565 scope.go:117] "RemoveContainer" containerID="5380372c9aa4b838395025205dfa7652af680d8e229cee15399a84e9eb011bd5"
Sep 15 22:29:06.064361 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:29:06.064341    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5380372c9aa4b838395025205dfa7652af680d8e229cee15399a84e9eb011bd5\": container with ID starting with 5380372c9aa4b838395025205dfa7652af680d8e229cee15399a84e9eb011bd5 not found: ID does not exist" containerID="5380372c9aa4b838395025205dfa7652af680d8e229cee15399a84e9eb011bd5"
Sep 15 22:29:06.064409 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:06.064372    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5380372c9aa4b838395025205dfa7652af680d8e229cee15399a84e9eb011bd5"} err="failed to get container status \"5380372c9aa4b838395025205dfa7652af680d8e229cee15399a84e9eb011bd5\": rpc error: code = NotFound desc = could not find container \"5380372c9aa4b838395025205dfa7652af680d8e229cee15399a84e9eb011bd5\": container with ID starting with 5380372c9aa4b838395025205dfa7652af680d8e229cee15399a84e9eb011bd5 not found: ID does not exist"
Sep 15 22:29:06.075115 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:06.075091    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b576979d9-2vkj4"]
Sep 15 22:29:06.079045 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:06.079008    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b576979d9-2vkj4"]
Sep 15 22:29:06.589114 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:06.589084    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07af1527-72f1-49e8-ac8e-54ab1c1bccdf" path="/var/lib/kubelet/pods/07af1527-72f1-49e8-ac8e-54ab1c1bccdf/volumes"
Sep 15 22:29:13.972746 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:13.972709    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vqrqt"
Sep 15 22:29:58.212018 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:58.211980    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 15 22:29:58.212626 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:58.212307    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" containerName="alertmanager" containerID="cri-o://6de639c5cc5c9efb7be92b8a53940a9d0f1cef31468453a7dce11ccd0d085f60" gracePeriod=120
Sep 15 22:29:58.212626 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:58.212364    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" containerName="kube-rbac-proxy-web" containerID="cri-o://800069f83fc7bd344ff60ed541b996e2f12d640aec2ce23972f59745a452282f" gracePeriod=120
Sep 15 22:29:58.212626 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:58.212399    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" containerName="prom-label-proxy" containerID="cri-o://3a7a467bbfacaa7990ae23a441089ec5402f170d0b98a28cbb3118377ad505e3" gracePeriod=120
Sep 15 22:29:58.212626 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:58.212396    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" containerName="config-reloader" containerID="cri-o://d65f66c31a26688b9d5e90698a36b89c7fa461decabf47a1004d3be8ea61298b" gracePeriod=120
Sep 15 22:29:58.212626 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:58.212352    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" containerName="kube-rbac-proxy-metric" containerID="cri-o://d40e6b5a0a6ce0ea253725ba7dab4e540542a02b930774178d7b11559261847e" gracePeriod=120
Sep 15 22:29:58.212626 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:58.212387    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" containerName="kube-rbac-proxy" containerID="cri-o://0e8f237ba6648c89f5d8f663d701b302eff7500b6c2adb38f37995e335aa0fca" gracePeriod=120
Sep 15 22:29:59.240563 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.240531    2565 generic.go:356] "Generic (PLEG): container finished" podID="9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" containerID="3a7a467bbfacaa7990ae23a441089ec5402f170d0b98a28cbb3118377ad505e3" exitCode=0
Sep 15 22:29:59.240563 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.240554    2565 generic.go:356] "Generic (PLEG): container finished" podID="9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" containerID="d40e6b5a0a6ce0ea253725ba7dab4e540542a02b930774178d7b11559261847e" exitCode=0
Sep 15 22:29:59.240563 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.240563    2565 generic.go:356] "Generic (PLEG): container finished" podID="9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" containerID="0e8f237ba6648c89f5d8f663d701b302eff7500b6c2adb38f37995e335aa0fca" exitCode=0
Sep 15 22:29:59.240563 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.240569    2565 generic.go:356] "Generic (PLEG): container finished" podID="9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" containerID="d65f66c31a26688b9d5e90698a36b89c7fa461decabf47a1004d3be8ea61298b" exitCode=0
Sep 15 22:29:59.241009 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.240575    2565 generic.go:356] "Generic (PLEG): container finished" podID="9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" containerID="6de639c5cc5c9efb7be92b8a53940a9d0f1cef31468453a7dce11ccd0d085f60" exitCode=0
Sep 15 22:29:59.241009 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.240604    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc","Type":"ContainerDied","Data":"3a7a467bbfacaa7990ae23a441089ec5402f170d0b98a28cbb3118377ad505e3"}
Sep 15 22:29:59.241009 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.240666    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc","Type":"ContainerDied","Data":"d40e6b5a0a6ce0ea253725ba7dab4e540542a02b930774178d7b11559261847e"}
Sep 15 22:29:59.241009 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.240692    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc","Type":"ContainerDied","Data":"0e8f237ba6648c89f5d8f663d701b302eff7500b6c2adb38f37995e335aa0fca"}
Sep 15 22:29:59.241009 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.240717    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc","Type":"ContainerDied","Data":"d65f66c31a26688b9d5e90698a36b89c7fa461decabf47a1004d3be8ea61298b"}
Sep 15 22:29:59.241009 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.240740    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc","Type":"ContainerDied","Data":"6de639c5cc5c9efb7be92b8a53940a9d0f1cef31468453a7dce11ccd0d085f60"}
Sep 15 22:29:59.451148 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.451126    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:29:59.537231 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.537206    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-config-volume\") pod \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") "
Sep 15 22:29:59.537392 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.537249    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-alertmanager-trusted-ca-bundle\") pod \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") "
Sep 15 22:29:59.537392 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.537290    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-alertmanager-main-db\") pod \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") "
Sep 15 22:29:59.537392 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.537315    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-cluster-tls-config\") pod \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") "
Sep 15 22:29:59.537392 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.537339    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-web-config\") pod \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") "
Sep 15 22:29:59.537392 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.537367    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-secret-alertmanager-main-tls\") pod \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") "
Sep 15 22:29:59.537651 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.537396    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p79zv\" (UniqueName: \"kubernetes.io/projected/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-kube-api-access-p79zv\") pod \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") "
Sep 15 22:29:59.537651 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.537427    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") "
Sep 15 22:29:59.537651 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.537465    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-secret-alertmanager-kube-rbac-proxy-web\") pod \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") "
Sep 15 22:29:59.537651 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.537489    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-tls-assets\") pod \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") "
Sep 15 22:29:59.537651 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.537520    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-config-out\") pod \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") "
Sep 15 22:29:59.537651 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.537551    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-secret-alertmanager-kube-rbac-proxy\") pod \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") "
Sep 15 22:29:59.537651 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.537586    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-metrics-client-ca\") pod \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\" (UID: \"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc\") "
Sep 15 22:29:59.538050 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.537705    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" (UID: "9fbcc674-9072-4d42-bbe1-3fc4289d9cbc"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:29:59.538050 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.537820    2565 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:29:59.538050 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.537909    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" (UID: "9fbcc674-9072-4d42-bbe1-3fc4289d9cbc"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:29:59.538211 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.538107    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" (UID: "9fbcc674-9072-4d42-bbe1-3fc4289d9cbc"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:29:59.540707 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.540655    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-config-volume" (OuterVolumeSpecName: "config-volume") pod "9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" (UID: "9fbcc674-9072-4d42-bbe1-3fc4289d9cbc"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:29:59.540825 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.540770    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-kube-api-access-p79zv" (OuterVolumeSpecName: "kube-api-access-p79zv") pod "9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" (UID: "9fbcc674-9072-4d42-bbe1-3fc4289d9cbc"). InnerVolumeSpecName "kube-api-access-p79zv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:29:59.540888 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.540870    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-config-out" (OuterVolumeSpecName: "config-out") pod "9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" (UID: "9fbcc674-9072-4d42-bbe1-3fc4289d9cbc"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:29:59.540947 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.540885    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" (UID: "9fbcc674-9072-4d42-bbe1-3fc4289d9cbc"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:29:59.541056 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.541010    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" (UID: "9fbcc674-9072-4d42-bbe1-3fc4289d9cbc"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:29:59.541134 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.541102    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" (UID: "9fbcc674-9072-4d42-bbe1-3fc4289d9cbc"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:29:59.541960 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.541934    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" (UID: "9fbcc674-9072-4d42-bbe1-3fc4289d9cbc"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:29:59.542208 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.542189    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" (UID: "9fbcc674-9072-4d42-bbe1-3fc4289d9cbc"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:29:59.544180 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.544152    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" (UID: "9fbcc674-9072-4d42-bbe1-3fc4289d9cbc"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:29:59.551149 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.551040    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-web-config" (OuterVolumeSpecName: "web-config") pod "9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" (UID: "9fbcc674-9072-4d42-bbe1-3fc4289d9cbc"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:29:59.638290 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.638249    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-metrics-client-ca\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:29:59.638290 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.638284    2565 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-config-volume\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:29:59.638290 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.638294    2565 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-alertmanager-main-db\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:29:59.638470 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.638304    2565 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-cluster-tls-config\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:29:59.638470 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.638313    2565 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-web-config\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:29:59.638470 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.638321    2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-secret-alertmanager-main-tls\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:29:59.638470 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.638329    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p79zv\" (UniqueName: \"kubernetes.io/projected/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-kube-api-access-p79zv\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:29:59.638470 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.638340    2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:29:59.638470 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.638350    2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:29:59.638470 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.638359    2565 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-tls-assets\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:29:59.638470 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.638366    2565 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-config-out\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:29:59.638470 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:29:59.638375    2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:30:00.247358 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.247326    2565 generic.go:356] "Generic (PLEG): container finished" podID="9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" containerID="800069f83fc7bd344ff60ed541b996e2f12d640aec2ce23972f59745a452282f" exitCode=0
Sep 15 22:30:00.247742 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.247418    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.247742 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.247412    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc","Type":"ContainerDied","Data":"800069f83fc7bd344ff60ed541b996e2f12d640aec2ce23972f59745a452282f"}
Sep 15 22:30:00.247742 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.247558    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9fbcc674-9072-4d42-bbe1-3fc4289d9cbc","Type":"ContainerDied","Data":"68861e74e3c06686b0e43bbc8cf74d17b91bc4661f7c52898cb11879039f9ba2"}
Sep 15 22:30:00.247742 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.247593    2565 scope.go:117] "RemoveContainer" containerID="3a7a467bbfacaa7990ae23a441089ec5402f170d0b98a28cbb3118377ad505e3"
Sep 15 22:30:00.255659 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.255638    2565 scope.go:117] "RemoveContainer" containerID="d40e6b5a0a6ce0ea253725ba7dab4e540542a02b930774178d7b11559261847e"
Sep 15 22:30:00.262284 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.262267    2565 scope.go:117] "RemoveContainer" containerID="0e8f237ba6648c89f5d8f663d701b302eff7500b6c2adb38f37995e335aa0fca"
Sep 15 22:30:00.268622 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.268605    2565 scope.go:117] "RemoveContainer" containerID="800069f83fc7bd344ff60ed541b996e2f12d640aec2ce23972f59745a452282f"
Sep 15 22:30:00.271253 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.271232    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 15 22:30:00.277735 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.277716    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 15 22:30:00.278056 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.278039    2565 scope.go:117] "RemoveContainer" containerID="d65f66c31a26688b9d5e90698a36b89c7fa461decabf47a1004d3be8ea61298b"
Sep 15 22:30:00.284316 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.284301    2565 scope.go:117] "RemoveContainer" containerID="6de639c5cc5c9efb7be92b8a53940a9d0f1cef31468453a7dce11ccd0d085f60"
Sep 15 22:30:00.290597 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.290582    2565 scope.go:117] "RemoveContainer" containerID="ddfa1b2b0d2ae63aa448b50a68c52f9ffa502533857aeb259d11ee2f90d9ff85"
Sep 15 22:30:00.296743 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.296722    2565 scope.go:117] "RemoveContainer" containerID="3a7a467bbfacaa7990ae23a441089ec5402f170d0b98a28cbb3118377ad505e3"
Sep 15 22:30:00.297015 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:30:00.296991    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a7a467bbfacaa7990ae23a441089ec5402f170d0b98a28cbb3118377ad505e3\": container with ID starting with 3a7a467bbfacaa7990ae23a441089ec5402f170d0b98a28cbb3118377ad505e3 not found: ID does not exist" containerID="3a7a467bbfacaa7990ae23a441089ec5402f170d0b98a28cbb3118377ad505e3"
Sep 15 22:30:00.297118 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.297040    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a7a467bbfacaa7990ae23a441089ec5402f170d0b98a28cbb3118377ad505e3"} err="failed to get container status \"3a7a467bbfacaa7990ae23a441089ec5402f170d0b98a28cbb3118377ad505e3\": rpc error: code = NotFound desc = could not find container \"3a7a467bbfacaa7990ae23a441089ec5402f170d0b98a28cbb3118377ad505e3\": container with ID starting with 3a7a467bbfacaa7990ae23a441089ec5402f170d0b98a28cbb3118377ad505e3 not found: ID does not exist"
Sep 15 22:30:00.297118 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.297064    2565 scope.go:117] "RemoveContainer" containerID="d40e6b5a0a6ce0ea253725ba7dab4e540542a02b930774178d7b11559261847e"
Sep 15 22:30:00.297310 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:30:00.297294    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d40e6b5a0a6ce0ea253725ba7dab4e540542a02b930774178d7b11559261847e\": container with ID starting with d40e6b5a0a6ce0ea253725ba7dab4e540542a02b930774178d7b11559261847e not found: ID does not exist" containerID="d40e6b5a0a6ce0ea253725ba7dab4e540542a02b930774178d7b11559261847e"
Sep 15 22:30:00.297373 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.297320    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d40e6b5a0a6ce0ea253725ba7dab4e540542a02b930774178d7b11559261847e"} err="failed to get container status \"d40e6b5a0a6ce0ea253725ba7dab4e540542a02b930774178d7b11559261847e\": rpc error: code = NotFound desc = could not find container \"d40e6b5a0a6ce0ea253725ba7dab4e540542a02b930774178d7b11559261847e\": container with ID starting with d40e6b5a0a6ce0ea253725ba7dab4e540542a02b930774178d7b11559261847e not found: ID does not exist"
Sep 15 22:30:00.297373 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.297340    2565 scope.go:117] "RemoveContainer" containerID="0e8f237ba6648c89f5d8f663d701b302eff7500b6c2adb38f37995e335aa0fca"
Sep 15 22:30:00.297597 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:30:00.297578    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e8f237ba6648c89f5d8f663d701b302eff7500b6c2adb38f37995e335aa0fca\": container with ID starting with 0e8f237ba6648c89f5d8f663d701b302eff7500b6c2adb38f37995e335aa0fca not found: ID does not exist" containerID="0e8f237ba6648c89f5d8f663d701b302eff7500b6c2adb38f37995e335aa0fca"
Sep 15 22:30:00.297638 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.297605    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e8f237ba6648c89f5d8f663d701b302eff7500b6c2adb38f37995e335aa0fca"} err="failed to get container status \"0e8f237ba6648c89f5d8f663d701b302eff7500b6c2adb38f37995e335aa0fca\": rpc error: code = NotFound desc = could not find container \"0e8f237ba6648c89f5d8f663d701b302eff7500b6c2adb38f37995e335aa0fca\": container with ID starting with 0e8f237ba6648c89f5d8f663d701b302eff7500b6c2adb38f37995e335aa0fca not found: ID does not exist"
Sep 15 22:30:00.297638 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.297623    2565 scope.go:117] "RemoveContainer" containerID="800069f83fc7bd344ff60ed541b996e2f12d640aec2ce23972f59745a452282f"
Sep 15 22:30:00.297850 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:30:00.297833    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"800069f83fc7bd344ff60ed541b996e2f12d640aec2ce23972f59745a452282f\": container with ID starting with 800069f83fc7bd344ff60ed541b996e2f12d640aec2ce23972f59745a452282f not found: ID does not exist" containerID="800069f83fc7bd344ff60ed541b996e2f12d640aec2ce23972f59745a452282f"
Sep 15 22:30:00.297910 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.297855    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"800069f83fc7bd344ff60ed541b996e2f12d640aec2ce23972f59745a452282f"} err="failed to get container status \"800069f83fc7bd344ff60ed541b996e2f12d640aec2ce23972f59745a452282f\": rpc error: code = NotFound desc = could not find container \"800069f83fc7bd344ff60ed541b996e2f12d640aec2ce23972f59745a452282f\": container with ID starting with 800069f83fc7bd344ff60ed541b996e2f12d640aec2ce23972f59745a452282f not found: ID does not exist"
Sep 15 22:30:00.297910 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.297875    2565 scope.go:117] "RemoveContainer" containerID="d65f66c31a26688b9d5e90698a36b89c7fa461decabf47a1004d3be8ea61298b"
Sep 15 22:30:00.298097 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:30:00.298078    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d65f66c31a26688b9d5e90698a36b89c7fa461decabf47a1004d3be8ea61298b\": container with ID starting with d65f66c31a26688b9d5e90698a36b89c7fa461decabf47a1004d3be8ea61298b not found: ID does not exist" containerID="d65f66c31a26688b9d5e90698a36b89c7fa461decabf47a1004d3be8ea61298b"
Sep 15 22:30:00.298183 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.298098    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d65f66c31a26688b9d5e90698a36b89c7fa461decabf47a1004d3be8ea61298b"} err="failed to get container status \"d65f66c31a26688b9d5e90698a36b89c7fa461decabf47a1004d3be8ea61298b\": rpc error: code = NotFound desc = could not find container \"d65f66c31a26688b9d5e90698a36b89c7fa461decabf47a1004d3be8ea61298b\": container with ID starting with d65f66c31a26688b9d5e90698a36b89c7fa461decabf47a1004d3be8ea61298b not found: ID does not exist"
Sep 15 22:30:00.298183 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.298122    2565 scope.go:117] "RemoveContainer" containerID="6de639c5cc5c9efb7be92b8a53940a9d0f1cef31468453a7dce11ccd0d085f60"
Sep 15 22:30:00.298380 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:30:00.298364    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6de639c5cc5c9efb7be92b8a53940a9d0f1cef31468453a7dce11ccd0d085f60\": container with ID starting with 6de639c5cc5c9efb7be92b8a53940a9d0f1cef31468453a7dce11ccd0d085f60 not found: ID does not exist" containerID="6de639c5cc5c9efb7be92b8a53940a9d0f1cef31468453a7dce11ccd0d085f60"
Sep 15 22:30:00.298419 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.298384    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6de639c5cc5c9efb7be92b8a53940a9d0f1cef31468453a7dce11ccd0d085f60"} err="failed to get container status \"6de639c5cc5c9efb7be92b8a53940a9d0f1cef31468453a7dce11ccd0d085f60\": rpc error: code = NotFound desc = could not find container \"6de639c5cc5c9efb7be92b8a53940a9d0f1cef31468453a7dce11ccd0d085f60\": container with ID starting with 6de639c5cc5c9efb7be92b8a53940a9d0f1cef31468453a7dce11ccd0d085f60 not found: ID does not exist"
Sep 15 22:30:00.298419 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.298397    2565 scope.go:117] "RemoveContainer" containerID="ddfa1b2b0d2ae63aa448b50a68c52f9ffa502533857aeb259d11ee2f90d9ff85"
Sep 15 22:30:00.298634 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:30:00.298616    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ddfa1b2b0d2ae63aa448b50a68c52f9ffa502533857aeb259d11ee2f90d9ff85\": container with ID starting with ddfa1b2b0d2ae63aa448b50a68c52f9ffa502533857aeb259d11ee2f90d9ff85 not found: ID does not exist" containerID="ddfa1b2b0d2ae63aa448b50a68c52f9ffa502533857aeb259d11ee2f90d9ff85"
Sep 15 22:30:00.298689 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.298642    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddfa1b2b0d2ae63aa448b50a68c52f9ffa502533857aeb259d11ee2f90d9ff85"} err="failed to get container status \"ddfa1b2b0d2ae63aa448b50a68c52f9ffa502533857aeb259d11ee2f90d9ff85\": rpc error: code = NotFound desc = could not find container \"ddfa1b2b0d2ae63aa448b50a68c52f9ffa502533857aeb259d11ee2f90d9ff85\": container with ID starting with ddfa1b2b0d2ae63aa448b50a68c52f9ffa502533857aeb259d11ee2f90d9ff85 not found: ID does not exist"
Sep 15 22:30:00.307158 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.307128    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 15 22:30:00.307407 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.307392    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07af1527-72f1-49e8-ac8e-54ab1c1bccdf" containerName="console"
Sep 15 22:30:00.307481 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.307410    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="07af1527-72f1-49e8-ac8e-54ab1c1bccdf" containerName="console"
Sep 15 22:30:00.307481 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.307428    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" containerName="init-config-reloader"
Sep 15 22:30:00.307481 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.307437    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" containerName="init-config-reloader"
Sep 15 22:30:00.307481 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.307449    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" containerName="config-reloader"
Sep 15 22:30:00.307481 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.307459    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" containerName="config-reloader"
Sep 15 22:30:00.307481 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.307467    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" containerName="kube-rbac-proxy"
Sep 15 22:30:00.307481 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.307476    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" containerName="kube-rbac-proxy"
Sep 15 22:30:00.307810 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.307485    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" containerName="kube-rbac-proxy-metric"
Sep 15 22:30:00.307810 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.307493    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" containerName="kube-rbac-proxy-metric"
Sep 15 22:30:00.307810 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.307502    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" containerName="prom-label-proxy"
Sep 15 22:30:00.307810 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.307511    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" containerName="prom-label-proxy"
Sep 15 22:30:00.307810 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.307528    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" containerName="alertmanager"
Sep 15 22:30:00.307810 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.307536    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" containerName="alertmanager"
Sep 15 22:30:00.307810 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.307553    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" containerName="kube-rbac-proxy-web"
Sep 15 22:30:00.307810 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.307561    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" containerName="kube-rbac-proxy-web"
Sep 15 22:30:00.307810 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.307623    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" containerName="config-reloader"
Sep 15 22:30:00.307810 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.307635    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" containerName="kube-rbac-proxy"
Sep 15 22:30:00.307810 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.307644    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" containerName="prom-label-proxy"
Sep 15 22:30:00.307810 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.307654    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" containerName="kube-rbac-proxy-metric"
Sep 15 22:30:00.307810 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.307664    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" containerName="kube-rbac-proxy-web"
Sep 15 22:30:00.307810 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.307674    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="07af1527-72f1-49e8-ac8e-54ab1c1bccdf" containerName="console"
Sep 15 22:30:00.307810 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.307681    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" containerName="alertmanager"
Sep 15 22:30:00.311004 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.310986    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.313094 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.313073    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 15 22:30:00.313194 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.313132    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 15 22:30:00.313194 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.313179    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 15 22:30:00.313312 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.313213    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 15 22:30:00.313481 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.313454    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 15 22:30:00.313548 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.313526    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 15 22:30:00.313597 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.313541    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 15 22:30:00.313597 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.313527    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-7h9vd\""
Sep 15 22:30:00.314085 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.314071    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 15 22:30:00.319251 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.319230    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 15 22:30:00.328118 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.328097    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 15 22:30:00.343093 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.343067    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qdws\" (UniqueName: \"kubernetes.io/projected/6543f8f8-8aee-4448-8534-fec54d43db35-kube-api-access-6qdws\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.343172 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.343102    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6543f8f8-8aee-4448-8534-fec54d43db35-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.343172 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.343127    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6543f8f8-8aee-4448-8534-fec54d43db35-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.343172 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.343150    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6543f8f8-8aee-4448-8534-fec54d43db35-config-out\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.343306 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.343175    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6543f8f8-8aee-4448-8534-fec54d43db35-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.343306 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.343194    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6543f8f8-8aee-4448-8534-fec54d43db35-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.343306 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.343208    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6543f8f8-8aee-4448-8534-fec54d43db35-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.343306 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.343246    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6543f8f8-8aee-4448-8534-fec54d43db35-config-volume\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.343306 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.343266    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6543f8f8-8aee-4448-8534-fec54d43db35-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.343306 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.343283    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6543f8f8-8aee-4448-8534-fec54d43db35-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.343306 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.343302    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6543f8f8-8aee-4448-8534-fec54d43db35-web-config\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.343529 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.343333    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6543f8f8-8aee-4448-8534-fec54d43db35-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.343529 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.343363    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6543f8f8-8aee-4448-8534-fec54d43db35-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.444607 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.444575    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6543f8f8-8aee-4448-8534-fec54d43db35-config-volume\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.444607 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.444608    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6543f8f8-8aee-4448-8534-fec54d43db35-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.444808 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.444626    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6543f8f8-8aee-4448-8534-fec54d43db35-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.444808 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.444644    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6543f8f8-8aee-4448-8534-fec54d43db35-web-config\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.444808 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.444659    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6543f8f8-8aee-4448-8534-fec54d43db35-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.444808 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.444685    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6543f8f8-8aee-4448-8534-fec54d43db35-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.444808 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.444750    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qdws\" (UniqueName: \"kubernetes.io/projected/6543f8f8-8aee-4448-8534-fec54d43db35-kube-api-access-6qdws\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.444808 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.444779    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6543f8f8-8aee-4448-8534-fec54d43db35-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.444808 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.444803    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6543f8f8-8aee-4448-8534-fec54d43db35-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.445181 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.444823    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6543f8f8-8aee-4448-8534-fec54d43db35-config-out\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.445181 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.444848    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6543f8f8-8aee-4448-8534-fec54d43db35-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.445181 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.444871    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6543f8f8-8aee-4448-8534-fec54d43db35-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.445181 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.444893    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6543f8f8-8aee-4448-8534-fec54d43db35-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.445181 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.445163    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6543f8f8-8aee-4448-8534-fec54d43db35-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.446090 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.446008    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6543f8f8-8aee-4448-8534-fec54d43db35-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.446790 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.446757    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6543f8f8-8aee-4448-8534-fec54d43db35-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.447740 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.447718    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6543f8f8-8aee-4448-8534-fec54d43db35-web-config\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.447837 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.447752    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6543f8f8-8aee-4448-8534-fec54d43db35-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.447917 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.447856    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6543f8f8-8aee-4448-8534-fec54d43db35-config-volume\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.447992 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.447971    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6543f8f8-8aee-4448-8534-fec54d43db35-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.448607 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.448584    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6543f8f8-8aee-4448-8534-fec54d43db35-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.448693 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.448616    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6543f8f8-8aee-4448-8534-fec54d43db35-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.449167 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.449149    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6543f8f8-8aee-4448-8534-fec54d43db35-config-out\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.449237 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.449221    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6543f8f8-8aee-4448-8534-fec54d43db35-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.449329 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.449310    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6543f8f8-8aee-4448-8534-fec54d43db35-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.453399 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.453382    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qdws\" (UniqueName: \"kubernetes.io/projected/6543f8f8-8aee-4448-8534-fec54d43db35-kube-api-access-6qdws\") pod \"alertmanager-main-0\" (UID: \"6543f8f8-8aee-4448-8534-fec54d43db35\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.592937 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.592899    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fbcc674-9072-4d42-bbe1-3fc4289d9cbc" path="/var/lib/kubelet/pods/9fbcc674-9072-4d42-bbe1-3fc4289d9cbc/volumes"
Sep 15 22:30:00.620393 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.620366    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 15 22:30:00.900653 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:00.900585    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 15 22:30:00.903797 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:30:00.903765    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6543f8f8_8aee_4448_8534_fec54d43db35.slice/crio-71ec7a2025698986581d93e1c7aba224455d173ee21c1b2ab6801a00584dc824 WatchSource:0}: Error finding container 71ec7a2025698986581d93e1c7aba224455d173ee21c1b2ab6801a00584dc824: Status 404 returned error can't find the container with id 71ec7a2025698986581d93e1c7aba224455d173ee21c1b2ab6801a00584dc824
Sep 15 22:30:01.253261 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:01.253183    2565 generic.go:356] "Generic (PLEG): container finished" podID="6543f8f8-8aee-4448-8534-fec54d43db35" containerID="3e6dc7e24aaa8a58e93cbcf359c6e4ec79ba467d934fbaa9fc6946e59fc79834" exitCode=0
Sep 15 22:30:01.253261 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:01.253250    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6543f8f8-8aee-4448-8534-fec54d43db35","Type":"ContainerDied","Data":"3e6dc7e24aaa8a58e93cbcf359c6e4ec79ba467d934fbaa9fc6946e59fc79834"}
Sep 15 22:30:01.253628 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:01.253304    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6543f8f8-8aee-4448-8534-fec54d43db35","Type":"ContainerStarted","Data":"71ec7a2025698986581d93e1c7aba224455d173ee21c1b2ab6801a00584dc824"}
Sep 15 22:30:02.201788 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.201753    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-9778f96f8-tr52c"]
Sep 15 22:30:02.205037 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.205002    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-9778f96f8-tr52c"
Sep 15 22:30:02.209996 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.209958    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-2ltnp\""
Sep 15 22:30:02.209996 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.209980    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 15 22:30:02.210194 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.210171    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 15 22:30:02.210294 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.210265    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 15 22:30:02.210583 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.210563    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 15 22:30:02.210583 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.210567    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 15 22:30:02.213658 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.213635    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 15 22:30:02.220647 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.220629    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-9778f96f8-tr52c"]
Sep 15 22:30:02.257230 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.257203    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb96bba3-1956-4f1f-8b67-3c7a06fcff76-metrics-client-ca\") pod \"telemeter-client-9778f96f8-tr52c\" (UID: \"fb96bba3-1956-4f1f-8b67-3c7a06fcff76\") " pod="openshift-monitoring/telemeter-client-9778f96f8-tr52c"
Sep 15 22:30:02.257637 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.257266    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2l6n\" (UniqueName: \"kubernetes.io/projected/fb96bba3-1956-4f1f-8b67-3c7a06fcff76-kube-api-access-l2l6n\") pod \"telemeter-client-9778f96f8-tr52c\" (UID: \"fb96bba3-1956-4f1f-8b67-3c7a06fcff76\") " pod="openshift-monitoring/telemeter-client-9778f96f8-tr52c"
Sep 15 22:30:02.257637 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.257308    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/fb96bba3-1956-4f1f-8b67-3c7a06fcff76-telemeter-client-tls\") pod \"telemeter-client-9778f96f8-tr52c\" (UID: \"fb96bba3-1956-4f1f-8b67-3c7a06fcff76\") " pod="openshift-monitoring/telemeter-client-9778f96f8-tr52c"
Sep 15 22:30:02.257637 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.257336    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb96bba3-1956-4f1f-8b67-3c7a06fcff76-serving-certs-ca-bundle\") pod \"telemeter-client-9778f96f8-tr52c\" (UID: \"fb96bba3-1956-4f1f-8b67-3c7a06fcff76\") " pod="openshift-monitoring/telemeter-client-9778f96f8-tr52c"
Sep 15 22:30:02.257637 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.257405    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/fb96bba3-1956-4f1f-8b67-3c7a06fcff76-federate-client-tls\") pod \"telemeter-client-9778f96f8-tr52c\" (UID: \"fb96bba3-1956-4f1f-8b67-3c7a06fcff76\") " pod="openshift-monitoring/telemeter-client-9778f96f8-tr52c"
Sep 15 22:30:02.257637 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.257435    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb96bba3-1956-4f1f-8b67-3c7a06fcff76-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9778f96f8-tr52c\" (UID: \"fb96bba3-1956-4f1f-8b67-3c7a06fcff76\") " pod="openshift-monitoring/telemeter-client-9778f96f8-tr52c"
Sep 15 22:30:02.257637 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.257489    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/fb96bba3-1956-4f1f-8b67-3c7a06fcff76-secret-telemeter-client\") pod \"telemeter-client-9778f96f8-tr52c\" (UID: \"fb96bba3-1956-4f1f-8b67-3c7a06fcff76\") " pod="openshift-monitoring/telemeter-client-9778f96f8-tr52c"
Sep 15 22:30:02.257637 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.257525    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fb96bba3-1956-4f1f-8b67-3c7a06fcff76-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9778f96f8-tr52c\" (UID: \"fb96bba3-1956-4f1f-8b67-3c7a06fcff76\") " pod="openshift-monitoring/telemeter-client-9778f96f8-tr52c"
Sep 15 22:30:02.260591 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.260570    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6543f8f8-8aee-4448-8534-fec54d43db35","Type":"ContainerStarted","Data":"1c3f16d488947717c4498361ee28b7de6c6f836ecbf621f9730986426f70affe"}
Sep 15 22:30:02.260673 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.260623    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6543f8f8-8aee-4448-8534-fec54d43db35","Type":"ContainerStarted","Data":"6de09119173af4201d6df2aa4f1e0312687d68b6abefd858df9aa56cb94ff2f7"}
Sep 15 22:30:02.260673 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.260649    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6543f8f8-8aee-4448-8534-fec54d43db35","Type":"ContainerStarted","Data":"a3ce4c6e524d558d32774e9b1d26d213c0d65971187a5bc4867e53435f4de7eb"}
Sep 15 22:30:02.260673 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.260672    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6543f8f8-8aee-4448-8534-fec54d43db35","Type":"ContainerStarted","Data":"7a506cf10adb73f497c0e5f22699671a27fe1d3e29287ed2a5586146e95184b6"}
Sep 15 22:30:02.260776 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.260697    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6543f8f8-8aee-4448-8534-fec54d43db35","Type":"ContainerStarted","Data":"c8b5ce070b871371be8f87e196992b2253db7d80827036c35a515b687cbc0983"}
Sep 15 22:30:02.260776 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.260730    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6543f8f8-8aee-4448-8534-fec54d43db35","Type":"ContainerStarted","Data":"c6ec1b4be2f3f618251210ab9590b27bdb30ccf1e75f9ff09fa7e52b737c3792"}
Sep 15 22:30:02.287945 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.287901    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.274622132 podStartE2EDuration="2.287882913s" totalImagesPullingTime="0s" totalInitContainerRuntime="13.260781ms" isStatefulPod=false podCreationTimestamp="2026-09-15 22:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:30:02.286713513 +0000 UTC m=+152.146424798" watchObservedRunningTime="2026-09-15 22:30:02.287882913 +0000 UTC m=+152.147594197"
Sep 15 22:30:02.358413 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.358376    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/fb96bba3-1956-4f1f-8b67-3c7a06fcff76-federate-client-tls\") pod \"telemeter-client-9778f96f8-tr52c\" (UID: \"fb96bba3-1956-4f1f-8b67-3c7a06fcff76\") " pod="openshift-monitoring/telemeter-client-9778f96f8-tr52c"
Sep 15 22:30:02.358413 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.358415    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb96bba3-1956-4f1f-8b67-3c7a06fcff76-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9778f96f8-tr52c\" (UID: \"fb96bba3-1956-4f1f-8b67-3c7a06fcff76\") " pod="openshift-monitoring/telemeter-client-9778f96f8-tr52c"
Sep 15 22:30:02.358634 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.358450    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/fb96bba3-1956-4f1f-8b67-3c7a06fcff76-secret-telemeter-client\") pod \"telemeter-client-9778f96f8-tr52c\" (UID: \"fb96bba3-1956-4f1f-8b67-3c7a06fcff76\") " pod="openshift-monitoring/telemeter-client-9778f96f8-tr52c"
Sep 15 22:30:02.358634 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.358478    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fb96bba3-1956-4f1f-8b67-3c7a06fcff76-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9778f96f8-tr52c\" (UID: \"fb96bba3-1956-4f1f-8b67-3c7a06fcff76\") " pod="openshift-monitoring/telemeter-client-9778f96f8-tr52c"
Sep 15 22:30:02.358634 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.358512    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb96bba3-1956-4f1f-8b67-3c7a06fcff76-metrics-client-ca\") pod \"telemeter-client-9778f96f8-tr52c\" (UID: \"fb96bba3-1956-4f1f-8b67-3c7a06fcff76\") " pod="openshift-monitoring/telemeter-client-9778f96f8-tr52c"
Sep 15 22:30:02.358634 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.358621    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2l6n\" (UniqueName: \"kubernetes.io/projected/fb96bba3-1956-4f1f-8b67-3c7a06fcff76-kube-api-access-l2l6n\") pod \"telemeter-client-9778f96f8-tr52c\" (UID: \"fb96bba3-1956-4f1f-8b67-3c7a06fcff76\") " pod="openshift-monitoring/telemeter-client-9778f96f8-tr52c"
Sep 15 22:30:02.358813 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.358667    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/fb96bba3-1956-4f1f-8b67-3c7a06fcff76-telemeter-client-tls\") pod \"telemeter-client-9778f96f8-tr52c\" (UID: \"fb96bba3-1956-4f1f-8b67-3c7a06fcff76\") " pod="openshift-monitoring/telemeter-client-9778f96f8-tr52c"
Sep 15 22:30:02.358813 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.358739    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb96bba3-1956-4f1f-8b67-3c7a06fcff76-serving-certs-ca-bundle\") pod \"telemeter-client-9778f96f8-tr52c\" (UID: \"fb96bba3-1956-4f1f-8b67-3c7a06fcff76\") " pod="openshift-monitoring/telemeter-client-9778f96f8-tr52c"
Sep 15 22:30:02.359584 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.359458    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb96bba3-1956-4f1f-8b67-3c7a06fcff76-serving-certs-ca-bundle\") pod \"telemeter-client-9778f96f8-tr52c\" (UID: \"fb96bba3-1956-4f1f-8b67-3c7a06fcff76\") " pod="openshift-monitoring/telemeter-client-9778f96f8-tr52c"
Sep 15 22:30:02.359995 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.359969    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb96bba3-1956-4f1f-8b67-3c7a06fcff76-metrics-client-ca\") pod \"telemeter-client-9778f96f8-tr52c\" (UID: \"fb96bba3-1956-4f1f-8b67-3c7a06fcff76\") " pod="openshift-monitoring/telemeter-client-9778f96f8-tr52c"
Sep 15 22:30:02.360283 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.360264    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb96bba3-1956-4f1f-8b67-3c7a06fcff76-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9778f96f8-tr52c\" (UID: \"fb96bba3-1956-4f1f-8b67-3c7a06fcff76\") " pod="openshift-monitoring/telemeter-client-9778f96f8-tr52c"
Sep 15 22:30:02.361110 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.361084    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fb96bba3-1956-4f1f-8b67-3c7a06fcff76-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9778f96f8-tr52c\" (UID: \"fb96bba3-1956-4f1f-8b67-3c7a06fcff76\") " pod="openshift-monitoring/telemeter-client-9778f96f8-tr52c"
Sep 15 22:30:02.361299 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.361281    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/fb96bba3-1956-4f1f-8b67-3c7a06fcff76-federate-client-tls\") pod \"telemeter-client-9778f96f8-tr52c\" (UID: \"fb96bba3-1956-4f1f-8b67-3c7a06fcff76\") " pod="openshift-monitoring/telemeter-client-9778f96f8-tr52c"
Sep 15 22:30:02.361545 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.361528    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/fb96bba3-1956-4f1f-8b67-3c7a06fcff76-secret-telemeter-client\") pod \"telemeter-client-9778f96f8-tr52c\" (UID: \"fb96bba3-1956-4f1f-8b67-3c7a06fcff76\") " pod="openshift-monitoring/telemeter-client-9778f96f8-tr52c"
Sep 15 22:30:02.361625 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.361609    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/fb96bba3-1956-4f1f-8b67-3c7a06fcff76-telemeter-client-tls\") pod \"telemeter-client-9778f96f8-tr52c\" (UID: \"fb96bba3-1956-4f1f-8b67-3c7a06fcff76\") " pod="openshift-monitoring/telemeter-client-9778f96f8-tr52c"
Sep 15 22:30:02.366397 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.366373    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2l6n\" (UniqueName: \"kubernetes.io/projected/fb96bba3-1956-4f1f-8b67-3c7a06fcff76-kube-api-access-l2l6n\") pod \"telemeter-client-9778f96f8-tr52c\" (UID: \"fb96bba3-1956-4f1f-8b67-3c7a06fcff76\") " pod="openshift-monitoring/telemeter-client-9778f96f8-tr52c"
Sep 15 22:30:02.517984 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.517904    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-9778f96f8-tr52c"
Sep 15 22:30:02.797480 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:02.797452    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-9778f96f8-tr52c"]
Sep 15 22:30:02.800277 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:30:02.800248    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb96bba3_1956_4f1f_8b67_3c7a06fcff76.slice/crio-6d7fb5022a2ea90c99fe84c0f93123eac85c0624e7e5d87ed9474ff279717410 WatchSource:0}: Error finding container 6d7fb5022a2ea90c99fe84c0f93123eac85c0624e7e5d87ed9474ff279717410: Status 404 returned error can't find the container with id 6d7fb5022a2ea90c99fe84c0f93123eac85c0624e7e5d87ed9474ff279717410
Sep 15 22:30:03.265017 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:03.264927    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9778f96f8-tr52c" event={"ID":"fb96bba3-1956-4f1f-8b67-3c7a06fcff76","Type":"ContainerStarted","Data":"6d7fb5022a2ea90c99fe84c0f93123eac85c0624e7e5d87ed9474ff279717410"}
Sep 15 22:30:05.273164 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:05.273131    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9778f96f8-tr52c" event={"ID":"fb96bba3-1956-4f1f-8b67-3c7a06fcff76","Type":"ContainerStarted","Data":"fcd86fe0c1d847c8e3b418bbc9dcfee0dc94039b0e5abf18909ca7004985ea58"}
Sep 15 22:30:05.273423 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:05.273238    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9778f96f8-tr52c" event={"ID":"fb96bba3-1956-4f1f-8b67-3c7a06fcff76","Type":"ContainerStarted","Data":"826f9b4f6656364fe28124a9d8f36cc93a6779f7d2f1de3d459efea4c0e675d7"}
Sep 15 22:30:06.278170 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:06.278132    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9778f96f8-tr52c" event={"ID":"fb96bba3-1956-4f1f-8b67-3c7a06fcff76","Type":"ContainerStarted","Data":"72fe43763dd0a34319934baab2e2b6ba29e77e2db2e18e710f4173f79513dc82"}
Sep 15 22:30:06.300351 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:06.300304    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-9778f96f8-tr52c" podStartSLOduration=2.006879269 podStartE2EDuration="4.300288749s" totalImagesPullingTime="2.29340948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:30:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:30:06.299203744 +0000 UTC m=+156.158915027" watchObservedRunningTime="2026-09-15 22:30:06.300288749 +0000 UTC m=+156.160000032"
Sep 15 22:30:06.397259 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:06.397229    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-749757d7bd-2sv2s"]
Sep 15 22:30:06.400290 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:06.400271    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-749757d7bd-2sv2s"
Sep 15 22:30:06.402260 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:06.402229    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 15 22:30:06.402260 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:06.402241    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 15 22:30:06.402465 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:06.402312    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 15 22:30:06.402668 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:06.402630    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 15 22:30:06.402791 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:06.402773    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4z6d6\""
Sep 15 22:30:06.402890 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:06.402839    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 15 22:30:06.402947 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:06.402895    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 15 22:30:06.403177 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:06.403165    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 15 22:30:06.408486 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:06.408467    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 15 22:30:06.414296 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:06.414276    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-749757d7bd-2sv2s"]
Sep 15 22:30:06.490082 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:06.490018    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a80b734d-2cd7-4b56-821b-7948966f4503-console-oauth-config\") pod \"console-749757d7bd-2sv2s\" (UID: \"a80b734d-2cd7-4b56-821b-7948966f4503\") " pod="openshift-console/console-749757d7bd-2sv2s"
Sep 15 22:30:06.490245 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:06.490094    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a80b734d-2cd7-4b56-821b-7948966f4503-oauth-serving-cert\") pod \"console-749757d7bd-2sv2s\" (UID: \"a80b734d-2cd7-4b56-821b-7948966f4503\") " pod="openshift-console/console-749757d7bd-2sv2s"
Sep 15 22:30:06.490245 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:06.490134    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a80b734d-2cd7-4b56-821b-7948966f4503-console-config\") pod \"console-749757d7bd-2sv2s\" (UID: \"a80b734d-2cd7-4b56-821b-7948966f4503\") " pod="openshift-console/console-749757d7bd-2sv2s"
Sep 15 22:30:06.490245 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:06.490158    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a80b734d-2cd7-4b56-821b-7948966f4503-trusted-ca-bundle\") pod \"console-749757d7bd-2sv2s\" (UID: \"a80b734d-2cd7-4b56-821b-7948966f4503\") " pod="openshift-console/console-749757d7bd-2sv2s"
Sep 15 22:30:06.490245 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:06.490213    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a80b734d-2cd7-4b56-821b-7948966f4503-service-ca\") pod \"console-749757d7bd-2sv2s\" (UID: \"a80b734d-2cd7-4b56-821b-7948966f4503\") " pod="openshift-console/console-749757d7bd-2sv2s"
Sep 15 22:30:06.490383 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:06.490250    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5d6wf\" (UniqueName: \"kubernetes.io/projected/a80b734d-2cd7-4b56-821b-7948966f4503-kube-api-access-5d6wf\") pod \"console-749757d7bd-2sv2s\" (UID: \"a80b734d-2cd7-4b56-821b-7948966f4503\") " pod="openshift-console/console-749757d7bd-2sv2s"
Sep 15 22:30:06.490383 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:06.490283    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a80b734d-2cd7-4b56-821b-7948966f4503-console-serving-cert\") pod \"console-749757d7bd-2sv2s\" (UID: \"a80b734d-2cd7-4b56-821b-7948966f4503\") " pod="openshift-console/console-749757d7bd-2sv2s"
Sep 15 22:30:06.591289 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:06.591262    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d6wf\" (UniqueName: \"kubernetes.io/projected/a80b734d-2cd7-4b56-821b-7948966f4503-kube-api-access-5d6wf\") pod \"console-749757d7bd-2sv2s\" (UID: \"a80b734d-2cd7-4b56-821b-7948966f4503\") " pod="openshift-console/console-749757d7bd-2sv2s"
Sep 15 22:30:06.591435 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:06.591299    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a80b734d-2cd7-4b56-821b-7948966f4503-console-serving-cert\") pod \"console-749757d7bd-2sv2s\" (UID: \"a80b734d-2cd7-4b56-821b-7948966f4503\") " pod="openshift-console/console-749757d7bd-2sv2s"
Sep 15 22:30:06.591435 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:06.591327    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a80b734d-2cd7-4b56-821b-7948966f4503-console-oauth-config\") pod \"console-749757d7bd-2sv2s\" (UID: \"a80b734d-2cd7-4b56-821b-7948966f4503\") " pod="openshift-console/console-749757d7bd-2sv2s"
Sep 15 22:30:06.591532 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:06.591444    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a80b734d-2cd7-4b56-821b-7948966f4503-oauth-serving-cert\") pod \"console-749757d7bd-2sv2s\" (UID: \"a80b734d-2cd7-4b56-821b-7948966f4503\") " pod="openshift-console/console-749757d7bd-2sv2s"
Sep 15 22:30:06.591532 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:06.591508    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a80b734d-2cd7-4b56-821b-7948966f4503-console-config\") pod \"console-749757d7bd-2sv2s\" (UID: \"a80b734d-2cd7-4b56-821b-7948966f4503\") " pod="openshift-console/console-749757d7bd-2sv2s"
Sep 15 22:30:06.591636 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:06.591537    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a80b734d-2cd7-4b56-821b-7948966f4503-trusted-ca-bundle\") pod \"console-749757d7bd-2sv2s\" (UID: \"a80b734d-2cd7-4b56-821b-7948966f4503\") " pod="openshift-console/console-749757d7bd-2sv2s"
Sep 15 22:30:06.591690 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:06.591671    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a80b734d-2cd7-4b56-821b-7948966f4503-service-ca\") pod \"console-749757d7bd-2sv2s\" (UID: \"a80b734d-2cd7-4b56-821b-7948966f4503\") " pod="openshift-console/console-749757d7bd-2sv2s"
Sep 15 22:30:06.592172 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:06.592151    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a80b734d-2cd7-4b56-821b-7948966f4503-oauth-serving-cert\") pod \"console-749757d7bd-2sv2s\" (UID: \"a80b734d-2cd7-4b56-821b-7948966f4503\") " pod="openshift-console/console-749757d7bd-2sv2s"
Sep 15 22:30:06.592296 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:06.592187    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a80b734d-2cd7-4b56-821b-7948966f4503-console-config\") pod \"console-749757d7bd-2sv2s\" (UID: \"a80b734d-2cd7-4b56-821b-7948966f4503\") " pod="openshift-console/console-749757d7bd-2sv2s"
Sep 15 22:30:06.592438 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:06.592415    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a80b734d-2cd7-4b56-821b-7948966f4503-service-ca\") pod \"console-749757d7bd-2sv2s\" (UID: \"a80b734d-2cd7-4b56-821b-7948966f4503\") " pod="openshift-console/console-749757d7bd-2sv2s"
Sep 15 22:30:06.592475 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:06.592428    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a80b734d-2cd7-4b56-821b-7948966f4503-trusted-ca-bundle\") pod \"console-749757d7bd-2sv2s\" (UID: \"a80b734d-2cd7-4b56-821b-7948966f4503\") " pod="openshift-console/console-749757d7bd-2sv2s"
Sep 15 22:30:06.593950 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:06.593924    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a80b734d-2cd7-4b56-821b-7948966f4503-console-oauth-config\") pod \"console-749757d7bd-2sv2s\" (UID: \"a80b734d-2cd7-4b56-821b-7948966f4503\") " pod="openshift-console/console-749757d7bd-2sv2s"
Sep 15 22:30:06.593950 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:06.593945    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a80b734d-2cd7-4b56-821b-7948966f4503-console-serving-cert\") pod \"console-749757d7bd-2sv2s\" (UID: \"a80b734d-2cd7-4b56-821b-7948966f4503\") " pod="openshift-console/console-749757d7bd-2sv2s"
Sep 15 22:30:06.600163 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:06.600142    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5d6wf\" (UniqueName: \"kubernetes.io/projected/a80b734d-2cd7-4b56-821b-7948966f4503-kube-api-access-5d6wf\") pod \"console-749757d7bd-2sv2s\" (UID: \"a80b734d-2cd7-4b56-821b-7948966f4503\") " pod="openshift-console/console-749757d7bd-2sv2s"
Sep 15 22:30:06.709969 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:06.709922    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-749757d7bd-2sv2s"
Sep 15 22:30:06.981525 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:06.981451    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-749757d7bd-2sv2s"]
Sep 15 22:30:06.984044 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:30:06.984001    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda80b734d_2cd7_4b56_821b_7948966f4503.slice/crio-9d6041dc520be2f1aa3f1c82ad17b286b7b073f2e7a9a3f6d2079aa62db0482e WatchSource:0}: Error finding container 9d6041dc520be2f1aa3f1c82ad17b286b7b073f2e7a9a3f6d2079aa62db0482e: Status 404 returned error can't find the container with id 9d6041dc520be2f1aa3f1c82ad17b286b7b073f2e7a9a3f6d2079aa62db0482e
Sep 15 22:30:07.284953 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:07.284912    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-749757d7bd-2sv2s" event={"ID":"a80b734d-2cd7-4b56-821b-7948966f4503","Type":"ContainerStarted","Data":"e3bd36f6cb42ca240315276d4876cac3806f5247e5be7306b1f6319799d01a99"}
Sep 15 22:30:07.285372 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:07.284975    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-749757d7bd-2sv2s" event={"ID":"a80b734d-2cd7-4b56-821b-7948966f4503","Type":"ContainerStarted","Data":"9d6041dc520be2f1aa3f1c82ad17b286b7b073f2e7a9a3f6d2079aa62db0482e"}
Sep 15 22:30:07.303337 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:07.303294    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-749757d7bd-2sv2s" podStartSLOduration=1.303278875 podStartE2EDuration="1.303278875s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:30:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:30:07.302127378 +0000 UTC m=+157.161838685" watchObservedRunningTime="2026-09-15 22:30:07.303278875 +0000 UTC m=+157.162990158"
Sep 15 22:30:16.710951 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:16.710899    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-749757d7bd-2sv2s"
Sep 15 22:30:16.711337 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:16.710979    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-749757d7bd-2sv2s"
Sep 15 22:30:16.715800 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:16.715781    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-749757d7bd-2sv2s"
Sep 15 22:30:17.325181 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:30:17.325156    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-749757d7bd-2sv2s"
Sep 15 22:31:17.728795 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:17.728764    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-9d47fc5c8-b5289"]
Sep 15 22:31:17.732284 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:17.732264    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9d47fc5c8-b5289"
Sep 15 22:31:17.752241 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:17.752216    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9d47fc5c8-b5289"]
Sep 15 22:31:17.818636 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:17.818605    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/88bbd734-5f2b-413e-8279-74862a9f00ad-console-oauth-config\") pod \"console-9d47fc5c8-b5289\" (UID: \"88bbd734-5f2b-413e-8279-74862a9f00ad\") " pod="openshift-console/console-9d47fc5c8-b5289"
Sep 15 22:31:17.818810 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:17.818642    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/88bbd734-5f2b-413e-8279-74862a9f00ad-service-ca\") pod \"console-9d47fc5c8-b5289\" (UID: \"88bbd734-5f2b-413e-8279-74862a9f00ad\") " pod="openshift-console/console-9d47fc5c8-b5289"
Sep 15 22:31:17.818810 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:17.818667    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njmbp\" (UniqueName: \"kubernetes.io/projected/88bbd734-5f2b-413e-8279-74862a9f00ad-kube-api-access-njmbp\") pod \"console-9d47fc5c8-b5289\" (UID: \"88bbd734-5f2b-413e-8279-74862a9f00ad\") " pod="openshift-console/console-9d47fc5c8-b5289"
Sep 15 22:31:17.818810 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:17.818731    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/88bbd734-5f2b-413e-8279-74862a9f00ad-console-config\") pod \"console-9d47fc5c8-b5289\" (UID: \"88bbd734-5f2b-413e-8279-74862a9f00ad\") " pod="openshift-console/console-9d47fc5c8-b5289"
Sep 15 22:31:17.818810 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:17.818796    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88bbd734-5f2b-413e-8279-74862a9f00ad-trusted-ca-bundle\") pod \"console-9d47fc5c8-b5289\" (UID: \"88bbd734-5f2b-413e-8279-74862a9f00ad\") " pod="openshift-console/console-9d47fc5c8-b5289"
Sep 15 22:31:17.819003 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:17.818848    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/88bbd734-5f2b-413e-8279-74862a9f00ad-oauth-serving-cert\") pod \"console-9d47fc5c8-b5289\" (UID: \"88bbd734-5f2b-413e-8279-74862a9f00ad\") " pod="openshift-console/console-9d47fc5c8-b5289"
Sep 15 22:31:17.819003 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:17.818884    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/88bbd734-5f2b-413e-8279-74862a9f00ad-console-serving-cert\") pod \"console-9d47fc5c8-b5289\" (UID: \"88bbd734-5f2b-413e-8279-74862a9f00ad\") " pod="openshift-console/console-9d47fc5c8-b5289"
Sep 15 22:31:17.920112 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:17.920074    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njmbp\" (UniqueName: \"kubernetes.io/projected/88bbd734-5f2b-413e-8279-74862a9f00ad-kube-api-access-njmbp\") pod \"console-9d47fc5c8-b5289\" (UID: \"88bbd734-5f2b-413e-8279-74862a9f00ad\") " pod="openshift-console/console-9d47fc5c8-b5289"
Sep 15 22:31:17.920112 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:17.920113    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/88bbd734-5f2b-413e-8279-74862a9f00ad-console-config\") pod \"console-9d47fc5c8-b5289\" (UID: \"88bbd734-5f2b-413e-8279-74862a9f00ad\") " pod="openshift-console/console-9d47fc5c8-b5289"
Sep 15 22:31:17.920294 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:17.920138    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88bbd734-5f2b-413e-8279-74862a9f00ad-trusted-ca-bundle\") pod \"console-9d47fc5c8-b5289\" (UID: \"88bbd734-5f2b-413e-8279-74862a9f00ad\") " pod="openshift-console/console-9d47fc5c8-b5289"
Sep 15 22:31:17.920294 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:17.920174    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/88bbd734-5f2b-413e-8279-74862a9f00ad-oauth-serving-cert\") pod \"console-9d47fc5c8-b5289\" (UID: \"88bbd734-5f2b-413e-8279-74862a9f00ad\") " pod="openshift-console/console-9d47fc5c8-b5289"
Sep 15 22:31:17.920294 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:17.920195    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/88bbd734-5f2b-413e-8279-74862a9f00ad-console-serving-cert\") pod \"console-9d47fc5c8-b5289\" (UID: \"88bbd734-5f2b-413e-8279-74862a9f00ad\") " pod="openshift-console/console-9d47fc5c8-b5289"
Sep 15 22:31:17.920294 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:17.920219    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/88bbd734-5f2b-413e-8279-74862a9f00ad-console-oauth-config\") pod \"console-9d47fc5c8-b5289\" (UID: \"88bbd734-5f2b-413e-8279-74862a9f00ad\") " pod="openshift-console/console-9d47fc5c8-b5289"
Sep 15 22:31:17.920455 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:17.920433    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/88bbd734-5f2b-413e-8279-74862a9f00ad-service-ca\") pod \"console-9d47fc5c8-b5289\" (UID: \"88bbd734-5f2b-413e-8279-74862a9f00ad\") " pod="openshift-console/console-9d47fc5c8-b5289"
Sep 15 22:31:17.920930 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:17.920900    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/88bbd734-5f2b-413e-8279-74862a9f00ad-console-config\") pod \"console-9d47fc5c8-b5289\" (UID: \"88bbd734-5f2b-413e-8279-74862a9f00ad\") " pod="openshift-console/console-9d47fc5c8-b5289"
Sep 15 22:31:17.920930 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:17.920925    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/88bbd734-5f2b-413e-8279-74862a9f00ad-oauth-serving-cert\") pod \"console-9d47fc5c8-b5289\" (UID: \"88bbd734-5f2b-413e-8279-74862a9f00ad\") " pod="openshift-console/console-9d47fc5c8-b5289"
Sep 15 22:31:17.921117 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:17.920973    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/88bbd734-5f2b-413e-8279-74862a9f00ad-service-ca\") pod \"console-9d47fc5c8-b5289\" (UID: \"88bbd734-5f2b-413e-8279-74862a9f00ad\") " pod="openshift-console/console-9d47fc5c8-b5289"
Sep 15 22:31:17.921163 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:17.921121    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88bbd734-5f2b-413e-8279-74862a9f00ad-trusted-ca-bundle\") pod \"console-9d47fc5c8-b5289\" (UID: \"88bbd734-5f2b-413e-8279-74862a9f00ad\") " pod="openshift-console/console-9d47fc5c8-b5289"
Sep 15 22:31:17.922650 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:17.922630    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/88bbd734-5f2b-413e-8279-74862a9f00ad-console-oauth-config\") pod \"console-9d47fc5c8-b5289\" (UID: \"88bbd734-5f2b-413e-8279-74862a9f00ad\") " pod="openshift-console/console-9d47fc5c8-b5289"
Sep 15 22:31:17.922715 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:17.922671    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/88bbd734-5f2b-413e-8279-74862a9f00ad-console-serving-cert\") pod \"console-9d47fc5c8-b5289\" (UID: \"88bbd734-5f2b-413e-8279-74862a9f00ad\") " pod="openshift-console/console-9d47fc5c8-b5289"
Sep 15 22:31:17.928939 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:17.928916    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njmbp\" (UniqueName: \"kubernetes.io/projected/88bbd734-5f2b-413e-8279-74862a9f00ad-kube-api-access-njmbp\") pod \"console-9d47fc5c8-b5289\" (UID: \"88bbd734-5f2b-413e-8279-74862a9f00ad\") " pod="openshift-console/console-9d47fc5c8-b5289"
Sep 15 22:31:18.040681 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:18.040653    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9d47fc5c8-b5289"
Sep 15 22:31:18.313889 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:18.313816    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9d47fc5c8-b5289"]
Sep 15 22:31:18.317038 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:31:18.316996    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88bbd734_5f2b_413e_8279_74862a9f00ad.slice/crio-daacb6f1207eab6615fc4ca4793207cc8b7b8d5581f76db2cbf5f8d82fa3d2db WatchSource:0}: Error finding container daacb6f1207eab6615fc4ca4793207cc8b7b8d5581f76db2cbf5f8d82fa3d2db: Status 404 returned error can't find the container with id daacb6f1207eab6615fc4ca4793207cc8b7b8d5581f76db2cbf5f8d82fa3d2db
Sep 15 22:31:18.533602 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:18.533565    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9d47fc5c8-b5289" event={"ID":"88bbd734-5f2b-413e-8279-74862a9f00ad","Type":"ContainerStarted","Data":"70c0bfc593a3967489e1211df57eeb07f0f7ab0071fa8d718ba7a1307dca731b"}
Sep 15 22:31:18.533761 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:18.533642    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9d47fc5c8-b5289" event={"ID":"88bbd734-5f2b-413e-8279-74862a9f00ad","Type":"ContainerStarted","Data":"daacb6f1207eab6615fc4ca4793207cc8b7b8d5581f76db2cbf5f8d82fa3d2db"}
Sep 15 22:31:18.552496 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:18.552452    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-9d47fc5c8-b5289" podStartSLOduration=1.5524370840000001 podStartE2EDuration="1.552437084s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:31:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:31:18.550896013 +0000 UTC m=+228.410607300" watchObservedRunningTime="2026-09-15 22:31:18.552437084 +0000 UTC m=+228.412148367"
Sep 15 22:31:28.040782 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:28.040748    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-9d47fc5c8-b5289"
Sep 15 22:31:28.041187 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:28.040824    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-9d47fc5c8-b5289"
Sep 15 22:31:28.045370 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:28.045348    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-9d47fc5c8-b5289"
Sep 15 22:31:28.572980 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:28.572952    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-9d47fc5c8-b5289"
Sep 15 22:31:28.620619 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:28.620591    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-749757d7bd-2sv2s"]
Sep 15 22:31:53.639655 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:53.639557    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-749757d7bd-2sv2s" podUID="a80b734d-2cd7-4b56-821b-7948966f4503" containerName="console" containerID="cri-o://e3bd36f6cb42ca240315276d4876cac3806f5247e5be7306b1f6319799d01a99" gracePeriod=15
Sep 15 22:31:53.880436 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:53.880413    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-749757d7bd-2sv2s"
Sep 15 22:31:53.992077 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:53.991987    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a80b734d-2cd7-4b56-821b-7948966f4503-oauth-serving-cert\") pod \"a80b734d-2cd7-4b56-821b-7948966f4503\" (UID: \"a80b734d-2cd7-4b56-821b-7948966f4503\") "
Sep 15 22:31:53.992077 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:53.992056    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a80b734d-2cd7-4b56-821b-7948966f4503-trusted-ca-bundle\") pod \"a80b734d-2cd7-4b56-821b-7948966f4503\" (UID: \"a80b734d-2cd7-4b56-821b-7948966f4503\") "
Sep 15 22:31:53.992250 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:53.992095    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a80b734d-2cd7-4b56-821b-7948966f4503-service-ca\") pod \"a80b734d-2cd7-4b56-821b-7948966f4503\" (UID: \"a80b734d-2cd7-4b56-821b-7948966f4503\") "
Sep 15 22:31:53.992250 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:53.992122    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a80b734d-2cd7-4b56-821b-7948966f4503-console-oauth-config\") pod \"a80b734d-2cd7-4b56-821b-7948966f4503\" (UID: \"a80b734d-2cd7-4b56-821b-7948966f4503\") "
Sep 15 22:31:53.992250 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:53.992154    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a80b734d-2cd7-4b56-821b-7948966f4503-console-config\") pod \"a80b734d-2cd7-4b56-821b-7948966f4503\" (UID: \"a80b734d-2cd7-4b56-821b-7948966f4503\") "
Sep 15 22:31:53.992250 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:53.992217    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5d6wf\" (UniqueName: \"kubernetes.io/projected/a80b734d-2cd7-4b56-821b-7948966f4503-kube-api-access-5d6wf\") pod \"a80b734d-2cd7-4b56-821b-7948966f4503\" (UID: \"a80b734d-2cd7-4b56-821b-7948966f4503\") "
Sep 15 22:31:53.992250 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:53.992242    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a80b734d-2cd7-4b56-821b-7948966f4503-console-serving-cert\") pod \"a80b734d-2cd7-4b56-821b-7948966f4503\" (UID: \"a80b734d-2cd7-4b56-821b-7948966f4503\") "
Sep 15 22:31:53.992491 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:53.992415    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a80b734d-2cd7-4b56-821b-7948966f4503-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a80b734d-2cd7-4b56-821b-7948966f4503" (UID: "a80b734d-2cd7-4b56-821b-7948966f4503"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:31:53.992547 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:53.992524    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a80b734d-2cd7-4b56-821b-7948966f4503-service-ca" (OuterVolumeSpecName: "service-ca") pod "a80b734d-2cd7-4b56-821b-7948966f4503" (UID: "a80b734d-2cd7-4b56-821b-7948966f4503"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:31:53.992608 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:53.992559    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a80b734d-2cd7-4b56-821b-7948966f4503-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a80b734d-2cd7-4b56-821b-7948966f4503" (UID: "a80b734d-2cd7-4b56-821b-7948966f4503"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:31:53.992902 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:53.992873    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a80b734d-2cd7-4b56-821b-7948966f4503-console-config" (OuterVolumeSpecName: "console-config") pod "a80b734d-2cd7-4b56-821b-7948966f4503" (UID: "a80b734d-2cd7-4b56-821b-7948966f4503"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:31:53.994298 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:53.994270    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a80b734d-2cd7-4b56-821b-7948966f4503-kube-api-access-5d6wf" (OuterVolumeSpecName: "kube-api-access-5d6wf") pod "a80b734d-2cd7-4b56-821b-7948966f4503" (UID: "a80b734d-2cd7-4b56-821b-7948966f4503"). InnerVolumeSpecName "kube-api-access-5d6wf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:31:53.994437 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:53.994408    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a80b734d-2cd7-4b56-821b-7948966f4503-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a80b734d-2cd7-4b56-821b-7948966f4503" (UID: "a80b734d-2cd7-4b56-821b-7948966f4503"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:31:53.994497 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:53.994432    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a80b734d-2cd7-4b56-821b-7948966f4503-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a80b734d-2cd7-4b56-821b-7948966f4503" (UID: "a80b734d-2cd7-4b56-821b-7948966f4503"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:31:54.092732 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:54.092702    2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a80b734d-2cd7-4b56-821b-7948966f4503-trusted-ca-bundle\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:31:54.092732 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:54.092725    2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a80b734d-2cd7-4b56-821b-7948966f4503-service-ca\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:31:54.092732 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:54.092734    2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a80b734d-2cd7-4b56-821b-7948966f4503-console-oauth-config\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:31:54.092910 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:54.092743    2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a80b734d-2cd7-4b56-821b-7948966f4503-console-config\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:31:54.092910 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:54.092754    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5d6wf\" (UniqueName: \"kubernetes.io/projected/a80b734d-2cd7-4b56-821b-7948966f4503-kube-api-access-5d6wf\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:31:54.092910 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:54.092762    2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a80b734d-2cd7-4b56-821b-7948966f4503-console-serving-cert\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:31:54.092910 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:54.092772    2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a80b734d-2cd7-4b56-821b-7948966f4503-oauth-serving-cert\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:31:54.659342 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:54.659303    2565 generic.go:356] "Generic (PLEG): container finished" podID="a80b734d-2cd7-4b56-821b-7948966f4503" containerID="e3bd36f6cb42ca240315276d4876cac3806f5247e5be7306b1f6319799d01a99" exitCode=0
Sep 15 22:31:54.659741 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:54.659352    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-749757d7bd-2sv2s"
Sep 15 22:31:54.659741 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:54.659376    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-749757d7bd-2sv2s" event={"ID":"a80b734d-2cd7-4b56-821b-7948966f4503","Type":"ContainerDied","Data":"e3bd36f6cb42ca240315276d4876cac3806f5247e5be7306b1f6319799d01a99"}
Sep 15 22:31:54.659741 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:54.659424    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-749757d7bd-2sv2s" event={"ID":"a80b734d-2cd7-4b56-821b-7948966f4503","Type":"ContainerDied","Data":"9d6041dc520be2f1aa3f1c82ad17b286b7b073f2e7a9a3f6d2079aa62db0482e"}
Sep 15 22:31:54.659741 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:54.659444    2565 scope.go:117] "RemoveContainer" containerID="e3bd36f6cb42ca240315276d4876cac3806f5247e5be7306b1f6319799d01a99"
Sep 15 22:31:54.667356 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:54.667335    2565 scope.go:117] "RemoveContainer" containerID="e3bd36f6cb42ca240315276d4876cac3806f5247e5be7306b1f6319799d01a99"
Sep 15 22:31:54.667626 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:31:54.667609    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3bd36f6cb42ca240315276d4876cac3806f5247e5be7306b1f6319799d01a99\": container with ID starting with e3bd36f6cb42ca240315276d4876cac3806f5247e5be7306b1f6319799d01a99 not found: ID does not exist" containerID="e3bd36f6cb42ca240315276d4876cac3806f5247e5be7306b1f6319799d01a99"
Sep 15 22:31:54.667690 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:54.667637    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3bd36f6cb42ca240315276d4876cac3806f5247e5be7306b1f6319799d01a99"} err="failed to get container status \"e3bd36f6cb42ca240315276d4876cac3806f5247e5be7306b1f6319799d01a99\": rpc error: code = NotFound desc = could not find container \"e3bd36f6cb42ca240315276d4876cac3806f5247e5be7306b1f6319799d01a99\": container with ID starting with e3bd36f6cb42ca240315276d4876cac3806f5247e5be7306b1f6319799d01a99 not found: ID does not exist"
Sep 15 22:31:54.674146 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:54.674120    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-749757d7bd-2sv2s"]
Sep 15 22:31:54.680000 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:54.679978    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-749757d7bd-2sv2s"]
Sep 15 22:31:56.589059 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:31:56.589006    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a80b734d-2cd7-4b56-821b-7948966f4503" path="/var/lib/kubelet/pods/a80b734d-2cd7-4b56-821b-7948966f4503/volumes"
Sep 15 22:32:18.374583 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:18.374542    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409rwnc"]
Sep 15 22:32:18.375241 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:18.375220    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a80b734d-2cd7-4b56-821b-7948966f4503" containerName="console"
Sep 15 22:32:18.375326 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:18.375245    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a80b734d-2cd7-4b56-821b-7948966f4503" containerName="console"
Sep 15 22:32:18.375375 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:18.375358    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a80b734d-2cd7-4b56-821b-7948966f4503" containerName="console"
Sep 15 22:32:18.380937 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:18.380914    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409rwnc"
Sep 15 22:32:18.384061 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:18.384039    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 15 22:32:18.384553 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:18.384535    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-68lxg\""
Sep 15 22:32:18.384616 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:18.384539    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 15 22:32:18.389784 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:18.389759    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409rwnc"]
Sep 15 22:32:18.479643 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:18.479606    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjblv\" (UniqueName: \"kubernetes.io/projected/144dbb43-9e1b-4e02-891d-3c09e83bfcfc-kube-api-access-xjblv\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409rwnc\" (UID: \"144dbb43-9e1b-4e02-891d-3c09e83bfcfc\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409rwnc"
Sep 15 22:32:18.479790 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:18.479660    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/144dbb43-9e1b-4e02-891d-3c09e83bfcfc-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409rwnc\" (UID: \"144dbb43-9e1b-4e02-891d-3c09e83bfcfc\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409rwnc"
Sep 15 22:32:18.479790 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:18.479687    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/144dbb43-9e1b-4e02-891d-3c09e83bfcfc-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409rwnc\" (UID: \"144dbb43-9e1b-4e02-891d-3c09e83bfcfc\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409rwnc"
Sep 15 22:32:18.580782 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:18.580746    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjblv\" (UniqueName: \"kubernetes.io/projected/144dbb43-9e1b-4e02-891d-3c09e83bfcfc-kube-api-access-xjblv\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409rwnc\" (UID: \"144dbb43-9e1b-4e02-891d-3c09e83bfcfc\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409rwnc"
Sep 15 22:32:18.580927 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:18.580808    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/144dbb43-9e1b-4e02-891d-3c09e83bfcfc-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409rwnc\" (UID: \"144dbb43-9e1b-4e02-891d-3c09e83bfcfc\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409rwnc"
Sep 15 22:32:18.580927 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:18.580832    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/144dbb43-9e1b-4e02-891d-3c09e83bfcfc-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409rwnc\" (UID: \"144dbb43-9e1b-4e02-891d-3c09e83bfcfc\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409rwnc"
Sep 15 22:32:18.581196 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:18.581176    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/144dbb43-9e1b-4e02-891d-3c09e83bfcfc-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409rwnc\" (UID: \"144dbb43-9e1b-4e02-891d-3c09e83bfcfc\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409rwnc"
Sep 15 22:32:18.581240 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:18.581213    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/144dbb43-9e1b-4e02-891d-3c09e83bfcfc-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409rwnc\" (UID: \"144dbb43-9e1b-4e02-891d-3c09e83bfcfc\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409rwnc"
Sep 15 22:32:18.590577 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:18.590550    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjblv\" (UniqueName: \"kubernetes.io/projected/144dbb43-9e1b-4e02-891d-3c09e83bfcfc-kube-api-access-xjblv\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409rwnc\" (UID: \"144dbb43-9e1b-4e02-891d-3c09e83bfcfc\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409rwnc"
Sep 15 22:32:18.690705 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:18.690637    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409rwnc"
Sep 15 22:32:18.962716 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:18.962628    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409rwnc"]
Sep 15 22:32:18.965543 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:32:18.965518    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod144dbb43_9e1b_4e02_891d_3c09e83bfcfc.slice/crio-7ee79c0d48369ac37a04092303d4b7d99f0cd354b191c3eb4f1d94e3c7e18160 WatchSource:0}: Error finding container 7ee79c0d48369ac37a04092303d4b7d99f0cd354b191c3eb4f1d94e3c7e18160: Status 404 returned error can't find the container with id 7ee79c0d48369ac37a04092303d4b7d99f0cd354b191c3eb4f1d94e3c7e18160
Sep 15 22:32:19.749522 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:19.749482    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409rwnc" event={"ID":"144dbb43-9e1b-4e02-891d-3c09e83bfcfc","Type":"ContainerStarted","Data":"7ee79c0d48369ac37a04092303d4b7d99f0cd354b191c3eb4f1d94e3c7e18160"}
Sep 15 22:32:25.775697 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:25.775613    2565 generic.go:356] "Generic (PLEG): container finished" podID="144dbb43-9e1b-4e02-891d-3c09e83bfcfc" containerID="4a699489f949c100dda235a8e6d4aaa01ace3f9bdc62bf1ca74d9625c12851d8" exitCode=0
Sep 15 22:32:25.776073 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:25.775705    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409rwnc" event={"ID":"144dbb43-9e1b-4e02-891d-3c09e83bfcfc","Type":"ContainerDied","Data":"4a699489f949c100dda235a8e6d4aaa01ace3f9bdc62bf1ca74d9625c12851d8"}
Sep 15 22:32:27.784611 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:27.784583    2565 generic.go:356] "Generic (PLEG): container finished" podID="144dbb43-9e1b-4e02-891d-3c09e83bfcfc" containerID="4dfe5a2c1feed932da110b51fbb851411f41374b143a54de243238b2602e472a" exitCode=0
Sep 15 22:32:27.784897 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:27.784665    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409rwnc" event={"ID":"144dbb43-9e1b-4e02-891d-3c09e83bfcfc","Type":"ContainerDied","Data":"4dfe5a2c1feed932da110b51fbb851411f41374b143a54de243238b2602e472a"}
Sep 15 22:32:30.553708 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:30.553475    2565 kubelet.go:1648] "Image garbage collection succeeded"
Sep 15 22:32:34.816717 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:34.816680    2565 generic.go:356] "Generic (PLEG): container finished" podID="144dbb43-9e1b-4e02-891d-3c09e83bfcfc" containerID="f21f531019827dbcc5e835290d7f4c2b7b3d22d72adc4fca32782b525be9dbae" exitCode=0
Sep 15 22:32:34.817186 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:34.816744    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409rwnc" event={"ID":"144dbb43-9e1b-4e02-891d-3c09e83bfcfc","Type":"ContainerDied","Data":"f21f531019827dbcc5e835290d7f4c2b7b3d22d72adc4fca32782b525be9dbae"}
Sep 15 22:32:35.945939 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:35.945915    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409rwnc"
Sep 15 22:32:36.030370 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:36.030344    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/144dbb43-9e1b-4e02-891d-3c09e83bfcfc-util\") pod \"144dbb43-9e1b-4e02-891d-3c09e83bfcfc\" (UID: \"144dbb43-9e1b-4e02-891d-3c09e83bfcfc\") "
Sep 15 22:32:36.030531 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:36.030381    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/144dbb43-9e1b-4e02-891d-3c09e83bfcfc-bundle\") pod \"144dbb43-9e1b-4e02-891d-3c09e83bfcfc\" (UID: \"144dbb43-9e1b-4e02-891d-3c09e83bfcfc\") "
Sep 15 22:32:36.030531 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:36.030453    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xjblv\" (UniqueName: \"kubernetes.io/projected/144dbb43-9e1b-4e02-891d-3c09e83bfcfc-kube-api-access-xjblv\") pod \"144dbb43-9e1b-4e02-891d-3c09e83bfcfc\" (UID: \"144dbb43-9e1b-4e02-891d-3c09e83bfcfc\") "
Sep 15 22:32:36.031041 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:36.030994    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/144dbb43-9e1b-4e02-891d-3c09e83bfcfc-bundle" (OuterVolumeSpecName: "bundle") pod "144dbb43-9e1b-4e02-891d-3c09e83bfcfc" (UID: "144dbb43-9e1b-4e02-891d-3c09e83bfcfc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:32:36.032626 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:36.032603    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/144dbb43-9e1b-4e02-891d-3c09e83bfcfc-kube-api-access-xjblv" (OuterVolumeSpecName: "kube-api-access-xjblv") pod "144dbb43-9e1b-4e02-891d-3c09e83bfcfc" (UID: "144dbb43-9e1b-4e02-891d-3c09e83bfcfc"). InnerVolumeSpecName "kube-api-access-xjblv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:32:36.034651 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:36.034628    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/144dbb43-9e1b-4e02-891d-3c09e83bfcfc-util" (OuterVolumeSpecName: "util") pod "144dbb43-9e1b-4e02-891d-3c09e83bfcfc" (UID: "144dbb43-9e1b-4e02-891d-3c09e83bfcfc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:32:36.131444 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:36.131379    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjblv\" (UniqueName: \"kubernetes.io/projected/144dbb43-9e1b-4e02-891d-3c09e83bfcfc-kube-api-access-xjblv\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:32:36.131444 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:36.131405    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/144dbb43-9e1b-4e02-891d-3c09e83bfcfc-util\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:32:36.131444 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:36.131415    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/144dbb43-9e1b-4e02-891d-3c09e83bfcfc-bundle\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:32:36.827258 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:36.827226    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409rwnc" event={"ID":"144dbb43-9e1b-4e02-891d-3c09e83bfcfc","Type":"ContainerDied","Data":"7ee79c0d48369ac37a04092303d4b7d99f0cd354b191c3eb4f1d94e3c7e18160"}
Sep 15 22:32:36.827258 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:36.827253    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409rwnc"
Sep 15 22:32:36.827456 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:36.827270    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7ee79c0d48369ac37a04092303d4b7d99f0cd354b191c3eb4f1d94e3c7e18160"
Sep 15 22:32:41.851648 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:41.851613    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fbglh"]
Sep 15 22:32:41.852016 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:41.851898    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="144dbb43-9e1b-4e02-891d-3c09e83bfcfc" containerName="util"
Sep 15 22:32:41.852016 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:41.851910    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="144dbb43-9e1b-4e02-891d-3c09e83bfcfc" containerName="util"
Sep 15 22:32:41.852016 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:41.851924    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="144dbb43-9e1b-4e02-891d-3c09e83bfcfc" containerName="pull"
Sep 15 22:32:41.852016 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:41.851930    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="144dbb43-9e1b-4e02-891d-3c09e83bfcfc" containerName="pull"
Sep 15 22:32:41.852016 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:41.851945    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="144dbb43-9e1b-4e02-891d-3c09e83bfcfc" containerName="extract"
Sep 15 22:32:41.852016 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:41.851951    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="144dbb43-9e1b-4e02-891d-3c09e83bfcfc" containerName="extract"
Sep 15 22:32:41.852016 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:41.851996    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="144dbb43-9e1b-4e02-891d-3c09e83bfcfc" containerName="extract"
Sep 15 22:32:41.855043 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:41.855008    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fbglh"
Sep 15 22:32:41.861263 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:41.860869    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-fgrck\""
Sep 15 22:32:41.861435 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:41.861414    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 15 22:32:41.861739 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:41.861719    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 15 22:32:41.875800 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:41.875770    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fbglh"]
Sep 15 22:32:41.975494 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:41.975458    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92h25\" (UniqueName: \"kubernetes.io/projected/3550bf84-1c32-4419-9772-593e423d4083-kube-api-access-92h25\") pod \"cert-manager-operator-controller-manager-55c444bf86-fbglh\" (UID: \"3550bf84-1c32-4419-9772-593e423d4083\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fbglh"
Sep 15 22:32:41.975494 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:41.975497    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3550bf84-1c32-4419-9772-593e423d4083-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-fbglh\" (UID: \"3550bf84-1c32-4419-9772-593e423d4083\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fbglh"
Sep 15 22:32:42.076904 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:42.076857    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92h25\" (UniqueName: \"kubernetes.io/projected/3550bf84-1c32-4419-9772-593e423d4083-kube-api-access-92h25\") pod \"cert-manager-operator-controller-manager-55c444bf86-fbglh\" (UID: \"3550bf84-1c32-4419-9772-593e423d4083\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fbglh"
Sep 15 22:32:42.076904 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:42.076908    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3550bf84-1c32-4419-9772-593e423d4083-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-fbglh\" (UID: \"3550bf84-1c32-4419-9772-593e423d4083\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fbglh"
Sep 15 22:32:42.077286 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:42.077270    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3550bf84-1c32-4419-9772-593e423d4083-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-fbglh\" (UID: \"3550bf84-1c32-4419-9772-593e423d4083\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fbglh"
Sep 15 22:32:42.089424 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:42.089392    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92h25\" (UniqueName: \"kubernetes.io/projected/3550bf84-1c32-4419-9772-593e423d4083-kube-api-access-92h25\") pod \"cert-manager-operator-controller-manager-55c444bf86-fbglh\" (UID: \"3550bf84-1c32-4419-9772-593e423d4083\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fbglh"
Sep 15 22:32:42.163428 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:42.163342    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fbglh"
Sep 15 22:32:42.445099 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:42.445009    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fbglh"]
Sep 15 22:32:42.450666 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:32:42.450630    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3550bf84_1c32_4419_9772_593e423d4083.slice/crio-1cce08d38c904bb4e04d764aba54fe7af805d229f1e31f06c0675edad7e1a946 WatchSource:0}: Error finding container 1cce08d38c904bb4e04d764aba54fe7af805d229f1e31f06c0675edad7e1a946: Status 404 returned error can't find the container with id 1cce08d38c904bb4e04d764aba54fe7af805d229f1e31f06c0675edad7e1a946
Sep 15 22:32:42.453360 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:42.453335    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 22:32:42.862318 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:42.862277    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fbglh" event={"ID":"3550bf84-1c32-4419-9772-593e423d4083","Type":"ContainerStarted","Data":"1cce08d38c904bb4e04d764aba54fe7af805d229f1e31f06c0675edad7e1a946"}
Sep 15 22:32:44.871943 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:44.871900    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fbglh" event={"ID":"3550bf84-1c32-4419-9772-593e423d4083","Type":"ContainerStarted","Data":"b56e3a7af8c1a3627a51675f4af148458e2bf13eb8423ab1b90684327790d276"}
Sep 15 22:32:44.908454 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:44.908403    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fbglh" podStartSLOduration=2.458375621 podStartE2EDuration="3.90838823s" totalImagesPullingTime="1.450012609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:32:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:32:44.907638441 +0000 UTC m=+314.767349725" watchObservedRunningTime="2026-09-15 22:32:44.90838823 +0000 UTC m=+314.768099513"
Sep 15 22:32:46.294703 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:46.294665    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd84rv"]
Sep 15 22:32:46.298248 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:46.298224    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd84rv"
Sep 15 22:32:46.301192 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:46.301172    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 15 22:32:46.301351 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:46.301333    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-68lxg\""
Sep 15 22:32:46.302341 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:46.302324    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 15 22:32:46.332071 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:46.332051    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd84rv"]
Sep 15 22:32:46.413559 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:46.413521    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b43ee5d8-abce-4741-8ca0-f73c695f2ee7-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd84rv\" (UID: \"b43ee5d8-abce-4741-8ca0-f73c695f2ee7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd84rv"
Sep 15 22:32:46.413727 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:46.413563    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ww5ns\" (UniqueName: \"kubernetes.io/projected/b43ee5d8-abce-4741-8ca0-f73c695f2ee7-kube-api-access-ww5ns\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd84rv\" (UID: \"b43ee5d8-abce-4741-8ca0-f73c695f2ee7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd84rv"
Sep 15 22:32:46.413727 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:46.413689    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b43ee5d8-abce-4741-8ca0-f73c695f2ee7-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd84rv\" (UID: \"b43ee5d8-abce-4741-8ca0-f73c695f2ee7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd84rv"
Sep 15 22:32:46.514327 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:46.514292    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b43ee5d8-abce-4741-8ca0-f73c695f2ee7-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd84rv\" (UID: \"b43ee5d8-abce-4741-8ca0-f73c695f2ee7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd84rv"
Sep 15 22:32:46.514327 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:46.514325    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ww5ns\" (UniqueName: \"kubernetes.io/projected/b43ee5d8-abce-4741-8ca0-f73c695f2ee7-kube-api-access-ww5ns\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd84rv\" (UID: \"b43ee5d8-abce-4741-8ca0-f73c695f2ee7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd84rv"
Sep 15 22:32:46.514508 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:46.514370    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b43ee5d8-abce-4741-8ca0-f73c695f2ee7-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd84rv\" (UID: \"b43ee5d8-abce-4741-8ca0-f73c695f2ee7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd84rv"
Sep 15 22:32:46.514752 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:46.514728    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b43ee5d8-abce-4741-8ca0-f73c695f2ee7-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd84rv\" (UID: \"b43ee5d8-abce-4741-8ca0-f73c695f2ee7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd84rv"
Sep 15 22:32:46.514786 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:46.514763    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b43ee5d8-abce-4741-8ca0-f73c695f2ee7-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd84rv\" (UID: \"b43ee5d8-abce-4741-8ca0-f73c695f2ee7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd84rv"
Sep 15 22:32:46.526210 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:46.526181    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ww5ns\" (UniqueName: \"kubernetes.io/projected/b43ee5d8-abce-4741-8ca0-f73c695f2ee7-kube-api-access-ww5ns\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd84rv\" (UID: \"b43ee5d8-abce-4741-8ca0-f73c695f2ee7\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd84rv"
Sep 15 22:32:46.607126 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:46.607056    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd84rv"
Sep 15 22:32:46.910396 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:46.910369    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd84rv"]
Sep 15 22:32:46.912607 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:32:46.912578    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb43ee5d8_abce_4741_8ca0_f73c695f2ee7.slice/crio-0ae48b3a67f1ee3334c502e29694ec900dc3c34d1c71aa7bb47ec56148930150 WatchSource:0}: Error finding container 0ae48b3a67f1ee3334c502e29694ec900dc3c34d1c71aa7bb47ec56148930150: Status 404 returned error can't find the container with id 0ae48b3a67f1ee3334c502e29694ec900dc3c34d1c71aa7bb47ec56148930150
Sep 15 22:32:47.628428 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:47.628397    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-thb2b"]
Sep 15 22:32:47.631533 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:47.631513    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-thb2b"
Sep 15 22:32:47.633655 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:47.633639    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 15 22:32:47.634211 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:47.634191    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 15 22:32:47.634250 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:47.634219    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-44zjd\""
Sep 15 22:32:47.647916 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:47.647894    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-thb2b"]
Sep 15 22:32:47.722712 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:47.722683    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ncdk\" (UniqueName: \"kubernetes.io/projected/1d364093-a365-44ee-b45c-6d17f9dc2dd1-kube-api-access-6ncdk\") pod \"cert-manager-cainjector-559cdd8c87-thb2b\" (UID: \"1d364093-a365-44ee-b45c-6d17f9dc2dd1\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-thb2b"
Sep 15 22:32:47.722852 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:47.722725    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1d364093-a365-44ee-b45c-6d17f9dc2dd1-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-thb2b\" (UID: \"1d364093-a365-44ee-b45c-6d17f9dc2dd1\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-thb2b"
Sep 15 22:32:47.823685 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:47.823606    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1d364093-a365-44ee-b45c-6d17f9dc2dd1-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-thb2b\" (UID: \"1d364093-a365-44ee-b45c-6d17f9dc2dd1\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-thb2b"
Sep 15 22:32:47.823838 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:47.823694    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ncdk\" (UniqueName: \"kubernetes.io/projected/1d364093-a365-44ee-b45c-6d17f9dc2dd1-kube-api-access-6ncdk\") pod \"cert-manager-cainjector-559cdd8c87-thb2b\" (UID: \"1d364093-a365-44ee-b45c-6d17f9dc2dd1\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-thb2b"
Sep 15 22:32:47.833188 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:47.833153    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ncdk\" (UniqueName: \"kubernetes.io/projected/1d364093-a365-44ee-b45c-6d17f9dc2dd1-kube-api-access-6ncdk\") pod \"cert-manager-cainjector-559cdd8c87-thb2b\" (UID: \"1d364093-a365-44ee-b45c-6d17f9dc2dd1\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-thb2b"
Sep 15 22:32:47.833283 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:47.833242    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1d364093-a365-44ee-b45c-6d17f9dc2dd1-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-thb2b\" (UID: \"1d364093-a365-44ee-b45c-6d17f9dc2dd1\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-thb2b"
Sep 15 22:32:47.888742 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:47.888715    2565 generic.go:356] "Generic (PLEG): container finished" podID="b43ee5d8-abce-4741-8ca0-f73c695f2ee7" containerID="ea58905650034eaea4615e6b35f844c1b75f129ce4f35444af1fd66e2fb27d65" exitCode=0
Sep 15 22:32:47.888877 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:47.888802    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd84rv" event={"ID":"b43ee5d8-abce-4741-8ca0-f73c695f2ee7","Type":"ContainerDied","Data":"ea58905650034eaea4615e6b35f844c1b75f129ce4f35444af1fd66e2fb27d65"}
Sep 15 22:32:47.888877 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:47.888849    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd84rv" event={"ID":"b43ee5d8-abce-4741-8ca0-f73c695f2ee7","Type":"ContainerStarted","Data":"0ae48b3a67f1ee3334c502e29694ec900dc3c34d1c71aa7bb47ec56148930150"}
Sep 15 22:32:47.951221 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:47.951192    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-thb2b"
Sep 15 22:32:48.225238 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:48.225213    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-thb2b"]
Sep 15 22:32:48.227540 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:32:48.227513    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1d364093_a365_44ee_b45c_6d17f9dc2dd1.slice/crio-ddd60f59a578f819adadf1af1b6fc9ff09e3841430a45b47fe97f72e63521cf4 WatchSource:0}: Error finding container ddd60f59a578f819adadf1af1b6fc9ff09e3841430a45b47fe97f72e63521cf4: Status 404 returned error can't find the container with id ddd60f59a578f819adadf1af1b6fc9ff09e3841430a45b47fe97f72e63521cf4
Sep 15 22:32:48.896087 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:48.896017    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-thb2b" event={"ID":"1d364093-a365-44ee-b45c-6d17f9dc2dd1","Type":"ContainerStarted","Data":"ddd60f59a578f819adadf1af1b6fc9ff09e3841430a45b47fe97f72e63521cf4"}
Sep 15 22:32:51.645799 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:51.645763    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-p2prs"]
Sep 15 22:32:51.648952 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:51.648934    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-p2prs"
Sep 15 22:32:51.650920 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:51.650902    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-ctgb5\""
Sep 15 22:32:51.658896 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:51.658875    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-p2prs"]
Sep 15 22:32:51.757690 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:51.757652    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7c9w\" (UniqueName: \"kubernetes.io/projected/7f100647-38cf-4583-9a1e-a9c960db9fd3-kube-api-access-h7c9w\") pod \"cert-manager-webhook-6486ffd997-p2prs\" (UID: \"7f100647-38cf-4583-9a1e-a9c960db9fd3\") " pod="cert-manager/cert-manager-webhook-6486ffd997-p2prs"
Sep 15 22:32:51.757850 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:51.757774    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7f100647-38cf-4583-9a1e-a9c960db9fd3-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-p2prs\" (UID: \"7f100647-38cf-4583-9a1e-a9c960db9fd3\") " pod="cert-manager/cert-manager-webhook-6486ffd997-p2prs"
Sep 15 22:32:51.858364 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:51.858310    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7f100647-38cf-4583-9a1e-a9c960db9fd3-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-p2prs\" (UID: \"7f100647-38cf-4583-9a1e-a9c960db9fd3\") " pod="cert-manager/cert-manager-webhook-6486ffd997-p2prs"
Sep 15 22:32:51.858538 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:51.858523    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7c9w\" (UniqueName: \"kubernetes.io/projected/7f100647-38cf-4583-9a1e-a9c960db9fd3-kube-api-access-h7c9w\") pod \"cert-manager-webhook-6486ffd997-p2prs\" (UID: \"7f100647-38cf-4583-9a1e-a9c960db9fd3\") " pod="cert-manager/cert-manager-webhook-6486ffd997-p2prs"
Sep 15 22:32:51.866936 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:51.866907    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7f100647-38cf-4583-9a1e-a9c960db9fd3-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-p2prs\" (UID: \"7f100647-38cf-4583-9a1e-a9c960db9fd3\") " pod="cert-manager/cert-manager-webhook-6486ffd997-p2prs"
Sep 15 22:32:51.867068 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:51.867010    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7c9w\" (UniqueName: \"kubernetes.io/projected/7f100647-38cf-4583-9a1e-a9c960db9fd3-kube-api-access-h7c9w\") pod \"cert-manager-webhook-6486ffd997-p2prs\" (UID: \"7f100647-38cf-4583-9a1e-a9c960db9fd3\") " pod="cert-manager/cert-manager-webhook-6486ffd997-p2prs"
Sep 15 22:32:51.910260 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:51.910171    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-thb2b" event={"ID":"1d364093-a365-44ee-b45c-6d17f9dc2dd1","Type":"ContainerStarted","Data":"c1f112fbb4f4cbce9f5524c676dcc1a985ccd8b93d7d61f64d846c7f4e971965"}
Sep 15 22:32:51.911964 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:51.911937    2565 generic.go:356] "Generic (PLEG): container finished" podID="b43ee5d8-abce-4741-8ca0-f73c695f2ee7" containerID="05f6fa638dee959508bfd7147ca234e8464b8fe0ad16e414dc345238b6bbd116" exitCode=0
Sep 15 22:32:51.912094 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:51.911979    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd84rv" event={"ID":"b43ee5d8-abce-4741-8ca0-f73c695f2ee7","Type":"ContainerDied","Data":"05f6fa638dee959508bfd7147ca234e8464b8fe0ad16e414dc345238b6bbd116"}
Sep 15 22:32:51.925478 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:51.925433    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-thb2b" podStartSLOduration=2.065345352 podStartE2EDuration="4.925417724s" totalImagesPullingTime="2.860072372s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:32:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:32:51.92367683 +0000 UTC m=+321.783388115" watchObservedRunningTime="2026-09-15 22:32:51.925417724 +0000 UTC m=+321.785129008"
Sep 15 22:32:51.957238 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:51.957215    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-p2prs"
Sep 15 22:32:52.239582 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:52.239559    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-p2prs"]
Sep 15 22:32:52.241858 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:32:52.241829    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7f100647_38cf_4583_9a1e_a9c960db9fd3.slice/crio-b9acefa23ee42663aecc044522785d80be77cc2065a50874063c8ac7c8ad1c87 WatchSource:0}: Error finding container b9acefa23ee42663aecc044522785d80be77cc2065a50874063c8ac7c8ad1c87: Status 404 returned error can't find the container with id b9acefa23ee42663aecc044522785d80be77cc2065a50874063c8ac7c8ad1c87
Sep 15 22:32:52.917737 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:52.917699    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-p2prs" event={"ID":"7f100647-38cf-4583-9a1e-a9c960db9fd3","Type":"ContainerStarted","Data":"33580136dd8700f479bbf5e1315e0b77a1eddb8d0bc952cfce0dcd2c20b28e8b"}
Sep 15 22:32:52.918184 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:52.917752    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-p2prs" event={"ID":"7f100647-38cf-4583-9a1e-a9c960db9fd3","Type":"ContainerStarted","Data":"b9acefa23ee42663aecc044522785d80be77cc2065a50874063c8ac7c8ad1c87"}
Sep 15 22:32:52.918184 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:52.917776    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-p2prs"
Sep 15 22:32:52.919755 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:52.919733    2565 generic.go:356] "Generic (PLEG): container finished" podID="b43ee5d8-abce-4741-8ca0-f73c695f2ee7" containerID="6fba91d262bb7eda1ee61b3909619797bb2aa78b5a206c11589a03494604708d" exitCode=0
Sep 15 22:32:52.919852 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:52.919792    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd84rv" event={"ID":"b43ee5d8-abce-4741-8ca0-f73c695f2ee7","Type":"ContainerDied","Data":"6fba91d262bb7eda1ee61b3909619797bb2aa78b5a206c11589a03494604708d"}
Sep 15 22:32:52.935277 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:52.935231    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-p2prs" podStartSLOduration=1.935216636 podStartE2EDuration="1.935216636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:32:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:32:52.934575441 +0000 UTC m=+322.794286725" watchObservedRunningTime="2026-09-15 22:32:52.935216636 +0000 UTC m=+322.794927920"
Sep 15 22:32:54.041662 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:54.041634    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd84rv"
Sep 15 22:32:54.179944 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:54.179860    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b43ee5d8-abce-4741-8ca0-f73c695f2ee7-util\") pod \"b43ee5d8-abce-4741-8ca0-f73c695f2ee7\" (UID: \"b43ee5d8-abce-4741-8ca0-f73c695f2ee7\") "
Sep 15 22:32:54.179944 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:54.179931    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b43ee5d8-abce-4741-8ca0-f73c695f2ee7-bundle\") pod \"b43ee5d8-abce-4741-8ca0-f73c695f2ee7\" (UID: \"b43ee5d8-abce-4741-8ca0-f73c695f2ee7\") "
Sep 15 22:32:54.180188 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:54.179953    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ww5ns\" (UniqueName: \"kubernetes.io/projected/b43ee5d8-abce-4741-8ca0-f73c695f2ee7-kube-api-access-ww5ns\") pod \"b43ee5d8-abce-4741-8ca0-f73c695f2ee7\" (UID: \"b43ee5d8-abce-4741-8ca0-f73c695f2ee7\") "
Sep 15 22:32:54.180349 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:54.180322    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b43ee5d8-abce-4741-8ca0-f73c695f2ee7-bundle" (OuterVolumeSpecName: "bundle") pod "b43ee5d8-abce-4741-8ca0-f73c695f2ee7" (UID: "b43ee5d8-abce-4741-8ca0-f73c695f2ee7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:32:54.182010 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:54.181987    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b43ee5d8-abce-4741-8ca0-f73c695f2ee7-kube-api-access-ww5ns" (OuterVolumeSpecName: "kube-api-access-ww5ns") pod "b43ee5d8-abce-4741-8ca0-f73c695f2ee7" (UID: "b43ee5d8-abce-4741-8ca0-f73c695f2ee7"). InnerVolumeSpecName "kube-api-access-ww5ns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:32:54.184552 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:54.184530    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b43ee5d8-abce-4741-8ca0-f73c695f2ee7-util" (OuterVolumeSpecName: "util") pod "b43ee5d8-abce-4741-8ca0-f73c695f2ee7" (UID: "b43ee5d8-abce-4741-8ca0-f73c695f2ee7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:32:54.280958 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:54.280928    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b43ee5d8-abce-4741-8ca0-f73c695f2ee7-util\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:32:54.280958 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:54.280952    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b43ee5d8-abce-4741-8ca0-f73c695f2ee7-bundle\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:32:54.280958 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:54.280963    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ww5ns\" (UniqueName: \"kubernetes.io/projected/b43ee5d8-abce-4741-8ca0-f73c695f2ee7-kube-api-access-ww5ns\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:32:54.928971 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:54.928871    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd84rv" event={"ID":"b43ee5d8-abce-4741-8ca0-f73c695f2ee7","Type":"ContainerDied","Data":"0ae48b3a67f1ee3334c502e29694ec900dc3c34d1c71aa7bb47ec56148930150"}
Sep 15 22:32:54.928971 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:54.928914    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd84rv"
Sep 15 22:32:54.928971 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:54.928939    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0ae48b3a67f1ee3334c502e29694ec900dc3c34d1c71aa7bb47ec56148930150"
Sep 15 22:32:58.925548 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:32:58.925520    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-p2prs"
Sep 15 22:33:05.893577 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:05.893540    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-xbttj"]
Sep 15 22:33:05.893940 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:05.893807    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b43ee5d8-abce-4741-8ca0-f73c695f2ee7" containerName="extract"
Sep 15 22:33:05.893940 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:05.893818    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b43ee5d8-abce-4741-8ca0-f73c695f2ee7" containerName="extract"
Sep 15 22:33:05.893940 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:05.893827    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b43ee5d8-abce-4741-8ca0-f73c695f2ee7" containerName="pull"
Sep 15 22:33:05.893940 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:05.893832    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b43ee5d8-abce-4741-8ca0-f73c695f2ee7" containerName="pull"
Sep 15 22:33:05.893940 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:05.893848    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b43ee5d8-abce-4741-8ca0-f73c695f2ee7" containerName="util"
Sep 15 22:33:05.893940 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:05.893853    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b43ee5d8-abce-4741-8ca0-f73c695f2ee7" containerName="util"
Sep 15 22:33:05.893940 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:05.893896    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b43ee5d8-abce-4741-8ca0-f73c695f2ee7" containerName="extract"
Sep 15 22:33:05.926059 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:05.926008    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-xbttj"]
Sep 15 22:33:05.926219 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:05.926152    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-xbttj"
Sep 15 22:33:05.928251 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:05.928230    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-mx54x\""
Sep 15 22:33:05.978856 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:05.976884    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/38092c89-df0d-451a-aed7-afd94059c88b-bound-sa-token\") pod \"cert-manager-55b4d954d5-xbttj\" (UID: \"38092c89-df0d-451a-aed7-afd94059c88b\") " pod="cert-manager/cert-manager-55b4d954d5-xbttj"
Sep 15 22:33:05.978856 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:05.976948    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhj7m\" (UniqueName: \"kubernetes.io/projected/38092c89-df0d-451a-aed7-afd94059c88b-kube-api-access-fhj7m\") pod \"cert-manager-55b4d954d5-xbttj\" (UID: \"38092c89-df0d-451a-aed7-afd94059c88b\") " pod="cert-manager/cert-manager-55b4d954d5-xbttj"
Sep 15 22:33:06.077926 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:06.077889    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/38092c89-df0d-451a-aed7-afd94059c88b-bound-sa-token\") pod \"cert-manager-55b4d954d5-xbttj\" (UID: \"38092c89-df0d-451a-aed7-afd94059c88b\") " pod="cert-manager/cert-manager-55b4d954d5-xbttj"
Sep 15 22:33:06.078154 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:06.077936    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhj7m\" (UniqueName: \"kubernetes.io/projected/38092c89-df0d-451a-aed7-afd94059c88b-kube-api-access-fhj7m\") pod \"cert-manager-55b4d954d5-xbttj\" (UID: \"38092c89-df0d-451a-aed7-afd94059c88b\") " pod="cert-manager/cert-manager-55b4d954d5-xbttj"
Sep 15 22:33:06.085626 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:06.085603    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/38092c89-df0d-451a-aed7-afd94059c88b-bound-sa-token\") pod \"cert-manager-55b4d954d5-xbttj\" (UID: \"38092c89-df0d-451a-aed7-afd94059c88b\") " pod="cert-manager/cert-manager-55b4d954d5-xbttj"
Sep 15 22:33:06.085738 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:06.085647    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhj7m\" (UniqueName: \"kubernetes.io/projected/38092c89-df0d-451a-aed7-afd94059c88b-kube-api-access-fhj7m\") pod \"cert-manager-55b4d954d5-xbttj\" (UID: \"38092c89-df0d-451a-aed7-afd94059c88b\") " pod="cert-manager/cert-manager-55b4d954d5-xbttj"
Sep 15 22:33:06.234557 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:06.234481    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-xbttj"
Sep 15 22:33:06.527800 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:06.527775    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-xbttj"]
Sep 15 22:33:06.530131 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:33:06.530097    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod38092c89_df0d_451a_aed7_afd94059c88b.slice/crio-2ff8f7b5442ebb1878b3cfdb6e8e4c2019e9fe98c11cf22d14e427e93e8990d6 WatchSource:0}: Error finding container 2ff8f7b5442ebb1878b3cfdb6e8e4c2019e9fe98c11cf22d14e427e93e8990d6: Status 404 returned error can't find the container with id 2ff8f7b5442ebb1878b3cfdb6e8e4c2019e9fe98c11cf22d14e427e93e8990d6
Sep 15 22:33:06.981658 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:06.981625    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-xbttj" event={"ID":"38092c89-df0d-451a-aed7-afd94059c88b","Type":"ContainerStarted","Data":"226c047585a9fbad4774f9dc9d5d5c5f0091c2e7ef7b7e06972119a3794357e3"}
Sep 15 22:33:06.981658 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:06.981666    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-xbttj" event={"ID":"38092c89-df0d-451a-aed7-afd94059c88b","Type":"ContainerStarted","Data":"2ff8f7b5442ebb1878b3cfdb6e8e4c2019e9fe98c11cf22d14e427e93e8990d6"}
Sep 15 22:33:06.998899 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:06.998848    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-xbttj" podStartSLOduration=1.998832219 podStartE2EDuration="1.998832219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:33:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:33:06.997681955 +0000 UTC m=+336.857393241" watchObservedRunningTime="2026-09-15 22:33:06.998832219 +0000 UTC m=+336.858543503"
Sep 15 22:33:10.858739 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:10.858707    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm8t8h"]
Sep 15 22:33:10.860984 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:10.860968    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm8t8h"
Sep 15 22:33:10.862964 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:10.862943    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 15 22:33:10.863097 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:10.863020    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-68lxg\""
Sep 15 22:33:10.863450 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:10.863437    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 15 22:33:10.871039 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:10.870992    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm8t8h"]
Sep 15 22:33:10.918234 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:10.918197    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d05554b-312b-47f7-9664-dba154b81009-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm8t8h\" (UID: \"4d05554b-312b-47f7-9664-dba154b81009\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm8t8h"
Sep 15 22:33:10.918383 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:10.918276    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2rn2\" (UniqueName: \"kubernetes.io/projected/4d05554b-312b-47f7-9664-dba154b81009-kube-api-access-z2rn2\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm8t8h\" (UID: \"4d05554b-312b-47f7-9664-dba154b81009\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm8t8h"
Sep 15 22:33:10.918383 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:10.918310    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d05554b-312b-47f7-9664-dba154b81009-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm8t8h\" (UID: \"4d05554b-312b-47f7-9664-dba154b81009\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm8t8h"
Sep 15 22:33:11.019382 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:11.019352    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d05554b-312b-47f7-9664-dba154b81009-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm8t8h\" (UID: \"4d05554b-312b-47f7-9664-dba154b81009\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm8t8h"
Sep 15 22:33:11.019543 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:11.019430    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2rn2\" (UniqueName: \"kubernetes.io/projected/4d05554b-312b-47f7-9664-dba154b81009-kube-api-access-z2rn2\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm8t8h\" (UID: \"4d05554b-312b-47f7-9664-dba154b81009\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm8t8h"
Sep 15 22:33:11.019543 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:11.019482    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d05554b-312b-47f7-9664-dba154b81009-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm8t8h\" (UID: \"4d05554b-312b-47f7-9664-dba154b81009\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm8t8h"
Sep 15 22:33:11.019844 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:11.019822    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d05554b-312b-47f7-9664-dba154b81009-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm8t8h\" (UID: \"4d05554b-312b-47f7-9664-dba154b81009\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm8t8h"
Sep 15 22:33:11.019888 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:11.019831    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d05554b-312b-47f7-9664-dba154b81009-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm8t8h\" (UID: \"4d05554b-312b-47f7-9664-dba154b81009\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm8t8h"
Sep 15 22:33:11.031118 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:11.031089    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2rn2\" (UniqueName: \"kubernetes.io/projected/4d05554b-312b-47f7-9664-dba154b81009-kube-api-access-z2rn2\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm8t8h\" (UID: \"4d05554b-312b-47f7-9664-dba154b81009\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm8t8h"
Sep 15 22:33:11.170854 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:11.170786    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm8t8h"
Sep 15 22:33:11.444980 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:11.444767    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm8t8h"]
Sep 15 22:33:11.447524 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:33:11.447497    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d05554b_312b_47f7_9664_dba154b81009.slice/crio-18677820598ef4dc1114bd2273bfb299634997b8353e8ec743d4465edaf95baa WatchSource:0}: Error finding container 18677820598ef4dc1114bd2273bfb299634997b8353e8ec743d4465edaf95baa: Status 404 returned error can't find the container with id 18677820598ef4dc1114bd2273bfb299634997b8353e8ec743d4465edaf95baa
Sep 15 22:33:12.004408 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:12.004371    2565 generic.go:356] "Generic (PLEG): container finished" podID="4d05554b-312b-47f7-9664-dba154b81009" containerID="f43ade1ebbf0ce9eb41bd839f9a8dabe569bc756a4c4134ce1d022487c3d11f7" exitCode=0
Sep 15 22:33:12.004806 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:12.004456    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm8t8h" event={"ID":"4d05554b-312b-47f7-9664-dba154b81009","Type":"ContainerDied","Data":"f43ade1ebbf0ce9eb41bd839f9a8dabe569bc756a4c4134ce1d022487c3d11f7"}
Sep 15 22:33:12.004806 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:12.004506    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm8t8h" event={"ID":"4d05554b-312b-47f7-9664-dba154b81009","Type":"ContainerStarted","Data":"18677820598ef4dc1114bd2273bfb299634997b8353e8ec743d4465edaf95baa"}
Sep 15 22:33:13.010699 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:13.010612    2565 generic.go:356] "Generic (PLEG): container finished" podID="4d05554b-312b-47f7-9664-dba154b81009" containerID="b9c669ff5eaf08a8b1f24080728cfd5a248e2cfed2b404a2635b688595a114a4" exitCode=0
Sep 15 22:33:13.010699 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:13.010664    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm8t8h" event={"ID":"4d05554b-312b-47f7-9664-dba154b81009","Type":"ContainerDied","Data":"b9c669ff5eaf08a8b1f24080728cfd5a248e2cfed2b404a2635b688595a114a4"}
Sep 15 22:33:14.017424 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:14.017388    2565 generic.go:356] "Generic (PLEG): container finished" podID="4d05554b-312b-47f7-9664-dba154b81009" containerID="15f308c1a04f0da0b2eb81ccbdc23e6adb00c555322067dfda9021540f3db981" exitCode=0
Sep 15 22:33:14.017798 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:14.017466    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm8t8h" event={"ID":"4d05554b-312b-47f7-9664-dba154b81009","Type":"ContainerDied","Data":"15f308c1a04f0da0b2eb81ccbdc23e6adb00c555322067dfda9021540f3db981"}
Sep 15 22:33:15.150618 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:15.150596    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm8t8h"
Sep 15 22:33:15.255561 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:15.255511    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z2rn2\" (UniqueName: \"kubernetes.io/projected/4d05554b-312b-47f7-9664-dba154b81009-kube-api-access-z2rn2\") pod \"4d05554b-312b-47f7-9664-dba154b81009\" (UID: \"4d05554b-312b-47f7-9664-dba154b81009\") "
Sep 15 22:33:15.255761 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:15.255593    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d05554b-312b-47f7-9664-dba154b81009-util\") pod \"4d05554b-312b-47f7-9664-dba154b81009\" (UID: \"4d05554b-312b-47f7-9664-dba154b81009\") "
Sep 15 22:33:15.255761 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:15.255657    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d05554b-312b-47f7-9664-dba154b81009-bundle\") pod \"4d05554b-312b-47f7-9664-dba154b81009\" (UID: \"4d05554b-312b-47f7-9664-dba154b81009\") "
Sep 15 22:33:15.256366 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:15.256340    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d05554b-312b-47f7-9664-dba154b81009-bundle" (OuterVolumeSpecName: "bundle") pod "4d05554b-312b-47f7-9664-dba154b81009" (UID: "4d05554b-312b-47f7-9664-dba154b81009"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:33:15.257569 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:15.257547    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d05554b-312b-47f7-9664-dba154b81009-kube-api-access-z2rn2" (OuterVolumeSpecName: "kube-api-access-z2rn2") pod "4d05554b-312b-47f7-9664-dba154b81009" (UID: "4d05554b-312b-47f7-9664-dba154b81009"). InnerVolumeSpecName "kube-api-access-z2rn2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:33:15.261279 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:15.261257    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d05554b-312b-47f7-9664-dba154b81009-util" (OuterVolumeSpecName: "util") pod "4d05554b-312b-47f7-9664-dba154b81009" (UID: "4d05554b-312b-47f7-9664-dba154b81009"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:33:15.356340 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:15.356307    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2rn2\" (UniqueName: \"kubernetes.io/projected/4d05554b-312b-47f7-9664-dba154b81009-kube-api-access-z2rn2\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:33:15.356340 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:15.356333    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d05554b-312b-47f7-9664-dba154b81009-util\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:33:15.356340 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:15.356342    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d05554b-312b-47f7-9664-dba154b81009-bundle\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:33:16.031853 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:16.031795    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm8t8h" event={"ID":"4d05554b-312b-47f7-9664-dba154b81009","Type":"ContainerDied","Data":"18677820598ef4dc1114bd2273bfb299634997b8353e8ec743d4465edaf95baa"}
Sep 15 22:33:16.032038 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:16.031889    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="18677820598ef4dc1114bd2273bfb299634997b8353e8ec743d4465edaf95baa"
Sep 15 22:33:16.032038 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:16.031820    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afm8t8h"
Sep 15 22:33:21.058100 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:21.058057    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0spn65"]
Sep 15 22:33:21.058464 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:21.058326    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d05554b-312b-47f7-9664-dba154b81009" containerName="extract"
Sep 15 22:33:21.058464 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:21.058335    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d05554b-312b-47f7-9664-dba154b81009" containerName="extract"
Sep 15 22:33:21.058464 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:21.058350    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d05554b-312b-47f7-9664-dba154b81009" containerName="util"
Sep 15 22:33:21.058464 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:21.058356    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d05554b-312b-47f7-9664-dba154b81009" containerName="util"
Sep 15 22:33:21.058464 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:21.058362    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d05554b-312b-47f7-9664-dba154b81009" containerName="pull"
Sep 15 22:33:21.058464 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:21.058367    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d05554b-312b-47f7-9664-dba154b81009" containerName="pull"
Sep 15 22:33:21.058464 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:21.058413    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d05554b-312b-47f7-9664-dba154b81009" containerName="extract"
Sep 15 22:33:21.061062 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:21.061046    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0spn65"
Sep 15 22:33:21.063270 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:21.063243    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 15 22:33:21.063270 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:21.063246    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-68lxg\""
Sep 15 22:33:21.063571 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:21.063533    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 15 22:33:21.070539 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:21.070519    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0spn65"]
Sep 15 22:33:21.103318 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:21.103284    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/038ae0d6-9915-4e94-9d7d-f273a95fff48-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0spn65\" (UID: \"038ae0d6-9915-4e94-9d7d-f273a95fff48\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0spn65"
Sep 15 22:33:21.103488 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:21.103333    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/038ae0d6-9915-4e94-9d7d-f273a95fff48-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0spn65\" (UID: \"038ae0d6-9915-4e94-9d7d-f273a95fff48\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0spn65"
Sep 15 22:33:21.103488 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:21.103355    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-228pb\" (UniqueName: \"kubernetes.io/projected/038ae0d6-9915-4e94-9d7d-f273a95fff48-kube-api-access-228pb\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0spn65\" (UID: \"038ae0d6-9915-4e94-9d7d-f273a95fff48\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0spn65"
Sep 15 22:33:21.204040 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:21.204004    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/038ae0d6-9915-4e94-9d7d-f273a95fff48-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0spn65\" (UID: \"038ae0d6-9915-4e94-9d7d-f273a95fff48\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0spn65"
Sep 15 22:33:21.204184 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:21.204081    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/038ae0d6-9915-4e94-9d7d-f273a95fff48-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0spn65\" (UID: \"038ae0d6-9915-4e94-9d7d-f273a95fff48\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0spn65"
Sep 15 22:33:21.204184 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:21.204111    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-228pb\" (UniqueName: \"kubernetes.io/projected/038ae0d6-9915-4e94-9d7d-f273a95fff48-kube-api-access-228pb\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0spn65\" (UID: \"038ae0d6-9915-4e94-9d7d-f273a95fff48\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0spn65"
Sep 15 22:33:21.204452 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:21.204424    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/038ae0d6-9915-4e94-9d7d-f273a95fff48-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0spn65\" (UID: \"038ae0d6-9915-4e94-9d7d-f273a95fff48\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0spn65"
Sep 15 22:33:21.204452 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:21.204446    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/038ae0d6-9915-4e94-9d7d-f273a95fff48-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0spn65\" (UID: \"038ae0d6-9915-4e94-9d7d-f273a95fff48\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0spn65"
Sep 15 22:33:21.212704 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:21.212675    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-228pb\" (UniqueName: \"kubernetes.io/projected/038ae0d6-9915-4e94-9d7d-f273a95fff48-kube-api-access-228pb\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0spn65\" (UID: \"038ae0d6-9915-4e94-9d7d-f273a95fff48\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0spn65"
Sep 15 22:33:21.370692 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:21.370619    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0spn65"
Sep 15 22:33:21.655192 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:21.655158    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0spn65"]
Sep 15 22:33:21.657356 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:33:21.657332    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod038ae0d6_9915_4e94_9d7d_f273a95fff48.slice/crio-2cbe6c3e024fed6f0288951233a0ee264aed1d3acb1a6b562256c7891fb57f33 WatchSource:0}: Error finding container 2cbe6c3e024fed6f0288951233a0ee264aed1d3acb1a6b562256c7891fb57f33: Status 404 returned error can't find the container with id 2cbe6c3e024fed6f0288951233a0ee264aed1d3acb1a6b562256c7891fb57f33
Sep 15 22:33:22.058585 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:22.058553    2565 generic.go:356] "Generic (PLEG): container finished" podID="038ae0d6-9915-4e94-9d7d-f273a95fff48" containerID="708ac72719dae410a4500927314850ee011b32223e0c439cfd226599a4937ee8" exitCode=0
Sep 15 22:33:22.059018 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:22.058592    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0spn65" event={"ID":"038ae0d6-9915-4e94-9d7d-f273a95fff48","Type":"ContainerDied","Data":"708ac72719dae410a4500927314850ee011b32223e0c439cfd226599a4937ee8"}
Sep 15 22:33:22.059018 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:22.058626    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0spn65" event={"ID":"038ae0d6-9915-4e94-9d7d-f273a95fff48","Type":"ContainerStarted","Data":"2cbe6c3e024fed6f0288951233a0ee264aed1d3acb1a6b562256c7891fb57f33"}
Sep 15 22:33:23.065077 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:23.065043    2565 generic.go:356] "Generic (PLEG): container finished" podID="038ae0d6-9915-4e94-9d7d-f273a95fff48" containerID="238808058d68374209324115c7c6d9e1b5b67c0a70f212162ec4c4102516f277" exitCode=0
Sep 15 22:33:23.065440 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:23.065113    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0spn65" event={"ID":"038ae0d6-9915-4e94-9d7d-f273a95fff48","Type":"ContainerDied","Data":"238808058d68374209324115c7c6d9e1b5b67c0a70f212162ec4c4102516f277"}
Sep 15 22:33:23.899503 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:23.899453    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-749d6986f8-lhz48"]
Sep 15 22:33:23.910092 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:23.910067    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-lhz48"
Sep 15 22:33:23.912175 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:23.912148    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 15 22:33:23.912309 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:23.912180    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 15 22:33:23.912309 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:23.912209    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 15 22:33:23.912309 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:23.912261    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 15 22:33:23.912478 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:23.912325    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-hh8gz\""
Sep 15 22:33:23.923700 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:23.923635    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-749d6986f8-lhz48"]
Sep 15 22:33:24.028879 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:24.028840    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0b727e71-070d-493c-9a92-cbf87f151291-apiservice-cert\") pod \"opendatahub-operator-controller-manager-749d6986f8-lhz48\" (UID: \"0b727e71-070d-493c-9a92-cbf87f151291\") " pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-lhz48"
Sep 15 22:33:24.029073 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:24.028889    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0b727e71-070d-493c-9a92-cbf87f151291-manifests\") pod \"opendatahub-operator-controller-manager-749d6986f8-lhz48\" (UID: \"0b727e71-070d-493c-9a92-cbf87f151291\") " pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-lhz48"
Sep 15 22:33:24.029073 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:24.028941    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b727e71-070d-493c-9a92-cbf87f151291-tmp\") pod \"opendatahub-operator-controller-manager-749d6986f8-lhz48\" (UID: \"0b727e71-070d-493c-9a92-cbf87f151291\") " pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-lhz48"
Sep 15 22:33:24.029175 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:24.029067    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0b727e71-070d-493c-9a92-cbf87f151291-webhook-cert\") pod \"opendatahub-operator-controller-manager-749d6986f8-lhz48\" (UID: \"0b727e71-070d-493c-9a92-cbf87f151291\") " pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-lhz48"
Sep 15 22:33:24.029175 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:24.029094    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrcql\" (UniqueName: \"kubernetes.io/projected/0b727e71-070d-493c-9a92-cbf87f151291-kube-api-access-hrcql\") pod \"opendatahub-operator-controller-manager-749d6986f8-lhz48\" (UID: \"0b727e71-070d-493c-9a92-cbf87f151291\") " pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-lhz48"
Sep 15 22:33:24.072109 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:24.072074    2565 generic.go:356] "Generic (PLEG): container finished" podID="038ae0d6-9915-4e94-9d7d-f273a95fff48" containerID="fe662b3de279208fa0be39192dcb1f36b2978dd816562d2617e46a17a99327ad" exitCode=0
Sep 15 22:33:24.072447 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:24.072122    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0spn65" event={"ID":"038ae0d6-9915-4e94-9d7d-f273a95fff48","Type":"ContainerDied","Data":"fe662b3de279208fa0be39192dcb1f36b2978dd816562d2617e46a17a99327ad"}
Sep 15 22:33:24.130041 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:24.129995    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0b727e71-070d-493c-9a92-cbf87f151291-webhook-cert\") pod \"opendatahub-operator-controller-manager-749d6986f8-lhz48\" (UID: \"0b727e71-070d-493c-9a92-cbf87f151291\") " pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-lhz48"
Sep 15 22:33:24.130228 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:24.130051    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrcql\" (UniqueName: \"kubernetes.io/projected/0b727e71-070d-493c-9a92-cbf87f151291-kube-api-access-hrcql\") pod \"opendatahub-operator-controller-manager-749d6986f8-lhz48\" (UID: \"0b727e71-070d-493c-9a92-cbf87f151291\") " pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-lhz48"
Sep 15 22:33:24.130228 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:24.130082    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0b727e71-070d-493c-9a92-cbf87f151291-apiservice-cert\") pod \"opendatahub-operator-controller-manager-749d6986f8-lhz48\" (UID: \"0b727e71-070d-493c-9a92-cbf87f151291\") " pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-lhz48"
Sep 15 22:33:24.130228 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:24.130106    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0b727e71-070d-493c-9a92-cbf87f151291-manifests\") pod \"opendatahub-operator-controller-manager-749d6986f8-lhz48\" (UID: \"0b727e71-070d-493c-9a92-cbf87f151291\") " pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-lhz48"
Sep 15 22:33:24.130228 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:24.130132    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b727e71-070d-493c-9a92-cbf87f151291-tmp\") pod \"opendatahub-operator-controller-manager-749d6986f8-lhz48\" (UID: \"0b727e71-070d-493c-9a92-cbf87f151291\") " pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-lhz48"
Sep 15 22:33:24.130592 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:24.130565    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0b727e71-070d-493c-9a92-cbf87f151291-manifests\") pod \"opendatahub-operator-controller-manager-749d6986f8-lhz48\" (UID: \"0b727e71-070d-493c-9a92-cbf87f151291\") " pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-lhz48"
Sep 15 22:33:24.132318 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:24.132299    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b727e71-070d-493c-9a92-cbf87f151291-tmp\") pod \"opendatahub-operator-controller-manager-749d6986f8-lhz48\" (UID: \"0b727e71-070d-493c-9a92-cbf87f151291\") " pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-lhz48"
Sep 15 22:33:24.132535 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:24.132513    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0b727e71-070d-493c-9a92-cbf87f151291-webhook-cert\") pod \"opendatahub-operator-controller-manager-749d6986f8-lhz48\" (UID: \"0b727e71-070d-493c-9a92-cbf87f151291\") " pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-lhz48"
Sep 15 22:33:24.132681 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:24.132661    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0b727e71-070d-493c-9a92-cbf87f151291-apiservice-cert\") pod \"opendatahub-operator-controller-manager-749d6986f8-lhz48\" (UID: \"0b727e71-070d-493c-9a92-cbf87f151291\") " pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-lhz48"
Sep 15 22:33:24.138422 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:24.138392    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrcql\" (UniqueName: \"kubernetes.io/projected/0b727e71-070d-493c-9a92-cbf87f151291-kube-api-access-hrcql\") pod \"opendatahub-operator-controller-manager-749d6986f8-lhz48\" (UID: \"0b727e71-070d-493c-9a92-cbf87f151291\") " pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-lhz48"
Sep 15 22:33:24.220177 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:24.220104    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-lhz48"
Sep 15 22:33:24.521136 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:24.521055    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-749d6986f8-lhz48"]
Sep 15 22:33:24.524168 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:33:24.524137    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b727e71_070d_493c_9a92_cbf87f151291.slice/crio-80f5c400a82ed25f083233edbb8ded06e5bf3d232f2a8aaa2e60b6421e5710e1 WatchSource:0}: Error finding container 80f5c400a82ed25f083233edbb8ded06e5bf3d232f2a8aaa2e60b6421e5710e1: Status 404 returned error can't find the container with id 80f5c400a82ed25f083233edbb8ded06e5bf3d232f2a8aaa2e60b6421e5710e1
Sep 15 22:33:25.080447 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:25.080367    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-lhz48" event={"ID":"0b727e71-070d-493c-9a92-cbf87f151291","Type":"ContainerStarted","Data":"80f5c400a82ed25f083233edbb8ded06e5bf3d232f2a8aaa2e60b6421e5710e1"}
Sep 15 22:33:25.238606 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:25.233406    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0spn65"
Sep 15 22:33:25.340515 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:25.340425    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-228pb\" (UniqueName: \"kubernetes.io/projected/038ae0d6-9915-4e94-9d7d-f273a95fff48-kube-api-access-228pb\") pod \"038ae0d6-9915-4e94-9d7d-f273a95fff48\" (UID: \"038ae0d6-9915-4e94-9d7d-f273a95fff48\") "
Sep 15 22:33:25.340690 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:25.340550    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/038ae0d6-9915-4e94-9d7d-f273a95fff48-util\") pod \"038ae0d6-9915-4e94-9d7d-f273a95fff48\" (UID: \"038ae0d6-9915-4e94-9d7d-f273a95fff48\") "
Sep 15 22:33:25.340690 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:25.340577    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/038ae0d6-9915-4e94-9d7d-f273a95fff48-bundle\") pod \"038ae0d6-9915-4e94-9d7d-f273a95fff48\" (UID: \"038ae0d6-9915-4e94-9d7d-f273a95fff48\") "
Sep 15 22:33:25.341684 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:25.341641    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/038ae0d6-9915-4e94-9d7d-f273a95fff48-bundle" (OuterVolumeSpecName: "bundle") pod "038ae0d6-9915-4e94-9d7d-f273a95fff48" (UID: "038ae0d6-9915-4e94-9d7d-f273a95fff48"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:33:25.345511 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:25.345481    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/038ae0d6-9915-4e94-9d7d-f273a95fff48-kube-api-access-228pb" (OuterVolumeSpecName: "kube-api-access-228pb") pod "038ae0d6-9915-4e94-9d7d-f273a95fff48" (UID: "038ae0d6-9915-4e94-9d7d-f273a95fff48"). InnerVolumeSpecName "kube-api-access-228pb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:33:25.349635 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:25.349605    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/038ae0d6-9915-4e94-9d7d-f273a95fff48-util" (OuterVolumeSpecName: "util") pod "038ae0d6-9915-4e94-9d7d-f273a95fff48" (UID: "038ae0d6-9915-4e94-9d7d-f273a95fff48"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:33:25.441370 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:25.441306    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/038ae0d6-9915-4e94-9d7d-f273a95fff48-util\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:33:25.441370 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:25.441335    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/038ae0d6-9915-4e94-9d7d-f273a95fff48-bundle\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:33:25.441370 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:25.441345    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-228pb\" (UniqueName: \"kubernetes.io/projected/038ae0d6-9915-4e94-9d7d-f273a95fff48-kube-api-access-228pb\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:33:26.089628 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:26.089593    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0spn65" event={"ID":"038ae0d6-9915-4e94-9d7d-f273a95fff48","Type":"ContainerDied","Data":"2cbe6c3e024fed6f0288951233a0ee264aed1d3acb1a6b562256c7891fb57f33"}
Sep 15 22:33:26.089628 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:26.089619    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0spn65"
Sep 15 22:33:26.090252 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:26.089643    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2cbe6c3e024fed6f0288951233a0ee264aed1d3acb1a6b562256c7891fb57f33"
Sep 15 22:33:28.104289 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:28.104252    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-lhz48" event={"ID":"0b727e71-070d-493c-9a92-cbf87f151291","Type":"ContainerStarted","Data":"6012c2c89bb06b466eee89f36c88dbff70fd436d5ac87dca0e9ca30b41c7492c"}
Sep 15 22:33:29.110921 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:29.110880    2565 generic.go:356] "Generic (PLEG): container finished" podID="0b727e71-070d-493c-9a92-cbf87f151291" containerID="6012c2c89bb06b466eee89f36c88dbff70fd436d5ac87dca0e9ca30b41c7492c" exitCode=0
Sep 15 22:33:29.111402 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:29.110960    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-lhz48" event={"ID":"0b727e71-070d-493c-9a92-cbf87f151291","Type":"ContainerDied","Data":"6012c2c89bb06b466eee89f36c88dbff70fd436d5ac87dca0e9ca30b41c7492c"}
Sep 15 22:33:30.117505 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:30.117469    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-lhz48" event={"ID":"0b727e71-070d-493c-9a92-cbf87f151291","Type":"ContainerStarted","Data":"702c5c654db82be4116dfd00559650ce461f80aba00203cfff7ee72ad4af361f"}
Sep 15 22:33:30.117987 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:30.117646    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-lhz48"
Sep 15 22:33:30.138474 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:30.138433    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-lhz48" podStartSLOduration=2.785504965 podStartE2EDuration="7.138420658s" totalImagesPullingTime="3.295123355s" totalInitContainerRuntime="1.057792338s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:33:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:33:30.137690285 +0000 UTC m=+359.997401568" watchObservedRunningTime="2026-09-15 22:33:30.138420658 +0000 UTC m=+359.998131942"
Sep 15 22:33:38.999345 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:38.999309    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-588b46b67f-jdbnq"]
Sep 15 22:33:38.999824 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:38.999604    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="038ae0d6-9915-4e94-9d7d-f273a95fff48" containerName="extract"
Sep 15 22:33:38.999824 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:38.999616    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="038ae0d6-9915-4e94-9d7d-f273a95fff48" containerName="extract"
Sep 15 22:33:38.999824 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:38.999627    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="038ae0d6-9915-4e94-9d7d-f273a95fff48" containerName="pull"
Sep 15 22:33:38.999824 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:38.999632    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="038ae0d6-9915-4e94-9d7d-f273a95fff48" containerName="pull"
Sep 15 22:33:38.999824 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:38.999645    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="038ae0d6-9915-4e94-9d7d-f273a95fff48" containerName="util"
Sep 15 22:33:38.999824 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:38.999651    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="038ae0d6-9915-4e94-9d7d-f273a95fff48" containerName="util"
Sep 15 22:33:38.999824 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:38.999703    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="038ae0d6-9915-4e94-9d7d-f273a95fff48" containerName="extract"
Sep 15 22:33:39.007964 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:39.007937    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-588b46b67f-jdbnq"
Sep 15 22:33:39.014368 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:39.014346    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-588b46b67f-jdbnq"]
Sep 15 22:33:39.169331 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:39.169305    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zl9pc\" (UniqueName: \"kubernetes.io/projected/b90747ad-4dbe-4fb1-84b4-a627e8083222-kube-api-access-zl9pc\") pod \"console-588b46b67f-jdbnq\" (UID: \"b90747ad-4dbe-4fb1-84b4-a627e8083222\") " pod="openshift-console/console-588b46b67f-jdbnq"
Sep 15 22:33:39.169507 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:39.169342    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b90747ad-4dbe-4fb1-84b4-a627e8083222-console-oauth-config\") pod \"console-588b46b67f-jdbnq\" (UID: \"b90747ad-4dbe-4fb1-84b4-a627e8083222\") " pod="openshift-console/console-588b46b67f-jdbnq"
Sep 15 22:33:39.169507 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:39.169359    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b90747ad-4dbe-4fb1-84b4-a627e8083222-service-ca\") pod \"console-588b46b67f-jdbnq\" (UID: \"b90747ad-4dbe-4fb1-84b4-a627e8083222\") " pod="openshift-console/console-588b46b67f-jdbnq"
Sep 15 22:33:39.169507 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:39.169406    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b90747ad-4dbe-4fb1-84b4-a627e8083222-console-serving-cert\") pod \"console-588b46b67f-jdbnq\" (UID: \"b90747ad-4dbe-4fb1-84b4-a627e8083222\") " pod="openshift-console/console-588b46b67f-jdbnq"
Sep 15 22:33:39.169507 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:39.169442    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b90747ad-4dbe-4fb1-84b4-a627e8083222-console-config\") pod \"console-588b46b67f-jdbnq\" (UID: \"b90747ad-4dbe-4fb1-84b4-a627e8083222\") " pod="openshift-console/console-588b46b67f-jdbnq"
Sep 15 22:33:39.169507 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:39.169460    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b90747ad-4dbe-4fb1-84b4-a627e8083222-trusted-ca-bundle\") pod \"console-588b46b67f-jdbnq\" (UID: \"b90747ad-4dbe-4fb1-84b4-a627e8083222\") " pod="openshift-console/console-588b46b67f-jdbnq"
Sep 15 22:33:39.169507 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:39.169497    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b90747ad-4dbe-4fb1-84b4-a627e8083222-oauth-serving-cert\") pod \"console-588b46b67f-jdbnq\" (UID: \"b90747ad-4dbe-4fb1-84b4-a627e8083222\") " pod="openshift-console/console-588b46b67f-jdbnq"
Sep 15 22:33:39.270570 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:39.270500    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zl9pc\" (UniqueName: \"kubernetes.io/projected/b90747ad-4dbe-4fb1-84b4-a627e8083222-kube-api-access-zl9pc\") pod \"console-588b46b67f-jdbnq\" (UID: \"b90747ad-4dbe-4fb1-84b4-a627e8083222\") " pod="openshift-console/console-588b46b67f-jdbnq"
Sep 15 22:33:39.270570 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:39.270539    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b90747ad-4dbe-4fb1-84b4-a627e8083222-console-oauth-config\") pod \"console-588b46b67f-jdbnq\" (UID: \"b90747ad-4dbe-4fb1-84b4-a627e8083222\") " pod="openshift-console/console-588b46b67f-jdbnq"
Sep 15 22:33:39.270570 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:39.270553    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b90747ad-4dbe-4fb1-84b4-a627e8083222-service-ca\") pod \"console-588b46b67f-jdbnq\" (UID: \"b90747ad-4dbe-4fb1-84b4-a627e8083222\") " pod="openshift-console/console-588b46b67f-jdbnq"
Sep 15 22:33:39.270763 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:39.270582    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b90747ad-4dbe-4fb1-84b4-a627e8083222-console-serving-cert\") pod \"console-588b46b67f-jdbnq\" (UID: \"b90747ad-4dbe-4fb1-84b4-a627e8083222\") " pod="openshift-console/console-588b46b67f-jdbnq"
Sep 15 22:33:39.270763 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:39.270612    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b90747ad-4dbe-4fb1-84b4-a627e8083222-console-config\") pod \"console-588b46b67f-jdbnq\" (UID: \"b90747ad-4dbe-4fb1-84b4-a627e8083222\") " pod="openshift-console/console-588b46b67f-jdbnq"
Sep 15 22:33:39.270763 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:39.270635    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b90747ad-4dbe-4fb1-84b4-a627e8083222-trusted-ca-bundle\") pod \"console-588b46b67f-jdbnq\" (UID: \"b90747ad-4dbe-4fb1-84b4-a627e8083222\") " pod="openshift-console/console-588b46b67f-jdbnq"
Sep 15 22:33:39.270763 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:39.270656    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b90747ad-4dbe-4fb1-84b4-a627e8083222-oauth-serving-cert\") pod \"console-588b46b67f-jdbnq\" (UID: \"b90747ad-4dbe-4fb1-84b4-a627e8083222\") " pod="openshift-console/console-588b46b67f-jdbnq"
Sep 15 22:33:39.271429 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:39.271407    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b90747ad-4dbe-4fb1-84b4-a627e8083222-console-config\") pod \"console-588b46b67f-jdbnq\" (UID: \"b90747ad-4dbe-4fb1-84b4-a627e8083222\") " pod="openshift-console/console-588b46b67f-jdbnq"
Sep 15 22:33:39.271546 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:39.271442    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b90747ad-4dbe-4fb1-84b4-a627e8083222-oauth-serving-cert\") pod \"console-588b46b67f-jdbnq\" (UID: \"b90747ad-4dbe-4fb1-84b4-a627e8083222\") " pod="openshift-console/console-588b46b67f-jdbnq"
Sep 15 22:33:39.271613 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:39.271578    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b90747ad-4dbe-4fb1-84b4-a627e8083222-service-ca\") pod \"console-588b46b67f-jdbnq\" (UID: \"b90747ad-4dbe-4fb1-84b4-a627e8083222\") " pod="openshift-console/console-588b46b67f-jdbnq"
Sep 15 22:33:39.271805 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:39.271783    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b90747ad-4dbe-4fb1-84b4-a627e8083222-trusted-ca-bundle\") pod \"console-588b46b67f-jdbnq\" (UID: \"b90747ad-4dbe-4fb1-84b4-a627e8083222\") " pod="openshift-console/console-588b46b67f-jdbnq"
Sep 15 22:33:39.273205 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:39.273186    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b90747ad-4dbe-4fb1-84b4-a627e8083222-console-serving-cert\") pod \"console-588b46b67f-jdbnq\" (UID: \"b90747ad-4dbe-4fb1-84b4-a627e8083222\") " pod="openshift-console/console-588b46b67f-jdbnq"
Sep 15 22:33:39.273302 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:39.273282    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b90747ad-4dbe-4fb1-84b4-a627e8083222-console-oauth-config\") pod \"console-588b46b67f-jdbnq\" (UID: \"b90747ad-4dbe-4fb1-84b4-a627e8083222\") " pod="openshift-console/console-588b46b67f-jdbnq"
Sep 15 22:33:39.280078 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:39.280061    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zl9pc\" (UniqueName: \"kubernetes.io/projected/b90747ad-4dbe-4fb1-84b4-a627e8083222-kube-api-access-zl9pc\") pod \"console-588b46b67f-jdbnq\" (UID: \"b90747ad-4dbe-4fb1-84b4-a627e8083222\") " pod="openshift-console/console-588b46b67f-jdbnq"
Sep 15 22:33:39.317512 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:39.317486    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-588b46b67f-jdbnq"
Sep 15 22:33:39.596575 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:39.596543    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-588b46b67f-jdbnq"]
Sep 15 22:33:39.598387 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:33:39.598363    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb90747ad_4dbe_4fb1_84b4_a627e8083222.slice/crio-ea1deccaabc6418c4355f6f391ea55bb9bc48b72d2ae366b1f7aa5f6fb35b1a3 WatchSource:0}: Error finding container ea1deccaabc6418c4355f6f391ea55bb9bc48b72d2ae366b1f7aa5f6fb35b1a3: Status 404 returned error can't find the container with id ea1deccaabc6418c4355f6f391ea55bb9bc48b72d2ae366b1f7aa5f6fb35b1a3
Sep 15 22:33:40.162748 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:40.162716    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-588b46b67f-jdbnq" event={"ID":"b90747ad-4dbe-4fb1-84b4-a627e8083222","Type":"ContainerStarted","Data":"703197bcb582172367380f8fe275500c3534b32eb19e5b1eb54f17664298c032"}
Sep 15 22:33:40.163138 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:40.162764    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-588b46b67f-jdbnq" event={"ID":"b90747ad-4dbe-4fb1-84b4-a627e8083222","Type":"ContainerStarted","Data":"ea1deccaabc6418c4355f6f391ea55bb9bc48b72d2ae366b1f7aa5f6fb35b1a3"}
Sep 15 22:33:40.184397 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:40.184334    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-588b46b67f-jdbnq" podStartSLOduration=2.1843138890000002 podStartE2EDuration="2.184313889s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:33:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:33:40.180909469 +0000 UTC m=+370.040620754" watchObservedRunningTime="2026-09-15 22:33:40.184313889 +0000 UTC m=+370.044025176"
Sep 15 22:33:41.123671 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:41.123640    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-749d6986f8-lhz48"
Sep 15 22:33:45.947165 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:45.947133    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7f98985fb-xjpv7"]
Sep 15 22:33:45.951542 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:45.951525    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7f98985fb-xjpv7"
Sep 15 22:33:45.953999 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:45.953975    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 15 22:33:45.954120 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:45.954042    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 15 22:33:45.954467 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:45.954258    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 15 22:33:45.954688 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:45.954665    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 15 22:33:45.962407 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:45.962313    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7f98985fb-xjpv7"]
Sep 15 22:33:46.127154 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:46.127117    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f177deb3-c155-477c-91fa-b189eac69537-tmp\") pod \"kube-auth-proxy-7f98985fb-xjpv7\" (UID: \"f177deb3-c155-477c-91fa-b189eac69537\") " pod="openshift-ingress/kube-auth-proxy-7f98985fb-xjpv7"
Sep 15 22:33:46.127323 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:46.127170    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx99n\" (UniqueName: \"kubernetes.io/projected/f177deb3-c155-477c-91fa-b189eac69537-kube-api-access-cx99n\") pod \"kube-auth-proxy-7f98985fb-xjpv7\" (UID: \"f177deb3-c155-477c-91fa-b189eac69537\") " pod="openshift-ingress/kube-auth-proxy-7f98985fb-xjpv7"
Sep 15 22:33:46.127323 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:46.127283    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f177deb3-c155-477c-91fa-b189eac69537-tls-certs\") pod \"kube-auth-proxy-7f98985fb-xjpv7\" (UID: \"f177deb3-c155-477c-91fa-b189eac69537\") " pod="openshift-ingress/kube-auth-proxy-7f98985fb-xjpv7"
Sep 15 22:33:46.227898 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:46.227813    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f177deb3-c155-477c-91fa-b189eac69537-tmp\") pod \"kube-auth-proxy-7f98985fb-xjpv7\" (UID: \"f177deb3-c155-477c-91fa-b189eac69537\") " pod="openshift-ingress/kube-auth-proxy-7f98985fb-xjpv7"
Sep 15 22:33:46.227898 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:46.227869    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx99n\" (UniqueName: \"kubernetes.io/projected/f177deb3-c155-477c-91fa-b189eac69537-kube-api-access-cx99n\") pod \"kube-auth-proxy-7f98985fb-xjpv7\" (UID: \"f177deb3-c155-477c-91fa-b189eac69537\") " pod="openshift-ingress/kube-auth-proxy-7f98985fb-xjpv7"
Sep 15 22:33:46.228143 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:46.227916    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f177deb3-c155-477c-91fa-b189eac69537-tls-certs\") pod \"kube-auth-proxy-7f98985fb-xjpv7\" (UID: \"f177deb3-c155-477c-91fa-b189eac69537\") " pod="openshift-ingress/kube-auth-proxy-7f98985fb-xjpv7"
Sep 15 22:33:46.230162 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:46.230138    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f177deb3-c155-477c-91fa-b189eac69537-tmp\") pod \"kube-auth-proxy-7f98985fb-xjpv7\" (UID: \"f177deb3-c155-477c-91fa-b189eac69537\") " pod="openshift-ingress/kube-auth-proxy-7f98985fb-xjpv7"
Sep 15 22:33:46.230399 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:46.230381    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f177deb3-c155-477c-91fa-b189eac69537-tls-certs\") pod \"kube-auth-proxy-7f98985fb-xjpv7\" (UID: \"f177deb3-c155-477c-91fa-b189eac69537\") " pod="openshift-ingress/kube-auth-proxy-7f98985fb-xjpv7"
Sep 15 22:33:46.236626 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:46.236603    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx99n\" (UniqueName: \"kubernetes.io/projected/f177deb3-c155-477c-91fa-b189eac69537-kube-api-access-cx99n\") pod \"kube-auth-proxy-7f98985fb-xjpv7\" (UID: \"f177deb3-c155-477c-91fa-b189eac69537\") " pod="openshift-ingress/kube-auth-proxy-7f98985fb-xjpv7"
Sep 15 22:33:46.267307 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:46.267281    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7f98985fb-xjpv7"
Sep 15 22:33:46.555130 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:46.555104    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7f98985fb-xjpv7"]
Sep 15 22:33:46.555889 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:33:46.555866    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf177deb3_c155_477c_91fa_b189eac69537.slice/crio-7cebbb55e52e2a0105b7e02b2736e547340a120c198f15213257cbf67f7bc80d WatchSource:0}: Error finding container 7cebbb55e52e2a0105b7e02b2736e547340a120c198f15213257cbf67f7bc80d: Status 404 returned error can't find the container with id 7cebbb55e52e2a0105b7e02b2736e547340a120c198f15213257cbf67f7bc80d
Sep 15 22:33:47.197702 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:47.197650    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7f98985fb-xjpv7" event={"ID":"f177deb3-c155-477c-91fa-b189eac69537","Type":"ContainerStarted","Data":"7cebbb55e52e2a0105b7e02b2736e547340a120c198f15213257cbf67f7bc80d"}
Sep 15 22:33:47.237670 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:47.237635    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg"]
Sep 15 22:33:47.241288 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:47.241263    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg"
Sep 15 22:33:47.243372 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:47.243352    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 15 22:33:47.243508 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:47.243432    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 15 22:33:47.243508 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:47.243433    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 15 22:33:47.243719 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:47.243698    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-fhmxk\""
Sep 15 22:33:47.243906 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:47.243873    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 15 22:33:47.261746 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:47.261719    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg"]
Sep 15 22:33:47.338505 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:47.338325    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-r4zwg\" (UID: \"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg"
Sep 15 22:33:47.338505 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:47.338406    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-r4zwg\" (UID: \"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg"
Sep 15 22:33:47.338505 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:47.338444    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-r4zwg\" (UID: \"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg"
Sep 15 22:33:47.338795 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:47.338513    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-r4zwg\" (UID: \"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg"
Sep 15 22:33:47.338795 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:47.338560    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-r4zwg\" (UID: \"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg"
Sep 15 22:33:47.338795 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:47.338591    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtx7v\" (UniqueName: \"kubernetes.io/projected/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-kube-api-access-wtx7v\") pod \"istiod-openshift-gateway-7b7995d5cd-r4zwg\" (UID: \"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg"
Sep 15 22:33:47.338795 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:47.338653    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-r4zwg\" (UID: \"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg"
Sep 15 22:33:47.440152 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:47.440115    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-r4zwg\" (UID: \"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg"
Sep 15 22:33:47.440321 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:47.440164    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtx7v\" (UniqueName: \"kubernetes.io/projected/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-kube-api-access-wtx7v\") pod \"istiod-openshift-gateway-7b7995d5cd-r4zwg\" (UID: \"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg"
Sep 15 22:33:47.440321 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:47.440192    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-r4zwg\" (UID: \"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg"
Sep 15 22:33:47.440321 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:47.440220    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-r4zwg\" (UID: \"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg"
Sep 15 22:33:47.440321 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:47.440259    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-r4zwg\" (UID: \"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg"
Sep 15 22:33:47.440321 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:47.440281    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-r4zwg\" (UID: \"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg"
Sep 15 22:33:47.440321 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:47.440321    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-r4zwg\" (UID: \"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg"
Sep 15 22:33:47.440908 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:47.440882    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-r4zwg\" (UID: \"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg"
Sep 15 22:33:47.443525 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:47.443405    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-r4zwg\" (UID: \"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg"
Sep 15 22:33:47.443525 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:47.443455    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-r4zwg\" (UID: \"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg"
Sep 15 22:33:47.443525 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:47.443453    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-r4zwg\" (UID: \"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg"
Sep 15 22:33:47.443870 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:47.443848    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-r4zwg\" (UID: \"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg"
Sep 15 22:33:47.454255 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:47.454145    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-r4zwg\" (UID: \"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg"
Sep 15 22:33:47.457931 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:47.457894    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtx7v\" (UniqueName: \"kubernetes.io/projected/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-kube-api-access-wtx7v\") pod \"istiod-openshift-gateway-7b7995d5cd-r4zwg\" (UID: \"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg"
Sep 15 22:33:47.553696 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:47.553661    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg"
Sep 15 22:33:47.847605 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:47.847567    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg"]
Sep 15 22:33:47.850384 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:33:47.850351    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77c1aed1_a0a7_4ddf_a3e9_d3e02d72b627.slice/crio-8a8c50e1671f4654b45d1a5a593bdd813ef2d73bd0e33f8b80526895c7da11ea WatchSource:0}: Error finding container 8a8c50e1671f4654b45d1a5a593bdd813ef2d73bd0e33f8b80526895c7da11ea: Status 404 returned error can't find the container with id 8a8c50e1671f4654b45d1a5a593bdd813ef2d73bd0e33f8b80526895c7da11ea
Sep 15 22:33:48.206529 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:48.206448    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg" event={"ID":"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627","Type":"ContainerStarted","Data":"8a8c50e1671f4654b45d1a5a593bdd813ef2d73bd0e33f8b80526895c7da11ea"}
Sep 15 22:33:49.317883 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:49.317849    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-588b46b67f-jdbnq"
Sep 15 22:33:49.318333 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:49.317946    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-588b46b67f-jdbnq"
Sep 15 22:33:49.323226 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:49.323202    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-588b46b67f-jdbnq"
Sep 15 22:33:50.219881 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:50.219845    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7f98985fb-xjpv7" event={"ID":"f177deb3-c155-477c-91fa-b189eac69537","Type":"ContainerStarted","Data":"06830ad3e891c3f1f20a3f3c894fb1a6410a883a3d10a691179a439e9805cee8"}
Sep 15 22:33:50.224380 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:50.224356    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-588b46b67f-jdbnq"
Sep 15 22:33:50.239693 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:50.239640    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7f98985fb-xjpv7" podStartSLOduration=2.330986072 podStartE2EDuration="5.239620877s" totalImagesPullingTime="2.908634805s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:33:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:33:50.236805599 +0000 UTC m=+380.096516896" watchObservedRunningTime="2026-09-15 22:33:50.239620877 +0000 UTC m=+380.099332163"
Sep 15 22:33:50.283124 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:50.283092    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9d47fc5c8-b5289"]
Sep 15 22:33:51.267249 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:51.267211    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 22:33:51.267636 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:51.267289    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 22:33:52.230536 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:52.230499    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg" event={"ID":"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627","Type":"ContainerStarted","Data":"c2c436f4c567f7a33f880453cf43649231d5208dfb0117d868f8bd1e4ce8e803"}
Sep 15 22:33:52.230700 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:52.230568    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg"
Sep 15 22:33:52.232312 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:52.232280    2565 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-r4zwg container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 22:33:52.232427 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:52.232338    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg" podUID="77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 22:33:52.265133 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:52.265067    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg" podStartSLOduration=1.8509609249999999 podStartE2EDuration="5.265046421s" totalImagesPullingTime="3.414085496s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:33:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:33:52.259343595 +0000 UTC m=+382.119054881" watchObservedRunningTime="2026-09-15 22:33:52.265046421 +0000 UTC m=+382.124757707"
Sep 15 22:33:53.235741 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:33:53.235712    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg"
Sep 15 22:34:03.997670 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:03.997633    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-t2pzq"]
Sep 15 22:34:04.004466 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:04.004441    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-45x2s"]
Sep 15 22:34:04.004600 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:04.004554    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-t2pzq"
Sep 15 22:34:04.006545 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:04.006522    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-45x2s"
Sep 15 22:34:04.006872 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:04.006856    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 15 22:34:04.006954 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:04.006882    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-j9zm7\""
Sep 15 22:34:04.010289 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:04.010256    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 15 22:34:04.010881 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:04.010864    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-kg2rv\""
Sep 15 22:34:04.020822 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:04.020801    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-45x2s"]
Sep 15 22:34:04.023301 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:04.023281    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-t2pzq"]
Sep 15 22:34:04.093497 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:04.093464    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gpn2\" (UniqueName: \"kubernetes.io/projected/c10794ba-ad48-471c-a78f-c2552e0d402f-kube-api-access-2gpn2\") pod \"kserve-controller-manager-d6f4b65b8-t2pzq\" (UID: \"c10794ba-ad48-471c-a78f-c2552e0d402f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-t2pzq"
Sep 15 22:34:04.093654 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:04.093518    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llxj6\" (UniqueName: \"kubernetes.io/projected/3dc56556-dd6c-42c1-9f68-816e78f5aa65-kube-api-access-llxj6\") pod \"llmisvc-controller-manager-777c6bd7d5-45x2s\" (UID: \"3dc56556-dd6c-42c1-9f68-816e78f5aa65\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-45x2s"
Sep 15 22:34:04.093654 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:04.093583    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3dc56556-dd6c-42c1-9f68-816e78f5aa65-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-45x2s\" (UID: \"3dc56556-dd6c-42c1-9f68-816e78f5aa65\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-45x2s"
Sep 15 22:34:04.093654 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:04.093604    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c10794ba-ad48-471c-a78f-c2552e0d402f-cert\") pod \"kserve-controller-manager-d6f4b65b8-t2pzq\" (UID: \"c10794ba-ad48-471c-a78f-c2552e0d402f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-t2pzq"
Sep 15 22:34:04.194400 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:04.194365    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gpn2\" (UniqueName: \"kubernetes.io/projected/c10794ba-ad48-471c-a78f-c2552e0d402f-kube-api-access-2gpn2\") pod \"kserve-controller-manager-d6f4b65b8-t2pzq\" (UID: \"c10794ba-ad48-471c-a78f-c2552e0d402f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-t2pzq"
Sep 15 22:34:04.194580 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:04.194416    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llxj6\" (UniqueName: \"kubernetes.io/projected/3dc56556-dd6c-42c1-9f68-816e78f5aa65-kube-api-access-llxj6\") pod \"llmisvc-controller-manager-777c6bd7d5-45x2s\" (UID: \"3dc56556-dd6c-42c1-9f68-816e78f5aa65\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-45x2s"
Sep 15 22:34:04.194580 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:04.194443    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3dc56556-dd6c-42c1-9f68-816e78f5aa65-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-45x2s\" (UID: \"3dc56556-dd6c-42c1-9f68-816e78f5aa65\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-45x2s"
Sep 15 22:34:04.194580 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:04.194562    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c10794ba-ad48-471c-a78f-c2552e0d402f-cert\") pod \"kserve-controller-manager-d6f4b65b8-t2pzq\" (UID: \"c10794ba-ad48-471c-a78f-c2552e0d402f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-t2pzq"
Sep 15 22:34:04.196912 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:04.196880    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c10794ba-ad48-471c-a78f-c2552e0d402f-cert\") pod \"kserve-controller-manager-d6f4b65b8-t2pzq\" (UID: \"c10794ba-ad48-471c-a78f-c2552e0d402f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-t2pzq"
Sep 15 22:34:04.196912 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:04.196904    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3dc56556-dd6c-42c1-9f68-816e78f5aa65-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-45x2s\" (UID: \"3dc56556-dd6c-42c1-9f68-816e78f5aa65\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-45x2s"
Sep 15 22:34:04.218846 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:04.218811    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gpn2\" (UniqueName: \"kubernetes.io/projected/c10794ba-ad48-471c-a78f-c2552e0d402f-kube-api-access-2gpn2\") pod \"kserve-controller-manager-d6f4b65b8-t2pzq\" (UID: \"c10794ba-ad48-471c-a78f-c2552e0d402f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-t2pzq"
Sep 15 22:34:04.218973 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:04.218919    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llxj6\" (UniqueName: \"kubernetes.io/projected/3dc56556-dd6c-42c1-9f68-816e78f5aa65-kube-api-access-llxj6\") pod \"llmisvc-controller-manager-777c6bd7d5-45x2s\" (UID: \"3dc56556-dd6c-42c1-9f68-816e78f5aa65\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-45x2s"
Sep 15 22:34:04.315948 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:04.315922    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-t2pzq"
Sep 15 22:34:04.323391 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:04.323352    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-45x2s"
Sep 15 22:34:04.591079 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:04.591045    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-t2pzq"]
Sep 15 22:34:04.592123 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:34:04.592097    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc10794ba_ad48_471c_a78f_c2552e0d402f.slice/crio-b5c0b0f46c7f54f0e492b14d9f7862ba1c1ed79046f833284390bbb2c7adb38e WatchSource:0}: Error finding container b5c0b0f46c7f54f0e492b14d9f7862ba1c1ed79046f833284390bbb2c7adb38e: Status 404 returned error can't find the container with id b5c0b0f46c7f54f0e492b14d9f7862ba1c1ed79046f833284390bbb2c7adb38e
Sep 15 22:34:04.606598 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:04.606573    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-45x2s"]
Sep 15 22:34:04.608358 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:34:04.608332    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3dc56556_dd6c_42c1_9f68_816e78f5aa65.slice/crio-bbacdadc65321f9484c970f4f4c1ac146d9b95f301a26aa823062bfff7851823 WatchSource:0}: Error finding container bbacdadc65321f9484c970f4f4c1ac146d9b95f301a26aa823062bfff7851823: Status 404 returned error can't find the container with id bbacdadc65321f9484c970f4f4c1ac146d9b95f301a26aa823062bfff7851823
Sep 15 22:34:05.292942 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:05.292904    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-t2pzq" event={"ID":"c10794ba-ad48-471c-a78f-c2552e0d402f","Type":"ContainerStarted","Data":"b5c0b0f46c7f54f0e492b14d9f7862ba1c1ed79046f833284390bbb2c7adb38e"}
Sep 15 22:34:05.294449 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:05.294423    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-45x2s" event={"ID":"3dc56556-dd6c-42c1-9f68-816e78f5aa65","Type":"ContainerStarted","Data":"bbacdadc65321f9484c970f4f4c1ac146d9b95f301a26aa823062bfff7851823"}
Sep 15 22:34:07.307751 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:07.307697    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-t2pzq" event={"ID":"c10794ba-ad48-471c-a78f-c2552e0d402f","Type":"ContainerStarted","Data":"467b6c4e9fbff3c787dc56c7e497c908bbd6bfdee4a3356724442b832e28fd08"}
Sep 15 22:34:07.308248 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:07.307793    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-t2pzq"
Sep 15 22:34:07.326399 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:07.326335    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-t2pzq" podStartSLOduration=2.189440308 podStartE2EDuration="4.326313686s" totalImagesPullingTime="2.136873378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:34:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:34:07.325612719 +0000 UTC m=+397.185324027" watchObservedRunningTime="2026-09-15 22:34:07.326313686 +0000 UTC m=+397.186024971"
Sep 15 22:34:07.457001 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:07.456973    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg"]
Sep 15 22:34:07.457324 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:07.457295    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg" podUID="77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627" containerName="discovery" containerID="cri-o://c2c436f4c567f7a33f880453cf43649231d5208dfb0117d868f8bd1e4ce8e803" gracePeriod=30
Sep 15 22:34:07.693244 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:07.693218    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg"
Sep 15 22:34:07.830817 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:07.830731    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-local-certs\") pod \"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627\" (UID: \"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627\") "
Sep 15 22:34:07.830817 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:07.830795    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-istio-kubeconfig\") pod \"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627\" (UID: \"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627\") "
Sep 15 22:34:07.830997 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:07.830829    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-istio-csr-ca-configmap\") pod \"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627\" (UID: \"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627\") "
Sep 15 22:34:07.830997 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:07.830875    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-cacerts\") pod \"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627\" (UID: \"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627\") "
Sep 15 22:34:07.830997 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:07.830901    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wtx7v\" (UniqueName: \"kubernetes.io/projected/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-kube-api-access-wtx7v\") pod \"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627\" (UID: \"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627\") "
Sep 15 22:34:07.830997 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:07.830965    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-istio-token\") pod \"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627\" (UID: \"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627\") "
Sep 15 22:34:07.830997 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:07.830990    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-istio-csr-dns-cert\") pod \"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627\" (UID: \"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627\") "
Sep 15 22:34:07.831368 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:07.831321    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627" (UID: "77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:34:07.833371 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:07.833338    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627" (UID: "77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:34:07.833485 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:07.833365    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-cacerts" (OuterVolumeSpecName: "cacerts") pod "77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627" (UID: "77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:34:07.833485 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:07.833375    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-kube-api-access-wtx7v" (OuterVolumeSpecName: "kube-api-access-wtx7v") pod "77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627" (UID: "77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627"). InnerVolumeSpecName "kube-api-access-wtx7v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:34:07.833614 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:07.833595    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-local-certs" (OuterVolumeSpecName: "local-certs") pod "77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627" (UID: "77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:34:07.833657 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:07.833623    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627" (UID: "77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:34:07.833702 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:07.833683    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-istio-token" (OuterVolumeSpecName: "istio-token") pod "77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627" (UID: "77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:34:07.931874 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:07.931843    2565 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-istio-csr-ca-configmap\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:34:07.931874 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:07.931868    2565 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-cacerts\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:34:07.931874 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:07.931878    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wtx7v\" (UniqueName: \"kubernetes.io/projected/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-kube-api-access-wtx7v\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:34:07.932112 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:07.931888    2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-istio-token\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:34:07.932112 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:07.931897    2565 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-istio-csr-dns-cert\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:34:07.932112 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:07.931905    2565 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-local-certs\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:34:07.932112 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:07.931913    2565 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627-istio-kubeconfig\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:34:08.314175 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:08.314133    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-45x2s" event={"ID":"3dc56556-dd6c-42c1-9f68-816e78f5aa65","Type":"ContainerStarted","Data":"fa02bb5510792f54f4294fb131597cc7d7f317510318ed449d8b846a02acc0bf"}
Sep 15 22:34:08.314612 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:08.314213    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-45x2s"
Sep 15 22:34:08.315617 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:08.315591    2565 generic.go:356] "Generic (PLEG): container finished" podID="77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627" containerID="c2c436f4c567f7a33f880453cf43649231d5208dfb0117d868f8bd1e4ce8e803" exitCode=0
Sep 15 22:34:08.315705 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:08.315633    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg"
Sep 15 22:34:08.315705 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:08.315663    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg" event={"ID":"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627","Type":"ContainerDied","Data":"c2c436f4c567f7a33f880453cf43649231d5208dfb0117d868f8bd1e4ce8e803"}
Sep 15 22:34:08.315772 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:08.315703    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg" event={"ID":"77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627","Type":"ContainerDied","Data":"8a8c50e1671f4654b45d1a5a593bdd813ef2d73bd0e33f8b80526895c7da11ea"}
Sep 15 22:34:08.315772 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:08.315729    2565 scope.go:117] "RemoveContainer" containerID="c2c436f4c567f7a33f880453cf43649231d5208dfb0117d868f8bd1e4ce8e803"
Sep 15 22:34:08.324564 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:08.324545    2565 scope.go:117] "RemoveContainer" containerID="c2c436f4c567f7a33f880453cf43649231d5208dfb0117d868f8bd1e4ce8e803"
Sep 15 22:34:08.324849 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:34:08.324828    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2c436f4c567f7a33f880453cf43649231d5208dfb0117d868f8bd1e4ce8e803\": container with ID starting with c2c436f4c567f7a33f880453cf43649231d5208dfb0117d868f8bd1e4ce8e803 not found: ID does not exist" containerID="c2c436f4c567f7a33f880453cf43649231d5208dfb0117d868f8bd1e4ce8e803"
Sep 15 22:34:08.324896 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:08.324857    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2c436f4c567f7a33f880453cf43649231d5208dfb0117d868f8bd1e4ce8e803"} err="failed to get container status \"c2c436f4c567f7a33f880453cf43649231d5208dfb0117d868f8bd1e4ce8e803\": rpc error: code = NotFound desc = could not find container \"c2c436f4c567f7a33f880453cf43649231d5208dfb0117d868f8bd1e4ce8e803\": container with ID starting with c2c436f4c567f7a33f880453cf43649231d5208dfb0117d868f8bd1e4ce8e803 not found: ID does not exist"
Sep 15 22:34:08.332877 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:08.332831    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-45x2s" podStartSLOduration=2.501379435 podStartE2EDuration="5.332819068s" totalImagesPullingTime="2.831439633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:34:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:34:08.331488216 +0000 UTC m=+398.191199508" watchObservedRunningTime="2026-09-15 22:34:08.332819068 +0000 UTC m=+398.192530351"
Sep 15 22:34:08.346347 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:08.346324    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg"]
Sep 15 22:34:08.350341 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:08.350322    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4zwg"]
Sep 15 22:34:08.589525 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:08.589433    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627" path="/var/lib/kubelet/pods/77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627/volumes"
Sep 15 22:34:15.309496 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:15.309453    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-9d47fc5c8-b5289" podUID="88bbd734-5f2b-413e-8279-74862a9f00ad" containerName="console" containerID="cri-o://70c0bfc593a3967489e1211df57eeb07f0f7ab0071fa8d718ba7a1307dca731b" gracePeriod=15
Sep 15 22:34:15.559837 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:15.559783    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9d47fc5c8-b5289"
Sep 15 22:34:15.696249 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:15.696219    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/88bbd734-5f2b-413e-8279-74862a9f00ad-oauth-serving-cert\") pod \"88bbd734-5f2b-413e-8279-74862a9f00ad\" (UID: \"88bbd734-5f2b-413e-8279-74862a9f00ad\") "
Sep 15 22:34:15.696249 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:15.696256    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/88bbd734-5f2b-413e-8279-74862a9f00ad-service-ca\") pod \"88bbd734-5f2b-413e-8279-74862a9f00ad\" (UID: \"88bbd734-5f2b-413e-8279-74862a9f00ad\") "
Sep 15 22:34:15.696492 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:15.696279    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/88bbd734-5f2b-413e-8279-74862a9f00ad-console-oauth-config\") pod \"88bbd734-5f2b-413e-8279-74862a9f00ad\" (UID: \"88bbd734-5f2b-413e-8279-74862a9f00ad\") "
Sep 15 22:34:15.696492 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:15.696301    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88bbd734-5f2b-413e-8279-74862a9f00ad-trusted-ca-bundle\") pod \"88bbd734-5f2b-413e-8279-74862a9f00ad\" (UID: \"88bbd734-5f2b-413e-8279-74862a9f00ad\") "
Sep 15 22:34:15.696492 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:15.696399    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njmbp\" (UniqueName: \"kubernetes.io/projected/88bbd734-5f2b-413e-8279-74862a9f00ad-kube-api-access-njmbp\") pod \"88bbd734-5f2b-413e-8279-74862a9f00ad\" (UID: \"88bbd734-5f2b-413e-8279-74862a9f00ad\") "
Sep 15 22:34:15.696492 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:15.696429    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/88bbd734-5f2b-413e-8279-74862a9f00ad-console-config\") pod \"88bbd734-5f2b-413e-8279-74862a9f00ad\" (UID: \"88bbd734-5f2b-413e-8279-74862a9f00ad\") "
Sep 15 22:34:15.696492 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:15.696464    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/88bbd734-5f2b-413e-8279-74862a9f00ad-console-serving-cert\") pod \"88bbd734-5f2b-413e-8279-74862a9f00ad\" (UID: \"88bbd734-5f2b-413e-8279-74862a9f00ad\") "
Sep 15 22:34:15.696940 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:15.696747    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88bbd734-5f2b-413e-8279-74862a9f00ad-service-ca" (OuterVolumeSpecName: "service-ca") pod "88bbd734-5f2b-413e-8279-74862a9f00ad" (UID: "88bbd734-5f2b-413e-8279-74862a9f00ad"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:34:15.696940 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:15.696891    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88bbd734-5f2b-413e-8279-74862a9f00ad-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "88bbd734-5f2b-413e-8279-74862a9f00ad" (UID: "88bbd734-5f2b-413e-8279-74862a9f00ad"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:34:15.696940 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:15.696900    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88bbd734-5f2b-413e-8279-74862a9f00ad-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "88bbd734-5f2b-413e-8279-74862a9f00ad" (UID: "88bbd734-5f2b-413e-8279-74862a9f00ad"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:34:15.697142 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:15.696986    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88bbd734-5f2b-413e-8279-74862a9f00ad-console-config" (OuterVolumeSpecName: "console-config") pod "88bbd734-5f2b-413e-8279-74862a9f00ad" (UID: "88bbd734-5f2b-413e-8279-74862a9f00ad"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:34:15.698434 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:15.698408    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88bbd734-5f2b-413e-8279-74862a9f00ad-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "88bbd734-5f2b-413e-8279-74862a9f00ad" (UID: "88bbd734-5f2b-413e-8279-74862a9f00ad"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:34:15.698580 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:15.698558    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88bbd734-5f2b-413e-8279-74862a9f00ad-kube-api-access-njmbp" (OuterVolumeSpecName: "kube-api-access-njmbp") pod "88bbd734-5f2b-413e-8279-74862a9f00ad" (UID: "88bbd734-5f2b-413e-8279-74862a9f00ad"). InnerVolumeSpecName "kube-api-access-njmbp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:34:15.698666 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:15.698643    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88bbd734-5f2b-413e-8279-74862a9f00ad-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "88bbd734-5f2b-413e-8279-74862a9f00ad" (UID: "88bbd734-5f2b-413e-8279-74862a9f00ad"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:34:15.797230 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:15.797193    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njmbp\" (UniqueName: \"kubernetes.io/projected/88bbd734-5f2b-413e-8279-74862a9f00ad-kube-api-access-njmbp\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:34:15.797230 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:15.797226    2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/88bbd734-5f2b-413e-8279-74862a9f00ad-console-config\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:34:15.797230 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:15.797235    2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/88bbd734-5f2b-413e-8279-74862a9f00ad-console-serving-cert\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:34:15.797441 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:15.797244    2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/88bbd734-5f2b-413e-8279-74862a9f00ad-oauth-serving-cert\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:34:15.797441 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:15.797252    2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/88bbd734-5f2b-413e-8279-74862a9f00ad-service-ca\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:34:15.797441 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:15.797261    2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/88bbd734-5f2b-413e-8279-74862a9f00ad-console-oauth-config\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:34:15.797441 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:15.797269    2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88bbd734-5f2b-413e-8279-74862a9f00ad-trusted-ca-bundle\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:34:16.353666 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:16.353632    2565 generic.go:356] "Generic (PLEG): container finished" podID="88bbd734-5f2b-413e-8279-74862a9f00ad" containerID="70c0bfc593a3967489e1211df57eeb07f0f7ab0071fa8d718ba7a1307dca731b" exitCode=0
Sep 15 22:34:16.354165 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:16.353688    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9d47fc5c8-b5289"
Sep 15 22:34:16.354165 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:16.353730    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9d47fc5c8-b5289" event={"ID":"88bbd734-5f2b-413e-8279-74862a9f00ad","Type":"ContainerDied","Data":"70c0bfc593a3967489e1211df57eeb07f0f7ab0071fa8d718ba7a1307dca731b"}
Sep 15 22:34:16.354165 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:16.353781    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9d47fc5c8-b5289" event={"ID":"88bbd734-5f2b-413e-8279-74862a9f00ad","Type":"ContainerDied","Data":"daacb6f1207eab6615fc4ca4793207cc8b7b8d5581f76db2cbf5f8d82fa3d2db"}
Sep 15 22:34:16.354165 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:16.353802    2565 scope.go:117] "RemoveContainer" containerID="70c0bfc593a3967489e1211df57eeb07f0f7ab0071fa8d718ba7a1307dca731b"
Sep 15 22:34:16.365166 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:16.364962    2565 scope.go:117] "RemoveContainer" containerID="70c0bfc593a3967489e1211df57eeb07f0f7ab0071fa8d718ba7a1307dca731b"
Sep 15 22:34:16.365445 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:34:16.365424    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70c0bfc593a3967489e1211df57eeb07f0f7ab0071fa8d718ba7a1307dca731b\": container with ID starting with 70c0bfc593a3967489e1211df57eeb07f0f7ab0071fa8d718ba7a1307dca731b not found: ID does not exist" containerID="70c0bfc593a3967489e1211df57eeb07f0f7ab0071fa8d718ba7a1307dca731b"
Sep 15 22:34:16.365514 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:16.365453    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70c0bfc593a3967489e1211df57eeb07f0f7ab0071fa8d718ba7a1307dca731b"} err="failed to get container status \"70c0bfc593a3967489e1211df57eeb07f0f7ab0071fa8d718ba7a1307dca731b\": rpc error: code = NotFound desc = could not find container \"70c0bfc593a3967489e1211df57eeb07f0f7ab0071fa8d718ba7a1307dca731b\": container with ID starting with 70c0bfc593a3967489e1211df57eeb07f0f7ab0071fa8d718ba7a1307dca731b not found: ID does not exist"
Sep 15 22:34:16.374585 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:16.374563    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9d47fc5c8-b5289"]
Sep 15 22:34:16.381109 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:16.381081    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-9d47fc5c8-b5289"]
Sep 15 22:34:16.590303 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:16.590271    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88bbd734-5f2b-413e-8279-74862a9f00ad" path="/var/lib/kubelet/pods/88bbd734-5f2b-413e-8279-74862a9f00ad/volumes"
Sep 15 22:34:38.321583 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:38.321546    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-t2pzq"
Sep 15 22:34:39.322959 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:39.322929    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-45x2s"
Sep 15 22:34:59.349499 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.349453    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847as9c4v"]
Sep 15 22:34:59.349891 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.349783    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88bbd734-5f2b-413e-8279-74862a9f00ad" containerName="console"
Sep 15 22:34:59.349891 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.349794    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="88bbd734-5f2b-413e-8279-74862a9f00ad" containerName="console"
Sep 15 22:34:59.349891 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.349810    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627" containerName="discovery"
Sep 15 22:34:59.349891 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.349815    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627" containerName="discovery"
Sep 15 22:34:59.349891 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.349873    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="88bbd734-5f2b-413e-8279-74862a9f00ad" containerName="console"
Sep 15 22:34:59.349891 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.349882    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="77c1aed1-a0a7-4ddf-a3e9-d3e02d72b627" containerName="discovery"
Sep 15 22:34:59.351668 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.351652    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847as9c4v"
Sep 15 22:34:59.357980 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.357945    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 15 22:34:59.357980 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.357976    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 15 22:34:59.358205 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.358125    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-68lxg\""
Sep 15 22:34:59.361893 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.361870    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847as9c4v"]
Sep 15 22:34:59.441317 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.441281    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d726k4h"]
Sep 15 22:34:59.443547 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.443527    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d726k4h"
Sep 15 22:34:59.451671 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.451644    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pct4s\" (UniqueName: \"kubernetes.io/projected/2c363002-2042-4ed2-b1d5-77b17f716629-kube-api-access-pct4s\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847as9c4v\" (UID: \"2c363002-2042-4ed2-b1d5-77b17f716629\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847as9c4v"
Sep 15 22:34:59.451769 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.451715    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c363002-2042-4ed2-b1d5-77b17f716629-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847as9c4v\" (UID: \"2c363002-2042-4ed2-b1d5-77b17f716629\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847as9c4v"
Sep 15 22:34:59.451826 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.451780    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c363002-2042-4ed2-b1d5-77b17f716629-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847as9c4v\" (UID: \"2c363002-2042-4ed2-b1d5-77b17f716629\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847as9c4v"
Sep 15 22:34:59.452909 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.452883    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d726k4h"]
Sep 15 22:34:59.543243 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.543211    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c8wcc9"]
Sep 15 22:34:59.545657 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.545637    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c8wcc9"
Sep 15 22:34:59.552573 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.552477    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/69a74253-3603-410a-8bc2-f36205eec6b4-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d726k4h\" (UID: \"69a74253-3603-410a-8bc2-f36205eec6b4\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d726k4h"
Sep 15 22:34:59.552573 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.552524    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pct4s\" (UniqueName: \"kubernetes.io/projected/2c363002-2042-4ed2-b1d5-77b17f716629-kube-api-access-pct4s\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847as9c4v\" (UID: \"2c363002-2042-4ed2-b1d5-77b17f716629\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847as9c4v"
Sep 15 22:34:59.552704 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.552594    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/69a74253-3603-410a-8bc2-f36205eec6b4-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d726k4h\" (UID: \"69a74253-3603-410a-8bc2-f36205eec6b4\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d726k4h"
Sep 15 22:34:59.552864 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.552837    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c363002-2042-4ed2-b1d5-77b17f716629-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847as9c4v\" (UID: \"2c363002-2042-4ed2-b1d5-77b17f716629\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847as9c4v"
Sep 15 22:34:59.552938 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.552891    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c363002-2042-4ed2-b1d5-77b17f716629-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847as9c4v\" (UID: \"2c363002-2042-4ed2-b1d5-77b17f716629\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847as9c4v"
Sep 15 22:34:59.553003 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.552943    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ls6rd\" (UniqueName: \"kubernetes.io/projected/69a74253-3603-410a-8bc2-f36205eec6b4-kube-api-access-ls6rd\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d726k4h\" (UID: \"69a74253-3603-410a-8bc2-f36205eec6b4\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d726k4h"
Sep 15 22:34:59.553392 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.553372    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c363002-2042-4ed2-b1d5-77b17f716629-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847as9c4v\" (UID: \"2c363002-2042-4ed2-b1d5-77b17f716629\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847as9c4v"
Sep 15 22:34:59.553498 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.553439    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c363002-2042-4ed2-b1d5-77b17f716629-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847as9c4v\" (UID: \"2c363002-2042-4ed2-b1d5-77b17f716629\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847as9c4v"
Sep 15 22:34:59.554854 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.554829    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c8wcc9"]
Sep 15 22:34:59.563509 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.563479    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pct4s\" (UniqueName: \"kubernetes.io/projected/2c363002-2042-4ed2-b1d5-77b17f716629-kube-api-access-pct4s\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847as9c4v\" (UID: \"2c363002-2042-4ed2-b1d5-77b17f716629\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847as9c4v"
Sep 15 22:34:59.641081 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.640991    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k45jp"]
Sep 15 22:34:59.643367 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.643351    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k45jp"
Sep 15 22:34:59.653390 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.653365    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k45jp"]
Sep 15 22:34:59.653535 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.653511    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/590faf13-cb38-4d1d-b79f-0099f61522b3-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c8wcc9\" (UID: \"590faf13-cb38-4d1d-b79f-0099f61522b3\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c8wcc9"
Sep 15 22:34:59.653597 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.653559    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ls6rd\" (UniqueName: \"kubernetes.io/projected/69a74253-3603-410a-8bc2-f36205eec6b4-kube-api-access-ls6rd\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d726k4h\" (UID: \"69a74253-3603-410a-8bc2-f36205eec6b4\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d726k4h"
Sep 15 22:34:59.653651 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.653604    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/69a74253-3603-410a-8bc2-f36205eec6b4-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d726k4h\" (UID: \"69a74253-3603-410a-8bc2-f36205eec6b4\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d726k4h"
Sep 15 22:34:59.653651 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.653645    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/590faf13-cb38-4d1d-b79f-0099f61522b3-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c8wcc9\" (UID: \"590faf13-cb38-4d1d-b79f-0099f61522b3\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c8wcc9"
Sep 15 22:34:59.653747 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.653674    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/69a74253-3603-410a-8bc2-f36205eec6b4-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d726k4h\" (UID: \"69a74253-3603-410a-8bc2-f36205eec6b4\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d726k4h"
Sep 15 22:34:59.653747 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.653722    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctkqs\" (UniqueName: \"kubernetes.io/projected/590faf13-cb38-4d1d-b79f-0099f61522b3-kube-api-access-ctkqs\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c8wcc9\" (UID: \"590faf13-cb38-4d1d-b79f-0099f61522b3\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c8wcc9"
Sep 15 22:34:59.654094 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.654072    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/69a74253-3603-410a-8bc2-f36205eec6b4-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d726k4h\" (UID: \"69a74253-3603-410a-8bc2-f36205eec6b4\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d726k4h"
Sep 15 22:34:59.654211 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.654108    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/69a74253-3603-410a-8bc2-f36205eec6b4-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d726k4h\" (UID: \"69a74253-3603-410a-8bc2-f36205eec6b4\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d726k4h"
Sep 15 22:34:59.661950 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.661920    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ls6rd\" (UniqueName: \"kubernetes.io/projected/69a74253-3603-410a-8bc2-f36205eec6b4-kube-api-access-ls6rd\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d726k4h\" (UID: \"69a74253-3603-410a-8bc2-f36205eec6b4\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d726k4h"
Sep 15 22:34:59.668051 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.668012    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847as9c4v"
Sep 15 22:34:59.753331 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.753301    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d726k4h"
Sep 15 22:34:59.754669 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.754644    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/590faf13-cb38-4d1d-b79f-0099f61522b3-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c8wcc9\" (UID: \"590faf13-cb38-4d1d-b79f-0099f61522b3\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c8wcc9"
Sep 15 22:34:59.754737 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.754686    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ecff6472-7d0f-4d6e-8a02-a8052effe07a-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k45jp\" (UID: \"ecff6472-7d0f-4d6e-8a02-a8052effe07a\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k45jp"
Sep 15 22:34:59.754737 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.754724    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ecff6472-7d0f-4d6e-8a02-a8052effe07a-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k45jp\" (UID: \"ecff6472-7d0f-4d6e-8a02-a8052effe07a\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k45jp"
Sep 15 22:34:59.754850 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.754763    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/590faf13-cb38-4d1d-b79f-0099f61522b3-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c8wcc9\" (UID: \"590faf13-cb38-4d1d-b79f-0099f61522b3\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c8wcc9"
Sep 15 22:34:59.754928 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.754905    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctkqs\" (UniqueName: \"kubernetes.io/projected/590faf13-cb38-4d1d-b79f-0099f61522b3-kube-api-access-ctkqs\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c8wcc9\" (UID: \"590faf13-cb38-4d1d-b79f-0099f61522b3\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c8wcc9"
Sep 15 22:34:59.754987 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.754958    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwb25\" (UniqueName: \"kubernetes.io/projected/ecff6472-7d0f-4d6e-8a02-a8052effe07a-kube-api-access-bwb25\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k45jp\" (UID: \"ecff6472-7d0f-4d6e-8a02-a8052effe07a\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k45jp"
Sep 15 22:34:59.755109 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.755090    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/590faf13-cb38-4d1d-b79f-0099f61522b3-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c8wcc9\" (UID: \"590faf13-cb38-4d1d-b79f-0099f61522b3\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c8wcc9"
Sep 15 22:34:59.755192 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.755107    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/590faf13-cb38-4d1d-b79f-0099f61522b3-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c8wcc9\" (UID: \"590faf13-cb38-4d1d-b79f-0099f61522b3\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c8wcc9"
Sep 15 22:34:59.763868 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.763844    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctkqs\" (UniqueName: \"kubernetes.io/projected/590faf13-cb38-4d1d-b79f-0099f61522b3-kube-api-access-ctkqs\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c8wcc9\" (UID: \"590faf13-cb38-4d1d-b79f-0099f61522b3\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c8wcc9"
Sep 15 22:34:59.855892 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.855860    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c8wcc9"
Sep 15 22:34:59.856094 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.856071    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwb25\" (UniqueName: \"kubernetes.io/projected/ecff6472-7d0f-4d6e-8a02-a8052effe07a-kube-api-access-bwb25\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k45jp\" (UID: \"ecff6472-7d0f-4d6e-8a02-a8052effe07a\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k45jp"
Sep 15 22:34:59.856152 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.856118    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ecff6472-7d0f-4d6e-8a02-a8052effe07a-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k45jp\" (UID: \"ecff6472-7d0f-4d6e-8a02-a8052effe07a\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k45jp"
Sep 15 22:34:59.856152 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.856144    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ecff6472-7d0f-4d6e-8a02-a8052effe07a-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k45jp\" (UID: \"ecff6472-7d0f-4d6e-8a02-a8052effe07a\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k45jp"
Sep 15 22:34:59.856472 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.856455    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ecff6472-7d0f-4d6e-8a02-a8052effe07a-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k45jp\" (UID: \"ecff6472-7d0f-4d6e-8a02-a8052effe07a\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k45jp"
Sep 15 22:34:59.856523 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.856480    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ecff6472-7d0f-4d6e-8a02-a8052effe07a-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k45jp\" (UID: \"ecff6472-7d0f-4d6e-8a02-a8052effe07a\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k45jp"
Sep 15 22:34:59.864688 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.864668    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwb25\" (UniqueName: \"kubernetes.io/projected/ecff6472-7d0f-4d6e-8a02-a8052effe07a-kube-api-access-bwb25\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k45jp\" (UID: \"ecff6472-7d0f-4d6e-8a02-a8052effe07a\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k45jp"
Sep 15 22:34:59.937857 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.937832    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847as9c4v"]
Sep 15 22:34:59.940007 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:34:59.939979    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c363002_2042_4ed2_b1d5_77b17f716629.slice/crio-c3b7af176d9cfd33bd625b27c6db930fa5fed91a38d46b01e9da4fbece2c8e40 WatchSource:0}: Error finding container c3b7af176d9cfd33bd625b27c6db930fa5fed91a38d46b01e9da4fbece2c8e40: Status 404 returned error can't find the container with id c3b7af176d9cfd33bd625b27c6db930fa5fed91a38d46b01e9da4fbece2c8e40
Sep 15 22:34:59.954815 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:34:59.954638    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k45jp"
Sep 15 22:35:00.028810 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:00.028771    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d726k4h"]
Sep 15 22:35:00.029566 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:35:00.029533    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69a74253_3603_410a_8bc2_f36205eec6b4.slice/crio-295681ae484cc66bba0b70b443fff085679f31eed6958a64c8d94b04a5683185 WatchSource:0}: Error finding container 295681ae484cc66bba0b70b443fff085679f31eed6958a64c8d94b04a5683185: Status 404 returned error can't find the container with id 295681ae484cc66bba0b70b443fff085679f31eed6958a64c8d94b04a5683185
Sep 15 22:35:00.126578 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:00.126557    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c8wcc9"]
Sep 15 22:35:00.129071 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:35:00.129049    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod590faf13_cb38_4d1d_b79f_0099f61522b3.slice/crio-5c1d040dc1e164a58705a23dcb7ee8e877652efa2a622043a2eac6128c40e0fe WatchSource:0}: Error finding container 5c1d040dc1e164a58705a23dcb7ee8e877652efa2a622043a2eac6128c40e0fe: Status 404 returned error can't find the container with id 5c1d040dc1e164a58705a23dcb7ee8e877652efa2a622043a2eac6128c40e0fe
Sep 15 22:35:00.226321 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:00.226177    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k45jp"]
Sep 15 22:35:00.287843 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:35:00.287804    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecff6472_7d0f_4d6e_8a02_a8052effe07a.slice/crio-91b2ebdb8e3a91f8520c0f06a9ec72b7e9cbd9876bac7f69e11b6b7834ed82b8 WatchSource:0}: Error finding container 91b2ebdb8e3a91f8520c0f06a9ec72b7e9cbd9876bac7f69e11b6b7834ed82b8: Status 404 returned error can't find the container with id 91b2ebdb8e3a91f8520c0f06a9ec72b7e9cbd9876bac7f69e11b6b7834ed82b8
Sep 15 22:35:00.559879 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:00.559849    2565 generic.go:356] "Generic (PLEG): container finished" podID="590faf13-cb38-4d1d-b79f-0099f61522b3" containerID="b94d3fc3dd14b08ca14756413cde5aed8b97613c2d6819bf4acf5ec849d35b94" exitCode=0
Sep 15 22:35:00.560293 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:00.559923    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c8wcc9" event={"ID":"590faf13-cb38-4d1d-b79f-0099f61522b3","Type":"ContainerDied","Data":"b94d3fc3dd14b08ca14756413cde5aed8b97613c2d6819bf4acf5ec849d35b94"}
Sep 15 22:35:00.560293 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:00.559962    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c8wcc9" event={"ID":"590faf13-cb38-4d1d-b79f-0099f61522b3","Type":"ContainerStarted","Data":"5c1d040dc1e164a58705a23dcb7ee8e877652efa2a622043a2eac6128c40e0fe"}
Sep 15 22:35:00.561644 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:00.561623    2565 generic.go:356] "Generic (PLEG): container finished" podID="2c363002-2042-4ed2-b1d5-77b17f716629" containerID="cd0646abb8877955ba08ac097aeb676e9339cfb214273850b5f6bfb5db1df371" exitCode=0
Sep 15 22:35:00.561756 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:00.561710    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847as9c4v" event={"ID":"2c363002-2042-4ed2-b1d5-77b17f716629","Type":"ContainerDied","Data":"cd0646abb8877955ba08ac097aeb676e9339cfb214273850b5f6bfb5db1df371"}
Sep 15 22:35:00.561831 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:00.561761    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847as9c4v" event={"ID":"2c363002-2042-4ed2-b1d5-77b17f716629","Type":"ContainerStarted","Data":"c3b7af176d9cfd33bd625b27c6db930fa5fed91a38d46b01e9da4fbece2c8e40"}
Sep 15 22:35:00.563872 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:00.563452    2565 generic.go:356] "Generic (PLEG): container finished" podID="69a74253-3603-410a-8bc2-f36205eec6b4" containerID="582de0860d991be5d4ca45852a1a842b3eec9af22c47c6a661e5bf6b8ba0025c" exitCode=0
Sep 15 22:35:00.563872 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:00.563480    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d726k4h" event={"ID":"69a74253-3603-410a-8bc2-f36205eec6b4","Type":"ContainerDied","Data":"582de0860d991be5d4ca45852a1a842b3eec9af22c47c6a661e5bf6b8ba0025c"}
Sep 15 22:35:00.563872 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:00.563524    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d726k4h" event={"ID":"69a74253-3603-410a-8bc2-f36205eec6b4","Type":"ContainerStarted","Data":"295681ae484cc66bba0b70b443fff085679f31eed6958a64c8d94b04a5683185"}
Sep 15 22:35:00.565389 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:00.565369    2565 generic.go:356] "Generic (PLEG): container finished" podID="ecff6472-7d0f-4d6e-8a02-a8052effe07a" containerID="7acc5ed0987c32301a514604b50e656f468c5aa0b94425913df48e98c673d4c2" exitCode=0
Sep 15 22:35:00.565494 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:00.565394    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k45jp" event={"ID":"ecff6472-7d0f-4d6e-8a02-a8052effe07a","Type":"ContainerDied","Data":"7acc5ed0987c32301a514604b50e656f468c5aa0b94425913df48e98c673d4c2"}
Sep 15 22:35:00.565494 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:00.565451    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k45jp" event={"ID":"ecff6472-7d0f-4d6e-8a02-a8052effe07a","Type":"ContainerStarted","Data":"91b2ebdb8e3a91f8520c0f06a9ec72b7e9cbd9876bac7f69e11b6b7834ed82b8"}
Sep 15 22:35:01.571788 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:01.571754    2565 generic.go:356] "Generic (PLEG): container finished" podID="ecff6472-7d0f-4d6e-8a02-a8052effe07a" containerID="226c04a29cca190ee15bb979ed522cb273da3583c4841327f8c601e2bcb0f268" exitCode=0
Sep 15 22:35:01.572228 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:01.571843    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k45jp" event={"ID":"ecff6472-7d0f-4d6e-8a02-a8052effe07a","Type":"ContainerDied","Data":"226c04a29cca190ee15bb979ed522cb273da3583c4841327f8c601e2bcb0f268"}
Sep 15 22:35:01.573779 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:01.573759    2565 generic.go:356] "Generic (PLEG): container finished" podID="69a74253-3603-410a-8bc2-f36205eec6b4" containerID="18d972e1a30783baf9ec92da47463b5735c0a69eb5bfb110e2a7a0b85fdda3e3" exitCode=0
Sep 15 22:35:01.573850 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:01.573802    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d726k4h" event={"ID":"69a74253-3603-410a-8bc2-f36205eec6b4","Type":"ContainerDied","Data":"18d972e1a30783baf9ec92da47463b5735c0a69eb5bfb110e2a7a0b85fdda3e3"}
Sep 15 22:35:02.581252 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:02.581176    2565 generic.go:356] "Generic (PLEG): container finished" podID="69a74253-3603-410a-8bc2-f36205eec6b4" containerID="f4a005da477d9c0a44e25be2faefe0bfc8fbfc69fbede02f4ea5f4ac8e0d078a" exitCode=0
Sep 15 22:35:02.581593 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:02.581254    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d726k4h" event={"ID":"69a74253-3603-410a-8bc2-f36205eec6b4","Type":"ContainerDied","Data":"f4a005da477d9c0a44e25be2faefe0bfc8fbfc69fbede02f4ea5f4ac8e0d078a"}
Sep 15 22:35:02.583441 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:02.583418    2565 generic.go:356] "Generic (PLEG): container finished" podID="ecff6472-7d0f-4d6e-8a02-a8052effe07a" containerID="2b7371cd74c2964e8312854ce5e4549a54b1bcaf5b24a2c7906f8858470c6f23" exitCode=0
Sep 15 22:35:02.583544 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:02.583501    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k45jp" event={"ID":"ecff6472-7d0f-4d6e-8a02-a8052effe07a","Type":"ContainerDied","Data":"2b7371cd74c2964e8312854ce5e4549a54b1bcaf5b24a2c7906f8858470c6f23"}
Sep 15 22:35:02.585269 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:02.585247    2565 generic.go:356] "Generic (PLEG): container finished" podID="590faf13-cb38-4d1d-b79f-0099f61522b3" containerID="7551607e59924ff66adfd037d26e0168cf491db617f7c573f2ab5338457a4882" exitCode=0
Sep 15 22:35:02.587418 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:02.587397    2565 generic.go:356] "Generic (PLEG): container finished" podID="2c363002-2042-4ed2-b1d5-77b17f716629" containerID="ef452ba15ace8e2160106077e7cbaa8dd304590d2da5f57e47c03e2f32c64f49" exitCode=0
Sep 15 22:35:02.591070 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:02.591050    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c8wcc9" event={"ID":"590faf13-cb38-4d1d-b79f-0099f61522b3","Type":"ContainerDied","Data":"7551607e59924ff66adfd037d26e0168cf491db617f7c573f2ab5338457a4882"}
Sep 15 22:35:02.591146 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:02.591087    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847as9c4v" event={"ID":"2c363002-2042-4ed2-b1d5-77b17f716629","Type":"ContainerDied","Data":"ef452ba15ace8e2160106077e7cbaa8dd304590d2da5f57e47c03e2f32c64f49"}
Sep 15 22:35:03.595090 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:03.595059    2565 generic.go:356] "Generic (PLEG): container finished" podID="590faf13-cb38-4d1d-b79f-0099f61522b3" containerID="ec4dd91e19833cb7f2d3dcaf2420e9c47217a46b4f95e824059403915559939c" exitCode=0
Sep 15 22:35:03.595488 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:03.595131    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c8wcc9" event={"ID":"590faf13-cb38-4d1d-b79f-0099f61522b3","Type":"ContainerDied","Data":"ec4dd91e19833cb7f2d3dcaf2420e9c47217a46b4f95e824059403915559939c"}
Sep 15 22:35:03.597245 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:03.597222    2565 generic.go:356] "Generic (PLEG): container finished" podID="2c363002-2042-4ed2-b1d5-77b17f716629" containerID="cb09de4267628e49046004939bb35eabcc184e38384449920d7f659e7d8fa857" exitCode=0
Sep 15 22:35:03.597340 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:03.597312    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847as9c4v" event={"ID":"2c363002-2042-4ed2-b1d5-77b17f716629","Type":"ContainerDied","Data":"cb09de4267628e49046004939bb35eabcc184e38384449920d7f659e7d8fa857"}
Sep 15 22:35:03.723130 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:03.723104    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d726k4h"
Sep 15 22:35:03.758565 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:03.758542    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k45jp"
Sep 15 22:35:03.789763 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:03.789739    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ls6rd\" (UniqueName: \"kubernetes.io/projected/69a74253-3603-410a-8bc2-f36205eec6b4-kube-api-access-ls6rd\") pod \"69a74253-3603-410a-8bc2-f36205eec6b4\" (UID: \"69a74253-3603-410a-8bc2-f36205eec6b4\") "
Sep 15 22:35:03.789882 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:03.789844    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/69a74253-3603-410a-8bc2-f36205eec6b4-util\") pod \"69a74253-3603-410a-8bc2-f36205eec6b4\" (UID: \"69a74253-3603-410a-8bc2-f36205eec6b4\") "
Sep 15 22:35:03.789919 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:03.789881    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/69a74253-3603-410a-8bc2-f36205eec6b4-bundle\") pod \"69a74253-3603-410a-8bc2-f36205eec6b4\" (UID: \"69a74253-3603-410a-8bc2-f36205eec6b4\") "
Sep 15 22:35:03.790485 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:03.790457    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69a74253-3603-410a-8bc2-f36205eec6b4-bundle" (OuterVolumeSpecName: "bundle") pod "69a74253-3603-410a-8bc2-f36205eec6b4" (UID: "69a74253-3603-410a-8bc2-f36205eec6b4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:35:03.791792 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:03.791769    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69a74253-3603-410a-8bc2-f36205eec6b4-kube-api-access-ls6rd" (OuterVolumeSpecName: "kube-api-access-ls6rd") pod "69a74253-3603-410a-8bc2-f36205eec6b4" (UID: "69a74253-3603-410a-8bc2-f36205eec6b4"). InnerVolumeSpecName "kube-api-access-ls6rd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:35:03.794913 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:03.794878    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69a74253-3603-410a-8bc2-f36205eec6b4-util" (OuterVolumeSpecName: "util") pod "69a74253-3603-410a-8bc2-f36205eec6b4" (UID: "69a74253-3603-410a-8bc2-f36205eec6b4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:35:03.890521 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:03.890445    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ecff6472-7d0f-4d6e-8a02-a8052effe07a-util\") pod \"ecff6472-7d0f-4d6e-8a02-a8052effe07a\" (UID: \"ecff6472-7d0f-4d6e-8a02-a8052effe07a\") "
Sep 15 22:35:03.890521 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:03.890522    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ecff6472-7d0f-4d6e-8a02-a8052effe07a-bundle\") pod \"ecff6472-7d0f-4d6e-8a02-a8052effe07a\" (UID: \"ecff6472-7d0f-4d6e-8a02-a8052effe07a\") "
Sep 15 22:35:03.890729 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:03.890552    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bwb25\" (UniqueName: \"kubernetes.io/projected/ecff6472-7d0f-4d6e-8a02-a8052effe07a-kube-api-access-bwb25\") pod \"ecff6472-7d0f-4d6e-8a02-a8052effe07a\" (UID: \"ecff6472-7d0f-4d6e-8a02-a8052effe07a\") "
Sep 15 22:35:03.890729 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:03.890686    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ls6rd\" (UniqueName: \"kubernetes.io/projected/69a74253-3603-410a-8bc2-f36205eec6b4-kube-api-access-ls6rd\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:35:03.890729 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:03.890696    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/69a74253-3603-410a-8bc2-f36205eec6b4-util\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:35:03.890729 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:03.890704    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/69a74253-3603-410a-8bc2-f36205eec6b4-bundle\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:35:03.890949 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:03.890913    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecff6472-7d0f-4d6e-8a02-a8052effe07a-bundle" (OuterVolumeSpecName: "bundle") pod "ecff6472-7d0f-4d6e-8a02-a8052effe07a" (UID: "ecff6472-7d0f-4d6e-8a02-a8052effe07a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:35:03.892567 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:03.892545    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecff6472-7d0f-4d6e-8a02-a8052effe07a-kube-api-access-bwb25" (OuterVolumeSpecName: "kube-api-access-bwb25") pod "ecff6472-7d0f-4d6e-8a02-a8052effe07a" (UID: "ecff6472-7d0f-4d6e-8a02-a8052effe07a"). InnerVolumeSpecName "kube-api-access-bwb25". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:35:03.895830 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:03.895793    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecff6472-7d0f-4d6e-8a02-a8052effe07a-util" (OuterVolumeSpecName: "util") pod "ecff6472-7d0f-4d6e-8a02-a8052effe07a" (UID: "ecff6472-7d0f-4d6e-8a02-a8052effe07a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:35:03.991315 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:03.991283    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ecff6472-7d0f-4d6e-8a02-a8052effe07a-util\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:35:03.991315 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:03.991308    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ecff6472-7d0f-4d6e-8a02-a8052effe07a-bundle\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:35:03.991315 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:03.991319    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bwb25\" (UniqueName: \"kubernetes.io/projected/ecff6472-7d0f-4d6e-8a02-a8052effe07a-kube-api-access-bwb25\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:35:04.604410 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:04.604385    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k45jp"
Sep 15 22:35:04.604739 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:04.604404    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6k45jp" event={"ID":"ecff6472-7d0f-4d6e-8a02-a8052effe07a","Type":"ContainerDied","Data":"91b2ebdb8e3a91f8520c0f06a9ec72b7e9cbd9876bac7f69e11b6b7834ed82b8"}
Sep 15 22:35:04.604739 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:04.604461    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="91b2ebdb8e3a91f8520c0f06a9ec72b7e9cbd9876bac7f69e11b6b7834ed82b8"
Sep 15 22:35:04.606484 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:04.606468    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d726k4h"
Sep 15 22:35:04.606594 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:04.606496    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d726k4h" event={"ID":"69a74253-3603-410a-8bc2-f36205eec6b4","Type":"ContainerDied","Data":"295681ae484cc66bba0b70b443fff085679f31eed6958a64c8d94b04a5683185"}
Sep 15 22:35:04.606594 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:04.606523    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="295681ae484cc66bba0b70b443fff085679f31eed6958a64c8d94b04a5683185"
Sep 15 22:35:04.739378 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:04.739356    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847as9c4v"
Sep 15 22:35:04.768476 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:04.768452    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c8wcc9"
Sep 15 22:35:04.795623 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:04.795598    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c363002-2042-4ed2-b1d5-77b17f716629-util\") pod \"2c363002-2042-4ed2-b1d5-77b17f716629\" (UID: \"2c363002-2042-4ed2-b1d5-77b17f716629\") "
Sep 15 22:35:04.795778 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:04.795717    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c363002-2042-4ed2-b1d5-77b17f716629-bundle\") pod \"2c363002-2042-4ed2-b1d5-77b17f716629\" (UID: \"2c363002-2042-4ed2-b1d5-77b17f716629\") "
Sep 15 22:35:04.795778 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:04.795757    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pct4s\" (UniqueName: \"kubernetes.io/projected/2c363002-2042-4ed2-b1d5-77b17f716629-kube-api-access-pct4s\") pod \"2c363002-2042-4ed2-b1d5-77b17f716629\" (UID: \"2c363002-2042-4ed2-b1d5-77b17f716629\") "
Sep 15 22:35:04.796322 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:04.796295    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c363002-2042-4ed2-b1d5-77b17f716629-bundle" (OuterVolumeSpecName: "bundle") pod "2c363002-2042-4ed2-b1d5-77b17f716629" (UID: "2c363002-2042-4ed2-b1d5-77b17f716629"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:35:04.798237 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:04.798213    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c363002-2042-4ed2-b1d5-77b17f716629-kube-api-access-pct4s" (OuterVolumeSpecName: "kube-api-access-pct4s") pod "2c363002-2042-4ed2-b1d5-77b17f716629" (UID: "2c363002-2042-4ed2-b1d5-77b17f716629"). InnerVolumeSpecName "kube-api-access-pct4s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:35:04.803060 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:04.803036    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c363002-2042-4ed2-b1d5-77b17f716629-util" (OuterVolumeSpecName: "util") pod "2c363002-2042-4ed2-b1d5-77b17f716629" (UID: "2c363002-2042-4ed2-b1d5-77b17f716629"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:35:04.896913 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:04.896846    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/590faf13-cb38-4d1d-b79f-0099f61522b3-util\") pod \"590faf13-cb38-4d1d-b79f-0099f61522b3\" (UID: \"590faf13-cb38-4d1d-b79f-0099f61522b3\") "
Sep 15 22:35:04.897049 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:04.896941    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ctkqs\" (UniqueName: \"kubernetes.io/projected/590faf13-cb38-4d1d-b79f-0099f61522b3-kube-api-access-ctkqs\") pod \"590faf13-cb38-4d1d-b79f-0099f61522b3\" (UID: \"590faf13-cb38-4d1d-b79f-0099f61522b3\") "
Sep 15 22:35:04.897049 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:04.896966    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/590faf13-cb38-4d1d-b79f-0099f61522b3-bundle\") pod \"590faf13-cb38-4d1d-b79f-0099f61522b3\" (UID: \"590faf13-cb38-4d1d-b79f-0099f61522b3\") "
Sep 15 22:35:04.897254 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:04.897237    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c363002-2042-4ed2-b1d5-77b17f716629-bundle\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:35:04.897302 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:04.897261    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pct4s\" (UniqueName: \"kubernetes.io/projected/2c363002-2042-4ed2-b1d5-77b17f716629-kube-api-access-pct4s\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:35:04.897302 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:04.897275    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c363002-2042-4ed2-b1d5-77b17f716629-util\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:35:04.897474 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:04.897441    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/590faf13-cb38-4d1d-b79f-0099f61522b3-bundle" (OuterVolumeSpecName: "bundle") pod "590faf13-cb38-4d1d-b79f-0099f61522b3" (UID: "590faf13-cb38-4d1d-b79f-0099f61522b3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:35:04.898953 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:04.898930    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/590faf13-cb38-4d1d-b79f-0099f61522b3-kube-api-access-ctkqs" (OuterVolumeSpecName: "kube-api-access-ctkqs") pod "590faf13-cb38-4d1d-b79f-0099f61522b3" (UID: "590faf13-cb38-4d1d-b79f-0099f61522b3"). InnerVolumeSpecName "kube-api-access-ctkqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:35:04.902388 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:04.902352    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/590faf13-cb38-4d1d-b79f-0099f61522b3-util" (OuterVolumeSpecName: "util") pod "590faf13-cb38-4d1d-b79f-0099f61522b3" (UID: "590faf13-cb38-4d1d-b79f-0099f61522b3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:35:04.998108 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:04.998069    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctkqs\" (UniqueName: \"kubernetes.io/projected/590faf13-cb38-4d1d-b79f-0099f61522b3-kube-api-access-ctkqs\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:35:04.998108 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:04.998105    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/590faf13-cb38-4d1d-b79f-0099f61522b3-bundle\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:35:04.998108 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:04.998115    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/590faf13-cb38-4d1d-b79f-0099f61522b3-util\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:35:05.614972 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:05.614941    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c8wcc9"
Sep 15 22:35:05.615409 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:05.614938    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c8wcc9" event={"ID":"590faf13-cb38-4d1d-b79f-0099f61522b3","Type":"ContainerDied","Data":"5c1d040dc1e164a58705a23dcb7ee8e877652efa2a622043a2eac6128c40e0fe"}
Sep 15 22:35:05.615409 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:05.615068    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5c1d040dc1e164a58705a23dcb7ee8e877652efa2a622043a2eac6128c40e0fe"
Sep 15 22:35:05.616921 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:05.616899    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847as9c4v" event={"ID":"2c363002-2042-4ed2-b1d5-77b17f716629","Type":"ContainerDied","Data":"c3b7af176d9cfd33bd625b27c6db930fa5fed91a38d46b01e9da4fbece2c8e40"}
Sep 15 22:35:05.617071 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:05.616934    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c3b7af176d9cfd33bd625b27c6db930fa5fed91a38d46b01e9da4fbece2c8e40"
Sep 15 22:35:05.617071 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:05.616907    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847as9c4v"
Sep 15 22:35:09.992107 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:09.992072    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-sd5tr"]
Sep 15 22:35:09.992474 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:09.992392    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c363002-2042-4ed2-b1d5-77b17f716629" containerName="extract"
Sep 15 22:35:09.992474 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:09.992404    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c363002-2042-4ed2-b1d5-77b17f716629" containerName="extract"
Sep 15 22:35:09.992474 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:09.992414    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69a74253-3603-410a-8bc2-f36205eec6b4" containerName="pull"
Sep 15 22:35:09.992474 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:09.992420    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="69a74253-3603-410a-8bc2-f36205eec6b4" containerName="pull"
Sep 15 22:35:09.992474 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:09.992427    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c363002-2042-4ed2-b1d5-77b17f716629" containerName="pull"
Sep 15 22:35:09.992474 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:09.992432    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c363002-2042-4ed2-b1d5-77b17f716629" containerName="pull"
Sep 15 22:35:09.992474 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:09.992438    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="590faf13-cb38-4d1d-b79f-0099f61522b3" containerName="pull"
Sep 15 22:35:09.992474 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:09.992443    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="590faf13-cb38-4d1d-b79f-0099f61522b3" containerName="pull"
Sep 15 22:35:09.992474 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:09.992451    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecff6472-7d0f-4d6e-8a02-a8052effe07a" containerName="pull"
Sep 15 22:35:09.992474 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:09.992457    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecff6472-7d0f-4d6e-8a02-a8052effe07a" containerName="pull"
Sep 15 22:35:09.992474 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:09.992464    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecff6472-7d0f-4d6e-8a02-a8052effe07a" containerName="util"
Sep 15 22:35:09.992474 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:09.992469    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecff6472-7d0f-4d6e-8a02-a8052effe07a" containerName="util"
Sep 15 22:35:09.992474 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:09.992477    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69a74253-3603-410a-8bc2-f36205eec6b4" containerName="util"
Sep 15 22:35:09.992829 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:09.992483    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="69a74253-3603-410a-8bc2-f36205eec6b4" containerName="util"
Sep 15 22:35:09.992829 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:09.992490    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="590faf13-cb38-4d1d-b79f-0099f61522b3" containerName="util"
Sep 15 22:35:09.992829 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:09.992495    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="590faf13-cb38-4d1d-b79f-0099f61522b3" containerName="util"
Sep 15 22:35:09.992829 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:09.992503    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecff6472-7d0f-4d6e-8a02-a8052effe07a" containerName="extract"
Sep 15 22:35:09.992829 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:09.992508    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecff6472-7d0f-4d6e-8a02-a8052effe07a" containerName="extract"
Sep 15 22:35:09.992829 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:09.992517    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69a74253-3603-410a-8bc2-f36205eec6b4" containerName="extract"
Sep 15 22:35:09.992829 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:09.992521    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="69a74253-3603-410a-8bc2-f36205eec6b4" containerName="extract"
Sep 15 22:35:09.992829 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:09.992529    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="590faf13-cb38-4d1d-b79f-0099f61522b3" containerName="extract"
Sep 15 22:35:09.992829 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:09.992533    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="590faf13-cb38-4d1d-b79f-0099f61522b3" containerName="extract"
Sep 15 22:35:09.992829 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:09.992539    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c363002-2042-4ed2-b1d5-77b17f716629" containerName="util"
Sep 15 22:35:09.992829 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:09.992544    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c363002-2042-4ed2-b1d5-77b17f716629" containerName="util"
Sep 15 22:35:09.992829 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:09.992591    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="590faf13-cb38-4d1d-b79f-0099f61522b3" containerName="extract"
Sep 15 22:35:09.992829 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:09.992599    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecff6472-7d0f-4d6e-8a02-a8052effe07a" containerName="extract"
Sep 15 22:35:09.992829 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:09.992604    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c363002-2042-4ed2-b1d5-77b17f716629" containerName="extract"
Sep 15 22:35:09.992829 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:09.992611    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="69a74253-3603-410a-8bc2-f36205eec6b4" containerName="extract"
Sep 15 22:35:09.996802 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:09.996784    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-sd5tr"
Sep 15 22:35:09.999777 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:09.999761    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 15 22:35:10.000500 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:10.000481    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 15 22:35:10.001257 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:10.001240    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-6swxf\""
Sep 15 22:35:10.015543 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:10.015519    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-sd5tr"]
Sep 15 22:35:10.142283 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:10.142244    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tc22\" (UniqueName: \"kubernetes.io/projected/2d37200f-d9ef-47dd-94c6-123e15982ea6-kube-api-access-8tc22\") pod \"limitador-operator-controller-manager-74495c69f8-sd5tr\" (UID: \"2d37200f-d9ef-47dd-94c6-123e15982ea6\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-sd5tr"
Sep 15 22:35:10.243065 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:10.242967    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tc22\" (UniqueName: \"kubernetes.io/projected/2d37200f-d9ef-47dd-94c6-123e15982ea6-kube-api-access-8tc22\") pod \"limitador-operator-controller-manager-74495c69f8-sd5tr\" (UID: \"2d37200f-d9ef-47dd-94c6-123e15982ea6\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-sd5tr"
Sep 15 22:35:10.254228 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:10.254196    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tc22\" (UniqueName: \"kubernetes.io/projected/2d37200f-d9ef-47dd-94c6-123e15982ea6-kube-api-access-8tc22\") pod \"limitador-operator-controller-manager-74495c69f8-sd5tr\" (UID: \"2d37200f-d9ef-47dd-94c6-123e15982ea6\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-sd5tr"
Sep 15 22:35:10.306689 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:10.306658    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-sd5tr"
Sep 15 22:35:10.581270 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:10.581232    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-sd5tr"]
Sep 15 22:35:10.583448 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:35:10.583423    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d37200f_d9ef_47dd_94c6_123e15982ea6.slice/crio-5ddb7572c4f76c05500a14a413d2f25e170090fe27875a6b5e28ee561711e282 WatchSource:0}: Error finding container 5ddb7572c4f76c05500a14a413d2f25e170090fe27875a6b5e28ee561711e282: Status 404 returned error can't find the container with id 5ddb7572c4f76c05500a14a413d2f25e170090fe27875a6b5e28ee561711e282
Sep 15 22:35:10.641728 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:10.641698    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-sd5tr" event={"ID":"2d37200f-d9ef-47dd-94c6-123e15982ea6","Type":"ContainerStarted","Data":"5ddb7572c4f76c05500a14a413d2f25e170090fe27875a6b5e28ee561711e282"}
Sep 15 22:35:12.861198 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:12.861159    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-fhn94"]
Sep 15 22:35:12.864843 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:12.864822    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-fhn94"
Sep 15 22:35:12.867802 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:12.867779    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-fkcv5\""
Sep 15 22:35:12.892417 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:12.892379    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-fhn94"]
Sep 15 22:35:12.966556 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:12.966516    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wjtq\" (UniqueName: \"kubernetes.io/projected/1f5dcd8e-b7f2-4f38-a96a-090ecad96a4f-kube-api-access-8wjtq\") pod \"authorino-operator-76b478bd78-fhn94\" (UID: \"1f5dcd8e-b7f2-4f38-a96a-090ecad96a4f\") " pod="kuadrant-system/authorino-operator-76b478bd78-fhn94"
Sep 15 22:35:13.067052 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:13.067016    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wjtq\" (UniqueName: \"kubernetes.io/projected/1f5dcd8e-b7f2-4f38-a96a-090ecad96a4f-kube-api-access-8wjtq\") pod \"authorino-operator-76b478bd78-fhn94\" (UID: \"1f5dcd8e-b7f2-4f38-a96a-090ecad96a4f\") " pod="kuadrant-system/authorino-operator-76b478bd78-fhn94"
Sep 15 22:35:13.079174 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:13.079139    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wjtq\" (UniqueName: \"kubernetes.io/projected/1f5dcd8e-b7f2-4f38-a96a-090ecad96a4f-kube-api-access-8wjtq\") pod \"authorino-operator-76b478bd78-fhn94\" (UID: \"1f5dcd8e-b7f2-4f38-a96a-090ecad96a4f\") " pod="kuadrant-system/authorino-operator-76b478bd78-fhn94"
Sep 15 22:35:13.179165 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:13.179075    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-fhn94"
Sep 15 22:35:13.453818 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:13.453793    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-fhn94"]
Sep 15 22:35:13.456391 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:35:13.456366    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f5dcd8e_b7f2_4f38_a96a_090ecad96a4f.slice/crio-2c0bd1274b198cfabffcc3472a9cde3199f1076349747e61b31437c2daa5f88b WatchSource:0}: Error finding container 2c0bd1274b198cfabffcc3472a9cde3199f1076349747e61b31437c2daa5f88b: Status 404 returned error can't find the container with id 2c0bd1274b198cfabffcc3472a9cde3199f1076349747e61b31437c2daa5f88b
Sep 15 22:35:13.660434 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:13.660403    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-sd5tr" event={"ID":"2d37200f-d9ef-47dd-94c6-123e15982ea6","Type":"ContainerStarted","Data":"ce1747204e89004bc9525d7d08a970e85636d2e02fbe125aac34e3431a829fd5"}
Sep 15 22:35:13.660622 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:13.660494    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-sd5tr"
Sep 15 22:35:13.661696 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:13.661675    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-fhn94" event={"ID":"1f5dcd8e-b7f2-4f38-a96a-090ecad96a4f","Type":"ContainerStarted","Data":"2c0bd1274b198cfabffcc3472a9cde3199f1076349747e61b31437c2daa5f88b"}
Sep 15 22:35:13.695157 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:13.695114    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-sd5tr" podStartSLOduration=2.246933615 podStartE2EDuration="4.695099391s" totalImagesPullingTime="2.448165776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:35:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:35:13.692246058 +0000 UTC m=+463.551957341" watchObservedRunningTime="2026-09-15 22:35:13.695099391 +0000 UTC m=+463.554810675"
Sep 15 22:35:15.675069 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:15.675018    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-fhn94" event={"ID":"1f5dcd8e-b7f2-4f38-a96a-090ecad96a4f","Type":"ContainerStarted","Data":"e2a0a8ac07de1740b1d3bc4dceca414f01c09c11c9ccd608bc9234c04b9573e8"}
Sep 15 22:35:15.675433 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:15.675104    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-fhn94"
Sep 15 22:35:15.709330 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:15.709283    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-fhn94" podStartSLOduration=2.364407135 podStartE2EDuration="3.709265533s" totalImagesPullingTime="1.344858398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:35:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:35:15.706165759 +0000 UTC m=+465.565877042" watchObservedRunningTime="2026-09-15 22:35:15.709265533 +0000 UTC m=+465.568976816"
Sep 15 22:35:17.398042 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:17.397986    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8jc9d"]
Sep 15 22:35:17.401548 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:17.401528    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8jc9d"
Sep 15 22:35:17.403805 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:17.403787    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-d7f88\""
Sep 15 22:35:17.421253 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:17.421225    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8jc9d"]
Sep 15 22:35:17.509007 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:17.508969    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5tx7\" (UniqueName: \"kubernetes.io/projected/132f7261-2e0c-4491-88f6-0c168d1554d0-kube-api-access-j5tx7\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8jc9d\" (UID: \"132f7261-2e0c-4491-88f6-0c168d1554d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8jc9d"
Sep 15 22:35:17.509007 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:17.509014    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/132f7261-2e0c-4491-88f6-0c168d1554d0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8jc9d\" (UID: \"132f7261-2e0c-4491-88f6-0c168d1554d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8jc9d"
Sep 15 22:35:17.609961 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:17.609922    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5tx7\" (UniqueName: \"kubernetes.io/projected/132f7261-2e0c-4491-88f6-0c168d1554d0-kube-api-access-j5tx7\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8jc9d\" (UID: \"132f7261-2e0c-4491-88f6-0c168d1554d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8jc9d"
Sep 15 22:35:17.609961 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:17.609968    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/132f7261-2e0c-4491-88f6-0c168d1554d0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8jc9d\" (UID: \"132f7261-2e0c-4491-88f6-0c168d1554d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8jc9d"
Sep 15 22:35:17.610334 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:17.610317    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/132f7261-2e0c-4491-88f6-0c168d1554d0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8jc9d\" (UID: \"132f7261-2e0c-4491-88f6-0c168d1554d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8jc9d"
Sep 15 22:35:17.624081 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:17.624045    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5tx7\" (UniqueName: \"kubernetes.io/projected/132f7261-2e0c-4491-88f6-0c168d1554d0-kube-api-access-j5tx7\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8jc9d\" (UID: \"132f7261-2e0c-4491-88f6-0c168d1554d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8jc9d"
Sep 15 22:35:17.711326 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:17.711246    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8jc9d"
Sep 15 22:35:17.997569 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:17.997536    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8jc9d"]
Sep 15 22:35:17.998749 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:35:17.998727    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod132f7261_2e0c_4491_88f6_0c168d1554d0.slice/crio-7f72e4d3b2e26ff301bc9ec8d4e539e4abd10b97cd7f96e96c54b614b49a9832 WatchSource:0}: Error finding container 7f72e4d3b2e26ff301bc9ec8d4e539e4abd10b97cd7f96e96c54b614b49a9832: Status 404 returned error can't find the container with id 7f72e4d3b2e26ff301bc9ec8d4e539e4abd10b97cd7f96e96c54b614b49a9832
Sep 15 22:35:18.692925 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:18.692888    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8jc9d" event={"ID":"132f7261-2e0c-4491-88f6-0c168d1554d0","Type":"ContainerStarted","Data":"7f72e4d3b2e26ff301bc9ec8d4e539e4abd10b97cd7f96e96c54b614b49a9832"}
Sep 15 22:35:22.716910 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:22.716878    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8jc9d" event={"ID":"132f7261-2e0c-4491-88f6-0c168d1554d0","Type":"ContainerStarted","Data":"e22ce7685e4c8d5b171995aaacc7120c64e5dd56d6dbdf5f1dbb39f344211ba0"}
Sep 15 22:35:22.717325 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:22.717011    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8jc9d"
Sep 15 22:35:22.736998 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:22.736942    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8jc9d" podStartSLOduration=1.934190786 podStartE2EDuration="5.736927118s" totalImagesPullingTime="3.802736332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:35:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:35:22.7357389 +0000 UTC m=+472.595450209" watchObservedRunningTime="2026-09-15 22:35:22.736927118 +0000 UTC m=+472.596638401"
Sep 15 22:35:24.367224 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:24.367186    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-f6c7t"]
Sep 15 22:35:24.370381 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:24.370360    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-f6c7t"
Sep 15 22:35:24.372393 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:24.372361    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 15 22:35:24.372475 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:24.372361    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 15 22:35:24.372475 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:24.372371    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-j5m44\""
Sep 15 22:35:24.382375 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:24.382347    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-f6c7t"]
Sep 15 22:35:24.470592 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:24.470561    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtc2h\" (UniqueName: \"kubernetes.io/projected/0d0c07b6-bfae-499a-abdd-e5220cb59c37-kube-api-access-wtc2h\") pod \"kuadrant-console-plugin-fbdd4f445-f6c7t\" (UID: \"0d0c07b6-bfae-499a-abdd-e5220cb59c37\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-f6c7t"
Sep 15 22:35:24.470776 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:24.470611    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0d0c07b6-bfae-499a-abdd-e5220cb59c37-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-f6c7t\" (UID: \"0d0c07b6-bfae-499a-abdd-e5220cb59c37\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-f6c7t"
Sep 15 22:35:24.470776 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:24.470633    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/0d0c07b6-bfae-499a-abdd-e5220cb59c37-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-f6c7t\" (UID: \"0d0c07b6-bfae-499a-abdd-e5220cb59c37\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-f6c7t"
Sep 15 22:35:24.572114 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:24.572076    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtc2h\" (UniqueName: \"kubernetes.io/projected/0d0c07b6-bfae-499a-abdd-e5220cb59c37-kube-api-access-wtc2h\") pod \"kuadrant-console-plugin-fbdd4f445-f6c7t\" (UID: \"0d0c07b6-bfae-499a-abdd-e5220cb59c37\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-f6c7t"
Sep 15 22:35:24.572283 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:24.572123    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0d0c07b6-bfae-499a-abdd-e5220cb59c37-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-f6c7t\" (UID: \"0d0c07b6-bfae-499a-abdd-e5220cb59c37\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-f6c7t"
Sep 15 22:35:24.572283 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:24.572153    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/0d0c07b6-bfae-499a-abdd-e5220cb59c37-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-f6c7t\" (UID: \"0d0c07b6-bfae-499a-abdd-e5220cb59c37\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-f6c7t"
Sep 15 22:35:24.572875 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:24.572854    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0d0c07b6-bfae-499a-abdd-e5220cb59c37-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-f6c7t\" (UID: \"0d0c07b6-bfae-499a-abdd-e5220cb59c37\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-f6c7t"
Sep 15 22:35:24.574575 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:24.574556    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/0d0c07b6-bfae-499a-abdd-e5220cb59c37-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-f6c7t\" (UID: \"0d0c07b6-bfae-499a-abdd-e5220cb59c37\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-f6c7t"
Sep 15 22:35:24.580131 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:24.580107    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtc2h\" (UniqueName: \"kubernetes.io/projected/0d0c07b6-bfae-499a-abdd-e5220cb59c37-kube-api-access-wtc2h\") pod \"kuadrant-console-plugin-fbdd4f445-f6c7t\" (UID: \"0d0c07b6-bfae-499a-abdd-e5220cb59c37\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-f6c7t"
Sep 15 22:35:24.671239 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:24.671146    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-sd5tr"
Sep 15 22:35:24.679634 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:24.679611    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-f6c7t"
Sep 15 22:35:24.990869 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:24.990836    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-f6c7t"]
Sep 15 22:35:24.992507 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:35:24.992478    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0d0c07b6_bfae_499a_abdd_e5220cb59c37.slice/crio-a98ba0e7384fb21d7cef79096ce10a606490fc2285f276445e3c0281b27aa585 WatchSource:0}: Error finding container a98ba0e7384fb21d7cef79096ce10a606490fc2285f276445e3c0281b27aa585: Status 404 returned error can't find the container with id a98ba0e7384fb21d7cef79096ce10a606490fc2285f276445e3c0281b27aa585
Sep 15 22:35:25.739723 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:25.739685    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-f6c7t" event={"ID":"0d0c07b6-bfae-499a-abdd-e5220cb59c37","Type":"ContainerStarted","Data":"a98ba0e7384fb21d7cef79096ce10a606490fc2285f276445e3c0281b27aa585"}
Sep 15 22:35:26.683082 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:26.683051    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-fhn94"
Sep 15 22:35:30.768903 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:30.768866    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-f6c7t" event={"ID":"0d0c07b6-bfae-499a-abdd-e5220cb59c37","Type":"ContainerStarted","Data":"c522700e574e88217002843c2782ecbca1974838a28219a7a8fd8eb3a8f8db3b"}
Sep 15 22:35:30.803123 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:30.803076    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-f6c7t" podStartSLOduration=2.071500857 podStartE2EDuration="6.803059937s" totalImagesPullingTime="4.73155908s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:35:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:35:30.801082169 +0000 UTC m=+480.660793444" watchObservedRunningTime="2026-09-15 22:35:30.803059937 +0000 UTC m=+480.662771220"
Sep 15 22:35:33.729078 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:33.729048    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8jc9d"
Sep 15 22:35:36.126966 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.126930    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8jc9d"]
Sep 15 22:35:36.127444 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.127148    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8jc9d" podUID="132f7261-2e0c-4491-88f6-0c168d1554d0" containerName="manager" containerID="cri-o://e22ce7685e4c8d5b171995aaacc7120c64e5dd56d6dbdf5f1dbb39f344211ba0" gracePeriod=2
Sep 15 22:35:36.134818 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.134786    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8jc9d"]
Sep 15 22:35:36.137417 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:35:36.137382    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-8jc9d\" is forbidden: User \"system:node:ip-10-0-128-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-208.ec2.internal' and this object" podUID="132f7261-2e0c-4491-88f6-0c168d1554d0" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8jc9d"
Sep 15 22:35:36.165923 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.165891    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-sd5tr"]
Sep 15 22:35:36.166255 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.166210    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-sd5tr" podUID="2d37200f-d9ef-47dd-94c6-123e15982ea6" containerName="manager" containerID="cri-o://ce1747204e89004bc9525d7d08a970e85636d2e02fbe125aac34e3431a829fd5" gracePeriod=2
Sep 15 22:35:36.188887 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.188855    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-sd5tr"]
Sep 15 22:35:36.193333 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:35:36.193301    2565 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"2d37200f-d9ef-47dd-94c6-123e15982ea6\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-74495c69f8-sd5tr\": pods \"limitador-operator-controller-manager-74495c69f8-sd5tr\" not found" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-sd5tr"
Sep 15 22:35:36.195733 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:35:36.195705    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-8jc9d\" is forbidden: User \"system:node:ip-10-0-128-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-208.ec2.internal' and this object" podUID="132f7261-2e0c-4491-88f6-0c168d1554d0" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8jc9d"
Sep 15 22:35:36.216392 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.216368    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-gc8cm"]
Sep 15 22:35:36.216705 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.216693    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d37200f-d9ef-47dd-94c6-123e15982ea6" containerName="manager"
Sep 15 22:35:36.216750 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.216706    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d37200f-d9ef-47dd-94c6-123e15982ea6" containerName="manager"
Sep 15 22:35:36.216750 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.216729    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="132f7261-2e0c-4491-88f6-0c168d1554d0" containerName="manager"
Sep 15 22:35:36.216750 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.216735    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="132f7261-2e0c-4491-88f6-0c168d1554d0" containerName="manager"
Sep 15 22:35:36.216839 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.216787    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="132f7261-2e0c-4491-88f6-0c168d1554d0" containerName="manager"
Sep 15 22:35:36.216839 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.216796    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d37200f-d9ef-47dd-94c6-123e15982ea6" containerName="manager"
Sep 15 22:35:36.219718 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.219700    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-gc8cm"
Sep 15 22:35:36.242706 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.242677    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-gc8cm"]
Sep 15 22:35:36.265154 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.265127    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bvgk\" (UniqueName: \"kubernetes.io/projected/6c109d0f-885d-407b-b630-12265bc44be9-kube-api-access-8bvgk\") pod \"limitador-operator-controller-manager-74495c69f8-gc8cm\" (UID: \"6c109d0f-885d-407b-b630-12265bc44be9\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-gc8cm"
Sep 15 22:35:36.272615 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:35:36.272586    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-sd5tr\" is forbidden: User \"system:node:ip-10-0-128-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-208.ec2.internal' and this object" podUID="2d37200f-d9ef-47dd-94c6-123e15982ea6" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-sd5tr"
Sep 15 22:35:36.276146 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:35:36.276111    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-8jc9d\" is forbidden: User \"system:node:ip-10-0-128-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-208.ec2.internal' and this object" podUID="132f7261-2e0c-4491-88f6-0c168d1554d0" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8jc9d"
Sep 15 22:35:36.365590 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.365562    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bvgk\" (UniqueName: \"kubernetes.io/projected/6c109d0f-885d-407b-b630-12265bc44be9-kube-api-access-8bvgk\") pod \"limitador-operator-controller-manager-74495c69f8-gc8cm\" (UID: \"6c109d0f-885d-407b-b630-12265bc44be9\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-gc8cm"
Sep 15 22:35:36.381531 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.381468    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bvgk\" (UniqueName: \"kubernetes.io/projected/6c109d0f-885d-407b-b630-12265bc44be9-kube-api-access-8bvgk\") pod \"limitador-operator-controller-manager-74495c69f8-gc8cm\" (UID: \"6c109d0f-885d-407b-b630-12265bc44be9\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-gc8cm"
Sep 15 22:35:36.406644 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.406622    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8jc9d"
Sep 15 22:35:36.409011 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:35:36.408983    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-sd5tr\" is forbidden: User \"system:node:ip-10-0-128-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-208.ec2.internal' and this object" podUID="2d37200f-d9ef-47dd-94c6-123e15982ea6" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-sd5tr"
Sep 15 22:35:36.409372 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.409349    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-sd5tr"
Sep 15 22:35:36.410664 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:35:36.410641    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-8jc9d\" is forbidden: User \"system:node:ip-10-0-128-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-208.ec2.internal' and this object" podUID="132f7261-2e0c-4491-88f6-0c168d1554d0" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8jc9d"
Sep 15 22:35:36.412439 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:35:36.412418    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-sd5tr\" is forbidden: User \"system:node:ip-10-0-128-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-208.ec2.internal' and this object" podUID="2d37200f-d9ef-47dd-94c6-123e15982ea6" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-sd5tr"
Sep 15 22:35:36.414419 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:35:36.414396    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-8jc9d\" is forbidden: User \"system:node:ip-10-0-128-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-208.ec2.internal' and this object" podUID="132f7261-2e0c-4491-88f6-0c168d1554d0" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8jc9d"
Sep 15 22:35:36.466129 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.466102    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/132f7261-2e0c-4491-88f6-0c168d1554d0-extensions-socket-volume\") pod \"132f7261-2e0c-4491-88f6-0c168d1554d0\" (UID: \"132f7261-2e0c-4491-88f6-0c168d1554d0\") "
Sep 15 22:35:36.466298 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.466151    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j5tx7\" (UniqueName: \"kubernetes.io/projected/132f7261-2e0c-4491-88f6-0c168d1554d0-kube-api-access-j5tx7\") pod \"132f7261-2e0c-4491-88f6-0c168d1554d0\" (UID: \"132f7261-2e0c-4491-88f6-0c168d1554d0\") "
Sep 15 22:35:36.466298 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.466180    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8tc22\" (UniqueName: \"kubernetes.io/projected/2d37200f-d9ef-47dd-94c6-123e15982ea6-kube-api-access-8tc22\") pod \"2d37200f-d9ef-47dd-94c6-123e15982ea6\" (UID: \"2d37200f-d9ef-47dd-94c6-123e15982ea6\") "
Sep 15 22:35:36.466587 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.466558    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/132f7261-2e0c-4491-88f6-0c168d1554d0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "132f7261-2e0c-4491-88f6-0c168d1554d0" (UID: "132f7261-2e0c-4491-88f6-0c168d1554d0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:35:36.467787 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.467764    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfdwh"]
Sep 15 22:35:36.468415 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.468395    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/132f7261-2e0c-4491-88f6-0c168d1554d0-kube-api-access-j5tx7" (OuterVolumeSpecName: "kube-api-access-j5tx7") pod "132f7261-2e0c-4491-88f6-0c168d1554d0" (UID: "132f7261-2e0c-4491-88f6-0c168d1554d0"). InnerVolumeSpecName "kube-api-access-j5tx7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:35:36.468505 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.468484    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d37200f-d9ef-47dd-94c6-123e15982ea6-kube-api-access-8tc22" (OuterVolumeSpecName: "kube-api-access-8tc22") pod "2d37200f-d9ef-47dd-94c6-123e15982ea6" (UID: "2d37200f-d9ef-47dd-94c6-123e15982ea6"). InnerVolumeSpecName "kube-api-access-8tc22". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:35:36.471628 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.471614    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfdwh"
Sep 15 22:35:36.474246 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:35:36.474217    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-sd5tr\" is forbidden: User \"system:node:ip-10-0-128-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-208.ec2.internal' and this object" podUID="2d37200f-d9ef-47dd-94c6-123e15982ea6" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-sd5tr"
Sep 15 22:35:36.476066 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:35:36.476044    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-8jc9d\" is forbidden: User \"system:node:ip-10-0-128-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-208.ec2.internal' and this object" podUID="132f7261-2e0c-4491-88f6-0c168d1554d0" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8jc9d"
Sep 15 22:35:36.486530 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.486503    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfdwh"]
Sep 15 22:35:36.566819 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.566786    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jdjg\" (UniqueName: \"kubernetes.io/projected/f2565baa-d682-4dce-90d3-c486dd114112-kube-api-access-6jdjg\") pod \"kuadrant-operator-controller-manager-5769f99d9f-wfdwh\" (UID: \"f2565baa-d682-4dce-90d3-c486dd114112\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfdwh"
Sep 15 22:35:36.566819 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.566821    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2565baa-d682-4dce-90d3-c486dd114112-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-wfdwh\" (UID: \"f2565baa-d682-4dce-90d3-c486dd114112\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfdwh"
Sep 15 22:35:36.567012 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.566861    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5tx7\" (UniqueName: \"kubernetes.io/projected/132f7261-2e0c-4491-88f6-0c168d1554d0-kube-api-access-j5tx7\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:35:36.567012 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.566875    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8tc22\" (UniqueName: \"kubernetes.io/projected/2d37200f-d9ef-47dd-94c6-123e15982ea6-kube-api-access-8tc22\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:35:36.567012 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.566884    2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/132f7261-2e0c-4491-88f6-0c168d1554d0-extensions-socket-volume\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:35:36.571714 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.571694    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-gc8cm"
Sep 15 22:35:36.591468 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.591439    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="132f7261-2e0c-4491-88f6-0c168d1554d0" path="/var/lib/kubelet/pods/132f7261-2e0c-4491-88f6-0c168d1554d0/volumes"
Sep 15 22:35:36.591765 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.591750    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d37200f-d9ef-47dd-94c6-123e15982ea6" path="/var/lib/kubelet/pods/2d37200f-d9ef-47dd-94c6-123e15982ea6/volumes"
Sep 15 22:35:36.667760 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.667688    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jdjg\" (UniqueName: \"kubernetes.io/projected/f2565baa-d682-4dce-90d3-c486dd114112-kube-api-access-6jdjg\") pod \"kuadrant-operator-controller-manager-5769f99d9f-wfdwh\" (UID: \"f2565baa-d682-4dce-90d3-c486dd114112\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfdwh"
Sep 15 22:35:36.667760 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.667726    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2565baa-d682-4dce-90d3-c486dd114112-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-wfdwh\" (UID: \"f2565baa-d682-4dce-90d3-c486dd114112\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfdwh"
Sep 15 22:35:36.668117 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.668098    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2565baa-d682-4dce-90d3-c486dd114112-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-wfdwh\" (UID: \"f2565baa-d682-4dce-90d3-c486dd114112\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfdwh"
Sep 15 22:35:36.682150 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.682129    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jdjg\" (UniqueName: \"kubernetes.io/projected/f2565baa-d682-4dce-90d3-c486dd114112-kube-api-access-6jdjg\") pod \"kuadrant-operator-controller-manager-5769f99d9f-wfdwh\" (UID: \"f2565baa-d682-4dce-90d3-c486dd114112\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfdwh"
Sep 15 22:35:36.781848 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.781819    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfdwh"
Sep 15 22:35:36.804619 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.804590    2565 generic.go:356] "Generic (PLEG): container finished" podID="2d37200f-d9ef-47dd-94c6-123e15982ea6" containerID="ce1747204e89004bc9525d7d08a970e85636d2e02fbe125aac34e3431a829fd5" exitCode=0
Sep 15 22:35:36.804751 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.804639    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-sd5tr"
Sep 15 22:35:36.804751 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.804690    2565 scope.go:117] "RemoveContainer" containerID="ce1747204e89004bc9525d7d08a970e85636d2e02fbe125aac34e3431a829fd5"
Sep 15 22:35:36.806698 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.806328    2565 generic.go:356] "Generic (PLEG): container finished" podID="132f7261-2e0c-4491-88f6-0c168d1554d0" containerID="e22ce7685e4c8d5b171995aaacc7120c64e5dd56d6dbdf5f1dbb39f344211ba0" exitCode=0
Sep 15 22:35:36.806698 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.806482    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8jc9d"
Sep 15 22:35:36.813590 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.813567    2565 scope.go:117] "RemoveContainer" containerID="ce1747204e89004bc9525d7d08a970e85636d2e02fbe125aac34e3431a829fd5"
Sep 15 22:35:36.813912 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:35:36.813882    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce1747204e89004bc9525d7d08a970e85636d2e02fbe125aac34e3431a829fd5\": container with ID starting with ce1747204e89004bc9525d7d08a970e85636d2e02fbe125aac34e3431a829fd5 not found: ID does not exist" containerID="ce1747204e89004bc9525d7d08a970e85636d2e02fbe125aac34e3431a829fd5"
Sep 15 22:35:36.813980 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.813921    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce1747204e89004bc9525d7d08a970e85636d2e02fbe125aac34e3431a829fd5"} err="failed to get container status \"ce1747204e89004bc9525d7d08a970e85636d2e02fbe125aac34e3431a829fd5\": rpc error: code = NotFound desc = could not find container \"ce1747204e89004bc9525d7d08a970e85636d2e02fbe125aac34e3431a829fd5\": container with ID starting with ce1747204e89004bc9525d7d08a970e85636d2e02fbe125aac34e3431a829fd5 not found: ID does not exist"
Sep 15 22:35:36.813980 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.813939    2565 scope.go:117] "RemoveContainer" containerID="e22ce7685e4c8d5b171995aaacc7120c64e5dd56d6dbdf5f1dbb39f344211ba0"
Sep 15 22:35:36.822399 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.822375    2565 scope.go:117] "RemoveContainer" containerID="e22ce7685e4c8d5b171995aaacc7120c64e5dd56d6dbdf5f1dbb39f344211ba0"
Sep 15 22:35:36.822738 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:35:36.822703    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e22ce7685e4c8d5b171995aaacc7120c64e5dd56d6dbdf5f1dbb39f344211ba0\": container with ID starting with e22ce7685e4c8d5b171995aaacc7120c64e5dd56d6dbdf5f1dbb39f344211ba0 not found: ID does not exist" containerID="e22ce7685e4c8d5b171995aaacc7120c64e5dd56d6dbdf5f1dbb39f344211ba0"
Sep 15 22:35:36.822839 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.822745    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e22ce7685e4c8d5b171995aaacc7120c64e5dd56d6dbdf5f1dbb39f344211ba0"} err="failed to get container status \"e22ce7685e4c8d5b171995aaacc7120c64e5dd56d6dbdf5f1dbb39f344211ba0\": rpc error: code = NotFound desc = could not find container \"e22ce7685e4c8d5b171995aaacc7120c64e5dd56d6dbdf5f1dbb39f344211ba0\": container with ID starting with e22ce7685e4c8d5b171995aaacc7120c64e5dd56d6dbdf5f1dbb39f344211ba0 not found: ID does not exist"
Sep 15 22:35:36.863956 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:36.859957    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-gc8cm"]
Sep 15 22:35:36.867717 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:35:36.867681    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c109d0f_885d_407b_b630_12265bc44be9.slice/crio-611ecf394a4060fc8c64375183c6fccafa4db27ac0be4b35e61549bfbbf961bc WatchSource:0}: Error finding container 611ecf394a4060fc8c64375183c6fccafa4db27ac0be4b35e61549bfbbf961bc: Status 404 returned error can't find the container with id 611ecf394a4060fc8c64375183c6fccafa4db27ac0be4b35e61549bfbbf961bc
Sep 15 22:35:37.082414 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:37.082385    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfdwh"]
Sep 15 22:35:37.083976 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:35:37.083951    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2565baa_d682_4dce_90d3_c486dd114112.slice/crio-5aa512b49b27276feb7500dbb6f0d9fd31ef1d5373063408b39b4ca885fe061a WatchSource:0}: Error finding container 5aa512b49b27276feb7500dbb6f0d9fd31ef1d5373063408b39b4ca885fe061a: Status 404 returned error can't find the container with id 5aa512b49b27276feb7500dbb6f0d9fd31ef1d5373063408b39b4ca885fe061a
Sep 15 22:35:37.815075 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:37.815017    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfdwh" event={"ID":"f2565baa-d682-4dce-90d3-c486dd114112","Type":"ContainerStarted","Data":"e090b86d29010bc75297f1a4ef68ea7e76351bc18813b10c61ff2bdbb47f94d9"}
Sep 15 22:35:37.815524 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:37.815099    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfdwh" event={"ID":"f2565baa-d682-4dce-90d3-c486dd114112","Type":"ContainerStarted","Data":"5aa512b49b27276feb7500dbb6f0d9fd31ef1d5373063408b39b4ca885fe061a"}
Sep 15 22:35:37.815524 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:37.815228    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfdwh"
Sep 15 22:35:37.816748 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:37.816723    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-gc8cm" event={"ID":"6c109d0f-885d-407b-b630-12265bc44be9","Type":"ContainerStarted","Data":"ab868773a7908bd76ab7ef3dc0344229c7ca697abf47170f554bf5d97e520c96"}
Sep 15 22:35:37.816878 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:37.816763    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-gc8cm" event={"ID":"6c109d0f-885d-407b-b630-12265bc44be9","Type":"ContainerStarted","Data":"611ecf394a4060fc8c64375183c6fccafa4db27ac0be4b35e61549bfbbf961bc"}
Sep 15 22:35:37.816878 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:37.816808    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-gc8cm"
Sep 15 22:35:37.842569 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:37.842517    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfdwh" podStartSLOduration=1.842504696 podStartE2EDuration="1.842504696s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:35:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:35:37.841082469 +0000 UTC m=+487.700793755" watchObservedRunningTime="2026-09-15 22:35:37.842504696 +0000 UTC m=+487.702215980"
Sep 15 22:35:37.861854 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:37.861813    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-gc8cm" podStartSLOduration=1.861799674 podStartE2EDuration="1.861799674s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:35:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:35:37.860166859 +0000 UTC m=+487.719878143" watchObservedRunningTime="2026-09-15 22:35:37.861799674 +0000 UTC m=+487.721510958"
Sep 15 22:35:48.824803 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:48.824763    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfdwh"
Sep 15 22:35:48.825218 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:35:48.825149    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-gc8cm"
Sep 15 22:36:09.418651 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:09.418615    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-4q5qn"]
Sep 15 22:36:09.440105 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:09.440072    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-4q5qn"]
Sep 15 22:36:09.440247 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:09.440177    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-4q5qn"
Sep 15 22:36:09.442209 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:09.442189    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 15 22:36:09.515817 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:09.515783    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-4q5qn"]
Sep 15 22:36:09.545052 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:09.545009    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9e556d38-dfaa-45fa-bb60-62358816ab33-config-file\") pod \"limitador-limitador-8588749977-4q5qn\" (UID: \"9e556d38-dfaa-45fa-bb60-62358816ab33\") " pod="kuadrant-system/limitador-limitador-8588749977-4q5qn"
Sep 15 22:36:09.545226 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:09.545061    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vh7gk\" (UniqueName: \"kubernetes.io/projected/9e556d38-dfaa-45fa-bb60-62358816ab33-kube-api-access-vh7gk\") pod \"limitador-limitador-8588749977-4q5qn\" (UID: \"9e556d38-dfaa-45fa-bb60-62358816ab33\") " pod="kuadrant-system/limitador-limitador-8588749977-4q5qn"
Sep 15 22:36:09.645804 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:09.645766    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9e556d38-dfaa-45fa-bb60-62358816ab33-config-file\") pod \"limitador-limitador-8588749977-4q5qn\" (UID: \"9e556d38-dfaa-45fa-bb60-62358816ab33\") " pod="kuadrant-system/limitador-limitador-8588749977-4q5qn"
Sep 15 22:36:09.645969 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:09.645810    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vh7gk\" (UniqueName: \"kubernetes.io/projected/9e556d38-dfaa-45fa-bb60-62358816ab33-kube-api-access-vh7gk\") pod \"limitador-limitador-8588749977-4q5qn\" (UID: \"9e556d38-dfaa-45fa-bb60-62358816ab33\") " pod="kuadrant-system/limitador-limitador-8588749977-4q5qn"
Sep 15 22:36:09.646480 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:09.646460    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9e556d38-dfaa-45fa-bb60-62358816ab33-config-file\") pod \"limitador-limitador-8588749977-4q5qn\" (UID: \"9e556d38-dfaa-45fa-bb60-62358816ab33\") " pod="kuadrant-system/limitador-limitador-8588749977-4q5qn"
Sep 15 22:36:09.654068 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:09.654043    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vh7gk\" (UniqueName: \"kubernetes.io/projected/9e556d38-dfaa-45fa-bb60-62358816ab33-kube-api-access-vh7gk\") pod \"limitador-limitador-8588749977-4q5qn\" (UID: \"9e556d38-dfaa-45fa-bb60-62358816ab33\") " pod="kuadrant-system/limitador-limitador-8588749977-4q5qn"
Sep 15 22:36:09.751061 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:09.750976    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-4q5qn"
Sep 15 22:36:10.026949 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:10.026921    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-4q5qn"]
Sep 15 22:36:10.028755 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:36:10.028734    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e556d38_dfaa_45fa_bb60_62358816ab33.slice/crio-1c97b06980678154ffa32b5b67d9a116de97ef34c5ee3ab07656ba5e41777a2e WatchSource:0}: Error finding container 1c97b06980678154ffa32b5b67d9a116de97ef34c5ee3ab07656ba5e41777a2e: Status 404 returned error can't find the container with id 1c97b06980678154ffa32b5b67d9a116de97ef34c5ee3ab07656ba5e41777a2e
Sep 15 22:36:10.234924 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:10.234893    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-z4p7q"]
Sep 15 22:36:10.239706 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:10.239682    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-z4p7q"
Sep 15 22:36:10.241699 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:10.241671    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-bjhfb\""
Sep 15 22:36:10.245320 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:10.245299    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-z4p7q"]
Sep 15 22:36:10.352431 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:10.352403    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5htfj\" (UniqueName: \"kubernetes.io/projected/3e4d9254-50c7-421b-b631-25812f50914e-kube-api-access-5htfj\") pod \"authorino-7bcfd8f9cd-z4p7q\" (UID: \"3e4d9254-50c7-421b-b631-25812f50914e\") " pod="kuadrant-system/authorino-7bcfd8f9cd-z4p7q"
Sep 15 22:36:10.453489 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:10.453458    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5htfj\" (UniqueName: \"kubernetes.io/projected/3e4d9254-50c7-421b-b631-25812f50914e-kube-api-access-5htfj\") pod \"authorino-7bcfd8f9cd-z4p7q\" (UID: \"3e4d9254-50c7-421b-b631-25812f50914e\") " pod="kuadrant-system/authorino-7bcfd8f9cd-z4p7q"
Sep 15 22:36:10.491668 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:10.491641    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5htfj\" (UniqueName: \"kubernetes.io/projected/3e4d9254-50c7-421b-b631-25812f50914e-kube-api-access-5htfj\") pod \"authorino-7bcfd8f9cd-z4p7q\" (UID: \"3e4d9254-50c7-421b-b631-25812f50914e\") " pod="kuadrant-system/authorino-7bcfd8f9cd-z4p7q"
Sep 15 22:36:10.548845 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:10.548806    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-w7kmd"]
Sep 15 22:36:10.550322 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:10.550300    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-z4p7q"
Sep 15 22:36:10.552475 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:10.552456    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-w7kmd"
Sep 15 22:36:10.560361 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:10.560337    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-w7kmd"]
Sep 15 22:36:10.654558 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:10.654527    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thgw6\" (UniqueName: \"kubernetes.io/projected/eaf42393-7af4-487e-8352-53496e924da3-kube-api-access-thgw6\") pod \"authorino-59c974bd48-w7kmd\" (UID: \"eaf42393-7af4-487e-8352-53496e924da3\") " pod="kuadrant-system/authorino-59c974bd48-w7kmd"
Sep 15 22:36:10.755974 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:10.755943    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thgw6\" (UniqueName: \"kubernetes.io/projected/eaf42393-7af4-487e-8352-53496e924da3-kube-api-access-thgw6\") pod \"authorino-59c974bd48-w7kmd\" (UID: \"eaf42393-7af4-487e-8352-53496e924da3\") " pod="kuadrant-system/authorino-59c974bd48-w7kmd"
Sep 15 22:36:10.764702 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:10.764675    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thgw6\" (UniqueName: \"kubernetes.io/projected/eaf42393-7af4-487e-8352-53496e924da3-kube-api-access-thgw6\") pod \"authorino-59c974bd48-w7kmd\" (UID: \"eaf42393-7af4-487e-8352-53496e924da3\") " pod="kuadrant-system/authorino-59c974bd48-w7kmd"
Sep 15 22:36:10.825766 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:10.825731    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-z4p7q"]
Sep 15 22:36:10.828469 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:36:10.828442    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3e4d9254_50c7_421b_b631_25812f50914e.slice/crio-fa42684662a0a6867ed54cff3d1fb4e4c3565eab21521d8b245f8e2222ce32a2 WatchSource:0}: Error finding container fa42684662a0a6867ed54cff3d1fb4e4c3565eab21521d8b245f8e2222ce32a2: Status 404 returned error can't find the container with id fa42684662a0a6867ed54cff3d1fb4e4c3565eab21521d8b245f8e2222ce32a2
Sep 15 22:36:10.863391 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:10.863323    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-w7kmd"
Sep 15 22:36:10.995696 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:10.995655    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-z4p7q" event={"ID":"3e4d9254-50c7-421b-b631-25812f50914e","Type":"ContainerStarted","Data":"fa42684662a0a6867ed54cff3d1fb4e4c3565eab21521d8b245f8e2222ce32a2"}
Sep 15 22:36:10.996880 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:10.996858    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-4q5qn" event={"ID":"9e556d38-dfaa-45fa-bb60-62358816ab33","Type":"ContainerStarted","Data":"1c97b06980678154ffa32b5b67d9a116de97ef34c5ee3ab07656ba5e41777a2e"}
Sep 15 22:36:11.135813 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:11.135790    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-w7kmd"]
Sep 15 22:36:11.138036 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:36:11.137997    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeaf42393_7af4_487e_8352_53496e924da3.slice/crio-2c5bf68f10f7243b794b5fa7e4b88b92398a53321c2e88d6bd41c26fb4c65935 WatchSource:0}: Error finding container 2c5bf68f10f7243b794b5fa7e4b88b92398a53321c2e88d6bd41c26fb4c65935: Status 404 returned error can't find the container with id 2c5bf68f10f7243b794b5fa7e4b88b92398a53321c2e88d6bd41c26fb4c65935
Sep 15 22:36:12.007471 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:12.007114    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-4q5qn" event={"ID":"9e556d38-dfaa-45fa-bb60-62358816ab33","Type":"ContainerStarted","Data":"f3ade59052ddc5933639c76469224b520874dcec2839be85c0dd5f2ffb8307b1"}
Sep 15 22:36:12.007471 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:12.007389    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-4q5qn"
Sep 15 22:36:12.008860 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:12.008832    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-w7kmd" event={"ID":"eaf42393-7af4-487e-8352-53496e924da3","Type":"ContainerStarted","Data":"2c5bf68f10f7243b794b5fa7e4b88b92398a53321c2e88d6bd41c26fb4c65935"}
Sep 15 22:36:12.024581 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:12.024527    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-4q5qn" podStartSLOduration=1.164947074 podStartE2EDuration="3.024506687s" totalImagesPullingTime="1.859559613s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:36:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:36:12.023326671 +0000 UTC m=+521.883037956" watchObservedRunningTime="2026-09-15 22:36:12.024506687 +0000 UTC m=+521.884217974"
Sep 15 22:36:14.023195 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:14.023144    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-z4p7q" event={"ID":"3e4d9254-50c7-421b-b631-25812f50914e","Type":"ContainerStarted","Data":"52138f117114c808a9496fb1ee48de1e7bd56064a88aa3090b187cf020b146fd"}
Sep 15 22:36:14.024600 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:14.024572    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-w7kmd" event={"ID":"eaf42393-7af4-487e-8352-53496e924da3","Type":"ContainerStarted","Data":"7c1cc70b687bcd29a6a30e91b15909bed3f21c3af9ab9bfa43368501e71de1cb"}
Sep 15 22:36:14.036477 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:14.036203    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-z4p7q" podStartSLOduration=1.85547521 podStartE2EDuration="4.036182925s" totalImagesPullingTime="2.180707715s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:36:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:36:14.035644559 +0000 UTC m=+523.895355844" watchObservedRunningTime="2026-09-15 22:36:14.036182925 +0000 UTC m=+523.895894209"
Sep 15 22:36:14.050407 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:14.050346    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-w7kmd" podStartSLOduration=2.175292759 podStartE2EDuration="4.05032718s" totalImagesPullingTime="1.875034421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:36:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:36:14.048327443 +0000 UTC m=+523.908038729" watchObservedRunningTime="2026-09-15 22:36:14.05032718 +0000 UTC m=+523.910038465"
Sep 15 22:36:14.075506 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:14.075474    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-z4p7q"]
Sep 15 22:36:16.034940 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:16.034902    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-z4p7q" podUID="3e4d9254-50c7-421b-b631-25812f50914e" containerName="authorino" containerID="cri-o://52138f117114c808a9496fb1ee48de1e7bd56064a88aa3090b187cf020b146fd" gracePeriod=30
Sep 15 22:36:16.278502 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:16.278478    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-z4p7q"
Sep 15 22:36:16.404064 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:16.403965    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5htfj\" (UniqueName: \"kubernetes.io/projected/3e4d9254-50c7-421b-b631-25812f50914e-kube-api-access-5htfj\") pod \"3e4d9254-50c7-421b-b631-25812f50914e\" (UID: \"3e4d9254-50c7-421b-b631-25812f50914e\") "
Sep 15 22:36:16.405985 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:16.405957    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e4d9254-50c7-421b-b631-25812f50914e-kube-api-access-5htfj" (OuterVolumeSpecName: "kube-api-access-5htfj") pod "3e4d9254-50c7-421b-b631-25812f50914e" (UID: "3e4d9254-50c7-421b-b631-25812f50914e"). InnerVolumeSpecName "kube-api-access-5htfj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:36:16.504803 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:16.504771    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5htfj\" (UniqueName: \"kubernetes.io/projected/3e4d9254-50c7-421b-b631-25812f50914e-kube-api-access-5htfj\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:36:17.041090 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:17.041058    2565 generic.go:356] "Generic (PLEG): container finished" podID="3e4d9254-50c7-421b-b631-25812f50914e" containerID="52138f117114c808a9496fb1ee48de1e7bd56064a88aa3090b187cf020b146fd" exitCode=0
Sep 15 22:36:17.041478 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:17.041110    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-z4p7q"
Sep 15 22:36:17.041478 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:17.041151    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-z4p7q" event={"ID":"3e4d9254-50c7-421b-b631-25812f50914e","Type":"ContainerDied","Data":"52138f117114c808a9496fb1ee48de1e7bd56064a88aa3090b187cf020b146fd"}
Sep 15 22:36:17.041478 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:17.041197    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-z4p7q" event={"ID":"3e4d9254-50c7-421b-b631-25812f50914e","Type":"ContainerDied","Data":"fa42684662a0a6867ed54cff3d1fb4e4c3565eab21521d8b245f8e2222ce32a2"}
Sep 15 22:36:17.041478 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:17.041216    2565 scope.go:117] "RemoveContainer" containerID="52138f117114c808a9496fb1ee48de1e7bd56064a88aa3090b187cf020b146fd"
Sep 15 22:36:17.049730 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:17.049713    2565 scope.go:117] "RemoveContainer" containerID="52138f117114c808a9496fb1ee48de1e7bd56064a88aa3090b187cf020b146fd"
Sep 15 22:36:17.050005 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:36:17.049983    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52138f117114c808a9496fb1ee48de1e7bd56064a88aa3090b187cf020b146fd\": container with ID starting with 52138f117114c808a9496fb1ee48de1e7bd56064a88aa3090b187cf020b146fd not found: ID does not exist" containerID="52138f117114c808a9496fb1ee48de1e7bd56064a88aa3090b187cf020b146fd"
Sep 15 22:36:17.050129 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:17.050010    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52138f117114c808a9496fb1ee48de1e7bd56064a88aa3090b187cf020b146fd"} err="failed to get container status \"52138f117114c808a9496fb1ee48de1e7bd56064a88aa3090b187cf020b146fd\": rpc error: code = NotFound desc = could not find container \"52138f117114c808a9496fb1ee48de1e7bd56064a88aa3090b187cf020b146fd\": container with ID starting with 52138f117114c808a9496fb1ee48de1e7bd56064a88aa3090b187cf020b146fd not found: ID does not exist"
Sep 15 22:36:17.060971 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:17.060938    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-z4p7q"]
Sep 15 22:36:17.065694 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:17.065666    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-z4p7q"]
Sep 15 22:36:18.590372 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:18.590341    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e4d9254-50c7-421b-b631-25812f50914e" path="/var/lib/kubelet/pods/3e4d9254-50c7-421b-b631-25812f50914e/volumes"
Sep 15 22:36:22.472059 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:22.472010    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-4q5qn"]
Sep 15 22:36:22.472490 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:22.472233    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-4q5qn" podUID="9e556d38-dfaa-45fa-bb60-62358816ab33" containerName="limitador" containerID="cri-o://f3ade59052ddc5933639c76469224b520874dcec2839be85c0dd5f2ffb8307b1" gracePeriod=30
Sep 15 22:36:22.473094 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:22.472937    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-4q5qn"
Sep 15 22:36:23.013704 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:23.013675    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-4q5qn"
Sep 15 22:36:23.075432 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:23.075335    2565 generic.go:356] "Generic (PLEG): container finished" podID="9e556d38-dfaa-45fa-bb60-62358816ab33" containerID="f3ade59052ddc5933639c76469224b520874dcec2839be85c0dd5f2ffb8307b1" exitCode=0
Sep 15 22:36:23.075432 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:23.075382    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-4q5qn"
Sep 15 22:36:23.075432 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:23.075423    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-4q5qn" event={"ID":"9e556d38-dfaa-45fa-bb60-62358816ab33","Type":"ContainerDied","Data":"f3ade59052ddc5933639c76469224b520874dcec2839be85c0dd5f2ffb8307b1"}
Sep 15 22:36:23.075694 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:23.075475    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-4q5qn" event={"ID":"9e556d38-dfaa-45fa-bb60-62358816ab33","Type":"ContainerDied","Data":"1c97b06980678154ffa32b5b67d9a116de97ef34c5ee3ab07656ba5e41777a2e"}
Sep 15 22:36:23.075694 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:23.075498    2565 scope.go:117] "RemoveContainer" containerID="f3ade59052ddc5933639c76469224b520874dcec2839be85c0dd5f2ffb8307b1"
Sep 15 22:36:23.084169 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:23.084151    2565 scope.go:117] "RemoveContainer" containerID="f3ade59052ddc5933639c76469224b520874dcec2839be85c0dd5f2ffb8307b1"
Sep 15 22:36:23.084435 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:36:23.084412    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3ade59052ddc5933639c76469224b520874dcec2839be85c0dd5f2ffb8307b1\": container with ID starting with f3ade59052ddc5933639c76469224b520874dcec2839be85c0dd5f2ffb8307b1 not found: ID does not exist" containerID="f3ade59052ddc5933639c76469224b520874dcec2839be85c0dd5f2ffb8307b1"
Sep 15 22:36:23.084489 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:23.084449    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3ade59052ddc5933639c76469224b520874dcec2839be85c0dd5f2ffb8307b1"} err="failed to get container status \"f3ade59052ddc5933639c76469224b520874dcec2839be85c0dd5f2ffb8307b1\": rpc error: code = NotFound desc = could not find container \"f3ade59052ddc5933639c76469224b520874dcec2839be85c0dd5f2ffb8307b1\": container with ID starting with f3ade59052ddc5933639c76469224b520874dcec2839be85c0dd5f2ffb8307b1 not found: ID does not exist"
Sep 15 22:36:23.160302 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:23.160264    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9e556d38-dfaa-45fa-bb60-62358816ab33-config-file\") pod \"9e556d38-dfaa-45fa-bb60-62358816ab33\" (UID: \"9e556d38-dfaa-45fa-bb60-62358816ab33\") "
Sep 15 22:36:23.160473 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:23.160330    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vh7gk\" (UniqueName: \"kubernetes.io/projected/9e556d38-dfaa-45fa-bb60-62358816ab33-kube-api-access-vh7gk\") pod \"9e556d38-dfaa-45fa-bb60-62358816ab33\" (UID: \"9e556d38-dfaa-45fa-bb60-62358816ab33\") "
Sep 15 22:36:23.160743 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:23.160720    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e556d38-dfaa-45fa-bb60-62358816ab33-config-file" (OuterVolumeSpecName: "config-file") pod "9e556d38-dfaa-45fa-bb60-62358816ab33" (UID: "9e556d38-dfaa-45fa-bb60-62358816ab33"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:36:23.162467 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:23.162446    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e556d38-dfaa-45fa-bb60-62358816ab33-kube-api-access-vh7gk" (OuterVolumeSpecName: "kube-api-access-vh7gk") pod "9e556d38-dfaa-45fa-bb60-62358816ab33" (UID: "9e556d38-dfaa-45fa-bb60-62358816ab33"). InnerVolumeSpecName "kube-api-access-vh7gk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:36:23.261783 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:23.261702    2565 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9e556d38-dfaa-45fa-bb60-62358816ab33-config-file\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:36:23.261783 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:23.261737    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vh7gk\" (UniqueName: \"kubernetes.io/projected/9e556d38-dfaa-45fa-bb60-62358816ab33-kube-api-access-vh7gk\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:36:23.395716 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:23.395683    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-4q5qn"]
Sep 15 22:36:23.401747 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:23.401719    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-4q5qn"]
Sep 15 22:36:24.590468 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:24.590434    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e556d38-dfaa-45fa-bb60-62358816ab33" path="/var/lib/kubelet/pods/9e556d38-dfaa-45fa-bb60-62358816ab33/volumes"
Sep 15 22:36:26.039791 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:26.039760    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-2drtx"]
Sep 15 22:36:26.040152 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:26.040142    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e556d38-dfaa-45fa-bb60-62358816ab33" containerName="limitador"
Sep 15 22:36:26.040193 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:26.040154    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e556d38-dfaa-45fa-bb60-62358816ab33" containerName="limitador"
Sep 15 22:36:26.040193 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:26.040175    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e4d9254-50c7-421b-b631-25812f50914e" containerName="authorino"
Sep 15 22:36:26.040193 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:26.040181    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e4d9254-50c7-421b-b631-25812f50914e" containerName="authorino"
Sep 15 22:36:26.040286 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:26.040237    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e4d9254-50c7-421b-b631-25812f50914e" containerName="authorino"
Sep 15 22:36:26.040286 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:26.040248    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e556d38-dfaa-45fa-bb60-62358816ab33" containerName="limitador"
Sep 15 22:36:26.043309 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:26.043280    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2drtx"
Sep 15 22:36:26.045623 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:26.045599    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 22:36:26.045748 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:26.045725    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 15 22:36:26.046007 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:26.045992    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 15 22:36:26.046126 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:26.046045    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-l98ql\""
Sep 15 22:36:26.058084 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:26.058061    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-2drtx"]
Sep 15 22:36:26.186960 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:26.186928    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdlzs\" (UniqueName: \"kubernetes.io/projected/2ce1a26d-3701-4c0a-96f8-463a1abb4384-kube-api-access-xdlzs\") pod \"postgres-69fb4594cf-2drtx\" (UID: \"2ce1a26d-3701-4c0a-96f8-463a1abb4384\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-2drtx"
Sep 15 22:36:26.187146 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:26.187019    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2ce1a26d-3701-4c0a-96f8-463a1abb4384-data\") pod \"postgres-69fb4594cf-2drtx\" (UID: \"2ce1a26d-3701-4c0a-96f8-463a1abb4384\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-2drtx"
Sep 15 22:36:26.288038 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:26.287995    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2ce1a26d-3701-4c0a-96f8-463a1abb4384-data\") pod \"postgres-69fb4594cf-2drtx\" (UID: \"2ce1a26d-3701-4c0a-96f8-463a1abb4384\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-2drtx"
Sep 15 22:36:26.288175 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:26.288070    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdlzs\" (UniqueName: \"kubernetes.io/projected/2ce1a26d-3701-4c0a-96f8-463a1abb4384-kube-api-access-xdlzs\") pod \"postgres-69fb4594cf-2drtx\" (UID: \"2ce1a26d-3701-4c0a-96f8-463a1abb4384\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-2drtx"
Sep 15 22:36:26.288491 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:26.288474    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2ce1a26d-3701-4c0a-96f8-463a1abb4384-data\") pod \"postgres-69fb4594cf-2drtx\" (UID: \"2ce1a26d-3701-4c0a-96f8-463a1abb4384\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-2drtx"
Sep 15 22:36:26.296494 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:26.296448    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdlzs\" (UniqueName: \"kubernetes.io/projected/2ce1a26d-3701-4c0a-96f8-463a1abb4384-kube-api-access-xdlzs\") pod \"postgres-69fb4594cf-2drtx\" (UID: \"2ce1a26d-3701-4c0a-96f8-463a1abb4384\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-2drtx"
Sep 15 22:36:26.354331 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:26.354302    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2drtx"
Sep 15 22:36:26.625335 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:26.625310    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-2drtx"]
Sep 15 22:36:26.627218 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:36:26.627189    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ce1a26d_3701_4c0a_96f8_463a1abb4384.slice/crio-37f9fe8d47929bb5aee0b3703e3808ebfd28eca92f4926987b253caf641f9ef0 WatchSource:0}: Error finding container 37f9fe8d47929bb5aee0b3703e3808ebfd28eca92f4926987b253caf641f9ef0: Status 404 returned error can't find the container with id 37f9fe8d47929bb5aee0b3703e3808ebfd28eca92f4926987b253caf641f9ef0
Sep 15 22:36:27.098034 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:27.097999    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2drtx" event={"ID":"2ce1a26d-3701-4c0a-96f8-463a1abb4384","Type":"ContainerStarted","Data":"37f9fe8d47929bb5aee0b3703e3808ebfd28eca92f4926987b253caf641f9ef0"}
Sep 15 22:36:29.113643 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:29.113608    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2drtx" event={"ID":"2ce1a26d-3701-4c0a-96f8-463a1abb4384","Type":"ContainerStarted","Data":"473c3e651ab67ecda470b3d0d6026244dbd9f265f76dd090aec7958575cd132b"}
Sep 15 22:36:29.114085 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:29.113708    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2drtx"
Sep 15 22:36:29.131040 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:29.130938    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2drtx" podStartSLOduration=0.853182446 podStartE2EDuration="3.130922996s" totalImagesPullingTime="2.27774055s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:36:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:36:29.130278622 +0000 UTC m=+538.989989905" watchObservedRunningTime="2026-09-15 22:36:29.130922996 +0000 UTC m=+538.990634280"
Sep 15 22:36:35.148623 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:35.148587    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2drtx"
Sep 15 22:36:35.697491 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:35.697457    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-cxj25"]
Sep 15 22:36:35.703487 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:35.703466    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-cxj25"
Sep 15 22:36:35.708518 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:35.708494    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-cxj25"]
Sep 15 22:36:35.765911 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:35.765678    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7t92l\" (UniqueName: \"kubernetes.io/projected/21667a75-308d-44f0-927c-b7eb2cc86122-kube-api-access-7t92l\") pod \"authorino-557f876f69-cxj25\" (UID: \"21667a75-308d-44f0-927c-b7eb2cc86122\") " pod="kuadrant-system/authorino-557f876f69-cxj25"
Sep 15 22:36:35.867708 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:35.867657    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7t92l\" (UniqueName: \"kubernetes.io/projected/21667a75-308d-44f0-927c-b7eb2cc86122-kube-api-access-7t92l\") pod \"authorino-557f876f69-cxj25\" (UID: \"21667a75-308d-44f0-927c-b7eb2cc86122\") " pod="kuadrant-system/authorino-557f876f69-cxj25"
Sep 15 22:36:35.876227 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:35.876200    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7t92l\" (UniqueName: \"kubernetes.io/projected/21667a75-308d-44f0-927c-b7eb2cc86122-kube-api-access-7t92l\") pod \"authorino-557f876f69-cxj25\" (UID: \"21667a75-308d-44f0-927c-b7eb2cc86122\") " pod="kuadrant-system/authorino-557f876f69-cxj25"
Sep 15 22:36:35.933432 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:35.933395    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-cxj25"]
Sep 15 22:36:35.933663 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:35.933652    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-cxj25"
Sep 15 22:36:35.963968 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:35.963901    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-84d49fbdf4-5pbn8"]
Sep 15 22:36:35.969784 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:35.969764    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84d49fbdf4-5pbn8"
Sep 15 22:36:35.974194 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:35.974143    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-84d49fbdf4-5pbn8"]
Sep 15 22:36:36.013793 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:36.013756    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-84d49fbdf4-5pbn8"]
Sep 15 22:36:36.014347 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:36:36.014198    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-hkgxc], unattached volumes=[], failed to process volumes=[kube-api-access-hkgxc]: context canceled" pod="kuadrant-system/authorino-84d49fbdf4-5pbn8" podUID="963b327b-1cbc-4ae3-9944-e1fc1e66bfaa"
Sep 15 22:36:36.045787 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:36.045752    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b49cb4fc6-gtq4g"]
Sep 15 22:36:36.050182 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:36.050162    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b49cb4fc6-gtq4g"
Sep 15 22:36:36.052325 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:36.052305    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 15 22:36:36.059441 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:36.059391    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b49cb4fc6-gtq4g"]
Sep 15 22:36:36.068865 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:36.068843    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slkw9\" (UniqueName: \"kubernetes.io/projected/715cfd0c-a5a2-46ad-9138-4714fc27781c-kube-api-access-slkw9\") pod \"authorino-b49cb4fc6-gtq4g\" (UID: \"715cfd0c-a5a2-46ad-9138-4714fc27781c\") " pod="kuadrant-system/authorino-b49cb4fc6-gtq4g"
Sep 15 22:36:36.068964 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:36.068881    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/715cfd0c-a5a2-46ad-9138-4714fc27781c-tls-cert\") pod \"authorino-b49cb4fc6-gtq4g\" (UID: \"715cfd0c-a5a2-46ad-9138-4714fc27781c\") " pod="kuadrant-system/authorino-b49cb4fc6-gtq4g"
Sep 15 22:36:36.068964 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:36.068940    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkgxc\" (UniqueName: \"kubernetes.io/projected/963b327b-1cbc-4ae3-9944-e1fc1e66bfaa-kube-api-access-hkgxc\") pod \"authorino-84d49fbdf4-5pbn8\" (UID: \"963b327b-1cbc-4ae3-9944-e1fc1e66bfaa\") " pod="kuadrant-system/authorino-84d49fbdf4-5pbn8"
Sep 15 22:36:36.150257 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:36.150221    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84d49fbdf4-5pbn8"
Sep 15 22:36:36.157647 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:36.157618    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84d49fbdf4-5pbn8"
Sep 15 22:36:36.169946 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:36.169922    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slkw9\" (UniqueName: \"kubernetes.io/projected/715cfd0c-a5a2-46ad-9138-4714fc27781c-kube-api-access-slkw9\") pod \"authorino-b49cb4fc6-gtq4g\" (UID: \"715cfd0c-a5a2-46ad-9138-4714fc27781c\") " pod="kuadrant-system/authorino-b49cb4fc6-gtq4g"
Sep 15 22:36:36.170081 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:36.169960    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/715cfd0c-a5a2-46ad-9138-4714fc27781c-tls-cert\") pod \"authorino-b49cb4fc6-gtq4g\" (UID: \"715cfd0c-a5a2-46ad-9138-4714fc27781c\") " pod="kuadrant-system/authorino-b49cb4fc6-gtq4g"
Sep 15 22:36:36.170081 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:36.170041    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkgxc\" (UniqueName: \"kubernetes.io/projected/963b327b-1cbc-4ae3-9944-e1fc1e66bfaa-kube-api-access-hkgxc\") pod \"authorino-84d49fbdf4-5pbn8\" (UID: \"963b327b-1cbc-4ae3-9944-e1fc1e66bfaa\") " pod="kuadrant-system/authorino-84d49fbdf4-5pbn8"
Sep 15 22:36:36.172470 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:36.172447    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/715cfd0c-a5a2-46ad-9138-4714fc27781c-tls-cert\") pod \"authorino-b49cb4fc6-gtq4g\" (UID: \"715cfd0c-a5a2-46ad-9138-4714fc27781c\") " pod="kuadrant-system/authorino-b49cb4fc6-gtq4g"
Sep 15 22:36:36.177305 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:36.177287    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slkw9\" (UniqueName: \"kubernetes.io/projected/715cfd0c-a5a2-46ad-9138-4714fc27781c-kube-api-access-slkw9\") pod \"authorino-b49cb4fc6-gtq4g\" (UID: \"715cfd0c-a5a2-46ad-9138-4714fc27781c\") " pod="kuadrant-system/authorino-b49cb4fc6-gtq4g"
Sep 15 22:36:36.177422 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:36.177405    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkgxc\" (UniqueName: \"kubernetes.io/projected/963b327b-1cbc-4ae3-9944-e1fc1e66bfaa-kube-api-access-hkgxc\") pod \"authorino-84d49fbdf4-5pbn8\" (UID: \"963b327b-1cbc-4ae3-9944-e1fc1e66bfaa\") " pod="kuadrant-system/authorino-84d49fbdf4-5pbn8"
Sep 15 22:36:36.211334 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:36.211312    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-cxj25"]
Sep 15 22:36:36.212888 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:36:36.212864    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod21667a75_308d_44f0_927c_b7eb2cc86122.slice/crio-8ac98e19f231c9d4f3667fe066df67479255ab9381cfe4f63ce9df2e657dff15 WatchSource:0}: Error finding container 8ac98e19f231c9d4f3667fe066df67479255ab9381cfe4f63ce9df2e657dff15: Status 404 returned error can't find the container with id 8ac98e19f231c9d4f3667fe066df67479255ab9381cfe4f63ce9df2e657dff15
Sep 15 22:36:36.270526 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:36.270500    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hkgxc\" (UniqueName: \"kubernetes.io/projected/963b327b-1cbc-4ae3-9944-e1fc1e66bfaa-kube-api-access-hkgxc\") pod \"963b327b-1cbc-4ae3-9944-e1fc1e66bfaa\" (UID: \"963b327b-1cbc-4ae3-9944-e1fc1e66bfaa\") "
Sep 15 22:36:36.272497 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:36.272470    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/963b327b-1cbc-4ae3-9944-e1fc1e66bfaa-kube-api-access-hkgxc" (OuterVolumeSpecName: "kube-api-access-hkgxc") pod "963b327b-1cbc-4ae3-9944-e1fc1e66bfaa" (UID: "963b327b-1cbc-4ae3-9944-e1fc1e66bfaa"). InnerVolumeSpecName "kube-api-access-hkgxc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:36:36.360365 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:36.360334    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b49cb4fc6-gtq4g"
Sep 15 22:36:36.371705 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:36.371682    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkgxc\" (UniqueName: \"kubernetes.io/projected/963b327b-1cbc-4ae3-9944-e1fc1e66bfaa-kube-api-access-hkgxc\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:36:36.632471 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:36.632441    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b49cb4fc6-gtq4g"]
Sep 15 22:36:36.635080 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:36:36.635049    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod715cfd0c_a5a2_46ad_9138_4714fc27781c.slice/crio-dfc63cff9a9b36f66874f955aa1692e435ee90d9ba2bafa09f078d659cec7097 WatchSource:0}: Error finding container dfc63cff9a9b36f66874f955aa1692e435ee90d9ba2bafa09f078d659cec7097: Status 404 returned error can't find the container with id dfc63cff9a9b36f66874f955aa1692e435ee90d9ba2bafa09f078d659cec7097
Sep 15 22:36:37.157088 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:37.156988    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-cxj25" event={"ID":"21667a75-308d-44f0-927c-b7eb2cc86122","Type":"ContainerStarted","Data":"a1541af71526f4b837dd60189915e59a23f3382309b3edaf60e4d8759392937a"}
Sep 15 22:36:37.157088 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:37.157014    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-557f876f69-cxj25" podUID="21667a75-308d-44f0-927c-b7eb2cc86122" containerName="authorino" containerID="cri-o://a1541af71526f4b837dd60189915e59a23f3382309b3edaf60e4d8759392937a" gracePeriod=30
Sep 15 22:36:37.157088 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:37.157059    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-cxj25" event={"ID":"21667a75-308d-44f0-927c-b7eb2cc86122","Type":"ContainerStarted","Data":"8ac98e19f231c9d4f3667fe066df67479255ab9381cfe4f63ce9df2e657dff15"}
Sep 15 22:36:37.159086 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:37.159055    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b49cb4fc6-gtq4g" event={"ID":"715cfd0c-a5a2-46ad-9138-4714fc27781c","Type":"ContainerStarted","Data":"6d32fe94339c421f0aeba1838ec582059acaea550b0ce5ba3248c29ff7ebf022"}
Sep 15 22:36:37.159217 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:37.159103    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b49cb4fc6-gtq4g" event={"ID":"715cfd0c-a5a2-46ad-9138-4714fc27781c","Type":"ContainerStarted","Data":"dfc63cff9a9b36f66874f955aa1692e435ee90d9ba2bafa09f078d659cec7097"}
Sep 15 22:36:37.159217 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:37.159067    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84d49fbdf4-5pbn8"
Sep 15 22:36:37.171677 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:37.171629    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-557f876f69-cxj25" podStartSLOduration=1.846615398 podStartE2EDuration="2.171612534s" totalImagesPullingTime="324.997136ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:36:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:36:37.170397977 +0000 UTC m=+547.030109262" watchObservedRunningTime="2026-09-15 22:36:37.171612534 +0000 UTC m=+547.031323822"
Sep 15 22:36:37.185525 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:37.185475    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b49cb4fc6-gtq4g" podStartSLOduration=0.752320238 podStartE2EDuration="1.185460152s" totalImagesPullingTime="433.139914ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:36:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:36:37.184641457 +0000 UTC m=+547.044352742" watchObservedRunningTime="2026-09-15 22:36:37.185460152 +0000 UTC m=+547.045171435"
Sep 15 22:36:37.185927 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:36:37.185904    2565 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod21667a75_308d_44f0_927c_b7eb2cc86122.slice/crio-a1541af71526f4b837dd60189915e59a23f3382309b3edaf60e4d8759392937a.scope\": RecentStats: unable to find data in memory cache]"
Sep 15 22:36:37.186038 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:36:37.185996    2565 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod21667a75_308d_44f0_927c_b7eb2cc86122.slice/crio-a1541af71526f4b837dd60189915e59a23f3382309b3edaf60e4d8759392937a.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod21667a75_308d_44f0_927c_b7eb2cc86122.slice/crio-conmon-a1541af71526f4b837dd60189915e59a23f3382309b3edaf60e4d8759392937a.scope\": RecentStats: unable to find data in memory cache]"
Sep 15 22:36:37.213008 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:37.212974    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-84d49fbdf4-5pbn8"]
Sep 15 22:36:37.219583 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:37.219529    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-84d49fbdf4-5pbn8"]
Sep 15 22:36:37.222275 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:37.222250    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-w7kmd"]
Sep 15 22:36:37.222435 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:37.222414    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-w7kmd" podUID="eaf42393-7af4-487e-8352-53496e924da3" containerName="authorino" containerID="cri-o://7c1cc70b687bcd29a6a30e91b15909bed3f21c3af9ab9bfa43368501e71de1cb" gracePeriod=30
Sep 15 22:36:37.473916 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:37.473896    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-cxj25"
Sep 15 22:36:37.483511 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:37.483479    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-w7kmd"
Sep 15 22:36:37.584249 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:37.584205    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7t92l\" (UniqueName: \"kubernetes.io/projected/21667a75-308d-44f0-927c-b7eb2cc86122-kube-api-access-7t92l\") pod \"21667a75-308d-44f0-927c-b7eb2cc86122\" (UID: \"21667a75-308d-44f0-927c-b7eb2cc86122\") "
Sep 15 22:36:37.584249 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:37.584248    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-thgw6\" (UniqueName: \"kubernetes.io/projected/eaf42393-7af4-487e-8352-53496e924da3-kube-api-access-thgw6\") pod \"eaf42393-7af4-487e-8352-53496e924da3\" (UID: \"eaf42393-7af4-487e-8352-53496e924da3\") "
Sep 15 22:36:37.586342 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:37.586310    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21667a75-308d-44f0-927c-b7eb2cc86122-kube-api-access-7t92l" (OuterVolumeSpecName: "kube-api-access-7t92l") pod "21667a75-308d-44f0-927c-b7eb2cc86122" (UID: "21667a75-308d-44f0-927c-b7eb2cc86122"). InnerVolumeSpecName "kube-api-access-7t92l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:36:37.586342 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:37.586315    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eaf42393-7af4-487e-8352-53496e924da3-kube-api-access-thgw6" (OuterVolumeSpecName: "kube-api-access-thgw6") pod "eaf42393-7af4-487e-8352-53496e924da3" (UID: "eaf42393-7af4-487e-8352-53496e924da3"). InnerVolumeSpecName "kube-api-access-thgw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:36:37.685643 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:37.685559    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7t92l\" (UniqueName: \"kubernetes.io/projected/21667a75-308d-44f0-927c-b7eb2cc86122-kube-api-access-7t92l\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:36:37.685643 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:37.685588    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thgw6\" (UniqueName: \"kubernetes.io/projected/eaf42393-7af4-487e-8352-53496e924da3-kube-api-access-thgw6\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:36:38.165172 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:38.165140    2565 generic.go:356] "Generic (PLEG): container finished" podID="21667a75-308d-44f0-927c-b7eb2cc86122" containerID="a1541af71526f4b837dd60189915e59a23f3382309b3edaf60e4d8759392937a" exitCode=0
Sep 15 22:36:38.165668 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:38.165203    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-cxj25"
Sep 15 22:36:38.165668 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:38.165232    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-cxj25" event={"ID":"21667a75-308d-44f0-927c-b7eb2cc86122","Type":"ContainerDied","Data":"a1541af71526f4b837dd60189915e59a23f3382309b3edaf60e4d8759392937a"}
Sep 15 22:36:38.165668 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:38.165274    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-cxj25" event={"ID":"21667a75-308d-44f0-927c-b7eb2cc86122","Type":"ContainerDied","Data":"8ac98e19f231c9d4f3667fe066df67479255ab9381cfe4f63ce9df2e657dff15"}
Sep 15 22:36:38.165668 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:38.165297    2565 scope.go:117] "RemoveContainer" containerID="a1541af71526f4b837dd60189915e59a23f3382309b3edaf60e4d8759392937a"
Sep 15 22:36:38.166720 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:38.166702    2565 generic.go:356] "Generic (PLEG): container finished" podID="eaf42393-7af4-487e-8352-53496e924da3" containerID="7c1cc70b687bcd29a6a30e91b15909bed3f21c3af9ab9bfa43368501e71de1cb" exitCode=0
Sep 15 22:36:38.166804 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:38.166757    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-w7kmd"
Sep 15 22:36:38.166804 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:38.166788    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-w7kmd" event={"ID":"eaf42393-7af4-487e-8352-53496e924da3","Type":"ContainerDied","Data":"7c1cc70b687bcd29a6a30e91b15909bed3f21c3af9ab9bfa43368501e71de1cb"}
Sep 15 22:36:38.166921 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:38.166837    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-w7kmd" event={"ID":"eaf42393-7af4-487e-8352-53496e924da3","Type":"ContainerDied","Data":"2c5bf68f10f7243b794b5fa7e4b88b92398a53321c2e88d6bd41c26fb4c65935"}
Sep 15 22:36:38.177001 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:38.176732    2565 scope.go:117] "RemoveContainer" containerID="a1541af71526f4b837dd60189915e59a23f3382309b3edaf60e4d8759392937a"
Sep 15 22:36:38.177172 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:36:38.177150    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1541af71526f4b837dd60189915e59a23f3382309b3edaf60e4d8759392937a\": container with ID starting with a1541af71526f4b837dd60189915e59a23f3382309b3edaf60e4d8759392937a not found: ID does not exist" containerID="a1541af71526f4b837dd60189915e59a23f3382309b3edaf60e4d8759392937a"
Sep 15 22:36:38.177237 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:38.177181    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1541af71526f4b837dd60189915e59a23f3382309b3edaf60e4d8759392937a"} err="failed to get container status \"a1541af71526f4b837dd60189915e59a23f3382309b3edaf60e4d8759392937a\": rpc error: code = NotFound desc = could not find container \"a1541af71526f4b837dd60189915e59a23f3382309b3edaf60e4d8759392937a\": container with ID starting with a1541af71526f4b837dd60189915e59a23f3382309b3edaf60e4d8759392937a not found: ID does not exist"
Sep 15 22:36:38.177237 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:38.177207    2565 scope.go:117] "RemoveContainer" containerID="7c1cc70b687bcd29a6a30e91b15909bed3f21c3af9ab9bfa43368501e71de1cb"
Sep 15 22:36:38.185253 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:38.185236    2565 scope.go:117] "RemoveContainer" containerID="7c1cc70b687bcd29a6a30e91b15909bed3f21c3af9ab9bfa43368501e71de1cb"
Sep 15 22:36:38.185503 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:36:38.185484    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c1cc70b687bcd29a6a30e91b15909bed3f21c3af9ab9bfa43368501e71de1cb\": container with ID starting with 7c1cc70b687bcd29a6a30e91b15909bed3f21c3af9ab9bfa43368501e71de1cb not found: ID does not exist" containerID="7c1cc70b687bcd29a6a30e91b15909bed3f21c3af9ab9bfa43368501e71de1cb"
Sep 15 22:36:38.185551 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:38.185508    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c1cc70b687bcd29a6a30e91b15909bed3f21c3af9ab9bfa43368501e71de1cb"} err="failed to get container status \"7c1cc70b687bcd29a6a30e91b15909bed3f21c3af9ab9bfa43368501e71de1cb\": rpc error: code = NotFound desc = could not find container \"7c1cc70b687bcd29a6a30e91b15909bed3f21c3af9ab9bfa43368501e71de1cb\": container with ID starting with 7c1cc70b687bcd29a6a30e91b15909bed3f21c3af9ab9bfa43368501e71de1cb not found: ID does not exist"
Sep 15 22:36:38.191980 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:38.191960    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-w7kmd"]
Sep 15 22:36:38.195972 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:38.195952    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-w7kmd"]
Sep 15 22:36:38.204461 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:38.204443    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-cxj25"]
Sep 15 22:36:38.208169 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:38.208145    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-cxj25"]
Sep 15 22:36:38.589738 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:38.589709    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21667a75-308d-44f0-927c-b7eb2cc86122" path="/var/lib/kubelet/pods/21667a75-308d-44f0-927c-b7eb2cc86122/volumes"
Sep 15 22:36:38.590056 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:38.590042    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="963b327b-1cbc-4ae3-9944-e1fc1e66bfaa" path="/var/lib/kubelet/pods/963b327b-1cbc-4ae3-9944-e1fc1e66bfaa/volumes"
Sep 15 22:36:38.590243 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:38.590232    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eaf42393-7af4-487e-8352-53496e924da3" path="/var/lib/kubelet/pods/eaf42393-7af4-487e-8352-53496e924da3/volumes"
Sep 15 22:36:42.018789 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:42.018751    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-586c574585-kr8gh"]
Sep 15 22:36:42.019377 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:42.019310    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21667a75-308d-44f0-927c-b7eb2cc86122" containerName="authorino"
Sep 15 22:36:42.019377 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:42.019331    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="21667a75-308d-44f0-927c-b7eb2cc86122" containerName="authorino"
Sep 15 22:36:42.019377 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:42.019348    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eaf42393-7af4-487e-8352-53496e924da3" containerName="authorino"
Sep 15 22:36:42.019377 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:42.019357    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="eaf42393-7af4-487e-8352-53496e924da3" containerName="authorino"
Sep 15 22:36:42.019681 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:42.019453    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="eaf42393-7af4-487e-8352-53496e924da3" containerName="authorino"
Sep 15 22:36:42.019681 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:42.019471    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="21667a75-308d-44f0-927c-b7eb2cc86122" containerName="authorino"
Sep 15 22:36:42.030070 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:42.030019    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-586c574585-kr8gh"
Sep 15 22:36:42.033434 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:42.033411    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 15 22:36:42.033830 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:42.033560    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 15 22:36:42.034052 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:42.034012    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-zbqvm\""
Sep 15 22:36:42.034245 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:42.034228    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 15 22:36:42.034361 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:42.034239    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-586c574585-kr8gh"]
Sep 15 22:36:42.121346 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:42.121315    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqz97\" (UniqueName: \"kubernetes.io/projected/01c2b94b-8d5e-4dad-8d40-54db13ce3b4b-kube-api-access-rqz97\") pod \"maas-controller-586c574585-kr8gh\" (UID: \"01c2b94b-8d5e-4dad-8d40-54db13ce3b4b\") " pod="opendatahub/maas-controller-586c574585-kr8gh"
Sep 15 22:36:42.121496 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:42.121351    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/01c2b94b-8d5e-4dad-8d40-54db13ce3b4b-webhook-cert\") pod \"maas-controller-586c574585-kr8gh\" (UID: \"01c2b94b-8d5e-4dad-8d40-54db13ce3b4b\") " pod="opendatahub/maas-controller-586c574585-kr8gh"
Sep 15 22:36:42.121496 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:42.121453    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01c2b94b-8d5e-4dad-8d40-54db13ce3b4b-metrics-certs\") pod \"maas-controller-586c574585-kr8gh\" (UID: \"01c2b94b-8d5e-4dad-8d40-54db13ce3b4b\") " pod="opendatahub/maas-controller-586c574585-kr8gh"
Sep 15 22:36:42.222591 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:42.222561    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01c2b94b-8d5e-4dad-8d40-54db13ce3b4b-metrics-certs\") pod \"maas-controller-586c574585-kr8gh\" (UID: \"01c2b94b-8d5e-4dad-8d40-54db13ce3b4b\") " pod="opendatahub/maas-controller-586c574585-kr8gh"
Sep 15 22:36:42.222773 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:42.222608    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqz97\" (UniqueName: \"kubernetes.io/projected/01c2b94b-8d5e-4dad-8d40-54db13ce3b4b-kube-api-access-rqz97\") pod \"maas-controller-586c574585-kr8gh\" (UID: \"01c2b94b-8d5e-4dad-8d40-54db13ce3b4b\") " pod="opendatahub/maas-controller-586c574585-kr8gh"
Sep 15 22:36:42.222773 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:42.222626    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/01c2b94b-8d5e-4dad-8d40-54db13ce3b4b-webhook-cert\") pod \"maas-controller-586c574585-kr8gh\" (UID: \"01c2b94b-8d5e-4dad-8d40-54db13ce3b4b\") " pod="opendatahub/maas-controller-586c574585-kr8gh"
Sep 15 22:36:42.222773 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:36:42.222735    2565 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 15 22:36:42.223062 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:36:42.222806    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01c2b94b-8d5e-4dad-8d40-54db13ce3b4b-webhook-cert podName:01c2b94b-8d5e-4dad-8d40-54db13ce3b4b nodeName:}" failed. No retries permitted until 2026-09-15 22:36:42.722790267 +0000 UTC m=+552.582501533 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/01c2b94b-8d5e-4dad-8d40-54db13ce3b4b-webhook-cert") pod "maas-controller-586c574585-kr8gh" (UID: "01c2b94b-8d5e-4dad-8d40-54db13ce3b4b") : secret "maas-controller-webhook-cert" not found
Sep 15 22:36:42.225224 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:42.225204    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01c2b94b-8d5e-4dad-8d40-54db13ce3b4b-metrics-certs\") pod \"maas-controller-586c574585-kr8gh\" (UID: \"01c2b94b-8d5e-4dad-8d40-54db13ce3b4b\") " pod="opendatahub/maas-controller-586c574585-kr8gh"
Sep 15 22:36:42.230415 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:42.230391    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqz97\" (UniqueName: \"kubernetes.io/projected/01c2b94b-8d5e-4dad-8d40-54db13ce3b4b-kube-api-access-rqz97\") pod \"maas-controller-586c574585-kr8gh\" (UID: \"01c2b94b-8d5e-4dad-8d40-54db13ce3b4b\") " pod="opendatahub/maas-controller-586c574585-kr8gh"
Sep 15 22:36:42.726278 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:42.726228    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/01c2b94b-8d5e-4dad-8d40-54db13ce3b4b-webhook-cert\") pod \"maas-controller-586c574585-kr8gh\" (UID: \"01c2b94b-8d5e-4dad-8d40-54db13ce3b4b\") " pod="opendatahub/maas-controller-586c574585-kr8gh"
Sep 15 22:36:42.728485 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:42.728466    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/01c2b94b-8d5e-4dad-8d40-54db13ce3b4b-webhook-cert\") pod \"maas-controller-586c574585-kr8gh\" (UID: \"01c2b94b-8d5e-4dad-8d40-54db13ce3b4b\") " pod="opendatahub/maas-controller-586c574585-kr8gh"
Sep 15 22:36:42.945560 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:42.945514    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-586c574585-kr8gh"
Sep 15 22:36:43.219529 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:43.219467    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-586c574585-kr8gh"]
Sep 15 22:36:43.221871 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:36:43.221840    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01c2b94b_8d5e_4dad_8d40_54db13ce3b4b.slice/crio-c5ebdd204c00a97738a50ff1d3b02c76ad07e11453262b989eaedaccf57491c5 WatchSource:0}: Error finding container c5ebdd204c00a97738a50ff1d3b02c76ad07e11453262b989eaedaccf57491c5: Status 404 returned error can't find the container with id c5ebdd204c00a97738a50ff1d3b02c76ad07e11453262b989eaedaccf57491c5
Sep 15 22:36:44.203790 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:44.203754    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-586c574585-kr8gh" event={"ID":"01c2b94b-8d5e-4dad-8d40-54db13ce3b4b","Type":"ContainerStarted","Data":"c5ebdd204c00a97738a50ff1d3b02c76ad07e11453262b989eaedaccf57491c5"}
Sep 15 22:36:46.216826 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:46.216788    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-586c574585-kr8gh" event={"ID":"01c2b94b-8d5e-4dad-8d40-54db13ce3b4b","Type":"ContainerStarted","Data":"10a6bdf36c0bc251b9b10c49ae2193067e9d78c66da15b2232c9e433f4d5b880"}
Sep 15 22:36:46.217234 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:46.216867    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-586c574585-kr8gh"
Sep 15 22:36:46.237354 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:46.237308    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-586c574585-kr8gh" podStartSLOduration=3.069486995 podStartE2EDuration="5.237293033s" totalImagesPullingTime="2.167806038s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:36:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:36:46.235262011 +0000 UTC m=+556.094973298" watchObservedRunningTime="2026-09-15 22:36:46.237293033 +0000 UTC m=+556.097004366"
Sep 15 22:36:56.633338 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:56.633298    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-586c574585-kr8gh"]
Sep 15 22:36:56.633759 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:56.633484    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-586c574585-kr8gh" podUID="01c2b94b-8d5e-4dad-8d40-54db13ce3b4b" containerName="manager" containerID="cri-o://10a6bdf36c0bc251b9b10c49ae2193067e9d78c66da15b2232c9e433f4d5b880" gracePeriod=30
Sep 15 22:36:56.635018 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:56.634992    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-586c574585-kr8gh"
Sep 15 22:36:56.882928 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:56.882904    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-586c574585-kr8gh"
Sep 15 22:36:56.955533 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:56.955459    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/01c2b94b-8d5e-4dad-8d40-54db13ce3b4b-webhook-cert\") pod \"01c2b94b-8d5e-4dad-8d40-54db13ce3b4b\" (UID: \"01c2b94b-8d5e-4dad-8d40-54db13ce3b4b\") "
Sep 15 22:36:56.955533 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:56.955496    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rqz97\" (UniqueName: \"kubernetes.io/projected/01c2b94b-8d5e-4dad-8d40-54db13ce3b4b-kube-api-access-rqz97\") pod \"01c2b94b-8d5e-4dad-8d40-54db13ce3b4b\" (UID: \"01c2b94b-8d5e-4dad-8d40-54db13ce3b4b\") "
Sep 15 22:36:56.955695 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:56.955565    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01c2b94b-8d5e-4dad-8d40-54db13ce3b4b-metrics-certs\") pod \"01c2b94b-8d5e-4dad-8d40-54db13ce3b4b\" (UID: \"01c2b94b-8d5e-4dad-8d40-54db13ce3b4b\") "
Sep 15 22:36:56.957654 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:56.957622    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01c2b94b-8d5e-4dad-8d40-54db13ce3b4b-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "01c2b94b-8d5e-4dad-8d40-54db13ce3b4b" (UID: "01c2b94b-8d5e-4dad-8d40-54db13ce3b4b"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:36:56.957759 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:56.957679    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01c2b94b-8d5e-4dad-8d40-54db13ce3b4b-kube-api-access-rqz97" (OuterVolumeSpecName: "kube-api-access-rqz97") pod "01c2b94b-8d5e-4dad-8d40-54db13ce3b4b" (UID: "01c2b94b-8d5e-4dad-8d40-54db13ce3b4b"). InnerVolumeSpecName "kube-api-access-rqz97". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:36:56.957759 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:56.957748    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01c2b94b-8d5e-4dad-8d40-54db13ce3b4b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "01c2b94b-8d5e-4dad-8d40-54db13ce3b4b" (UID: "01c2b94b-8d5e-4dad-8d40-54db13ce3b4b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:36:57.056534 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:57.056504    2565 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/01c2b94b-8d5e-4dad-8d40-54db13ce3b4b-webhook-cert\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:36:57.056534 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:57.056529    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rqz97\" (UniqueName: \"kubernetes.io/projected/01c2b94b-8d5e-4dad-8d40-54db13ce3b4b-kube-api-access-rqz97\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:36:57.056534 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:57.056539    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01c2b94b-8d5e-4dad-8d40-54db13ce3b4b-metrics-certs\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:36:57.278690 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:57.278603    2565 generic.go:356] "Generic (PLEG): container finished" podID="01c2b94b-8d5e-4dad-8d40-54db13ce3b4b" containerID="10a6bdf36c0bc251b9b10c49ae2193067e9d78c66da15b2232c9e433f4d5b880" exitCode=0
Sep 15 22:36:57.278690 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:57.278652    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-586c574585-kr8gh"
Sep 15 22:36:57.278858 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:57.278692    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-586c574585-kr8gh" event={"ID":"01c2b94b-8d5e-4dad-8d40-54db13ce3b4b","Type":"ContainerDied","Data":"10a6bdf36c0bc251b9b10c49ae2193067e9d78c66da15b2232c9e433f4d5b880"}
Sep 15 22:36:57.278858 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:57.278745    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-586c574585-kr8gh" event={"ID":"01c2b94b-8d5e-4dad-8d40-54db13ce3b4b","Type":"ContainerDied","Data":"c5ebdd204c00a97738a50ff1d3b02c76ad07e11453262b989eaedaccf57491c5"}
Sep 15 22:36:57.278858 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:57.278770    2565 scope.go:117] "RemoveContainer" containerID="10a6bdf36c0bc251b9b10c49ae2193067e9d78c66da15b2232c9e433f4d5b880"
Sep 15 22:36:57.288169 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:57.288152    2565 scope.go:117] "RemoveContainer" containerID="10a6bdf36c0bc251b9b10c49ae2193067e9d78c66da15b2232c9e433f4d5b880"
Sep 15 22:36:57.288443 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:36:57.288423    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10a6bdf36c0bc251b9b10c49ae2193067e9d78c66da15b2232c9e433f4d5b880\": container with ID starting with 10a6bdf36c0bc251b9b10c49ae2193067e9d78c66da15b2232c9e433f4d5b880 not found: ID does not exist" containerID="10a6bdf36c0bc251b9b10c49ae2193067e9d78c66da15b2232c9e433f4d5b880"
Sep 15 22:36:57.288497 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:57.288452    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10a6bdf36c0bc251b9b10c49ae2193067e9d78c66da15b2232c9e433f4d5b880"} err="failed to get container status \"10a6bdf36c0bc251b9b10c49ae2193067e9d78c66da15b2232c9e433f4d5b880\": rpc error: code = NotFound desc = could not find container \"10a6bdf36c0bc251b9b10c49ae2193067e9d78c66da15b2232c9e433f4d5b880\": container with ID starting with 10a6bdf36c0bc251b9b10c49ae2193067e9d78c66da15b2232c9e433f4d5b880 not found: ID does not exist"
Sep 15 22:36:57.299795 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:57.299766    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-586c574585-kr8gh"]
Sep 15 22:36:57.305700 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:57.305679    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-586c574585-kr8gh"]
Sep 15 22:36:58.590306 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:36:58.590270    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01c2b94b-8d5e-4dad-8d40-54db13ce3b4b" path="/var/lib/kubelet/pods/01c2b94b-8d5e-4dad-8d40-54db13ce3b4b/volumes"
Sep 15 22:37:18.846207 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:18.846175    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-98857cbf4-hpw2w"]
Sep 15 22:37:18.846667 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:18.846535    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01c2b94b-8d5e-4dad-8d40-54db13ce3b4b" containerName="manager"
Sep 15 22:37:18.846667 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:18.846548    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="01c2b94b-8d5e-4dad-8d40-54db13ce3b4b" containerName="manager"
Sep 15 22:37:18.846667 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:18.846647    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="01c2b94b-8d5e-4dad-8d40-54db13ce3b4b" containerName="manager"
Sep 15 22:37:18.852487 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:18.852463    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-98857cbf4-hpw2w"
Sep 15 22:37:18.855348 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:18.855316    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 15 22:37:18.855755 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:18.855355    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tsrzq\""
Sep 15 22:37:18.856141 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:18.855489    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 15 22:37:18.865673 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:18.865505    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-98857cbf4-hpw2w"]
Sep 15 22:37:19.034372 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:19.034336    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34b70b32-cf8b-4b40-9882-dfa432ee13d4-metrics-certs\") pod \"maas-api-98857cbf4-hpw2w\" (UID: \"34b70b32-cf8b-4b40-9882-dfa432ee13d4\") " pod="odh-ai-gateway-infra/maas-api-98857cbf4-hpw2w"
Sep 15 22:37:19.034372 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:19.034376    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tv8gb\" (UniqueName: \"kubernetes.io/projected/34b70b32-cf8b-4b40-9882-dfa432ee13d4-kube-api-access-tv8gb\") pod \"maas-api-98857cbf4-hpw2w\" (UID: \"34b70b32-cf8b-4b40-9882-dfa432ee13d4\") " pod="odh-ai-gateway-infra/maas-api-98857cbf4-hpw2w"
Sep 15 22:37:19.034580 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:19.034413    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34b70b32-cf8b-4b40-9882-dfa432ee13d4-maas-api-tls\") pod \"maas-api-98857cbf4-hpw2w\" (UID: \"34b70b32-cf8b-4b40-9882-dfa432ee13d4\") " pod="odh-ai-gateway-infra/maas-api-98857cbf4-hpw2w"
Sep 15 22:37:19.034580 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:19.034453    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34b70b32-cf8b-4b40-9882-dfa432ee13d4-openshift-service-ca\") pod \"maas-api-98857cbf4-hpw2w\" (UID: \"34b70b32-cf8b-4b40-9882-dfa432ee13d4\") " pod="odh-ai-gateway-infra/maas-api-98857cbf4-hpw2w"
Sep 15 22:37:19.135713 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:19.135637    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34b70b32-cf8b-4b40-9882-dfa432ee13d4-openshift-service-ca\") pod \"maas-api-98857cbf4-hpw2w\" (UID: \"34b70b32-cf8b-4b40-9882-dfa432ee13d4\") " pod="odh-ai-gateway-infra/maas-api-98857cbf4-hpw2w"
Sep 15 22:37:19.135713 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:19.135686    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34b70b32-cf8b-4b40-9882-dfa432ee13d4-metrics-certs\") pod \"maas-api-98857cbf4-hpw2w\" (UID: \"34b70b32-cf8b-4b40-9882-dfa432ee13d4\") " pod="odh-ai-gateway-infra/maas-api-98857cbf4-hpw2w"
Sep 15 22:37:19.135713 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:19.135709    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv8gb\" (UniqueName: \"kubernetes.io/projected/34b70b32-cf8b-4b40-9882-dfa432ee13d4-kube-api-access-tv8gb\") pod \"maas-api-98857cbf4-hpw2w\" (UID: \"34b70b32-cf8b-4b40-9882-dfa432ee13d4\") " pod="odh-ai-gateway-infra/maas-api-98857cbf4-hpw2w"
Sep 15 22:37:19.135973 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:19.135742    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34b70b32-cf8b-4b40-9882-dfa432ee13d4-maas-api-tls\") pod \"maas-api-98857cbf4-hpw2w\" (UID: \"34b70b32-cf8b-4b40-9882-dfa432ee13d4\") " pod="odh-ai-gateway-infra/maas-api-98857cbf4-hpw2w"
Sep 15 22:37:19.135973 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:37:19.135844    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found
Sep 15 22:37:19.135973 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:37:19.135917    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34b70b32-cf8b-4b40-9882-dfa432ee13d4-metrics-certs podName:34b70b32-cf8b-4b40-9882-dfa432ee13d4 nodeName:}" failed. No retries permitted until 2026-09-15 22:37:19.635897121 +0000 UTC m=+589.495608385 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/34b70b32-cf8b-4b40-9882-dfa432ee13d4-metrics-certs") pod "maas-api-98857cbf4-hpw2w" (UID: "34b70b32-cf8b-4b40-9882-dfa432ee13d4") : secret "maas-api-metrics-tls" not found
Sep 15 22:37:19.136272 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:19.136252    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34b70b32-cf8b-4b40-9882-dfa432ee13d4-openshift-service-ca\") pod \"maas-api-98857cbf4-hpw2w\" (UID: \"34b70b32-cf8b-4b40-9882-dfa432ee13d4\") " pod="odh-ai-gateway-infra/maas-api-98857cbf4-hpw2w"
Sep 15 22:37:19.138035 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:19.138010    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34b70b32-cf8b-4b40-9882-dfa432ee13d4-maas-api-tls\") pod \"maas-api-98857cbf4-hpw2w\" (UID: \"34b70b32-cf8b-4b40-9882-dfa432ee13d4\") " pod="odh-ai-gateway-infra/maas-api-98857cbf4-hpw2w"
Sep 15 22:37:19.144244 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:19.144220    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv8gb\" (UniqueName: \"kubernetes.io/projected/34b70b32-cf8b-4b40-9882-dfa432ee13d4-kube-api-access-tv8gb\") pod \"maas-api-98857cbf4-hpw2w\" (UID: \"34b70b32-cf8b-4b40-9882-dfa432ee13d4\") " pod="odh-ai-gateway-infra/maas-api-98857cbf4-hpw2w"
Sep 15 22:37:19.366802 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:19.366769    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-fgpqt"]
Sep 15 22:37:19.370330 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:19.370306    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fgpqt"
Sep 15 22:37:19.373892 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:19.373869    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 15 22:37:19.384090 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:19.384064    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-fgpqt"]
Sep 15 22:37:19.539268 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:19.539226    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd29c53d-5d08-47be-ba67-0e959c0ee312-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-fgpqt\" (UID: \"dd29c53d-5d08-47be-ba67-0e959c0ee312\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fgpqt"
Sep 15 22:37:19.539418 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:19.539366    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7wdk\" (UniqueName: \"kubernetes.io/projected/dd29c53d-5d08-47be-ba67-0e959c0ee312-kube-api-access-z7wdk\") pod \"payload-pre-processing-59fbc47bf8-fgpqt\" (UID: \"dd29c53d-5d08-47be-ba67-0e959c0ee312\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fgpqt"
Sep 15 22:37:19.640634 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:19.640598    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd29c53d-5d08-47be-ba67-0e959c0ee312-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-fgpqt\" (UID: \"dd29c53d-5d08-47be-ba67-0e959c0ee312\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fgpqt"
Sep 15 22:37:19.640824 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:19.640689    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7wdk\" (UniqueName: \"kubernetes.io/projected/dd29c53d-5d08-47be-ba67-0e959c0ee312-kube-api-access-z7wdk\") pod \"payload-pre-processing-59fbc47bf8-fgpqt\" (UID: \"dd29c53d-5d08-47be-ba67-0e959c0ee312\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fgpqt"
Sep 15 22:37:19.640824 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:19.640737    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34b70b32-cf8b-4b40-9882-dfa432ee13d4-metrics-certs\") pod \"maas-api-98857cbf4-hpw2w\" (UID: \"34b70b32-cf8b-4b40-9882-dfa432ee13d4\") " pod="odh-ai-gateway-infra/maas-api-98857cbf4-hpw2w"
Sep 15 22:37:19.641338 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:19.641314    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd29c53d-5d08-47be-ba67-0e959c0ee312-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-fgpqt\" (UID: \"dd29c53d-5d08-47be-ba67-0e959c0ee312\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fgpqt"
Sep 15 22:37:19.643132 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:19.643109    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34b70b32-cf8b-4b40-9882-dfa432ee13d4-metrics-certs\") pod \"maas-api-98857cbf4-hpw2w\" (UID: \"34b70b32-cf8b-4b40-9882-dfa432ee13d4\") " pod="odh-ai-gateway-infra/maas-api-98857cbf4-hpw2w"
Sep 15 22:37:19.653228 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:19.653200    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7wdk\" (UniqueName: \"kubernetes.io/projected/dd29c53d-5d08-47be-ba67-0e959c0ee312-kube-api-access-z7wdk\") pod \"payload-pre-processing-59fbc47bf8-fgpqt\" (UID: \"dd29c53d-5d08-47be-ba67-0e959c0ee312\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fgpqt"
Sep 15 22:37:19.682089 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:19.682058    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fgpqt"
Sep 15 22:37:19.771583 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:19.771206    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-98857cbf4-hpw2w"
Sep 15 22:37:19.965259 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:19.965233    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-fgpqt"]
Sep 15 22:37:19.967673 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:37:19.967644    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd29c53d_5d08_47be_ba67_0e959c0ee312.slice/crio-b3448a27bd53c98af606485423596f78c477e4978ff669d3818426d8f607a09d WatchSource:0}: Error finding container b3448a27bd53c98af606485423596f78c477e4978ff669d3818426d8f607a09d: Status 404 returned error can't find the container with id b3448a27bd53c98af606485423596f78c477e4978ff669d3818426d8f607a09d
Sep 15 22:37:20.048990 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:20.048968    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-98857cbf4-hpw2w"]
Sep 15 22:37:20.050801 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:37:20.050773    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34b70b32_cf8b_4b40_9882_dfa432ee13d4.slice/crio-92c8d1d6e0b7310ac4764beed36c81e3030285bf6889d25081312648fb641079 WatchSource:0}: Error finding container 92c8d1d6e0b7310ac4764beed36c81e3030285bf6889d25081312648fb641079: Status 404 returned error can't find the container with id 92c8d1d6e0b7310ac4764beed36c81e3030285bf6889d25081312648fb641079
Sep 15 22:37:20.408485 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:20.408451    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fgpqt" event={"ID":"dd29c53d-5d08-47be-ba67-0e959c0ee312","Type":"ContainerStarted","Data":"b3448a27bd53c98af606485423596f78c477e4978ff669d3818426d8f607a09d"}
Sep 15 22:37:20.409826 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:20.409796    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-98857cbf4-hpw2w" event={"ID":"34b70b32-cf8b-4b40-9882-dfa432ee13d4","Type":"ContainerStarted","Data":"92c8d1d6e0b7310ac4764beed36c81e3030285bf6889d25081312648fb641079"}
Sep 15 22:37:22.423954 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:22.423912    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-98857cbf4-hpw2w" event={"ID":"34b70b32-cf8b-4b40-9882-dfa432ee13d4","Type":"ContainerStarted","Data":"e20cacf4f7c79a3c5f77ddc94e1ee01bb27c15ee29738e04990e61878e84dccd"}
Sep 15 22:37:22.424444 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:22.424057    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-98857cbf4-hpw2w"
Sep 15 22:37:22.425546 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:22.425511    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fgpqt" event={"ID":"dd29c53d-5d08-47be-ba67-0e959c0ee312","Type":"ContainerStarted","Data":"5e8b7a877142af5c52a8b470dfb3925c94c37bd22580c6edefbc1cf2256f89d8"}
Sep 15 22:37:22.425640 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:22.425620    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fgpqt"
Sep 15 22:37:22.441477 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:22.441431    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-98857cbf4-hpw2w" podStartSLOduration=2.777153756 podStartE2EDuration="4.441414107s" totalImagesPullingTime="1.664260351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:37:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:37:22.440651943 +0000 UTC m=+592.300363230" watchObservedRunningTime="2026-09-15 22:37:22.441414107 +0000 UTC m=+592.301125524"
Sep 15 22:37:22.457242 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:22.457208    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fgpqt" podStartSLOduration=1.713200063 podStartE2EDuration="3.457196296s" totalImagesPullingTime="1.743996233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:37:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:37:22.455621613 +0000 UTC m=+592.315332898" watchObservedRunningTime="2026-09-15 22:37:22.457196296 +0000 UTC m=+592.316907579"
Sep 15 22:37:28.436838 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:28.436810    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-98857cbf4-hpw2w"
Sep 15 22:37:33.433077 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:33.433045    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fgpqt"
Sep 15 22:37:46.864593 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:46.864560    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-k4ln9"]
Sep 15 22:37:46.868039 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:46.867992    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k4ln9"
Sep 15 22:37:46.870480 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:46.870458    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 15 22:37:46.870480 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:46.870476    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 15 22:37:46.870669 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:46.870461    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 15 22:37:46.870669 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:46.870463    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-582dz\""
Sep 15 22:37:46.878261 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:46.878239    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-k4ln9"]
Sep 15 22:37:46.973274 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:46.973240    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c4d06c9-88c6-421a-ba3b-97bcc86e667e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k4ln9\" (UID: \"4c4d06c9-88c6-421a-ba3b-97bcc86e667e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k4ln9"
Sep 15 22:37:46.973274 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:46.973273    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66sd2\" (UniqueName: \"kubernetes.io/projected/4c4d06c9-88c6-421a-ba3b-97bcc86e667e-kube-api-access-66sd2\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k4ln9\" (UID: \"4c4d06c9-88c6-421a-ba3b-97bcc86e667e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k4ln9"
Sep 15 22:37:46.973464 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:46.973305    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c4d06c9-88c6-421a-ba3b-97bcc86e667e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k4ln9\" (UID: \"4c4d06c9-88c6-421a-ba3b-97bcc86e667e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k4ln9"
Sep 15 22:37:46.973464 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:46.973353    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c4d06c9-88c6-421a-ba3b-97bcc86e667e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k4ln9\" (UID: \"4c4d06c9-88c6-421a-ba3b-97bcc86e667e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k4ln9"
Sep 15 22:37:46.973464 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:46.973414    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c4d06c9-88c6-421a-ba3b-97bcc86e667e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k4ln9\" (UID: \"4c4d06c9-88c6-421a-ba3b-97bcc86e667e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k4ln9"
Sep 15 22:37:46.973464 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:46.973430    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c4d06c9-88c6-421a-ba3b-97bcc86e667e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k4ln9\" (UID: \"4c4d06c9-88c6-421a-ba3b-97bcc86e667e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k4ln9"
Sep 15 22:37:47.074648 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:47.074616    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c4d06c9-88c6-421a-ba3b-97bcc86e667e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k4ln9\" (UID: \"4c4d06c9-88c6-421a-ba3b-97bcc86e667e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k4ln9"
Sep 15 22:37:47.074648 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:47.074651    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c4d06c9-88c6-421a-ba3b-97bcc86e667e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k4ln9\" (UID: \"4c4d06c9-88c6-421a-ba3b-97bcc86e667e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k4ln9"
Sep 15 22:37:47.074841 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:47.074680    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c4d06c9-88c6-421a-ba3b-97bcc86e667e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k4ln9\" (UID: \"4c4d06c9-88c6-421a-ba3b-97bcc86e667e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k4ln9"
Sep 15 22:37:47.074841 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:47.074704    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c4d06c9-88c6-421a-ba3b-97bcc86e667e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k4ln9\" (UID: \"4c4d06c9-88c6-421a-ba3b-97bcc86e667e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k4ln9"
Sep 15 22:37:47.074841 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:47.074770    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c4d06c9-88c6-421a-ba3b-97bcc86e667e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k4ln9\" (UID: \"4c4d06c9-88c6-421a-ba3b-97bcc86e667e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k4ln9"
Sep 15 22:37:47.074841 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:47.074793    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66sd2\" (UniqueName: \"kubernetes.io/projected/4c4d06c9-88c6-421a-ba3b-97bcc86e667e-kube-api-access-66sd2\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k4ln9\" (UID: \"4c4d06c9-88c6-421a-ba3b-97bcc86e667e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k4ln9"
Sep 15 22:37:47.075070 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:47.075014    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c4d06c9-88c6-421a-ba3b-97bcc86e667e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k4ln9\" (UID: \"4c4d06c9-88c6-421a-ba3b-97bcc86e667e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k4ln9"
Sep 15 22:37:47.075149 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:47.075106    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c4d06c9-88c6-421a-ba3b-97bcc86e667e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k4ln9\" (UID: \"4c4d06c9-88c6-421a-ba3b-97bcc86e667e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k4ln9"
Sep 15 22:37:47.075233 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:47.075218    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c4d06c9-88c6-421a-ba3b-97bcc86e667e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k4ln9\" (UID: \"4c4d06c9-88c6-421a-ba3b-97bcc86e667e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k4ln9"
Sep 15 22:37:47.077101 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:47.077082    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c4d06c9-88c6-421a-ba3b-97bcc86e667e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k4ln9\" (UID: \"4c4d06c9-88c6-421a-ba3b-97bcc86e667e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k4ln9"
Sep 15 22:37:47.077239 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:47.077224    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c4d06c9-88c6-421a-ba3b-97bcc86e667e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k4ln9\" (UID: \"4c4d06c9-88c6-421a-ba3b-97bcc86e667e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k4ln9"
Sep 15 22:37:47.083374 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:47.083352    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66sd2\" (UniqueName: \"kubernetes.io/projected/4c4d06c9-88c6-421a-ba3b-97bcc86e667e-kube-api-access-66sd2\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k4ln9\" (UID: \"4c4d06c9-88c6-421a-ba3b-97bcc86e667e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k4ln9"
Sep 15 22:37:47.180196 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:47.180130    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k4ln9"
Sep 15 22:37:47.457546 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:47.457512    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-k4ln9"]
Sep 15 22:37:47.457869 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:37:47.457842    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c4d06c9_88c6_421a_ba3b_97bcc86e667e.slice/crio-796a9901cdfe07bf5b228d92a7bac0ebb9190737866f2e9b35662cfb2b9cd3c5 WatchSource:0}: Error finding container 796a9901cdfe07bf5b228d92a7bac0ebb9190737866f2e9b35662cfb2b9cd3c5: Status 404 returned error can't find the container with id 796a9901cdfe07bf5b228d92a7bac0ebb9190737866f2e9b35662cfb2b9cd3c5
Sep 15 22:37:47.459332 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:47.459312    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 22:37:47.565583 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:47.565548    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k4ln9" event={"ID":"4c4d06c9-88c6-421a-ba3b-97bcc86e667e","Type":"ContainerStarted","Data":"796a9901cdfe07bf5b228d92a7bac0ebb9190737866f2e9b35662cfb2b9cd3c5"}
Sep 15 22:37:48.573298 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:48.573270    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k4ln9" event={"ID":"4c4d06c9-88c6-421a-ba3b-97bcc86e667e","Type":"ContainerStarted","Data":"d538229da0917a6720f0d76e04e1acc9765e90b0aafa383f194e679fb2321fc9"}
Sep 15 22:37:48.595812 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:48.595766    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k4ln9" podStartSLOduration=1.554418261 podStartE2EDuration="2.595750132s" totalImagesPullingTime="1.041331871s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:37:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:37:48.594715001 +0000 UTC m=+618.454426285" watchObservedRunningTime="2026-09-15 22:37:48.595750132 +0000 UTC m=+618.455461415"
Sep 15 22:37:57.180481 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:57.180450    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k4ln9"
Sep 15 22:37:57.180938 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:57.180500    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k4ln9"
Sep 15 22:37:57.193317 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:57.193294    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k4ln9"
Sep 15 22:37:57.631924 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:37:57.631894    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k4ln9"
Sep 15 22:38:16.045064 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:16.045020    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-r5shq"]
Sep 15 22:38:16.049133 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:16.049111    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r5shq"
Sep 15 22:38:16.051140 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:16.051117    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 15 22:38:16.060934 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:16.060909    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-r5shq"]
Sep 15 22:38:16.121869 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:16.121834    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b49ca52-8bb2-4446-9479-21c426c881cc-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r5shq\" (UID: \"0b49ca52-8bb2-4446-9479-21c426c881cc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r5shq"
Sep 15 22:38:16.122016 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:16.121878    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcp25\" (UniqueName: \"kubernetes.io/projected/0b49ca52-8bb2-4446-9479-21c426c881cc-kube-api-access-fcp25\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r5shq\" (UID: \"0b49ca52-8bb2-4446-9479-21c426c881cc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r5shq"
Sep 15 22:38:16.122016 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:16.121904    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b49ca52-8bb2-4446-9479-21c426c881cc-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r5shq\" (UID: \"0b49ca52-8bb2-4446-9479-21c426c881cc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r5shq"
Sep 15 22:38:16.122016 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:16.121927    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b49ca52-8bb2-4446-9479-21c426c881cc-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r5shq\" (UID: \"0b49ca52-8bb2-4446-9479-21c426c881cc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r5shq"
Sep 15 22:38:16.122016 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:16.121954    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b49ca52-8bb2-4446-9479-21c426c881cc-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r5shq\" (UID: \"0b49ca52-8bb2-4446-9479-21c426c881cc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r5shq"
Sep 15 22:38:16.122190 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:16.122058    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b49ca52-8bb2-4446-9479-21c426c881cc-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r5shq\" (UID: \"0b49ca52-8bb2-4446-9479-21c426c881cc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r5shq"
Sep 15 22:38:16.223070 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:16.223021    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b49ca52-8bb2-4446-9479-21c426c881cc-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r5shq\" (UID: \"0b49ca52-8bb2-4446-9479-21c426c881cc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r5shq"
Sep 15 22:38:16.223252 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:16.223088    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b49ca52-8bb2-4446-9479-21c426c881cc-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r5shq\" (UID: \"0b49ca52-8bb2-4446-9479-21c426c881cc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r5shq"
Sep 15 22:38:16.223252 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:16.223125    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcp25\" (UniqueName: \"kubernetes.io/projected/0b49ca52-8bb2-4446-9479-21c426c881cc-kube-api-access-fcp25\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r5shq\" (UID: \"0b49ca52-8bb2-4446-9479-21c426c881cc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r5shq"
Sep 15 22:38:16.223252 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:16.223162    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b49ca52-8bb2-4446-9479-21c426c881cc-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r5shq\" (UID: \"0b49ca52-8bb2-4446-9479-21c426c881cc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r5shq"
Sep 15 22:38:16.223252 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:16.223187    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b49ca52-8bb2-4446-9479-21c426c881cc-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r5shq\" (UID: \"0b49ca52-8bb2-4446-9479-21c426c881cc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r5shq"
Sep 15 22:38:16.223252 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:16.223220    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b49ca52-8bb2-4446-9479-21c426c881cc-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r5shq\" (UID: \"0b49ca52-8bb2-4446-9479-21c426c881cc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r5shq"
Sep 15 22:38:16.223546 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:16.223444    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b49ca52-8bb2-4446-9479-21c426c881cc-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r5shq\" (UID: \"0b49ca52-8bb2-4446-9479-21c426c881cc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r5shq"
Sep 15 22:38:16.223546 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:16.223484    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b49ca52-8bb2-4446-9479-21c426c881cc-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r5shq\" (UID: \"0b49ca52-8bb2-4446-9479-21c426c881cc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r5shq"
Sep 15 22:38:16.223546 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:16.223508    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b49ca52-8bb2-4446-9479-21c426c881cc-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r5shq\" (UID: \"0b49ca52-8bb2-4446-9479-21c426c881cc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r5shq"
Sep 15 22:38:16.225482 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:16.225460    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b49ca52-8bb2-4446-9479-21c426c881cc-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r5shq\" (UID: \"0b49ca52-8bb2-4446-9479-21c426c881cc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r5shq"
Sep 15 22:38:16.225585 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:16.225566    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b49ca52-8bb2-4446-9479-21c426c881cc-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r5shq\" (UID: \"0b49ca52-8bb2-4446-9479-21c426c881cc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r5shq"
Sep 15 22:38:16.230783 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:16.230763    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcp25\" (UniqueName: \"kubernetes.io/projected/0b49ca52-8bb2-4446-9479-21c426c881cc-kube-api-access-fcp25\") pod \"e2e-trlp-test-simulated-kserve-598466548d-r5shq\" (UID: \"0b49ca52-8bb2-4446-9479-21c426c881cc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r5shq"
Sep 15 22:38:16.359298 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:16.359220    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r5shq"
Sep 15 22:38:16.636343 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:16.636312    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-r5shq"]
Sep 15 22:38:16.637822 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:38:16.637798    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b49ca52_8bb2_4446_9479_21c426c881cc.slice/crio-eb62f21cc5d34941fe027095fd71963a3e9f06300dbcabab455c02f4e8382119 WatchSource:0}: Error finding container eb62f21cc5d34941fe027095fd71963a3e9f06300dbcabab455c02f4e8382119: Status 404 returned error can't find the container with id eb62f21cc5d34941fe027095fd71963a3e9f06300dbcabab455c02f4e8382119
Sep 15 22:38:16.724506 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:16.724472    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r5shq" event={"ID":"0b49ca52-8bb2-4446-9479-21c426c881cc","Type":"ContainerStarted","Data":"eb62f21cc5d34941fe027095fd71963a3e9f06300dbcabab455c02f4e8382119"}
Sep 15 22:38:17.734073 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:17.734019    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r5shq" event={"ID":"0b49ca52-8bb2-4446-9479-21c426c881cc","Type":"ContainerStarted","Data":"5ddf023ace5ea1697623fdbf680c77167368df15664e348de95c108fdc36012d"}
Sep 15 22:38:17.751043 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:17.750963    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r5shq" podStartSLOduration=1.545932069 podStartE2EDuration="1.750941571s" totalImagesPullingTime="205.009502ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:38:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:38:17.750074467 +0000 UTC m=+647.609785748" watchObservedRunningTime="2026-09-15 22:38:17.750941571 +0000 UTC m=+647.610652856"
Sep 15 22:38:26.262087 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:26.262054    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7"]
Sep 15 22:38:26.265464 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:26.265442    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7"
Sep 15 22:38:26.267791 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:26.267771    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 15 22:38:26.276067 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:26.276039    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7"]
Sep 15 22:38:26.316067 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:26.316008    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnqgl\" (UniqueName: \"kubernetes.io/projected/eabf69af-5154-4d83-8f0e-9a072852b1ad-kube-api-access-wnqgl\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7\" (UID: \"eabf69af-5154-4d83-8f0e-9a072852b1ad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7"
Sep 15 22:38:26.316194 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:26.316119    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eabf69af-5154-4d83-8f0e-9a072852b1ad-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7\" (UID: \"eabf69af-5154-4d83-8f0e-9a072852b1ad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7"
Sep 15 22:38:26.316194 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:26.316161    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eabf69af-5154-4d83-8f0e-9a072852b1ad-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7\" (UID: \"eabf69af-5154-4d83-8f0e-9a072852b1ad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7"
Sep 15 22:38:26.316300 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:26.316200    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eabf69af-5154-4d83-8f0e-9a072852b1ad-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7\" (UID: \"eabf69af-5154-4d83-8f0e-9a072852b1ad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7"
Sep 15 22:38:26.316300 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:26.316273    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eabf69af-5154-4d83-8f0e-9a072852b1ad-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7\" (UID: \"eabf69af-5154-4d83-8f0e-9a072852b1ad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7"
Sep 15 22:38:26.316386 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:26.316322    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eabf69af-5154-4d83-8f0e-9a072852b1ad-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7\" (UID: \"eabf69af-5154-4d83-8f0e-9a072852b1ad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7"
Sep 15 22:38:26.359633 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:26.359610    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r5shq"
Sep 15 22:38:26.359776 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:26.359703    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r5shq"
Sep 15 22:38:26.372226 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:26.372204    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r5shq"
Sep 15 22:38:26.417348 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:26.417319    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnqgl\" (UniqueName: \"kubernetes.io/projected/eabf69af-5154-4d83-8f0e-9a072852b1ad-kube-api-access-wnqgl\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7\" (UID: \"eabf69af-5154-4d83-8f0e-9a072852b1ad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7"
Sep 15 22:38:26.417516 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:26.417478    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eabf69af-5154-4d83-8f0e-9a072852b1ad-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7\" (UID: \"eabf69af-5154-4d83-8f0e-9a072852b1ad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7"
Sep 15 22:38:26.417586 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:26.417538    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eabf69af-5154-4d83-8f0e-9a072852b1ad-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7\" (UID: \"eabf69af-5154-4d83-8f0e-9a072852b1ad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7"
Sep 15 22:38:26.417643 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:26.417597    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eabf69af-5154-4d83-8f0e-9a072852b1ad-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7\" (UID: \"eabf69af-5154-4d83-8f0e-9a072852b1ad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7"
Sep 15 22:38:26.417753 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:26.417731    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eabf69af-5154-4d83-8f0e-9a072852b1ad-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7\" (UID: \"eabf69af-5154-4d83-8f0e-9a072852b1ad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7"
Sep 15 22:38:26.417839 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:26.417776    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eabf69af-5154-4d83-8f0e-9a072852b1ad-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7\" (UID: \"eabf69af-5154-4d83-8f0e-9a072852b1ad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7"
Sep 15 22:38:26.418156 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:26.418133    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eabf69af-5154-4d83-8f0e-9a072852b1ad-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7\" (UID: \"eabf69af-5154-4d83-8f0e-9a072852b1ad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7"
Sep 15 22:38:26.418241 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:26.418165    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eabf69af-5154-4d83-8f0e-9a072852b1ad-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7\" (UID: \"eabf69af-5154-4d83-8f0e-9a072852b1ad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7"
Sep 15 22:38:26.418241 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:26.418192    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eabf69af-5154-4d83-8f0e-9a072852b1ad-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7\" (UID: \"eabf69af-5154-4d83-8f0e-9a072852b1ad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7"
Sep 15 22:38:26.419977 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:26.419958    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eabf69af-5154-4d83-8f0e-9a072852b1ad-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7\" (UID: \"eabf69af-5154-4d83-8f0e-9a072852b1ad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7"
Sep 15 22:38:26.419977 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:26.419965    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eabf69af-5154-4d83-8f0e-9a072852b1ad-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7\" (UID: \"eabf69af-5154-4d83-8f0e-9a072852b1ad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7"
Sep 15 22:38:26.425726 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:26.425702    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnqgl\" (UniqueName: \"kubernetes.io/projected/eabf69af-5154-4d83-8f0e-9a072852b1ad-kube-api-access-wnqgl\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7\" (UID: \"eabf69af-5154-4d83-8f0e-9a072852b1ad\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7"
Sep 15 22:38:26.577393 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:26.577360    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7"
Sep 15 22:38:26.792628 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:26.792598    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-r5shq"
Sep 15 22:38:26.854369 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:26.854343    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7"]
Sep 15 22:38:26.856222 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:38:26.856191    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeabf69af_5154_4d83_8f0e_9a072852b1ad.slice/crio-d574670c7bcf54ab5e8e2af44eca6b5ca687fe8176399036a186113819806417 WatchSource:0}: Error finding container d574670c7bcf54ab5e8e2af44eca6b5ca687fe8176399036a186113819806417: Status 404 returned error can't find the container with id d574670c7bcf54ab5e8e2af44eca6b5ca687fe8176399036a186113819806417
Sep 15 22:38:27.788994 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:27.788956    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7" event={"ID":"eabf69af-5154-4d83-8f0e-9a072852b1ad","Type":"ContainerStarted","Data":"da6801fe9f1e76539299445fbfb021508818f5ee0cb8a8def977d88af3d9dae7"}
Sep 15 22:38:27.789369 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:27.789061    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7" event={"ID":"eabf69af-5154-4d83-8f0e-9a072852b1ad","Type":"ContainerStarted","Data":"d574670c7bcf54ab5e8e2af44eca6b5ca687fe8176399036a186113819806417"}
Sep 15 22:38:27.810837 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:27.810778    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7" podStartSLOduration=1.622386804 podStartE2EDuration="1.810756662s" totalImagesPullingTime="188.369858ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:38:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:38:27.808864531 +0000 UTC m=+657.668575816" watchObservedRunningTime="2026-09-15 22:38:27.810756662 +0000 UTC m=+657.670467947"
Sep 15 22:38:36.578037 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:36.578001    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7"
Sep 15 22:38:36.578490 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:36.578077    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7"
Sep 15 22:38:36.591299 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:36.591276    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7"
Sep 15 22:38:36.853156 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:36.853073    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7"
Sep 15 22:38:41.757182 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:41.757143    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-cwtbq"]
Sep 15 22:38:41.761291 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:41.761266    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwtbq"
Sep 15 22:38:41.763333 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:41.763312    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 15 22:38:41.771251 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:41.771219    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-cwtbq"]
Sep 15 22:38:41.858444 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:41.858412    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7dn9\" (UniqueName: \"kubernetes.io/projected/91a9a4da-7b05-4220-b3c0-61364b9ef198-kube-api-access-t7dn9\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cwtbq\" (UID: \"91a9a4da-7b05-4220-b3c0-61364b9ef198\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwtbq"
Sep 15 22:38:41.858649 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:41.858468    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91a9a4da-7b05-4220-b3c0-61364b9ef198-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cwtbq\" (UID: \"91a9a4da-7b05-4220-b3c0-61364b9ef198\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwtbq"
Sep 15 22:38:41.858649 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:41.858501    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91a9a4da-7b05-4220-b3c0-61364b9ef198-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cwtbq\" (UID: \"91a9a4da-7b05-4220-b3c0-61364b9ef198\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwtbq"
Sep 15 22:38:41.858649 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:41.858576    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91a9a4da-7b05-4220-b3c0-61364b9ef198-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cwtbq\" (UID: \"91a9a4da-7b05-4220-b3c0-61364b9ef198\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwtbq"
Sep 15 22:38:41.858649 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:41.858615    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91a9a4da-7b05-4220-b3c0-61364b9ef198-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cwtbq\" (UID: \"91a9a4da-7b05-4220-b3c0-61364b9ef198\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwtbq"
Sep 15 22:38:41.858826 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:41.858685    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91a9a4da-7b05-4220-b3c0-61364b9ef198-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cwtbq\" (UID: \"91a9a4da-7b05-4220-b3c0-61364b9ef198\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwtbq"
Sep 15 22:38:41.959622 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:41.959589    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7dn9\" (UniqueName: \"kubernetes.io/projected/91a9a4da-7b05-4220-b3c0-61364b9ef198-kube-api-access-t7dn9\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cwtbq\" (UID: \"91a9a4da-7b05-4220-b3c0-61364b9ef198\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwtbq"
Sep 15 22:38:41.959927 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:41.959638    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91a9a4da-7b05-4220-b3c0-61364b9ef198-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cwtbq\" (UID: \"91a9a4da-7b05-4220-b3c0-61364b9ef198\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwtbq"
Sep 15 22:38:41.959927 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:41.959755    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91a9a4da-7b05-4220-b3c0-61364b9ef198-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cwtbq\" (UID: \"91a9a4da-7b05-4220-b3c0-61364b9ef198\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwtbq"
Sep 15 22:38:41.959927 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:41.959809    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91a9a4da-7b05-4220-b3c0-61364b9ef198-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cwtbq\" (UID: \"91a9a4da-7b05-4220-b3c0-61364b9ef198\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwtbq"
Sep 15 22:38:41.959927 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:41.959838    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91a9a4da-7b05-4220-b3c0-61364b9ef198-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cwtbq\" (UID: \"91a9a4da-7b05-4220-b3c0-61364b9ef198\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwtbq"
Sep 15 22:38:41.959927 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:41.959891    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91a9a4da-7b05-4220-b3c0-61364b9ef198-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cwtbq\" (UID: \"91a9a4da-7b05-4220-b3c0-61364b9ef198\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwtbq"
Sep 15 22:38:41.960206 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:41.960118    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91a9a4da-7b05-4220-b3c0-61364b9ef198-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cwtbq\" (UID: \"91a9a4da-7b05-4220-b3c0-61364b9ef198\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwtbq"
Sep 15 22:38:41.960206 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:41.960177    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91a9a4da-7b05-4220-b3c0-61364b9ef198-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cwtbq\" (UID: \"91a9a4da-7b05-4220-b3c0-61364b9ef198\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwtbq"
Sep 15 22:38:41.960304 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:41.960286    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91a9a4da-7b05-4220-b3c0-61364b9ef198-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cwtbq\" (UID: \"91a9a4da-7b05-4220-b3c0-61364b9ef198\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwtbq"
Sep 15 22:38:41.961866 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:41.961840    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91a9a4da-7b05-4220-b3c0-61364b9ef198-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cwtbq\" (UID: \"91a9a4da-7b05-4220-b3c0-61364b9ef198\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwtbq"
Sep 15 22:38:41.962285 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:41.962267    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91a9a4da-7b05-4220-b3c0-61364b9ef198-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cwtbq\" (UID: \"91a9a4da-7b05-4220-b3c0-61364b9ef198\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwtbq"
Sep 15 22:38:41.967162 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:41.967140    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7dn9\" (UniqueName: \"kubernetes.io/projected/91a9a4da-7b05-4220-b3c0-61364b9ef198-kube-api-access-t7dn9\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cwtbq\" (UID: \"91a9a4da-7b05-4220-b3c0-61364b9ef198\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwtbq"
Sep 15 22:38:42.072710 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:42.072664    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwtbq"
Sep 15 22:38:42.349348 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:42.349318    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-cwtbq"]
Sep 15 22:38:42.350560 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:38:42.350536    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91a9a4da_7b05_4220_b3c0_61364b9ef198.slice/crio-257575469a93a27fb625517a1b9311dd1604ee01059d9dcbc8c8019caae12c42 WatchSource:0}: Error finding container 257575469a93a27fb625517a1b9311dd1604ee01059d9dcbc8c8019caae12c42: Status 404 returned error can't find the container with id 257575469a93a27fb625517a1b9311dd1604ee01059d9dcbc8c8019caae12c42
Sep 15 22:38:42.877354 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:42.877316    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwtbq" event={"ID":"91a9a4da-7b05-4220-b3c0-61364b9ef198","Type":"ContainerStarted","Data":"4769228e7f5d0ae0e50de63efff202d1234648ea73aa971c20bddc4f7907fabc"}
Sep 15 22:38:42.877733 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:42.877381    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwtbq" event={"ID":"91a9a4da-7b05-4220-b3c0-61364b9ef198","Type":"ContainerStarted","Data":"257575469a93a27fb625517a1b9311dd1604ee01059d9dcbc8c8019caae12c42"}
Sep 15 22:38:42.895381 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:42.895335    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwtbq" podStartSLOduration=1.713671994 podStartE2EDuration="1.895320572s" totalImagesPullingTime="181.648578ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:38:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:38:42.893354637 +0000 UTC m=+672.753065947" watchObservedRunningTime="2026-09-15 22:38:42.895320572 +0000 UTC m=+672.755031857"
Sep 15 22:38:52.073432 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:52.073398    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwtbq"
Sep 15 22:38:52.073790 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:52.073456    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwtbq"
Sep 15 22:38:52.085783 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:52.085757    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwtbq"
Sep 15 22:38:52.949257 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:38:52.949229    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cwtbq"
Sep 15 22:40:03.364472 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:03.364435    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c4b5f44c7-25fnj"]
Sep 15 22:40:03.367883 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:03.367864    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c4b5f44c7-25fnj"
Sep 15 22:40:03.375290 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:03.375184    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c4b5f44c7-25fnj"]
Sep 15 22:40:03.492417 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:03.492382    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv8km\" (UniqueName: \"kubernetes.io/projected/59d5908c-cac0-4888-9de9-a1c390b68e55-kube-api-access-xv8km\") pod \"authorino-c4b5f44c7-25fnj\" (UID: \"59d5908c-cac0-4888-9de9-a1c390b68e55\") " pod="kuadrant-system/authorino-c4b5f44c7-25fnj"
Sep 15 22:40:03.492572 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:03.492428    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/59d5908c-cac0-4888-9de9-a1c390b68e55-tls-cert\") pod \"authorino-c4b5f44c7-25fnj\" (UID: \"59d5908c-cac0-4888-9de9-a1c390b68e55\") " pod="kuadrant-system/authorino-c4b5f44c7-25fnj"
Sep 15 22:40:03.592812 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:03.592779    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xv8km\" (UniqueName: \"kubernetes.io/projected/59d5908c-cac0-4888-9de9-a1c390b68e55-kube-api-access-xv8km\") pod \"authorino-c4b5f44c7-25fnj\" (UID: \"59d5908c-cac0-4888-9de9-a1c390b68e55\") " pod="kuadrant-system/authorino-c4b5f44c7-25fnj"
Sep 15 22:40:03.592978 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:03.592825    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/59d5908c-cac0-4888-9de9-a1c390b68e55-tls-cert\") pod \"authorino-c4b5f44c7-25fnj\" (UID: \"59d5908c-cac0-4888-9de9-a1c390b68e55\") " pod="kuadrant-system/authorino-c4b5f44c7-25fnj"
Sep 15 22:40:03.595152 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:03.595126    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/59d5908c-cac0-4888-9de9-a1c390b68e55-tls-cert\") pod \"authorino-c4b5f44c7-25fnj\" (UID: \"59d5908c-cac0-4888-9de9-a1c390b68e55\") " pod="kuadrant-system/authorino-c4b5f44c7-25fnj"
Sep 15 22:40:03.600632 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:03.600611    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv8km\" (UniqueName: \"kubernetes.io/projected/59d5908c-cac0-4888-9de9-a1c390b68e55-kube-api-access-xv8km\") pod \"authorino-c4b5f44c7-25fnj\" (UID: \"59d5908c-cac0-4888-9de9-a1c390b68e55\") " pod="kuadrant-system/authorino-c4b5f44c7-25fnj"
Sep 15 22:40:03.678707 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:03.678636    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c4b5f44c7-25fnj"
Sep 15 22:40:03.950062 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:03.949955    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c4b5f44c7-25fnj"]
Sep 15 22:40:03.952902 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:40:03.952872    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod59d5908c_cac0_4888_9de9_a1c390b68e55.slice/crio-f8dfc44b8d313b3ddb24dba1dccde3789ab48787f02c6de6a2b8ef1d54afc7af WatchSource:0}: Error finding container f8dfc44b8d313b3ddb24dba1dccde3789ab48787f02c6de6a2b8ef1d54afc7af: Status 404 returned error can't find the container with id f8dfc44b8d313b3ddb24dba1dccde3789ab48787f02c6de6a2b8ef1d54afc7af
Sep 15 22:40:04.355663 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:04.355624    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c4b5f44c7-25fnj" event={"ID":"59d5908c-cac0-4888-9de9-a1c390b68e55","Type":"ContainerStarted","Data":"f8dfc44b8d313b3ddb24dba1dccde3789ab48787f02c6de6a2b8ef1d54afc7af"}
Sep 15 22:40:05.362801 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:05.362760    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c4b5f44c7-25fnj" event={"ID":"59d5908c-cac0-4888-9de9-a1c390b68e55","Type":"ContainerStarted","Data":"b40df567d8d9b26ad7cde6e2bd0b3de4b69b59a5d5c9f417428766fed8231c0d"}
Sep 15 22:40:05.379523 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:05.379464    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c4b5f44c7-25fnj" podStartSLOduration=1.396910108 podStartE2EDuration="2.379442148s" totalImagesPullingTime="982.53204ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:40:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:40:05.376495326 +0000 UTC m=+755.236206610" watchObservedRunningTime="2026-09-15 22:40:05.379442148 +0000 UTC m=+755.239153433"
Sep 15 22:40:05.409004 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:05.407980    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b49cb4fc6-gtq4g"]
Sep 15 22:40:05.409004 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:05.408294    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b49cb4fc6-gtq4g" podUID="715cfd0c-a5a2-46ad-9138-4714fc27781c" containerName="authorino" containerID="cri-o://6d32fe94339c421f0aeba1838ec582059acaea550b0ce5ba3248c29ff7ebf022" gracePeriod=30
Sep 15 22:40:05.676535 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:05.676508    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b49cb4fc6-gtq4g"
Sep 15 22:40:05.814051 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:05.813989    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-slkw9\" (UniqueName: \"kubernetes.io/projected/715cfd0c-a5a2-46ad-9138-4714fc27781c-kube-api-access-slkw9\") pod \"715cfd0c-a5a2-46ad-9138-4714fc27781c\" (UID: \"715cfd0c-a5a2-46ad-9138-4714fc27781c\") "
Sep 15 22:40:05.814247 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:05.814072    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/715cfd0c-a5a2-46ad-9138-4714fc27781c-tls-cert\") pod \"715cfd0c-a5a2-46ad-9138-4714fc27781c\" (UID: \"715cfd0c-a5a2-46ad-9138-4714fc27781c\") "
Sep 15 22:40:05.816154 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:05.816120    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/715cfd0c-a5a2-46ad-9138-4714fc27781c-kube-api-access-slkw9" (OuterVolumeSpecName: "kube-api-access-slkw9") pod "715cfd0c-a5a2-46ad-9138-4714fc27781c" (UID: "715cfd0c-a5a2-46ad-9138-4714fc27781c"). InnerVolumeSpecName "kube-api-access-slkw9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:40:05.824598 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:05.824570    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/715cfd0c-a5a2-46ad-9138-4714fc27781c-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "715cfd0c-a5a2-46ad-9138-4714fc27781c" (UID: "715cfd0c-a5a2-46ad-9138-4714fc27781c"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:40:05.915254 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:05.915178    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-slkw9\" (UniqueName: \"kubernetes.io/projected/715cfd0c-a5a2-46ad-9138-4714fc27781c-kube-api-access-slkw9\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:40:05.915254 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:05.915205    2565 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/715cfd0c-a5a2-46ad-9138-4714fc27781c-tls-cert\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:40:06.371519 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:06.371478    2565 generic.go:356] "Generic (PLEG): container finished" podID="715cfd0c-a5a2-46ad-9138-4714fc27781c" containerID="6d32fe94339c421f0aeba1838ec582059acaea550b0ce5ba3248c29ff7ebf022" exitCode=0
Sep 15 22:40:06.371519 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:06.371523    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b49cb4fc6-gtq4g"
Sep 15 22:40:06.371964 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:06.371593    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b49cb4fc6-gtq4g" event={"ID":"715cfd0c-a5a2-46ad-9138-4714fc27781c","Type":"ContainerDied","Data":"6d32fe94339c421f0aeba1838ec582059acaea550b0ce5ba3248c29ff7ebf022"}
Sep 15 22:40:06.371964 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:06.371634    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b49cb4fc6-gtq4g" event={"ID":"715cfd0c-a5a2-46ad-9138-4714fc27781c","Type":"ContainerDied","Data":"dfc63cff9a9b36f66874f955aa1692e435ee90d9ba2bafa09f078d659cec7097"}
Sep 15 22:40:06.371964 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:06.371652    2565 scope.go:117] "RemoveContainer" containerID="6d32fe94339c421f0aeba1838ec582059acaea550b0ce5ba3248c29ff7ebf022"
Sep 15 22:40:06.387649 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:06.387622    2565 scope.go:117] "RemoveContainer" containerID="6d32fe94339c421f0aeba1838ec582059acaea550b0ce5ba3248c29ff7ebf022"
Sep 15 22:40:06.387954 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:40:06.387934    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d32fe94339c421f0aeba1838ec582059acaea550b0ce5ba3248c29ff7ebf022\": container with ID starting with 6d32fe94339c421f0aeba1838ec582059acaea550b0ce5ba3248c29ff7ebf022 not found: ID does not exist" containerID="6d32fe94339c421f0aeba1838ec582059acaea550b0ce5ba3248c29ff7ebf022"
Sep 15 22:40:06.388008 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:06.387962    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d32fe94339c421f0aeba1838ec582059acaea550b0ce5ba3248c29ff7ebf022"} err="failed to get container status \"6d32fe94339c421f0aeba1838ec582059acaea550b0ce5ba3248c29ff7ebf022\": rpc error: code = NotFound desc = could not find container \"6d32fe94339c421f0aeba1838ec582059acaea550b0ce5ba3248c29ff7ebf022\": container with ID starting with 6d32fe94339c421f0aeba1838ec582059acaea550b0ce5ba3248c29ff7ebf022 not found: ID does not exist"
Sep 15 22:40:06.397259 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:06.397229    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b49cb4fc6-gtq4g"]
Sep 15 22:40:06.404825 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:06.404801    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b49cb4fc6-gtq4g"]
Sep 15 22:40:06.591585 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:06.591554    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="715cfd0c-a5a2-46ad-9138-4714fc27781c" path="/var/lib/kubelet/pods/715cfd0c-a5a2-46ad-9138-4714fc27781c/volumes"
Sep 15 22:40:26.939448 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:26.939414    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4"]
Sep 15 22:40:26.939941 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:26.939923    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="715cfd0c-a5a2-46ad-9138-4714fc27781c" containerName="authorino"
Sep 15 22:40:26.940008 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:26.939943    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="715cfd0c-a5a2-46ad-9138-4714fc27781c" containerName="authorino"
Sep 15 22:40:26.940061 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:26.940017    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="715cfd0c-a5a2-46ad-9138-4714fc27781c" containerName="authorino"
Sep 15 22:40:26.943222 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:26.943204    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4"
Sep 15 22:40:26.945650 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:26.945629    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-fffdae-serving-cert\""
Sep 15 22:40:26.954630 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:26.954609    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4"]
Sep 15 22:40:27.000758 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:27.000733    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z68th\" (UniqueName: \"kubernetes.io/projected/743d1809-ef85-4780-88df-646139103059-kube-api-access-z68th\") pod \"maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4\" (UID: \"743d1809-ef85-4780-88df-646139103059\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4"
Sep 15 22:40:27.000854 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:27.000798    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/743d1809-ef85-4780-88df-646139103059-maas-api-tls\") pod \"maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4\" (UID: \"743d1809-ef85-4780-88df-646139103059\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4"
Sep 15 22:40:27.000854 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:27.000846    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/743d1809-ef85-4780-88df-646139103059-openshift-service-ca\") pod \"maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4\" (UID: \"743d1809-ef85-4780-88df-646139103059\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4"
Sep 15 22:40:27.000973 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:27.000928    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/743d1809-ef85-4780-88df-646139103059-metrics-certs\") pod \"maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4\" (UID: \"743d1809-ef85-4780-88df-646139103059\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4"
Sep 15 22:40:27.101549 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:27.101514    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/743d1809-ef85-4780-88df-646139103059-metrics-certs\") pod \"maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4\" (UID: \"743d1809-ef85-4780-88df-646139103059\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4"
Sep 15 22:40:27.101721 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:27.101625    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z68th\" (UniqueName: \"kubernetes.io/projected/743d1809-ef85-4780-88df-646139103059-kube-api-access-z68th\") pod \"maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4\" (UID: \"743d1809-ef85-4780-88df-646139103059\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4"
Sep 15 22:40:27.101721 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:27.101668    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/743d1809-ef85-4780-88df-646139103059-maas-api-tls\") pod \"maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4\" (UID: \"743d1809-ef85-4780-88df-646139103059\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4"
Sep 15 22:40:27.101721 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:27.101707    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/743d1809-ef85-4780-88df-646139103059-openshift-service-ca\") pod \"maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4\" (UID: \"743d1809-ef85-4780-88df-646139103059\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4"
Sep 15 22:40:27.101883 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:40:27.101792    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-fffdae-serving-cert: secret "maas-api-e2e-inf-a-fffdae-serving-cert" not found
Sep 15 22:40:27.101946 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:40:27.101881    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/743d1809-ef85-4780-88df-646139103059-maas-api-tls podName:743d1809-ef85-4780-88df-646139103059 nodeName:}" failed. No retries permitted until 2026-09-15 22:40:27.601859932 +0000 UTC m=+777.461571211 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/743d1809-ef85-4780-88df-646139103059-maas-api-tls") pod "maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4" (UID: "743d1809-ef85-4780-88df-646139103059") : secret "maas-api-e2e-inf-a-fffdae-serving-cert" not found
Sep 15 22:40:27.102497 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:27.102470    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/743d1809-ef85-4780-88df-646139103059-openshift-service-ca\") pod \"maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4\" (UID: \"743d1809-ef85-4780-88df-646139103059\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4"
Sep 15 22:40:27.104298 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:27.104270    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/743d1809-ef85-4780-88df-646139103059-metrics-certs\") pod \"maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4\" (UID: \"743d1809-ef85-4780-88df-646139103059\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4"
Sep 15 22:40:27.109819 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:27.109791    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z68th\" (UniqueName: \"kubernetes.io/projected/743d1809-ef85-4780-88df-646139103059-kube-api-access-z68th\") pod \"maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4\" (UID: \"743d1809-ef85-4780-88df-646139103059\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4"
Sep 15 22:40:27.605398 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:27.605359    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/743d1809-ef85-4780-88df-646139103059-maas-api-tls\") pod \"maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4\" (UID: \"743d1809-ef85-4780-88df-646139103059\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4"
Sep 15 22:40:27.608613 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:27.608584    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/743d1809-ef85-4780-88df-646139103059-maas-api-tls\") pod \"maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4\" (UID: \"743d1809-ef85-4780-88df-646139103059\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4"
Sep 15 22:40:27.677170 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:27.677127    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-fffdae-846d8cb8b-9bsts"]
Sep 15 22:40:27.681267 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:27.681247    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-fffdae-846d8cb8b-9bsts"
Sep 15 22:40:27.683295 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:27.683276    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-fffdae-dockercfg-m6tcd\""
Sep 15 22:40:27.683384 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:27.683309    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-fffdae\""
Sep 15 22:40:27.693037 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:27.693001    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-fffdae-846d8cb8b-9bsts"]
Sep 15 22:40:27.705891 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:27.705816    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mksxd\" (UniqueName: \"kubernetes.io/projected/f52bbe42-b0f6-4b1a-bfff-a8df7b5550ba-kube-api-access-mksxd\") pod \"payload-pre-processing-e2e-inf-a-fffdae-846d8cb8b-9bsts\" (UID: \"f52bbe42-b0f6-4b1a-bfff-a8df7b5550ba\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-fffdae-846d8cb8b-9bsts"
Sep 15 22:40:27.706019 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:27.705895    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f52bbe42-b0f6-4b1a-bfff-a8df7b5550ba-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-fffdae-846d8cb8b-9bsts\" (UID: \"f52bbe42-b0f6-4b1a-bfff-a8df7b5550ba\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-fffdae-846d8cb8b-9bsts"
Sep 15 22:40:27.806949 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:27.806917    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mksxd\" (UniqueName: \"kubernetes.io/projected/f52bbe42-b0f6-4b1a-bfff-a8df7b5550ba-kube-api-access-mksxd\") pod \"payload-pre-processing-e2e-inf-a-fffdae-846d8cb8b-9bsts\" (UID: \"f52bbe42-b0f6-4b1a-bfff-a8df7b5550ba\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-fffdae-846d8cb8b-9bsts"
Sep 15 22:40:27.807138 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:27.807014    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f52bbe42-b0f6-4b1a-bfff-a8df7b5550ba-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-fffdae-846d8cb8b-9bsts\" (UID: \"f52bbe42-b0f6-4b1a-bfff-a8df7b5550ba\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-fffdae-846d8cb8b-9bsts"
Sep 15 22:40:27.807854 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:27.807829    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f52bbe42-b0f6-4b1a-bfff-a8df7b5550ba-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-fffdae-846d8cb8b-9bsts\" (UID: \"f52bbe42-b0f6-4b1a-bfff-a8df7b5550ba\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-fffdae-846d8cb8b-9bsts"
Sep 15 22:40:27.818732 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:27.818711    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mksxd\" (UniqueName: \"kubernetes.io/projected/f52bbe42-b0f6-4b1a-bfff-a8df7b5550ba-kube-api-access-mksxd\") pod \"payload-pre-processing-e2e-inf-a-fffdae-846d8cb8b-9bsts\" (UID: \"f52bbe42-b0f6-4b1a-bfff-a8df7b5550ba\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-fffdae-846d8cb8b-9bsts"
Sep 15 22:40:27.856628 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:27.856576    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4"
Sep 15 22:40:27.993367 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:27.993333    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-fffdae-846d8cb8b-9bsts"
Sep 15 22:40:28.129741 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:28.129691    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4"]
Sep 15 22:40:28.131720 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:40:28.131696    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod743d1809_ef85_4780_88df_646139103059.slice/crio-83d4a32bcf3c40b2580a66f33c24c7828ab3d732f33f444233c507ea96488163 WatchSource:0}: Error finding container 83d4a32bcf3c40b2580a66f33c24c7828ab3d732f33f444233c507ea96488163: Status 404 returned error can't find the container with id 83d4a32bcf3c40b2580a66f33c24c7828ab3d732f33f444233c507ea96488163
Sep 15 22:40:28.162433 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:28.162404    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx"]
Sep 15 22:40:28.167234 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:28.167215    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx"
Sep 15 22:40:28.169207 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:28.169187    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-03f258-serving-cert\""
Sep 15 22:40:28.177140 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:28.177079    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx"]
Sep 15 22:40:28.210897 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:28.210866    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqj59\" (UniqueName: \"kubernetes.io/projected/ad7fe46d-f88b-434d-ad47-92372e7c1ba8-kube-api-access-gqj59\") pod \"maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx\" (UID: \"ad7fe46d-f88b-434d-ad47-92372e7c1ba8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx"
Sep 15 22:40:28.211066 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:28.210909    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad7fe46d-f88b-434d-ad47-92372e7c1ba8-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx\" (UID: \"ad7fe46d-f88b-434d-ad47-92372e7c1ba8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx"
Sep 15 22:40:28.211066 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:28.211018    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad7fe46d-f88b-434d-ad47-92372e7c1ba8-metrics-certs\") pod \"maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx\" (UID: \"ad7fe46d-f88b-434d-ad47-92372e7c1ba8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx"
Sep 15 22:40:28.211210 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:28.211093    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad7fe46d-f88b-434d-ad47-92372e7c1ba8-maas-api-tls\") pod \"maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx\" (UID: \"ad7fe46d-f88b-434d-ad47-92372e7c1ba8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx"
Sep 15 22:40:28.270096 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:28.270072    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-fffdae-846d8cb8b-9bsts"]
Sep 15 22:40:28.271069 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:40:28.271018    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf52bbe42_b0f6_4b1a_bfff_a8df7b5550ba.slice/crio-7d01df4d943e1b47920be5a72633de49b12c74380d0fe249b9e21aff44493594 WatchSource:0}: Error finding container 7d01df4d943e1b47920be5a72633de49b12c74380d0fe249b9e21aff44493594: Status 404 returned error can't find the container with id 7d01df4d943e1b47920be5a72633de49b12c74380d0fe249b9e21aff44493594
Sep 15 22:40:28.311939 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:28.311911    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad7fe46d-f88b-434d-ad47-92372e7c1ba8-metrics-certs\") pod \"maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx\" (UID: \"ad7fe46d-f88b-434d-ad47-92372e7c1ba8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx"
Sep 15 22:40:28.312130 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:28.311969    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad7fe46d-f88b-434d-ad47-92372e7c1ba8-maas-api-tls\") pod \"maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx\" (UID: \"ad7fe46d-f88b-434d-ad47-92372e7c1ba8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx"
Sep 15 22:40:28.312130 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:28.312008    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqj59\" (UniqueName: \"kubernetes.io/projected/ad7fe46d-f88b-434d-ad47-92372e7c1ba8-kube-api-access-gqj59\") pod \"maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx\" (UID: \"ad7fe46d-f88b-434d-ad47-92372e7c1ba8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx"
Sep 15 22:40:28.312130 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:28.312086    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad7fe46d-f88b-434d-ad47-92372e7c1ba8-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx\" (UID: \"ad7fe46d-f88b-434d-ad47-92372e7c1ba8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx"
Sep 15 22:40:28.312840 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:28.312813    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad7fe46d-f88b-434d-ad47-92372e7c1ba8-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx\" (UID: \"ad7fe46d-f88b-434d-ad47-92372e7c1ba8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx"
Sep 15 22:40:28.314718 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:28.314695    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad7fe46d-f88b-434d-ad47-92372e7c1ba8-metrics-certs\") pod \"maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx\" (UID: \"ad7fe46d-f88b-434d-ad47-92372e7c1ba8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx"
Sep 15 22:40:28.314992 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:28.314959    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad7fe46d-f88b-434d-ad47-92372e7c1ba8-maas-api-tls\") pod \"maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx\" (UID: \"ad7fe46d-f88b-434d-ad47-92372e7c1ba8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx"
Sep 15 22:40:28.320763 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:28.320745    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqj59\" (UniqueName: \"kubernetes.io/projected/ad7fe46d-f88b-434d-ad47-92372e7c1ba8-kube-api-access-gqj59\") pod \"maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx\" (UID: \"ad7fe46d-f88b-434d-ad47-92372e7c1ba8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx"
Sep 15 22:40:28.480842 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:28.480767    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx"
Sep 15 22:40:28.505081 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:28.505012    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4" event={"ID":"743d1809-ef85-4780-88df-646139103059","Type":"ContainerStarted","Data":"83d4a32bcf3c40b2580a66f33c24c7828ab3d732f33f444233c507ea96488163"}
Sep 15 22:40:28.507330 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:28.507119    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-fffdae-846d8cb8b-9bsts" event={"ID":"f52bbe42-b0f6-4b1a-bfff-a8df7b5550ba","Type":"ContainerStarted","Data":"06975fd91b3d085301bc520018eb42107159f28347d416c79cd6f305f388757f"}
Sep 15 22:40:28.507330 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:28.507172    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-fffdae-846d8cb8b-9bsts" event={"ID":"f52bbe42-b0f6-4b1a-bfff-a8df7b5550ba","Type":"ContainerStarted","Data":"7d01df4d943e1b47920be5a72633de49b12c74380d0fe249b9e21aff44493594"}
Sep 15 22:40:28.507330 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:28.507250    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-fffdae-846d8cb8b-9bsts"
Sep 15 22:40:28.531534 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:28.531477    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-fffdae-846d8cb8b-9bsts" podStartSLOduration=1.5314571 podStartE2EDuration="1.5314571s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:40:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:40:28.530128909 +0000 UTC m=+778.389840197" watchObservedRunningTime="2026-09-15 22:40:28.5314571 +0000 UTC m=+778.391168385"
Sep 15 22:40:28.759918 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:28.759892    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx"]
Sep 15 22:40:28.761327 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:40:28.761297    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad7fe46d_f88b_434d_ad47_92372e7c1ba8.slice/crio-7864b10a0f4763f7fc86d4ef56c1c5bc1ee98cdfecf31bd0dd4e25873bfea88b WatchSource:0}: Error finding container 7864b10a0f4763f7fc86d4ef56c1c5bc1ee98cdfecf31bd0dd4e25873bfea88b: Status 404 returned error can't find the container with id 7864b10a0f4763f7fc86d4ef56c1c5bc1ee98cdfecf31bd0dd4e25873bfea88b
Sep 15 22:40:29.519361 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:29.519262    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx" event={"ID":"ad7fe46d-f88b-434d-ad47-92372e7c1ba8","Type":"ContainerStarted","Data":"52f45ad3405551add58a4e76312f8798edbb8736ad1c169dd70c38a286b5fa1d"}
Sep 15 22:40:29.519361 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:29.519336    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx" event={"ID":"ad7fe46d-f88b-434d-ad47-92372e7c1ba8","Type":"ContainerStarted","Data":"7864b10a0f4763f7fc86d4ef56c1c5bc1ee98cdfecf31bd0dd4e25873bfea88b"}
Sep 15 22:40:29.519361 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:29.519361    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx"
Sep 15 22:40:29.521668 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:29.521634    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4" event={"ID":"743d1809-ef85-4780-88df-646139103059","Type":"ContainerStarted","Data":"9af813dc1a955f4172451173cebeae63e0c63c14a1b61ffe38565672c9aadf1f"}
Sep 15 22:40:29.521795 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:29.521710    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4"
Sep 15 22:40:29.540915 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:29.540872    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx" podStartSLOduration=1.12645137 podStartE2EDuration="1.540854844s" totalImagesPullingTime="414.403474ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:40:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:40:29.540328704 +0000 UTC m=+779.400039997" watchObservedRunningTime="2026-09-15 22:40:29.540854844 +0000 UTC m=+779.400566132"
Sep 15 22:40:29.559880 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:29.559829    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4" podStartSLOduration=3.093104592 podStartE2EDuration="3.559811946s" totalImagesPullingTime="466.707354ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:40:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:40:29.557165402 +0000 UTC m=+779.416876699" watchObservedRunningTime="2026-09-15 22:40:29.559811946 +0000 UTC m=+779.419523230"
Sep 15 22:40:30.169611 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:30.169575    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-fe020d-6dfcbdd58f-2vjb6"]
Sep 15 22:40:30.173404 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:30.173385    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-fe020d-6dfcbdd58f-2vjb6"
Sep 15 22:40:30.175471 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:30.175449    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-fe020d\""
Sep 15 22:40:30.175471 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:30.175460    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-fe020d-dockercfg-5d8d6\""
Sep 15 22:40:30.181817 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:30.181796    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-fe020d-6dfcbdd58f-2vjb6"]
Sep 15 22:40:30.225265 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:30.225231    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mc4d\" (UniqueName: \"kubernetes.io/projected/caaa5987-ca96-4c13-937d-fa83342f056c-kube-api-access-6mc4d\") pod \"payload-processing-e2e-worker-w0-fe020d-6dfcbdd58f-2vjb6\" (UID: \"caaa5987-ca96-4c13-937d-fa83342f056c\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-fe020d-6dfcbdd58f-2vjb6"
Sep 15 22:40:30.225265 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:30.225269    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/caaa5987-ca96-4c13-937d-fa83342f056c-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-fe020d-6dfcbdd58f-2vjb6\" (UID: \"caaa5987-ca96-4c13-937d-fa83342f056c\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-fe020d-6dfcbdd58f-2vjb6"
Sep 15 22:40:30.326488 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:30.326451    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mc4d\" (UniqueName: \"kubernetes.io/projected/caaa5987-ca96-4c13-937d-fa83342f056c-kube-api-access-6mc4d\") pod \"payload-processing-e2e-worker-w0-fe020d-6dfcbdd58f-2vjb6\" (UID: \"caaa5987-ca96-4c13-937d-fa83342f056c\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-fe020d-6dfcbdd58f-2vjb6"
Sep 15 22:40:30.326488 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:30.326489    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/caaa5987-ca96-4c13-937d-fa83342f056c-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-fe020d-6dfcbdd58f-2vjb6\" (UID: \"caaa5987-ca96-4c13-937d-fa83342f056c\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-fe020d-6dfcbdd58f-2vjb6"
Sep 15 22:40:30.327227 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:30.327209    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/caaa5987-ca96-4c13-937d-fa83342f056c-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-fe020d-6dfcbdd58f-2vjb6\" (UID: \"caaa5987-ca96-4c13-937d-fa83342f056c\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-fe020d-6dfcbdd58f-2vjb6"
Sep 15 22:40:30.337887 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:30.337857    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mc4d\" (UniqueName: \"kubernetes.io/projected/caaa5987-ca96-4c13-937d-fa83342f056c-kube-api-access-6mc4d\") pod \"payload-processing-e2e-worker-w0-fe020d-6dfcbdd58f-2vjb6\" (UID: \"caaa5987-ca96-4c13-937d-fa83342f056c\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-fe020d-6dfcbdd58f-2vjb6"
Sep 15 22:40:30.485055 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:30.484948    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-fe020d-6dfcbdd58f-2vjb6"
Sep 15 22:40:30.760593 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:30.760563    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-fe020d-6dfcbdd58f-2vjb6"]
Sep 15 22:40:30.761752 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:40:30.761722    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaaa5987_ca96_4c13_937d_fa83342f056c.slice/crio-51892ff2208df6a290aa572541c808ff1ad3ac6e0f9265c772a2e45c5bdb6367 WatchSource:0}: Error finding container 51892ff2208df6a290aa572541c808ff1ad3ac6e0f9265c772a2e45c5bdb6367: Status 404 returned error can't find the container with id 51892ff2208df6a290aa572541c808ff1ad3ac6e0f9265c772a2e45c5bdb6367
Sep 15 22:40:31.537595 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:31.537558    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-fe020d-6dfcbdd58f-2vjb6" event={"ID":"caaa5987-ca96-4c13-937d-fa83342f056c","Type":"ContainerStarted","Data":"12dcb2e13132c33f0f6adad25a30293b5af8de9168916e4f096037320c15dad3"}
Sep 15 22:40:31.537780 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:31.537616    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-fe020d-6dfcbdd58f-2vjb6" event={"ID":"caaa5987-ca96-4c13-937d-fa83342f056c","Type":"ContainerStarted","Data":"51892ff2208df6a290aa572541c808ff1ad3ac6e0f9265c772a2e45c5bdb6367"}
Sep 15 22:40:31.537780 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:31.537675    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-fe020d-6dfcbdd58f-2vjb6"
Sep 15 22:40:31.558443 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:31.558391    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-fe020d-6dfcbdd58f-2vjb6" podStartSLOduration=1.558372362 podStartE2EDuration="1.558372362s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:40:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:40:31.55403849 +0000 UTC m=+781.413749768" watchObservedRunningTime="2026-09-15 22:40:31.558372362 +0000 UTC m=+781.418083648"
Sep 15 22:40:35.535416 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:35.535380    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4"
Sep 15 22:40:35.535997 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:35.535476    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx"
Sep 15 22:40:37.291523 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:37.291478    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm"]
Sep 15 22:40:37.295410 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:37.295391    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm"
Sep 15 22:40:37.297216 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:37.297198    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-33ff1a95-serving-cert\""
Sep 15 22:40:37.303441 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:37.303418    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm"]
Sep 15 22:40:37.395576 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:37.395541    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff825dda-452f-473f-b582-b17257d5b019-openshift-service-ca\") pod \"maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm\" (UID: \"ff825dda-452f-473f-b582-b17257d5b019\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm"
Sep 15 22:40:37.395744 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:37.395671    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff825dda-452f-473f-b582-b17257d5b019-metrics-certs\") pod \"maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm\" (UID: \"ff825dda-452f-473f-b582-b17257d5b019\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm"
Sep 15 22:40:37.395744 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:37.395717    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59ztk\" (UniqueName: \"kubernetes.io/projected/ff825dda-452f-473f-b582-b17257d5b019-kube-api-access-59ztk\") pod \"maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm\" (UID: \"ff825dda-452f-473f-b582-b17257d5b019\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm"
Sep 15 22:40:37.395819 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:37.395746    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff825dda-452f-473f-b582-b17257d5b019-maas-api-tls\") pod \"maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm\" (UID: \"ff825dda-452f-473f-b582-b17257d5b019\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm"
Sep 15 22:40:37.497246 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:37.497214    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff825dda-452f-473f-b582-b17257d5b019-openshift-service-ca\") pod \"maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm\" (UID: \"ff825dda-452f-473f-b582-b17257d5b019\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm"
Sep 15 22:40:37.497409 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:37.497376    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff825dda-452f-473f-b582-b17257d5b019-metrics-certs\") pod \"maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm\" (UID: \"ff825dda-452f-473f-b582-b17257d5b019\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm"
Sep 15 22:40:37.497462 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:37.497425    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59ztk\" (UniqueName: \"kubernetes.io/projected/ff825dda-452f-473f-b582-b17257d5b019-kube-api-access-59ztk\") pod \"maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm\" (UID: \"ff825dda-452f-473f-b582-b17257d5b019\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm"
Sep 15 22:40:37.497462 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:37.497452    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff825dda-452f-473f-b582-b17257d5b019-maas-api-tls\") pod \"maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm\" (UID: \"ff825dda-452f-473f-b582-b17257d5b019\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm"
Sep 15 22:40:37.497884 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:37.497861    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff825dda-452f-473f-b582-b17257d5b019-openshift-service-ca\") pod \"maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm\" (UID: \"ff825dda-452f-473f-b582-b17257d5b019\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm"
Sep 15 22:40:37.499714 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:37.499697    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff825dda-452f-473f-b582-b17257d5b019-metrics-certs\") pod \"maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm\" (UID: \"ff825dda-452f-473f-b582-b17257d5b019\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm"
Sep 15 22:40:37.499865 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:37.499845    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff825dda-452f-473f-b582-b17257d5b019-maas-api-tls\") pod \"maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm\" (UID: \"ff825dda-452f-473f-b582-b17257d5b019\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm"
Sep 15 22:40:37.506778 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:37.506747    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59ztk\" (UniqueName: \"kubernetes.io/projected/ff825dda-452f-473f-b582-b17257d5b019-kube-api-access-59ztk\") pod \"maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm\" (UID: \"ff825dda-452f-473f-b582-b17257d5b019\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm"
Sep 15 22:40:37.608512 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:37.608431    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm"
Sep 15 22:40:37.886984 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:37.886957    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm"]
Sep 15 22:40:37.888850 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:40:37.888821    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff825dda_452f_473f_b582_b17257d5b019.slice/crio-d23a9eea5016e1e84b30f7aea92887a63263ace197c67f7699df507fce3bf662 WatchSource:0}: Error finding container d23a9eea5016e1e84b30f7aea92887a63263ace197c67f7699df507fce3bf662: Status 404 returned error can't find the container with id d23a9eea5016e1e84b30f7aea92887a63263ace197c67f7699df507fce3bf662
Sep 15 22:40:38.579684 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:38.579651    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm" event={"ID":"ff825dda-452f-473f-b582-b17257d5b019","Type":"ContainerStarted","Data":"f2d0e2d38e2d32fd518ad03df7c215d944f0804b9c2c1c7d9d4865b98b40cc36"}
Sep 15 22:40:38.580061 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:38.579703    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm" event={"ID":"ff825dda-452f-473f-b582-b17257d5b019","Type":"ContainerStarted","Data":"d23a9eea5016e1e84b30f7aea92887a63263ace197c67f7699df507fce3bf662"}
Sep 15 22:40:38.580061 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:38.579770    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm"
Sep 15 22:40:38.602092 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:38.602019    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm" podStartSLOduration=1.2304266990000001 podStartE2EDuration="1.601999232s" totalImagesPullingTime="371.572533ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:40:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:40:38.599183488 +0000 UTC m=+788.458894776" watchObservedRunningTime="2026-09-15 22:40:38.601999232 +0000 UTC m=+788.461710518"
Sep 15 22:40:39.523937 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:39.523908    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-fffdae-846d8cb8b-9bsts"
Sep 15 22:40:39.891586 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:39.891554    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg"]
Sep 15 22:40:39.895358 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:39.895338    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg"
Sep 15 22:40:39.897564 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:39.897545    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-b08870-serving-cert\""
Sep 15 22:40:39.908359 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:39.908337    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg"]
Sep 15 22:40:39.920163 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:39.920132    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/885fa4e1-068e-4306-b350-ed16cc000595-metrics-certs\") pod \"maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg\" (UID: \"885fa4e1-068e-4306-b350-ed16cc000595\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg"
Sep 15 22:40:39.920272 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:39.920239    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/885fa4e1-068e-4306-b350-ed16cc000595-openshift-service-ca\") pod \"maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg\" (UID: \"885fa4e1-068e-4306-b350-ed16cc000595\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg"
Sep 15 22:40:39.920356 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:39.920272    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/885fa4e1-068e-4306-b350-ed16cc000595-maas-api-tls\") pod \"maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg\" (UID: \"885fa4e1-068e-4306-b350-ed16cc000595\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg"
Sep 15 22:40:39.920356 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:39.920296    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vrpx\" (UniqueName: \"kubernetes.io/projected/885fa4e1-068e-4306-b350-ed16cc000595-kube-api-access-8vrpx\") pod \"maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg\" (UID: \"885fa4e1-068e-4306-b350-ed16cc000595\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg"
Sep 15 22:40:40.020996 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:40.020960    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/885fa4e1-068e-4306-b350-ed16cc000595-metrics-certs\") pod \"maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg\" (UID: \"885fa4e1-068e-4306-b350-ed16cc000595\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg"
Sep 15 22:40:40.021167 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:40.021112    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/885fa4e1-068e-4306-b350-ed16cc000595-openshift-service-ca\") pod \"maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg\" (UID: \"885fa4e1-068e-4306-b350-ed16cc000595\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg"
Sep 15 22:40:40.021167 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:40.021149    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/885fa4e1-068e-4306-b350-ed16cc000595-maas-api-tls\") pod \"maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg\" (UID: \"885fa4e1-068e-4306-b350-ed16cc000595\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg"
Sep 15 22:40:40.021277 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:40.021178    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vrpx\" (UniqueName: \"kubernetes.io/projected/885fa4e1-068e-4306-b350-ed16cc000595-kube-api-access-8vrpx\") pod \"maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg\" (UID: \"885fa4e1-068e-4306-b350-ed16cc000595\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg"
Sep 15 22:40:40.021327 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:40:40.021281    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-b08870-serving-cert: secret "maas-api-e2e-autores-b08870-serving-cert" not found
Sep 15 22:40:40.021370 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:40:40.021359    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/885fa4e1-068e-4306-b350-ed16cc000595-maas-api-tls podName:885fa4e1-068e-4306-b350-ed16cc000595 nodeName:}" failed. No retries permitted until 2026-09-15 22:40:40.521339021 +0000 UTC m=+790.381050302 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/885fa4e1-068e-4306-b350-ed16cc000595-maas-api-tls") pod "maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg" (UID: "885fa4e1-068e-4306-b350-ed16cc000595") : secret "maas-api-e2e-autores-b08870-serving-cert" not found
Sep 15 22:40:40.021911 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:40.021882    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/885fa4e1-068e-4306-b350-ed16cc000595-openshift-service-ca\") pod \"maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg\" (UID: \"885fa4e1-068e-4306-b350-ed16cc000595\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg"
Sep 15 22:40:40.023950 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:40.023926    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/885fa4e1-068e-4306-b350-ed16cc000595-metrics-certs\") pod \"maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg\" (UID: \"885fa4e1-068e-4306-b350-ed16cc000595\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg"
Sep 15 22:40:40.028675 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:40.028652    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vrpx\" (UniqueName: \"kubernetes.io/projected/885fa4e1-068e-4306-b350-ed16cc000595-kube-api-access-8vrpx\") pod \"maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg\" (UID: \"885fa4e1-068e-4306-b350-ed16cc000595\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg"
Sep 15 22:40:40.526688 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:40.526649    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/885fa4e1-068e-4306-b350-ed16cc000595-maas-api-tls\") pod \"maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg\" (UID: \"885fa4e1-068e-4306-b350-ed16cc000595\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg"
Sep 15 22:40:40.529116 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:40.529091    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/885fa4e1-068e-4306-b350-ed16cc000595-maas-api-tls\") pod \"maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg\" (UID: \"885fa4e1-068e-4306-b350-ed16cc000595\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg"
Sep 15 22:40:40.778375 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:40.778301    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-b08870-659658b6b6-hjhhz"]
Sep 15 22:40:40.782449 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:40.782427    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-b08870-659658b6b6-hjhhz"
Sep 15 22:40:40.784952 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:40.784924    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-b08870\""
Sep 15 22:40:40.794664 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:40.794637    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-b08870-659658b6b6-hjhhz"]
Sep 15 22:40:40.809385 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:40.809355    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg"
Sep 15 22:40:40.829458 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:40.829431    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2032f779-1d90-415a-b785-f9662989cf34-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-b08870-659658b6b6-hjhhz\" (UID: \"2032f779-1d90-415a-b785-f9662989cf34\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-b08870-659658b6b6-hjhhz"
Sep 15 22:40:40.829563 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:40.829474    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dv4qq\" (UniqueName: \"kubernetes.io/projected/2032f779-1d90-415a-b785-f9662989cf34-kube-api-access-dv4qq\") pod \"payload-pre-processing-e2e-autores-b08870-659658b6b6-hjhhz\" (UID: \"2032f779-1d90-415a-b785-f9662989cf34\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-b08870-659658b6b6-hjhhz"
Sep 15 22:40:40.930034 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:40.930001    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2032f779-1d90-415a-b785-f9662989cf34-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-b08870-659658b6b6-hjhhz\" (UID: \"2032f779-1d90-415a-b785-f9662989cf34\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-b08870-659658b6b6-hjhhz"
Sep 15 22:40:40.930388 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:40.930070    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dv4qq\" (UniqueName: \"kubernetes.io/projected/2032f779-1d90-415a-b785-f9662989cf34-kube-api-access-dv4qq\") pod \"payload-pre-processing-e2e-autores-b08870-659658b6b6-hjhhz\" (UID: \"2032f779-1d90-415a-b785-f9662989cf34\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-b08870-659658b6b6-hjhhz"
Sep 15 22:40:40.930733 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:40.930714    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2032f779-1d90-415a-b785-f9662989cf34-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-b08870-659658b6b6-hjhhz\" (UID: \"2032f779-1d90-415a-b785-f9662989cf34\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-b08870-659658b6b6-hjhhz"
Sep 15 22:40:40.942574 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:40.942548    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dv4qq\" (UniqueName: \"kubernetes.io/projected/2032f779-1d90-415a-b785-f9662989cf34-kube-api-access-dv4qq\") pod \"payload-pre-processing-e2e-autores-b08870-659658b6b6-hjhhz\" (UID: \"2032f779-1d90-415a-b785-f9662989cf34\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-b08870-659658b6b6-hjhhz"
Sep 15 22:40:41.092342 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:41.092317    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg"]
Sep 15 22:40:41.093537 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:40:41.093510    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod885fa4e1_068e_4306_b350_ed16cc000595.slice/crio-868f92e4fab8058974ee469801558cb641435eebd15a2df9f52728bbd36d82d1 WatchSource:0}: Error finding container 868f92e4fab8058974ee469801558cb641435eebd15a2df9f52728bbd36d82d1: Status 404 returned error can't find the container with id 868f92e4fab8058974ee469801558cb641435eebd15a2df9f52728bbd36d82d1
Sep 15 22:40:41.096438 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:41.096419    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-b08870-659658b6b6-hjhhz"
Sep 15 22:40:41.373726 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:41.373696    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-b08870-659658b6b6-hjhhz"]
Sep 15 22:40:41.375663 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:40:41.375637    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2032f779_1d90_415a_b785_f9662989cf34.slice/crio-07d9276c1ec48e3b17aeac910fa0557ecac09acb3e43db3a21d4029ae0ba93f9 WatchSource:0}: Error finding container 07d9276c1ec48e3b17aeac910fa0557ecac09acb3e43db3a21d4029ae0ba93f9: Status 404 returned error can't find the container with id 07d9276c1ec48e3b17aeac910fa0557ecac09acb3e43db3a21d4029ae0ba93f9
Sep 15 22:40:41.600251 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:41.600215    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg" event={"ID":"885fa4e1-068e-4306-b350-ed16cc000595","Type":"ContainerStarted","Data":"47c2b2428278c08435902b28599052d91bea296eeb4b85bd31e653485705b432"}
Sep 15 22:40:41.600424 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:41.600276    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg" event={"ID":"885fa4e1-068e-4306-b350-ed16cc000595","Type":"ContainerStarted","Data":"868f92e4fab8058974ee469801558cb641435eebd15a2df9f52728bbd36d82d1"}
Sep 15 22:40:41.600424 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:41.600410    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg"
Sep 15 22:40:41.608811 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:41.608777    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-b08870-659658b6b6-hjhhz" event={"ID":"2032f779-1d90-415a-b785-f9662989cf34","Type":"ContainerStarted","Data":"c9c6ba3301e642cecdff98fddf2854772e5d30effa52543a47adb5c2debf43d4"}
Sep 15 22:40:41.608932 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:41.608836    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-b08870-659658b6b6-hjhhz"
Sep 15 22:40:41.608932 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:41.608857    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-b08870-659658b6b6-hjhhz" event={"ID":"2032f779-1d90-415a-b785-f9662989cf34","Type":"ContainerStarted","Data":"07d9276c1ec48e3b17aeac910fa0557ecac09acb3e43db3a21d4029ae0ba93f9"}
Sep 15 22:40:41.627976 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:41.627874    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg" podStartSLOduration=2.233045713 podStartE2EDuration="2.627847066s" totalImagesPullingTime="394.801353ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:40:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:40:41.62623991 +0000 UTC m=+791.485951196" watchObservedRunningTime="2026-09-15 22:40:41.627847066 +0000 UTC m=+791.487558351"
Sep 15 22:40:41.688069 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:41.688005    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-b08870-659658b6b6-hjhhz" podStartSLOduration=1.687988525 podStartE2EDuration="1.687988525s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:40:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:40:41.685512355 +0000 UTC m=+791.545223640" watchObservedRunningTime="2026-09-15 22:40:41.687988525 +0000 UTC m=+791.547699809"
Sep 15 22:40:42.545095 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:42.545058    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-fe020d-6dfcbdd58f-2vjb6"
Sep 15 22:40:44.595348 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:44.595319    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm"
Sep 15 22:40:47.622038 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:47.622007    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg"
Sep 15 22:40:52.618864 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:52.618835    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-b08870-659658b6b6-hjhhz"
Sep 15 22:40:53.675539 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:53.675463    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-765e6d-7cc87d5854-kdqmd"]
Sep 15 22:40:53.679546 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:53.679525    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-765e6d-7cc87d5854-kdqmd"
Sep 15 22:40:53.681890 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:53.681829    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-765e6d\""
Sep 15 22:40:53.682205 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:53.682178    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-765e6d-dockercfg-l8mkj\""
Sep 15 22:40:53.688497 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:53.688471    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-765e6d-7cc87d5854-kdqmd"]
Sep 15 22:40:53.755984 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:53.755956    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec30af64-6264-4f7a-9ff2-b3be5d20b42a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-765e6d-7cc87d5854-kdqmd\" (UID: \"ec30af64-6264-4f7a-9ff2-b3be5d20b42a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-765e6d-7cc87d5854-kdqmd"
Sep 15 22:40:53.756149 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:53.756073    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bf74\" (UniqueName: \"kubernetes.io/projected/ec30af64-6264-4f7a-9ff2-b3be5d20b42a-kube-api-access-8bf74\") pod \"payload-pre-processing-e2e-inf-b-765e6d-7cc87d5854-kdqmd\" (UID: \"ec30af64-6264-4f7a-9ff2-b3be5d20b42a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-765e6d-7cc87d5854-kdqmd"
Sep 15 22:40:53.857332 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:53.857300    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec30af64-6264-4f7a-9ff2-b3be5d20b42a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-765e6d-7cc87d5854-kdqmd\" (UID: \"ec30af64-6264-4f7a-9ff2-b3be5d20b42a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-765e6d-7cc87d5854-kdqmd"
Sep 15 22:40:53.857490 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:53.857393    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bf74\" (UniqueName: \"kubernetes.io/projected/ec30af64-6264-4f7a-9ff2-b3be5d20b42a-kube-api-access-8bf74\") pod \"payload-pre-processing-e2e-inf-b-765e6d-7cc87d5854-kdqmd\" (UID: \"ec30af64-6264-4f7a-9ff2-b3be5d20b42a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-765e6d-7cc87d5854-kdqmd"
Sep 15 22:40:53.857972 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:53.857951    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec30af64-6264-4f7a-9ff2-b3be5d20b42a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-765e6d-7cc87d5854-kdqmd\" (UID: \"ec30af64-6264-4f7a-9ff2-b3be5d20b42a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-765e6d-7cc87d5854-kdqmd"
Sep 15 22:40:53.865674 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:53.865642    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bf74\" (UniqueName: \"kubernetes.io/projected/ec30af64-6264-4f7a-9ff2-b3be5d20b42a-kube-api-access-8bf74\") pod \"payload-pre-processing-e2e-inf-b-765e6d-7cc87d5854-kdqmd\" (UID: \"ec30af64-6264-4f7a-9ff2-b3be5d20b42a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-765e6d-7cc87d5854-kdqmd"
Sep 15 22:40:53.992563 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:53.992479    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-765e6d-7cc87d5854-kdqmd"
Sep 15 22:40:54.270795 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:54.270765    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-765e6d-7cc87d5854-kdqmd"]
Sep 15 22:40:54.272211 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:40:54.272184    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec30af64_6264_4f7a_9ff2_b3be5d20b42a.slice/crio-38002ef4fd9638b6602a85579095b47b6d6b8d867e879d96593d2ecb07542666 WatchSource:0}: Error finding container 38002ef4fd9638b6602a85579095b47b6d6b8d867e879d96593d2ecb07542666: Status 404 returned error can't find the container with id 38002ef4fd9638b6602a85579095b47b6d6b8d867e879d96593d2ecb07542666
Sep 15 22:40:54.688759 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:54.688724    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-765e6d-7cc87d5854-kdqmd" event={"ID":"ec30af64-6264-4f7a-9ff2-b3be5d20b42a","Type":"ContainerStarted","Data":"16d9782645be11bb0d4036c0f1f3c34ea5a30341e2e1aaaebd50fb09089e7e39"}
Sep 15 22:40:54.689149 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:54.688775    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-765e6d-7cc87d5854-kdqmd"
Sep 15 22:40:54.689149 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:54.688796    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-765e6d-7cc87d5854-kdqmd" event={"ID":"ec30af64-6264-4f7a-9ff2-b3be5d20b42a","Type":"ContainerStarted","Data":"38002ef4fd9638b6602a85579095b47b6d6b8d867e879d96593d2ecb07542666"}
Sep 15 22:40:54.707614 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:40:54.707534    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-765e6d-7cc87d5854-kdqmd" podStartSLOduration=1.7075153410000001 podStartE2EDuration="1.707515341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:40:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:40:54.702743555 +0000 UTC m=+804.562454842" watchObservedRunningTime="2026-09-15 22:40:54.707515341 +0000 UTC m=+804.567226626"
Sep 15 22:41:00.967936 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:00.967900    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm"]
Sep 15 22:41:00.968427 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:00.968278    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm" podUID="ff825dda-452f-473f-b582-b17257d5b019" containerName="maas-api" containerID="cri-o://f2d0e2d38e2d32fd518ad03df7c215d944f0804b9c2c1c7d9d4865b98b40cc36" gracePeriod=30
Sep 15 22:41:01.227677 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:01.227615    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm"
Sep 15 22:41:01.330377 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:01.330332    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff825dda-452f-473f-b582-b17257d5b019-openshift-service-ca\") pod \"ff825dda-452f-473f-b582-b17257d5b019\" (UID: \"ff825dda-452f-473f-b582-b17257d5b019\") "
Sep 15 22:41:01.330537 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:01.330394    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff825dda-452f-473f-b582-b17257d5b019-maas-api-tls\") pod \"ff825dda-452f-473f-b582-b17257d5b019\" (UID: \"ff825dda-452f-473f-b582-b17257d5b019\") "
Sep 15 22:41:01.330537 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:01.330431    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-59ztk\" (UniqueName: \"kubernetes.io/projected/ff825dda-452f-473f-b582-b17257d5b019-kube-api-access-59ztk\") pod \"ff825dda-452f-473f-b582-b17257d5b019\" (UID: \"ff825dda-452f-473f-b582-b17257d5b019\") "
Sep 15 22:41:01.330537 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:01.330509    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff825dda-452f-473f-b582-b17257d5b019-metrics-certs\") pod \"ff825dda-452f-473f-b582-b17257d5b019\" (UID: \"ff825dda-452f-473f-b582-b17257d5b019\") "
Sep 15 22:41:01.331193 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:01.331162    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff825dda-452f-473f-b582-b17257d5b019-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ff825dda-452f-473f-b582-b17257d5b019" (UID: "ff825dda-452f-473f-b582-b17257d5b019"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:41:01.334414 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:01.334388    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff825dda-452f-473f-b582-b17257d5b019-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ff825dda-452f-473f-b582-b17257d5b019" (UID: "ff825dda-452f-473f-b582-b17257d5b019"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:41:01.335848 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:01.335822    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff825dda-452f-473f-b582-b17257d5b019-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ff825dda-452f-473f-b582-b17257d5b019" (UID: "ff825dda-452f-473f-b582-b17257d5b019"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:41:01.337847 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:01.337648    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff825dda-452f-473f-b582-b17257d5b019-kube-api-access-59ztk" (OuterVolumeSpecName: "kube-api-access-59ztk") pod "ff825dda-452f-473f-b582-b17257d5b019" (UID: "ff825dda-452f-473f-b582-b17257d5b019"). InnerVolumeSpecName "kube-api-access-59ztk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:41:01.431219 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:01.431186    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff825dda-452f-473f-b582-b17257d5b019-metrics-certs\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:41:01.431219 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:01.431211    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff825dda-452f-473f-b582-b17257d5b019-openshift-service-ca\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:41:01.431219 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:01.431222    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff825dda-452f-473f-b582-b17257d5b019-maas-api-tls\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:41:01.431499 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:01.431232    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59ztk\" (UniqueName: \"kubernetes.io/projected/ff825dda-452f-473f-b582-b17257d5b019-kube-api-access-59ztk\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:41:01.733956 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:01.733916    2565 generic.go:356] "Generic (PLEG): container finished" podID="ff825dda-452f-473f-b582-b17257d5b019" containerID="f2d0e2d38e2d32fd518ad03df7c215d944f0804b9c2c1c7d9d4865b98b40cc36" exitCode=0
Sep 15 22:41:01.734136 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:01.733965    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm"
Sep 15 22:41:01.734136 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:01.733962    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm" event={"ID":"ff825dda-452f-473f-b582-b17257d5b019","Type":"ContainerDied","Data":"f2d0e2d38e2d32fd518ad03df7c215d944f0804b9c2c1c7d9d4865b98b40cc36"}
Sep 15 22:41:01.734136 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:01.734099    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm" event={"ID":"ff825dda-452f-473f-b582-b17257d5b019","Type":"ContainerDied","Data":"d23a9eea5016e1e84b30f7aea92887a63263ace197c67f7699df507fce3bf662"}
Sep 15 22:41:01.734136 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:01.734129    2565 scope.go:117] "RemoveContainer" containerID="f2d0e2d38e2d32fd518ad03df7c215d944f0804b9c2c1c7d9d4865b98b40cc36"
Sep 15 22:41:01.744591 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:01.744573    2565 scope.go:117] "RemoveContainer" containerID="f2d0e2d38e2d32fd518ad03df7c215d944f0804b9c2c1c7d9d4865b98b40cc36"
Sep 15 22:41:01.744867 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:41:01.744833    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2d0e2d38e2d32fd518ad03df7c215d944f0804b9c2c1c7d9d4865b98b40cc36\": container with ID starting with f2d0e2d38e2d32fd518ad03df7c215d944f0804b9c2c1c7d9d4865b98b40cc36 not found: ID does not exist" containerID="f2d0e2d38e2d32fd518ad03df7c215d944f0804b9c2c1c7d9d4865b98b40cc36"
Sep 15 22:41:01.744906 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:01.744876    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2d0e2d38e2d32fd518ad03df7c215d944f0804b9c2c1c7d9d4865b98b40cc36"} err="failed to get container status \"f2d0e2d38e2d32fd518ad03df7c215d944f0804b9c2c1c7d9d4865b98b40cc36\": rpc error: code = NotFound desc = could not find container \"f2d0e2d38e2d32fd518ad03df7c215d944f0804b9c2c1c7d9d4865b98b40cc36\": container with ID starting with f2d0e2d38e2d32fd518ad03df7c215d944f0804b9c2c1c7d9d4865b98b40cc36 not found: ID does not exist"
Sep 15 22:41:01.756115 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:01.756088    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm"]
Sep 15 22:41:01.764796 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:01.764775    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-33ff1a95-844c967dc7-jn4tm"]
Sep 15 22:41:02.590996 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:02.590959    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff825dda-452f-473f-b582-b17257d5b019" path="/var/lib/kubelet/pods/ff825dda-452f-473f-b582-b17257d5b019/volumes"
Sep 15 22:41:05.697490 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:05.697455    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-765e6d-7cc87d5854-kdqmd"
Sep 15 22:41:07.590123 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:07.590085    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9"]
Sep 15 22:41:07.590882 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:07.590545    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff825dda-452f-473f-b582-b17257d5b019" containerName="maas-api"
Sep 15 22:41:07.590882 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:07.590557    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff825dda-452f-473f-b582-b17257d5b019" containerName="maas-api"
Sep 15 22:41:07.590882 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:07.590631    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff825dda-452f-473f-b582-b17257d5b019" containerName="maas-api"
Sep 15 22:41:07.595427 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:07.595407    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9"
Sep 15 22:41:07.597979 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:07.597955    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-36d266fd-serving-cert\""
Sep 15 22:41:07.618500 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:07.618476    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9"]
Sep 15 22:41:07.693016 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:07.692987    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/274784a9-0b11-41a2-a9af-a5d1d5a30cfd-metrics-certs\") pod \"maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9\" (UID: \"274784a9-0b11-41a2-a9af-a5d1d5a30cfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9"
Sep 15 22:41:07.693191 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:07.693059    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/274784a9-0b11-41a2-a9af-a5d1d5a30cfd-openshift-service-ca\") pod \"maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9\" (UID: \"274784a9-0b11-41a2-a9af-a5d1d5a30cfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9"
Sep 15 22:41:07.693191 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:07.693100    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/274784a9-0b11-41a2-a9af-a5d1d5a30cfd-maas-api-tls\") pod \"maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9\" (UID: \"274784a9-0b11-41a2-a9af-a5d1d5a30cfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9"
Sep 15 22:41:07.693191 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:07.693136    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p68fx\" (UniqueName: \"kubernetes.io/projected/274784a9-0b11-41a2-a9af-a5d1d5a30cfd-kube-api-access-p68fx\") pod \"maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9\" (UID: \"274784a9-0b11-41a2-a9af-a5d1d5a30cfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9"
Sep 15 22:41:07.794470 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:07.794436    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/274784a9-0b11-41a2-a9af-a5d1d5a30cfd-metrics-certs\") pod \"maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9\" (UID: \"274784a9-0b11-41a2-a9af-a5d1d5a30cfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9"
Sep 15 22:41:07.794631 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:07.794481    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/274784a9-0b11-41a2-a9af-a5d1d5a30cfd-openshift-service-ca\") pod \"maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9\" (UID: \"274784a9-0b11-41a2-a9af-a5d1d5a30cfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9"
Sep 15 22:41:07.794631 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:07.794508    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/274784a9-0b11-41a2-a9af-a5d1d5a30cfd-maas-api-tls\") pod \"maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9\" (UID: \"274784a9-0b11-41a2-a9af-a5d1d5a30cfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9"
Sep 15 22:41:07.794631 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:07.794535    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p68fx\" (UniqueName: \"kubernetes.io/projected/274784a9-0b11-41a2-a9af-a5d1d5a30cfd-kube-api-access-p68fx\") pod \"maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9\" (UID: \"274784a9-0b11-41a2-a9af-a5d1d5a30cfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9"
Sep 15 22:41:07.795225 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:07.795198    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/274784a9-0b11-41a2-a9af-a5d1d5a30cfd-openshift-service-ca\") pod \"maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9\" (UID: \"274784a9-0b11-41a2-a9af-a5d1d5a30cfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9"
Sep 15 22:41:07.796689 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:07.796661    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/274784a9-0b11-41a2-a9af-a5d1d5a30cfd-metrics-certs\") pod \"maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9\" (UID: \"274784a9-0b11-41a2-a9af-a5d1d5a30cfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9"
Sep 15 22:41:07.796689 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:07.796686    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/274784a9-0b11-41a2-a9af-a5d1d5a30cfd-maas-api-tls\") pod \"maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9\" (UID: \"274784a9-0b11-41a2-a9af-a5d1d5a30cfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9"
Sep 15 22:41:07.802438 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:07.802411    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p68fx\" (UniqueName: \"kubernetes.io/projected/274784a9-0b11-41a2-a9af-a5d1d5a30cfd-kube-api-access-p68fx\") pod \"maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9\" (UID: \"274784a9-0b11-41a2-a9af-a5d1d5a30cfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9"
Sep 15 22:41:07.906993 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:07.906925    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9"
Sep 15 22:41:08.183290 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:08.183249    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9"]
Sep 15 22:41:08.185067 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:41:08.185036    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod274784a9_0b11_41a2_a9af_a5d1d5a30cfd.slice/crio-2dbb4296763e5e1d5ba121eac60328a53c6907fb6a9c2589e0bb46469a881a75 WatchSource:0}: Error finding container 2dbb4296763e5e1d5ba121eac60328a53c6907fb6a9c2589e0bb46469a881a75: Status 404 returned error can't find the container with id 2dbb4296763e5e1d5ba121eac60328a53c6907fb6a9c2589e0bb46469a881a75
Sep 15 22:41:08.377232 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:08.377191    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-36d266fd-569bd5574-5vl8j"]
Sep 15 22:41:08.381757 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:08.381731    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-36d266fd-569bd5574-5vl8j"
Sep 15 22:41:08.384058 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:08.384017    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-36d266fd\""
Sep 15 22:41:08.384178 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:08.384017    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-36d266fd-dockercfg-lptfz\""
Sep 15 22:41:08.393921 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:08.393888    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-36d266fd-569bd5574-5vl8j"]
Sep 15 22:41:08.501762 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:08.501678    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/513f648d-6987-4fb6-aaa8-da9d9c47d45f-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-36d266fd-569bd5574-5vl8j\" (UID: \"513f648d-6987-4fb6-aaa8-da9d9c47d45f\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-36d266fd-569bd5574-5vl8j"
Sep 15 22:41:08.502038 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:08.502001    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9l8kb\" (UniqueName: \"kubernetes.io/projected/513f648d-6987-4fb6-aaa8-da9d9c47d45f-kube-api-access-9l8kb\") pod \"payload-pre-processing-e2e-migrate-36d266fd-569bd5574-5vl8j\" (UID: \"513f648d-6987-4fb6-aaa8-da9d9c47d45f\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-36d266fd-569bd5574-5vl8j"
Sep 15 22:41:08.603697 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:08.603655    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/513f648d-6987-4fb6-aaa8-da9d9c47d45f-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-36d266fd-569bd5574-5vl8j\" (UID: \"513f648d-6987-4fb6-aaa8-da9d9c47d45f\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-36d266fd-569bd5574-5vl8j"
Sep 15 22:41:08.604217 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:08.603713    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l8kb\" (UniqueName: \"kubernetes.io/projected/513f648d-6987-4fb6-aaa8-da9d9c47d45f-kube-api-access-9l8kb\") pod \"payload-pre-processing-e2e-migrate-36d266fd-569bd5574-5vl8j\" (UID: \"513f648d-6987-4fb6-aaa8-da9d9c47d45f\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-36d266fd-569bd5574-5vl8j"
Sep 15 22:41:08.604733 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:08.604711    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/513f648d-6987-4fb6-aaa8-da9d9c47d45f-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-36d266fd-569bd5574-5vl8j\" (UID: \"513f648d-6987-4fb6-aaa8-da9d9c47d45f\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-36d266fd-569bd5574-5vl8j"
Sep 15 22:41:08.612171 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:08.612149    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9l8kb\" (UniqueName: \"kubernetes.io/projected/513f648d-6987-4fb6-aaa8-da9d9c47d45f-kube-api-access-9l8kb\") pod \"payload-pre-processing-e2e-migrate-36d266fd-569bd5574-5vl8j\" (UID: \"513f648d-6987-4fb6-aaa8-da9d9c47d45f\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-36d266fd-569bd5574-5vl8j"
Sep 15 22:41:08.702214 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:08.702185    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-36d266fd-569bd5574-5vl8j"
Sep 15 22:41:08.778155 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:08.778123    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9" event={"ID":"274784a9-0b11-41a2-a9af-a5d1d5a30cfd","Type":"ContainerStarted","Data":"8865637fb87a77fa8310385c47f94c80c79f0acda43392874d25e23c0de2204c"}
Sep 15 22:41:08.778294 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:08.778181    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9" event={"ID":"274784a9-0b11-41a2-a9af-a5d1d5a30cfd","Type":"ContainerStarted","Data":"2dbb4296763e5e1d5ba121eac60328a53c6907fb6a9c2589e0bb46469a881a75"}
Sep 15 22:41:08.778294 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:08.778212    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9"
Sep 15 22:41:08.801458 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:08.801389    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9" podStartSLOduration=1.472952988 podStartE2EDuration="1.801369162s" totalImagesPullingTime="328.416174ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:41:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:41:08.798219776 +0000 UTC m=+818.657931059" watchObservedRunningTime="2026-09-15 22:41:08.801369162 +0000 UTC m=+818.661080446"
Sep 15 22:41:08.978938 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:08.978907    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-36d266fd-569bd5574-5vl8j"]
Sep 15 22:41:08.981060 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:41:08.981017    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod513f648d_6987_4fb6_aaa8_da9d9c47d45f.slice/crio-61ac7f0fe8489016742b1a47c65f27882d84ce23ba022101bd6a2387f4801c91 WatchSource:0}: Error finding container 61ac7f0fe8489016742b1a47c65f27882d84ce23ba022101bd6a2387f4801c91: Status 404 returned error can't find the container with id 61ac7f0fe8489016742b1a47c65f27882d84ce23ba022101bd6a2387f4801c91
Sep 15 22:41:09.785671 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:09.785639    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-36d266fd-569bd5574-5vl8j" event={"ID":"513f648d-6987-4fb6-aaa8-da9d9c47d45f","Type":"ContainerStarted","Data":"59f2934a05640e5fe0fb6fc9fe5ae9bdcfaf03deb9623e2f182855dcd2884f79"}
Sep 15 22:41:09.786065 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:09.785837    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-36d266fd-569bd5574-5vl8j"
Sep 15 22:41:09.786065 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:09.785861    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-36d266fd-569bd5574-5vl8j" event={"ID":"513f648d-6987-4fb6-aaa8-da9d9c47d45f","Type":"ContainerStarted","Data":"61ac7f0fe8489016742b1a47c65f27882d84ce23ba022101bd6a2387f4801c91"}
Sep 15 22:41:09.804204 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:09.804150    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-36d266fd-569bd5574-5vl8j" podStartSLOduration=1.804134678 podStartE2EDuration="1.804134678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:41:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:41:09.800559878 +0000 UTC m=+819.660271159" watchObservedRunningTime="2026-09-15 22:41:09.804134678 +0000 UTC m=+819.663845961"
Sep 15 22:41:14.791316 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:14.791283    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9"
Sep 15 22:41:20.794431 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:20.794400    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-36d266fd-569bd5574-5vl8j"
Sep 15 22:41:46.339483 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:46.339448    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9"]
Sep 15 22:41:46.339971 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:46.339755    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9" podUID="274784a9-0b11-41a2-a9af-a5d1d5a30cfd" containerName="maas-api" containerID="cri-o://8865637fb87a77fa8310385c47f94c80c79f0acda43392874d25e23c0de2204c" gracePeriod=30
Sep 15 22:41:46.412307 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:46.412274    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-36d266fd-569bd5574-5vl8j"]
Sep 15 22:41:46.412552 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:46.412522    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-36d266fd-569bd5574-5vl8j" podUID="513f648d-6987-4fb6-aaa8-da9d9c47d45f" containerName="payload-pre-processing" containerID="cri-o://59f2934a05640e5fe0fb6fc9fe5ae9bdcfaf03deb9623e2f182855dcd2884f79" gracePeriod=30
Sep 15 22:41:46.669487 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:46.669462    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9"
Sep 15 22:41:46.709859 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:46.709835    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-36d266fd-569bd5574-5vl8j"
Sep 15 22:41:46.740276 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:46.740253    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p68fx\" (UniqueName: \"kubernetes.io/projected/274784a9-0b11-41a2-a9af-a5d1d5a30cfd-kube-api-access-p68fx\") pod \"274784a9-0b11-41a2-a9af-a5d1d5a30cfd\" (UID: \"274784a9-0b11-41a2-a9af-a5d1d5a30cfd\") "
Sep 15 22:41:46.740409 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:46.740373    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/274784a9-0b11-41a2-a9af-a5d1d5a30cfd-metrics-certs\") pod \"274784a9-0b11-41a2-a9af-a5d1d5a30cfd\" (UID: \"274784a9-0b11-41a2-a9af-a5d1d5a30cfd\") "
Sep 15 22:41:46.740409 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:46.740393    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/274784a9-0b11-41a2-a9af-a5d1d5a30cfd-openshift-service-ca\") pod \"274784a9-0b11-41a2-a9af-a5d1d5a30cfd\" (UID: \"274784a9-0b11-41a2-a9af-a5d1d5a30cfd\") "
Sep 15 22:41:46.740502 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:46.740424    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/274784a9-0b11-41a2-a9af-a5d1d5a30cfd-maas-api-tls\") pod \"274784a9-0b11-41a2-a9af-a5d1d5a30cfd\" (UID: \"274784a9-0b11-41a2-a9af-a5d1d5a30cfd\") "
Sep 15 22:41:46.741168 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:46.741139    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/274784a9-0b11-41a2-a9af-a5d1d5a30cfd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "274784a9-0b11-41a2-a9af-a5d1d5a30cfd" (UID: "274784a9-0b11-41a2-a9af-a5d1d5a30cfd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:41:46.742449 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:46.742423    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/274784a9-0b11-41a2-a9af-a5d1d5a30cfd-kube-api-access-p68fx" (OuterVolumeSpecName: "kube-api-access-p68fx") pod "274784a9-0b11-41a2-a9af-a5d1d5a30cfd" (UID: "274784a9-0b11-41a2-a9af-a5d1d5a30cfd"). InnerVolumeSpecName "kube-api-access-p68fx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:41:46.742928 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:46.742906    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/274784a9-0b11-41a2-a9af-a5d1d5a30cfd-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "274784a9-0b11-41a2-a9af-a5d1d5a30cfd" (UID: "274784a9-0b11-41a2-a9af-a5d1d5a30cfd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:41:46.742995 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:46.742926    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/274784a9-0b11-41a2-a9af-a5d1d5a30cfd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "274784a9-0b11-41a2-a9af-a5d1d5a30cfd" (UID: "274784a9-0b11-41a2-a9af-a5d1d5a30cfd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:41:46.841229 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:46.841203    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9l8kb\" (UniqueName: \"kubernetes.io/projected/513f648d-6987-4fb6-aaa8-da9d9c47d45f-kube-api-access-9l8kb\") pod \"513f648d-6987-4fb6-aaa8-da9d9c47d45f\" (UID: \"513f648d-6987-4fb6-aaa8-da9d9c47d45f\") "
Sep 15 22:41:46.841386 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:46.841251    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/513f648d-6987-4fb6-aaa8-da9d9c47d45f-plugins-config-volume\") pod \"513f648d-6987-4fb6-aaa8-da9d9c47d45f\" (UID: \"513f648d-6987-4fb6-aaa8-da9d9c47d45f\") "
Sep 15 22:41:46.841547 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:46.841533    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/274784a9-0b11-41a2-a9af-a5d1d5a30cfd-metrics-certs\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:41:46.841610 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:46.841550    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/274784a9-0b11-41a2-a9af-a5d1d5a30cfd-openshift-service-ca\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:41:46.841610 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:46.841562    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/274784a9-0b11-41a2-a9af-a5d1d5a30cfd-maas-api-tls\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:41:46.841610 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:46.841571    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p68fx\" (UniqueName: \"kubernetes.io/projected/274784a9-0b11-41a2-a9af-a5d1d5a30cfd-kube-api-access-p68fx\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:41:46.841705 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:46.841602    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/513f648d-6987-4fb6-aaa8-da9d9c47d45f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "513f648d-6987-4fb6-aaa8-da9d9c47d45f" (UID: "513f648d-6987-4fb6-aaa8-da9d9c47d45f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:41:46.843263 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:46.843241    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/513f648d-6987-4fb6-aaa8-da9d9c47d45f-kube-api-access-9l8kb" (OuterVolumeSpecName: "kube-api-access-9l8kb") pod "513f648d-6987-4fb6-aaa8-da9d9c47d45f" (UID: "513f648d-6987-4fb6-aaa8-da9d9c47d45f"). InnerVolumeSpecName "kube-api-access-9l8kb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:41:46.942976 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:46.942918    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9l8kb\" (UniqueName: \"kubernetes.io/projected/513f648d-6987-4fb6-aaa8-da9d9c47d45f-kube-api-access-9l8kb\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:41:46.942976 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:46.942941    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/513f648d-6987-4fb6-aaa8-da9d9c47d45f-plugins-config-volume\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:41:47.006178 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:47.006146    2565 generic.go:356] "Generic (PLEG): container finished" podID="274784a9-0b11-41a2-a9af-a5d1d5a30cfd" containerID="8865637fb87a77fa8310385c47f94c80c79f0acda43392874d25e23c0de2204c" exitCode=0
Sep 15 22:41:47.006322 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:47.006193    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9"
Sep 15 22:41:47.006322 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:47.006236    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9" event={"ID":"274784a9-0b11-41a2-a9af-a5d1d5a30cfd","Type":"ContainerDied","Data":"8865637fb87a77fa8310385c47f94c80c79f0acda43392874d25e23c0de2204c"}
Sep 15 22:41:47.006446 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:47.006360    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9" event={"ID":"274784a9-0b11-41a2-a9af-a5d1d5a30cfd","Type":"ContainerDied","Data":"2dbb4296763e5e1d5ba121eac60328a53c6907fb6a9c2589e0bb46469a881a75"}
Sep 15 22:41:47.006446 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:47.006389    2565 scope.go:117] "RemoveContainer" containerID="8865637fb87a77fa8310385c47f94c80c79f0acda43392874d25e23c0de2204c"
Sep 15 22:41:47.008216 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:47.008182    2565 generic.go:356] "Generic (PLEG): container finished" podID="513f648d-6987-4fb6-aaa8-da9d9c47d45f" containerID="59f2934a05640e5fe0fb6fc9fe5ae9bdcfaf03deb9623e2f182855dcd2884f79" exitCode=0
Sep 15 22:41:47.008320 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:47.008227    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-36d266fd-569bd5574-5vl8j"
Sep 15 22:41:47.008320 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:47.008234    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-36d266fd-569bd5574-5vl8j" event={"ID":"513f648d-6987-4fb6-aaa8-da9d9c47d45f","Type":"ContainerDied","Data":"59f2934a05640e5fe0fb6fc9fe5ae9bdcfaf03deb9623e2f182855dcd2884f79"}
Sep 15 22:41:47.008320 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:47.008286    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-36d266fd-569bd5574-5vl8j" event={"ID":"513f648d-6987-4fb6-aaa8-da9d9c47d45f","Type":"ContainerDied","Data":"61ac7f0fe8489016742b1a47c65f27882d84ce23ba022101bd6a2387f4801c91"}
Sep 15 22:41:47.021942 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:47.021447    2565 scope.go:117] "RemoveContainer" containerID="8865637fb87a77fa8310385c47f94c80c79f0acda43392874d25e23c0de2204c"
Sep 15 22:41:47.021942 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:41:47.021774    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8865637fb87a77fa8310385c47f94c80c79f0acda43392874d25e23c0de2204c\": container with ID starting with 8865637fb87a77fa8310385c47f94c80c79f0acda43392874d25e23c0de2204c not found: ID does not exist" containerID="8865637fb87a77fa8310385c47f94c80c79f0acda43392874d25e23c0de2204c"
Sep 15 22:41:47.021942 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:47.021807    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8865637fb87a77fa8310385c47f94c80c79f0acda43392874d25e23c0de2204c"} err="failed to get container status \"8865637fb87a77fa8310385c47f94c80c79f0acda43392874d25e23c0de2204c\": rpc error: code = NotFound desc = could not find container \"8865637fb87a77fa8310385c47f94c80c79f0acda43392874d25e23c0de2204c\": container with ID starting with 8865637fb87a77fa8310385c47f94c80c79f0acda43392874d25e23c0de2204c not found: ID does not exist"
Sep 15 22:41:47.021942 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:47.021854    2565 scope.go:117] "RemoveContainer" containerID="59f2934a05640e5fe0fb6fc9fe5ae9bdcfaf03deb9623e2f182855dcd2884f79"
Sep 15 22:41:47.030508 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:47.030492    2565 scope.go:117] "RemoveContainer" containerID="59f2934a05640e5fe0fb6fc9fe5ae9bdcfaf03deb9623e2f182855dcd2884f79"
Sep 15 22:41:47.030766 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:41:47.030751    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59f2934a05640e5fe0fb6fc9fe5ae9bdcfaf03deb9623e2f182855dcd2884f79\": container with ID starting with 59f2934a05640e5fe0fb6fc9fe5ae9bdcfaf03deb9623e2f182855dcd2884f79 not found: ID does not exist" containerID="59f2934a05640e5fe0fb6fc9fe5ae9bdcfaf03deb9623e2f182855dcd2884f79"
Sep 15 22:41:47.030818 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:47.030772    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59f2934a05640e5fe0fb6fc9fe5ae9bdcfaf03deb9623e2f182855dcd2884f79"} err="failed to get container status \"59f2934a05640e5fe0fb6fc9fe5ae9bdcfaf03deb9623e2f182855dcd2884f79\": rpc error: code = NotFound desc = could not find container \"59f2934a05640e5fe0fb6fc9fe5ae9bdcfaf03deb9623e2f182855dcd2884f79\": container with ID starting with 59f2934a05640e5fe0fb6fc9fe5ae9bdcfaf03deb9623e2f182855dcd2884f79 not found: ID does not exist"
Sep 15 22:41:47.036826 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:47.036805    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9"]
Sep 15 22:41:47.041020 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:47.041002    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-36d266fd-55cb6688c9-nnkt9"]
Sep 15 22:41:47.050075 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:47.050057    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-36d266fd-569bd5574-5vl8j"]
Sep 15 22:41:47.057948 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:47.057929    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-36d266fd-569bd5574-5vl8j"]
Sep 15 22:41:48.590654 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:48.590622    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="274784a9-0b11-41a2-a9af-a5d1d5a30cfd" path="/var/lib/kubelet/pods/274784a9-0b11-41a2-a9af-a5d1d5a30cfd/volumes"
Sep 15 22:41:48.591001 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:48.590983    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="513f648d-6987-4fb6-aaa8-da9d9c47d45f" path="/var/lib/kubelet/pods/513f648d-6987-4fb6-aaa8-da9d9c47d45f/volumes"
Sep 15 22:41:51.935795 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:51.935762    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t"]
Sep 15 22:41:51.936295 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:51.936276    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="274784a9-0b11-41a2-a9af-a5d1d5a30cfd" containerName="maas-api"
Sep 15 22:41:51.936295 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:51.936296    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="274784a9-0b11-41a2-a9af-a5d1d5a30cfd" containerName="maas-api"
Sep 15 22:41:51.936423 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:51.936316    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="513f648d-6987-4fb6-aaa8-da9d9c47d45f" containerName="payload-pre-processing"
Sep 15 22:41:51.936423 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:51.936322    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="513f648d-6987-4fb6-aaa8-da9d9c47d45f" containerName="payload-pre-processing"
Sep 15 22:41:51.936423 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:51.936404    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="274784a9-0b11-41a2-a9af-a5d1d5a30cfd" containerName="maas-api"
Sep 15 22:41:51.936423 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:51.936414    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="513f648d-6987-4fb6-aaa8-da9d9c47d45f" containerName="payload-pre-processing"
Sep 15 22:41:51.941136 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:51.941110    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t"
Sep 15 22:41:51.942998 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:51.942978    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-93550aef-serving-cert\""
Sep 15 22:41:51.954466 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:51.953951    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t"]
Sep 15 22:41:52.090887 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:52.090847    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f516cdea-ee92-43d0-8728-87fed4bc544a-metrics-certs\") pod \"maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t\" (UID: \"f516cdea-ee92-43d0-8728-87fed4bc544a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t"
Sep 15 22:41:52.091065 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:52.090933    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zz7j\" (UniqueName: \"kubernetes.io/projected/f516cdea-ee92-43d0-8728-87fed4bc544a-kube-api-access-8zz7j\") pod \"maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t\" (UID: \"f516cdea-ee92-43d0-8728-87fed4bc544a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t"
Sep 15 22:41:52.091065 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:52.090959    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f516cdea-ee92-43d0-8728-87fed4bc544a-maas-api-tls\") pod \"maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t\" (UID: \"f516cdea-ee92-43d0-8728-87fed4bc544a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t"
Sep 15 22:41:52.091065 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:52.090981    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f516cdea-ee92-43d0-8728-87fed4bc544a-openshift-service-ca\") pod \"maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t\" (UID: \"f516cdea-ee92-43d0-8728-87fed4bc544a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t"
Sep 15 22:41:52.192182 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:52.192105    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f516cdea-ee92-43d0-8728-87fed4bc544a-metrics-certs\") pod \"maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t\" (UID: \"f516cdea-ee92-43d0-8728-87fed4bc544a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t"
Sep 15 22:41:52.192182 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:52.192170    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zz7j\" (UniqueName: \"kubernetes.io/projected/f516cdea-ee92-43d0-8728-87fed4bc544a-kube-api-access-8zz7j\") pod \"maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t\" (UID: \"f516cdea-ee92-43d0-8728-87fed4bc544a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t"
Sep 15 22:41:52.192401 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:52.192195    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f516cdea-ee92-43d0-8728-87fed4bc544a-maas-api-tls\") pod \"maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t\" (UID: \"f516cdea-ee92-43d0-8728-87fed4bc544a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t"
Sep 15 22:41:52.192401 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:52.192212    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f516cdea-ee92-43d0-8728-87fed4bc544a-openshift-service-ca\") pod \"maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t\" (UID: \"f516cdea-ee92-43d0-8728-87fed4bc544a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t"
Sep 15 22:41:52.192928 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:52.192902    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f516cdea-ee92-43d0-8728-87fed4bc544a-openshift-service-ca\") pod \"maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t\" (UID: \"f516cdea-ee92-43d0-8728-87fed4bc544a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t"
Sep 15 22:41:52.194624 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:52.194603    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f516cdea-ee92-43d0-8728-87fed4bc544a-metrics-certs\") pod \"maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t\" (UID: \"f516cdea-ee92-43d0-8728-87fed4bc544a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t"
Sep 15 22:41:52.194704 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:52.194631    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f516cdea-ee92-43d0-8728-87fed4bc544a-maas-api-tls\") pod \"maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t\" (UID: \"f516cdea-ee92-43d0-8728-87fed4bc544a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t"
Sep 15 22:41:52.199521 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:52.199497    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zz7j\" (UniqueName: \"kubernetes.io/projected/f516cdea-ee92-43d0-8728-87fed4bc544a-kube-api-access-8zz7j\") pod \"maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t\" (UID: \"f516cdea-ee92-43d0-8728-87fed4bc544a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t"
Sep 15 22:41:52.257388 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:52.257361    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t"
Sep 15 22:41:52.536070 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:52.536041    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t"]
Sep 15 22:41:52.537724 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:41:52.537698    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf516cdea_ee92_43d0_8728_87fed4bc544a.slice/crio-ea17f9e6db0246c0346c09cb33747efd1f2857dfe16ae55daf6087a3a0c35ea0 WatchSource:0}: Error finding container ea17f9e6db0246c0346c09cb33747efd1f2857dfe16ae55daf6087a3a0c35ea0: Status 404 returned error can't find the container with id ea17f9e6db0246c0346c09cb33747efd1f2857dfe16ae55daf6087a3a0c35ea0
Sep 15 22:41:53.047082 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:53.047046    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t" event={"ID":"f516cdea-ee92-43d0-8728-87fed4bc544a","Type":"ContainerStarted","Data":"ed77c1725dc9c6b5db0d1134bc41ab6f862ef0a6336bc3c8019aecda269ce709"}
Sep 15 22:41:53.047509 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:53.047112    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t" event={"ID":"f516cdea-ee92-43d0-8728-87fed4bc544a","Type":"ContainerStarted","Data":"ea17f9e6db0246c0346c09cb33747efd1f2857dfe16ae55daf6087a3a0c35ea0"}
Sep 15 22:41:53.047509 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:53.047146    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t"
Sep 15 22:41:53.066097 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:53.066053    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t" podStartSLOduration=1.6979629649999999 podStartE2EDuration="2.066037045s" totalImagesPullingTime="368.07408ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:41:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:41:53.062946941 +0000 UTC m=+862.922658226" watchObservedRunningTime="2026-09-15 22:41:53.066037045 +0000 UTC m=+862.925748320"
Sep 15 22:41:59.058528 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:41:59.058496    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t"
Sep 15 22:42:21.735919 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:21.735886    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t"]
Sep 15 22:42:21.736457 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:21.736143    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t" podUID="f516cdea-ee92-43d0-8728-87fed4bc544a" containerName="maas-api" containerID="cri-o://ed77c1725dc9c6b5db0d1134bc41ab6f862ef0a6336bc3c8019aecda269ce709" gracePeriod=30
Sep 15 22:42:21.996594 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:21.996542    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t"
Sep 15 22:42:22.087819 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:22.087784    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8zz7j\" (UniqueName: \"kubernetes.io/projected/f516cdea-ee92-43d0-8728-87fed4bc544a-kube-api-access-8zz7j\") pod \"f516cdea-ee92-43d0-8728-87fed4bc544a\" (UID: \"f516cdea-ee92-43d0-8728-87fed4bc544a\") "
Sep 15 22:42:22.087995 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:22.087849    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f516cdea-ee92-43d0-8728-87fed4bc544a-openshift-service-ca\") pod \"f516cdea-ee92-43d0-8728-87fed4bc544a\" (UID: \"f516cdea-ee92-43d0-8728-87fed4bc544a\") "
Sep 15 22:42:22.087995 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:22.087932    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f516cdea-ee92-43d0-8728-87fed4bc544a-metrics-certs\") pod \"f516cdea-ee92-43d0-8728-87fed4bc544a\" (UID: \"f516cdea-ee92-43d0-8728-87fed4bc544a\") "
Sep 15 22:42:22.088160 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:22.088049    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f516cdea-ee92-43d0-8728-87fed4bc544a-maas-api-tls\") pod \"f516cdea-ee92-43d0-8728-87fed4bc544a\" (UID: \"f516cdea-ee92-43d0-8728-87fed4bc544a\") "
Sep 15 22:42:22.088242 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:22.088215    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f516cdea-ee92-43d0-8728-87fed4bc544a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f516cdea-ee92-43d0-8728-87fed4bc544a" (UID: "f516cdea-ee92-43d0-8728-87fed4bc544a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:42:22.088422 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:22.088407    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f516cdea-ee92-43d0-8728-87fed4bc544a-openshift-service-ca\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:42:22.090126 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:22.090104    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f516cdea-ee92-43d0-8728-87fed4bc544a-kube-api-access-8zz7j" (OuterVolumeSpecName: "kube-api-access-8zz7j") pod "f516cdea-ee92-43d0-8728-87fed4bc544a" (UID: "f516cdea-ee92-43d0-8728-87fed4bc544a"). InnerVolumeSpecName "kube-api-access-8zz7j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:42:22.090372 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:22.090351    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f516cdea-ee92-43d0-8728-87fed4bc544a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f516cdea-ee92-43d0-8728-87fed4bc544a" (UID: "f516cdea-ee92-43d0-8728-87fed4bc544a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:42:22.090435 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:22.090366    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f516cdea-ee92-43d0-8728-87fed4bc544a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f516cdea-ee92-43d0-8728-87fed4bc544a" (UID: "f516cdea-ee92-43d0-8728-87fed4bc544a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:42:22.189000 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:22.188958    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f516cdea-ee92-43d0-8728-87fed4bc544a-maas-api-tls\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:42:22.189000 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:22.188996    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8zz7j\" (UniqueName: \"kubernetes.io/projected/f516cdea-ee92-43d0-8728-87fed4bc544a-kube-api-access-8zz7j\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:42:22.189000 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:22.189006    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f516cdea-ee92-43d0-8728-87fed4bc544a-metrics-certs\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:42:22.232599 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:22.232574    2565 generic.go:356] "Generic (PLEG): container finished" podID="f516cdea-ee92-43d0-8728-87fed4bc544a" containerID="ed77c1725dc9c6b5db0d1134bc41ab6f862ef0a6336bc3c8019aecda269ce709" exitCode=0
Sep 15 22:42:22.232715 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:22.232619    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t"
Sep 15 22:42:22.232715 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:22.232660    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t" event={"ID":"f516cdea-ee92-43d0-8728-87fed4bc544a","Type":"ContainerDied","Data":"ed77c1725dc9c6b5db0d1134bc41ab6f862ef0a6336bc3c8019aecda269ce709"}
Sep 15 22:42:22.232783 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:22.232715    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t" event={"ID":"f516cdea-ee92-43d0-8728-87fed4bc544a","Type":"ContainerDied","Data":"ea17f9e6db0246c0346c09cb33747efd1f2857dfe16ae55daf6087a3a0c35ea0"}
Sep 15 22:42:22.232783 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:22.232741    2565 scope.go:117] "RemoveContainer" containerID="ed77c1725dc9c6b5db0d1134bc41ab6f862ef0a6336bc3c8019aecda269ce709"
Sep 15 22:42:22.242075 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:22.242058    2565 scope.go:117] "RemoveContainer" containerID="ed77c1725dc9c6b5db0d1134bc41ab6f862ef0a6336bc3c8019aecda269ce709"
Sep 15 22:42:22.242354 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:42:22.242335    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed77c1725dc9c6b5db0d1134bc41ab6f862ef0a6336bc3c8019aecda269ce709\": container with ID starting with ed77c1725dc9c6b5db0d1134bc41ab6f862ef0a6336bc3c8019aecda269ce709 not found: ID does not exist" containerID="ed77c1725dc9c6b5db0d1134bc41ab6f862ef0a6336bc3c8019aecda269ce709"
Sep 15 22:42:22.242448 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:22.242365    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed77c1725dc9c6b5db0d1134bc41ab6f862ef0a6336bc3c8019aecda269ce709"} err="failed to get container status \"ed77c1725dc9c6b5db0d1134bc41ab6f862ef0a6336bc3c8019aecda269ce709\": rpc error: code = NotFound desc = could not find container \"ed77c1725dc9c6b5db0d1134bc41ab6f862ef0a6336bc3c8019aecda269ce709\": container with ID starting with ed77c1725dc9c6b5db0d1134bc41ab6f862ef0a6336bc3c8019aecda269ce709 not found: ID does not exist"
Sep 15 22:42:22.253353 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:22.253265    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t"]
Sep 15 22:42:22.256743 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:22.256724    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-93550aef-68dc8b954f-bzj2t"]
Sep 15 22:42:22.591151 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:22.591123    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f516cdea-ee92-43d0-8728-87fed4bc544a" path="/var/lib/kubelet/pods/f516cdea-ee92-43d0-8728-87fed4bc544a/volumes"
Sep 15 22:42:31.035688 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.035656    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm"]
Sep 15 22:42:31.036357 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.036335    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f516cdea-ee92-43d0-8728-87fed4bc544a" containerName="maas-api"
Sep 15 22:42:31.036462 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.036359    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f516cdea-ee92-43d0-8728-87fed4bc544a" containerName="maas-api"
Sep 15 22:42:31.036521 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.036472    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f516cdea-ee92-43d0-8728-87fed4bc544a" containerName="maas-api"
Sep 15 22:42:31.041217 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.041197    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm"
Sep 15 22:42:31.043194 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.043168    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-2d1828d0-serving-cert\""
Sep 15 22:42:31.054007 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.053983    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm"]
Sep 15 22:42:31.174856 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.174821    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/343536b9-0d5e-48e7-bac0-160c01578579-metrics-certs\") pod \"maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm\" (UID: \"343536b9-0d5e-48e7-bac0-160c01578579\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm"
Sep 15 22:42:31.175019 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.174883    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/343536b9-0d5e-48e7-bac0-160c01578579-openshift-service-ca\") pod \"maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm\" (UID: \"343536b9-0d5e-48e7-bac0-160c01578579\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm"
Sep 15 22:42:31.175019 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.174952    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2s56k\" (UniqueName: \"kubernetes.io/projected/343536b9-0d5e-48e7-bac0-160c01578579-kube-api-access-2s56k\") pod \"maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm\" (UID: \"343536b9-0d5e-48e7-bac0-160c01578579\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm"
Sep 15 22:42:31.175019 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.174991    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/343536b9-0d5e-48e7-bac0-160c01578579-maas-api-tls\") pod \"maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm\" (UID: \"343536b9-0d5e-48e7-bac0-160c01578579\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm"
Sep 15 22:42:31.275861 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.275829    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2s56k\" (UniqueName: \"kubernetes.io/projected/343536b9-0d5e-48e7-bac0-160c01578579-kube-api-access-2s56k\") pod \"maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm\" (UID: \"343536b9-0d5e-48e7-bac0-160c01578579\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm"
Sep 15 22:42:31.276077 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.275868    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/343536b9-0d5e-48e7-bac0-160c01578579-maas-api-tls\") pod \"maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm\" (UID: \"343536b9-0d5e-48e7-bac0-160c01578579\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm"
Sep 15 22:42:31.276077 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.275933    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/343536b9-0d5e-48e7-bac0-160c01578579-metrics-certs\") pod \"maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm\" (UID: \"343536b9-0d5e-48e7-bac0-160c01578579\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm"
Sep 15 22:42:31.276077 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.275979    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/343536b9-0d5e-48e7-bac0-160c01578579-openshift-service-ca\") pod \"maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm\" (UID: \"343536b9-0d5e-48e7-bac0-160c01578579\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm"
Sep 15 22:42:31.276256 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:42:31.276094    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-2d1828d0-serving-cert: secret "maas-api-e2e-derive-2d1828d0-serving-cert" not found
Sep 15 22:42:31.276256 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:42:31.276178    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/343536b9-0d5e-48e7-bac0-160c01578579-maas-api-tls podName:343536b9-0d5e-48e7-bac0-160c01578579 nodeName:}" failed. No retries permitted until 2026-09-15 22:42:31.77615932 +0000 UTC m=+901.635870589 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/343536b9-0d5e-48e7-bac0-160c01578579-maas-api-tls") pod "maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm" (UID: "343536b9-0d5e-48e7-bac0-160c01578579") : secret "maas-api-e2e-derive-2d1828d0-serving-cert" not found
Sep 15 22:42:31.276747 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.276729    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/343536b9-0d5e-48e7-bac0-160c01578579-openshift-service-ca\") pod \"maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm\" (UID: \"343536b9-0d5e-48e7-bac0-160c01578579\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm"
Sep 15 22:42:31.278323 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.278303    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/343536b9-0d5e-48e7-bac0-160c01578579-metrics-certs\") pod \"maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm\" (UID: \"343536b9-0d5e-48e7-bac0-160c01578579\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm"
Sep 15 22:42:31.284759 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.284736    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2s56k\" (UniqueName: \"kubernetes.io/projected/343536b9-0d5e-48e7-bac0-160c01578579-kube-api-access-2s56k\") pod \"maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm\" (UID: \"343536b9-0d5e-48e7-bac0-160c01578579\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm"
Sep 15 22:42:31.681345 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.681274    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2d1828d0-f9969ddf4-rdltm"]
Sep 15 22:42:31.685002 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.684986    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-2d1828d0-f9969ddf4-rdltm"
Sep 15 22:42:31.687162 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.687135    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-2d1828d0\""
Sep 15 22:42:31.687295 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.687166    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-2d1828d0-dockercfg-wz8fg\""
Sep 15 22:42:31.693244 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.693222    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2d1828d0-7644bcc9df-qv7cp"]
Sep 15 22:42:31.696712 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.696693    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-2d1828d0-7644bcc9df-qv7cp"
Sep 15 22:42:31.705260 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.705235    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2d1828d0-f9969ddf4-rdltm"]
Sep 15 22:42:31.721320 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.721296    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2d1828d0-7644bcc9df-qv7cp"]
Sep 15 22:42:31.781324 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.781296    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/343536b9-0d5e-48e7-bac0-160c01578579-maas-api-tls\") pod \"maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm\" (UID: \"343536b9-0d5e-48e7-bac0-160c01578579\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm"
Sep 15 22:42:31.781470 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.781344    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf3c9028-4caa-441f-a47b-b7effd50bc33-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-2d1828d0-f9969ddf4-rdltm\" (UID: \"cf3c9028-4caa-441f-a47b-b7effd50bc33\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2d1828d0-f9969ddf4-rdltm"
Sep 15 22:42:31.781470 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.781367    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfln5\" (UniqueName: \"kubernetes.io/projected/cf3c9028-4caa-441f-a47b-b7effd50bc33-kube-api-access-wfln5\") pod \"payload-pre-processing-e2e-derive-2d1828d0-f9969ddf4-rdltm\" (UID: \"cf3c9028-4caa-441f-a47b-b7effd50bc33\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2d1828d0-f9969ddf4-rdltm"
Sep 15 22:42:31.783682 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.783662    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/343536b9-0d5e-48e7-bac0-160c01578579-maas-api-tls\") pod \"maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm\" (UID: \"343536b9-0d5e-48e7-bac0-160c01578579\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm"
Sep 15 22:42:31.882717 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.882685    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e496a66d-e62c-4a86-809f-7f1a67541c2b-plugins-config-volume\") pod \"payload-processing-e2e-derive-2d1828d0-7644bcc9df-qv7cp\" (UID: \"e496a66d-e62c-4a86-809f-7f1a67541c2b\") " pod="openshift-ingress/payload-processing-e2e-derive-2d1828d0-7644bcc9df-qv7cp"
Sep 15 22:42:31.882884 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.882762    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf3c9028-4caa-441f-a47b-b7effd50bc33-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-2d1828d0-f9969ddf4-rdltm\" (UID: \"cf3c9028-4caa-441f-a47b-b7effd50bc33\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2d1828d0-f9969ddf4-rdltm"
Sep 15 22:42:31.882884 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.882782    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfln5\" (UniqueName: \"kubernetes.io/projected/cf3c9028-4caa-441f-a47b-b7effd50bc33-kube-api-access-wfln5\") pod \"payload-pre-processing-e2e-derive-2d1828d0-f9969ddf4-rdltm\" (UID: \"cf3c9028-4caa-441f-a47b-b7effd50bc33\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2d1828d0-f9969ddf4-rdltm"
Sep 15 22:42:31.882884 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.882831    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xwls\" (UniqueName: \"kubernetes.io/projected/e496a66d-e62c-4a86-809f-7f1a67541c2b-kube-api-access-6xwls\") pod \"payload-processing-e2e-derive-2d1828d0-7644bcc9df-qv7cp\" (UID: \"e496a66d-e62c-4a86-809f-7f1a67541c2b\") " pod="openshift-ingress/payload-processing-e2e-derive-2d1828d0-7644bcc9df-qv7cp"
Sep 15 22:42:31.883467 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.883446    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf3c9028-4caa-441f-a47b-b7effd50bc33-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-2d1828d0-f9969ddf4-rdltm\" (UID: \"cf3c9028-4caa-441f-a47b-b7effd50bc33\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2d1828d0-f9969ddf4-rdltm"
Sep 15 22:42:31.890701 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.890683    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfln5\" (UniqueName: \"kubernetes.io/projected/cf3c9028-4caa-441f-a47b-b7effd50bc33-kube-api-access-wfln5\") pod \"payload-pre-processing-e2e-derive-2d1828d0-f9969ddf4-rdltm\" (UID: \"cf3c9028-4caa-441f-a47b-b7effd50bc33\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2d1828d0-f9969ddf4-rdltm"
Sep 15 22:42:31.954910 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.954850    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm"
Sep 15 22:42:31.983761 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.983732    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xwls\" (UniqueName: \"kubernetes.io/projected/e496a66d-e62c-4a86-809f-7f1a67541c2b-kube-api-access-6xwls\") pod \"payload-processing-e2e-derive-2d1828d0-7644bcc9df-qv7cp\" (UID: \"e496a66d-e62c-4a86-809f-7f1a67541c2b\") " pod="openshift-ingress/payload-processing-e2e-derive-2d1828d0-7644bcc9df-qv7cp"
Sep 15 22:42:31.983878 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.983779    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e496a66d-e62c-4a86-809f-7f1a67541c2b-plugins-config-volume\") pod \"payload-processing-e2e-derive-2d1828d0-7644bcc9df-qv7cp\" (UID: \"e496a66d-e62c-4a86-809f-7f1a67541c2b\") " pod="openshift-ingress/payload-processing-e2e-derive-2d1828d0-7644bcc9df-qv7cp"
Sep 15 22:42:31.984466 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.984443    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e496a66d-e62c-4a86-809f-7f1a67541c2b-plugins-config-volume\") pod \"payload-processing-e2e-derive-2d1828d0-7644bcc9df-qv7cp\" (UID: \"e496a66d-e62c-4a86-809f-7f1a67541c2b\") " pod="openshift-ingress/payload-processing-e2e-derive-2d1828d0-7644bcc9df-qv7cp"
Sep 15 22:42:31.991308 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.991287    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xwls\" (UniqueName: \"kubernetes.io/projected/e496a66d-e62c-4a86-809f-7f1a67541c2b-kube-api-access-6xwls\") pod \"payload-processing-e2e-derive-2d1828d0-7644bcc9df-qv7cp\" (UID: \"e496a66d-e62c-4a86-809f-7f1a67541c2b\") " pod="openshift-ingress/payload-processing-e2e-derive-2d1828d0-7644bcc9df-qv7cp"
Sep 15 22:42:31.996144 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:31.996124    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-2d1828d0-f9969ddf4-rdltm"
Sep 15 22:42:32.008705 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:32.008087    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-2d1828d0-7644bcc9df-qv7cp"
Sep 15 22:42:32.231995 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:32.231969    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm"]
Sep 15 22:42:32.232883 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:42:32.232859    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod343536b9_0d5e_48e7_bac0_160c01578579.slice/crio-3c694acc02a12810fa1de346d61cd9ea8e07d649d5b1d17d13566db5f9b73884 WatchSource:0}: Error finding container 3c694acc02a12810fa1de346d61cd9ea8e07d649d5b1d17d13566db5f9b73884: Status 404 returned error can't find the container with id 3c694acc02a12810fa1de346d61cd9ea8e07d649d5b1d17d13566db5f9b73884
Sep 15 22:42:32.294124 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:32.293658    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2d1828d0-f9969ddf4-rdltm"]
Sep 15 22:42:32.295521 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:42:32.295496    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf3c9028_4caa_441f_a47b_b7effd50bc33.slice/crio-eb7ddf5da499fe88075212979385601c79e968997fd28ccfd68513880518b7cb WatchSource:0}: Error finding container eb7ddf5da499fe88075212979385601c79e968997fd28ccfd68513880518b7cb: Status 404 returned error can't find the container with id eb7ddf5da499fe88075212979385601c79e968997fd28ccfd68513880518b7cb
Sep 15 22:42:32.297379 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:32.297319    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm" event={"ID":"343536b9-0d5e-48e7-bac0-160c01578579","Type":"ContainerStarted","Data":"3c694acc02a12810fa1de346d61cd9ea8e07d649d5b1d17d13566db5f9b73884"}
Sep 15 22:42:32.297847 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:32.297823    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2d1828d0-7644bcc9df-qv7cp"]
Sep 15 22:42:32.297925 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:42:32.297838    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode496a66d_e62c_4a86_809f_7f1a67541c2b.slice/crio-3d509ea41524f1d358148bf9132ed1683404ae99ccb4b664e8b8998d1c57c78f WatchSource:0}: Error finding container 3d509ea41524f1d358148bf9132ed1683404ae99ccb4b664e8b8998d1c57c78f: Status 404 returned error can't find the container with id 3d509ea41524f1d358148bf9132ed1683404ae99ccb4b664e8b8998d1c57c78f
Sep 15 22:42:33.306593 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:33.306554    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-2d1828d0-f9969ddf4-rdltm" event={"ID":"cf3c9028-4caa-441f-a47b-b7effd50bc33","Type":"ContainerStarted","Data":"02c4100f8debd242e96e14344f53f1d57fafe2ff55c32333b5903e0d6a8a37e8"}
Sep 15 22:42:33.307078 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:33.306624    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-2d1828d0-f9969ddf4-rdltm" event={"ID":"cf3c9028-4caa-441f-a47b-b7effd50bc33","Type":"ContainerStarted","Data":"eb7ddf5da499fe88075212979385601c79e968997fd28ccfd68513880518b7cb"}
Sep 15 22:42:33.307078 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:33.306816    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-2d1828d0-f9969ddf4-rdltm"
Sep 15 22:42:33.308569 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:33.308539    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-2d1828d0-7644bcc9df-qv7cp" event={"ID":"e496a66d-e62c-4a86-809f-7f1a67541c2b","Type":"ContainerStarted","Data":"76d1cc662255aeeaf26f8078d45dbd4e14cd1a38e253bf5ad617e505340c8b2c"}
Sep 15 22:42:33.308717 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:33.308590    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-2d1828d0-7644bcc9df-qv7cp" event={"ID":"e496a66d-e62c-4a86-809f-7f1a67541c2b","Type":"ContainerStarted","Data":"3d509ea41524f1d358148bf9132ed1683404ae99ccb4b664e8b8998d1c57c78f"}
Sep 15 22:42:33.308717 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:33.308621    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-2d1828d0-7644bcc9df-qv7cp"
Sep 15 22:42:33.310706 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:33.310686    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm" event={"ID":"343536b9-0d5e-48e7-bac0-160c01578579","Type":"ContainerStarted","Data":"ba64e8a3ee9bb35f6eece470483fea4a2120acb495650bed8692a0fa28ef93ac"}
Sep 15 22:42:33.310818 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:33.310802    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm"
Sep 15 22:42:33.323992 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:33.323945    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-2d1828d0-f9969ddf4-rdltm" podStartSLOduration=2.323931989 podStartE2EDuration="2.323931989s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:42:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:42:33.321654099 +0000 UTC m=+903.181365383" watchObservedRunningTime="2026-09-15 22:42:33.323931989 +0000 UTC m=+903.183643273"
Sep 15 22:42:33.338982 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:33.338939    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm" podStartSLOduration=1.992047346 podStartE2EDuration="2.338926037s" totalImagesPullingTime="346.878691ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:42:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:42:33.336113577 +0000 UTC m=+903.195824863" watchObservedRunningTime="2026-09-15 22:42:33.338926037 +0000 UTC m=+903.198637352"
Sep 15 22:42:33.350639 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:33.350608    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-2d1828d0-7644bcc9df-qv7cp" podStartSLOduration=2.350597382 podStartE2EDuration="2.350597382s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:42:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:42:33.350440896 +0000 UTC m=+903.210152182" watchObservedRunningTime="2026-09-15 22:42:33.350597382 +0000 UTC m=+903.210308662"
Sep 15 22:42:39.323654 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:39.323628    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm"
Sep 15 22:42:44.321093 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:44.321050    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-2d1828d0-7644bcc9df-qv7cp"
Sep 15 22:42:44.321547 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:42:44.321144    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-2d1828d0-f9969ddf4-rdltm"
Sep 15 22:43:08.153258 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.153217    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm"]
Sep 15 22:43:08.155977 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.153480    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm" podUID="343536b9-0d5e-48e7-bac0-160c01578579" containerName="maas-api" containerID="cri-o://ba64e8a3ee9bb35f6eece470483fea4a2120acb495650bed8692a0fa28ef93ac" gracePeriod=30
Sep 15 22:43:08.199210 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.199180    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2d1828d0-7644bcc9df-qv7cp"]
Sep 15 22:43:08.199491 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.199447    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-2d1828d0-7644bcc9df-qv7cp" podUID="e496a66d-e62c-4a86-809f-7f1a67541c2b" containerName="payload-processing" containerID="cri-o://76d1cc662255aeeaf26f8078d45dbd4e14cd1a38e253bf5ad617e505340c8b2c" gracePeriod=30
Sep 15 22:43:08.210993 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.210964    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2d1828d0-f9969ddf4-rdltm"]
Sep 15 22:43:08.211238 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.211211    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-2d1828d0-f9969ddf4-rdltm" podUID="cf3c9028-4caa-441f-a47b-b7effd50bc33" containerName="payload-pre-processing" containerID="cri-o://02c4100f8debd242e96e14344f53f1d57fafe2ff55c32333b5903e0d6a8a37e8" gracePeriod=30
Sep 15 22:43:08.477750 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.477719    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-2d1828d0-7644bcc9df-qv7cp"
Sep 15 22:43:08.538016 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.537906    2565 generic.go:356] "Generic (PLEG): container finished" podID="cf3c9028-4caa-441f-a47b-b7effd50bc33" containerID="02c4100f8debd242e96e14344f53f1d57fafe2ff55c32333b5903e0d6a8a37e8" exitCode=0
Sep 15 22:43:08.538338 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.538264    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-2d1828d0-f9969ddf4-rdltm" event={"ID":"cf3c9028-4caa-441f-a47b-b7effd50bc33","Type":"ContainerDied","Data":"02c4100f8debd242e96e14344f53f1d57fafe2ff55c32333b5903e0d6a8a37e8"}
Sep 15 22:43:08.540838 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.540147    2565 generic.go:356] "Generic (PLEG): container finished" podID="e496a66d-e62c-4a86-809f-7f1a67541c2b" containerID="76d1cc662255aeeaf26f8078d45dbd4e14cd1a38e253bf5ad617e505340c8b2c" exitCode=0
Sep 15 22:43:08.540838 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.540281    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-2d1828d0-7644bcc9df-qv7cp" event={"ID":"e496a66d-e62c-4a86-809f-7f1a67541c2b","Type":"ContainerDied","Data":"76d1cc662255aeeaf26f8078d45dbd4e14cd1a38e253bf5ad617e505340c8b2c"}
Sep 15 22:43:08.540838 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.540321    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-2d1828d0-7644bcc9df-qv7cp" event={"ID":"e496a66d-e62c-4a86-809f-7f1a67541c2b","Type":"ContainerDied","Data":"3d509ea41524f1d358148bf9132ed1683404ae99ccb4b664e8b8998d1c57c78f"}
Sep 15 22:43:08.540838 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.540352    2565 scope.go:117] "RemoveContainer" containerID="76d1cc662255aeeaf26f8078d45dbd4e14cd1a38e253bf5ad617e505340c8b2c"
Sep 15 22:43:08.540838 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.540498    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-2d1828d0-7644bcc9df-qv7cp"
Sep 15 22:43:08.541335 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.541317    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm"
Sep 15 22:43:08.547007 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.545817    2565 generic.go:356] "Generic (PLEG): container finished" podID="343536b9-0d5e-48e7-bac0-160c01578579" containerID="ba64e8a3ee9bb35f6eece470483fea4a2120acb495650bed8692a0fa28ef93ac" exitCode=0
Sep 15 22:43:08.547007 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.545887    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm" event={"ID":"343536b9-0d5e-48e7-bac0-160c01578579","Type":"ContainerDied","Data":"ba64e8a3ee9bb35f6eece470483fea4a2120acb495650bed8692a0fa28ef93ac"}
Sep 15 22:43:08.547007 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.545953    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm" event={"ID":"343536b9-0d5e-48e7-bac0-160c01578579","Type":"ContainerDied","Data":"3c694acc02a12810fa1de346d61cd9ea8e07d649d5b1d17d13566db5f9b73884"}
Sep 15 22:43:08.556639 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.556618    2565 scope.go:117] "RemoveContainer" containerID="76d1cc662255aeeaf26f8078d45dbd4e14cd1a38e253bf5ad617e505340c8b2c"
Sep 15 22:43:08.556882 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:43:08.556862    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76d1cc662255aeeaf26f8078d45dbd4e14cd1a38e253bf5ad617e505340c8b2c\": container with ID starting with 76d1cc662255aeeaf26f8078d45dbd4e14cd1a38e253bf5ad617e505340c8b2c not found: ID does not exist" containerID="76d1cc662255aeeaf26f8078d45dbd4e14cd1a38e253bf5ad617e505340c8b2c"
Sep 15 22:43:08.556956 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.556889    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76d1cc662255aeeaf26f8078d45dbd4e14cd1a38e253bf5ad617e505340c8b2c"} err="failed to get container status \"76d1cc662255aeeaf26f8078d45dbd4e14cd1a38e253bf5ad617e505340c8b2c\": rpc error: code = NotFound desc = could not find container \"76d1cc662255aeeaf26f8078d45dbd4e14cd1a38e253bf5ad617e505340c8b2c\": container with ID starting with 76d1cc662255aeeaf26f8078d45dbd4e14cd1a38e253bf5ad617e505340c8b2c not found: ID does not exist"
Sep 15 22:43:08.556956 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.556905    2565 scope.go:117] "RemoveContainer" containerID="ba64e8a3ee9bb35f6eece470483fea4a2120acb495650bed8692a0fa28ef93ac"
Sep 15 22:43:08.559765 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.559743    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-2d1828d0-f9969ddf4-rdltm"
Sep 15 22:43:08.565321 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.565306    2565 scope.go:117] "RemoveContainer" containerID="ba64e8a3ee9bb35f6eece470483fea4a2120acb495650bed8692a0fa28ef93ac"
Sep 15 22:43:08.565579 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:43:08.565562    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba64e8a3ee9bb35f6eece470483fea4a2120acb495650bed8692a0fa28ef93ac\": container with ID starting with ba64e8a3ee9bb35f6eece470483fea4a2120acb495650bed8692a0fa28ef93ac not found: ID does not exist" containerID="ba64e8a3ee9bb35f6eece470483fea4a2120acb495650bed8692a0fa28ef93ac"
Sep 15 22:43:08.565643 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.565585    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba64e8a3ee9bb35f6eece470483fea4a2120acb495650bed8692a0fa28ef93ac"} err="failed to get container status \"ba64e8a3ee9bb35f6eece470483fea4a2120acb495650bed8692a0fa28ef93ac\": rpc error: code = NotFound desc = could not find container \"ba64e8a3ee9bb35f6eece470483fea4a2120acb495650bed8692a0fa28ef93ac\": container with ID starting with ba64e8a3ee9bb35f6eece470483fea4a2120acb495650bed8692a0fa28ef93ac not found: ID does not exist"
Sep 15 22:43:08.631508 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.631481    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e496a66d-e62c-4a86-809f-7f1a67541c2b-plugins-config-volume\") pod \"e496a66d-e62c-4a86-809f-7f1a67541c2b\" (UID: \"e496a66d-e62c-4a86-809f-7f1a67541c2b\") "
Sep 15 22:43:08.631644 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.631538    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6xwls\" (UniqueName: \"kubernetes.io/projected/e496a66d-e62c-4a86-809f-7f1a67541c2b-kube-api-access-6xwls\") pod \"e496a66d-e62c-4a86-809f-7f1a67541c2b\" (UID: \"e496a66d-e62c-4a86-809f-7f1a67541c2b\") "
Sep 15 22:43:08.631644 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.631587    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/343536b9-0d5e-48e7-bac0-160c01578579-openshift-service-ca\") pod \"343536b9-0d5e-48e7-bac0-160c01578579\" (UID: \"343536b9-0d5e-48e7-bac0-160c01578579\") "
Sep 15 22:43:08.631756 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.631648    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wfln5\" (UniqueName: \"kubernetes.io/projected/cf3c9028-4caa-441f-a47b-b7effd50bc33-kube-api-access-wfln5\") pod \"cf3c9028-4caa-441f-a47b-b7effd50bc33\" (UID: \"cf3c9028-4caa-441f-a47b-b7effd50bc33\") "
Sep 15 22:43:08.631812 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.631770    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/343536b9-0d5e-48e7-bac0-160c01578579-metrics-certs\") pod \"343536b9-0d5e-48e7-bac0-160c01578579\" (UID: \"343536b9-0d5e-48e7-bac0-160c01578579\") "
Sep 15 22:43:08.631812 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.631807    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf3c9028-4caa-441f-a47b-b7effd50bc33-plugins-config-volume\") pod \"cf3c9028-4caa-441f-a47b-b7effd50bc33\" (UID: \"cf3c9028-4caa-441f-a47b-b7effd50bc33\") "
Sep 15 22:43:08.631924 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.631829    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2s56k\" (UniqueName: \"kubernetes.io/projected/343536b9-0d5e-48e7-bac0-160c01578579-kube-api-access-2s56k\") pod \"343536b9-0d5e-48e7-bac0-160c01578579\" (UID: \"343536b9-0d5e-48e7-bac0-160c01578579\") "
Sep 15 22:43:08.631924 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.631856    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/343536b9-0d5e-48e7-bac0-160c01578579-maas-api-tls\") pod \"343536b9-0d5e-48e7-bac0-160c01578579\" (UID: \"343536b9-0d5e-48e7-bac0-160c01578579\") "
Sep 15 22:43:08.631924 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.631884    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e496a66d-e62c-4a86-809f-7f1a67541c2b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e496a66d-e62c-4a86-809f-7f1a67541c2b" (UID: "e496a66d-e62c-4a86-809f-7f1a67541c2b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:43:08.632113 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.631937    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/343536b9-0d5e-48e7-bac0-160c01578579-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "343536b9-0d5e-48e7-bac0-160c01578579" (UID: "343536b9-0d5e-48e7-bac0-160c01578579"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:43:08.632244 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.632220    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e496a66d-e62c-4a86-809f-7f1a67541c2b-plugins-config-volume\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:43:08.632396 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.632253    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/343536b9-0d5e-48e7-bac0-160c01578579-openshift-service-ca\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:43:08.632396 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.632217    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf3c9028-4caa-441f-a47b-b7effd50bc33-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cf3c9028-4caa-441f-a47b-b7effd50bc33" (UID: "cf3c9028-4caa-441f-a47b-b7effd50bc33"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:43:08.634064 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.634007    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/343536b9-0d5e-48e7-bac0-160c01578579-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "343536b9-0d5e-48e7-bac0-160c01578579" (UID: "343536b9-0d5e-48e7-bac0-160c01578579"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:43:08.634157 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.634037    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/343536b9-0d5e-48e7-bac0-160c01578579-kube-api-access-2s56k" (OuterVolumeSpecName: "kube-api-access-2s56k") pod "343536b9-0d5e-48e7-bac0-160c01578579" (UID: "343536b9-0d5e-48e7-bac0-160c01578579"). InnerVolumeSpecName "kube-api-access-2s56k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:43:08.634157 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.634138    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e496a66d-e62c-4a86-809f-7f1a67541c2b-kube-api-access-6xwls" (OuterVolumeSpecName: "kube-api-access-6xwls") pod "e496a66d-e62c-4a86-809f-7f1a67541c2b" (UID: "e496a66d-e62c-4a86-809f-7f1a67541c2b"). InnerVolumeSpecName "kube-api-access-6xwls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:43:08.634300 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.634276    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/343536b9-0d5e-48e7-bac0-160c01578579-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "343536b9-0d5e-48e7-bac0-160c01578579" (UID: "343536b9-0d5e-48e7-bac0-160c01578579"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:43:08.634387 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.634327    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf3c9028-4caa-441f-a47b-b7effd50bc33-kube-api-access-wfln5" (OuterVolumeSpecName: "kube-api-access-wfln5") pod "cf3c9028-4caa-441f-a47b-b7effd50bc33" (UID: "cf3c9028-4caa-441f-a47b-b7effd50bc33"). InnerVolumeSpecName "kube-api-access-wfln5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:43:08.733508 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.733454    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfln5\" (UniqueName: \"kubernetes.io/projected/cf3c9028-4caa-441f-a47b-b7effd50bc33-kube-api-access-wfln5\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:43:08.733508 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.733474    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/343536b9-0d5e-48e7-bac0-160c01578579-metrics-certs\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:43:08.733508 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.733485    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf3c9028-4caa-441f-a47b-b7effd50bc33-plugins-config-volume\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:43:08.733508 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.733494    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2s56k\" (UniqueName: \"kubernetes.io/projected/343536b9-0d5e-48e7-bac0-160c01578579-kube-api-access-2s56k\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:43:08.733508 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.733503    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/343536b9-0d5e-48e7-bac0-160c01578579-maas-api-tls\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:43:08.733508 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.733511    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xwls\" (UniqueName: \"kubernetes.io/projected/e496a66d-e62c-4a86-809f-7f1a67541c2b-kube-api-access-6xwls\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:43:08.862748 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.862721    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2d1828d0-7644bcc9df-qv7cp"]
Sep 15 22:43:08.866935 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:08.866913    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2d1828d0-7644bcc9df-qv7cp"]
Sep 15 22:43:09.553067 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:09.553011    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-2d1828d0-f9969ddf4-rdltm"
Sep 15 22:43:09.553067 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:09.553050    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-2d1828d0-f9969ddf4-rdltm" event={"ID":"cf3c9028-4caa-441f-a47b-b7effd50bc33","Type":"ContainerDied","Data":"eb7ddf5da499fe88075212979385601c79e968997fd28ccfd68513880518b7cb"}
Sep 15 22:43:09.553615 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:09.553115    2565 scope.go:117] "RemoveContainer" containerID="02c4100f8debd242e96e14344f53f1d57fafe2ff55c32333b5903e0d6a8a37e8"
Sep 15 22:43:09.555331 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:09.555306    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm"
Sep 15 22:43:09.575896 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:09.575876    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2d1828d0-f9969ddf4-rdltm"]
Sep 15 22:43:09.579593 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:09.579575    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2d1828d0-f9969ddf4-rdltm"]
Sep 15 22:43:09.601940 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:09.601919    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm"]
Sep 15 22:43:09.609140 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:09.609112    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2d1828d0-76bbf5fdcc-mxgcm"]
Sep 15 22:43:10.591070 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:10.591036    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="343536b9-0d5e-48e7-bac0-160c01578579" path="/var/lib/kubelet/pods/343536b9-0d5e-48e7-bac0-160c01578579/volumes"
Sep 15 22:43:10.591478 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:10.591463    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf3c9028-4caa-441f-a47b-b7effd50bc33" path="/var/lib/kubelet/pods/cf3c9028-4caa-441f-a47b-b7effd50bc33/volumes"
Sep 15 22:43:10.591832 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:10.591819    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e496a66d-e62c-4a86-809f-7f1a67541c2b" path="/var/lib/kubelet/pods/e496a66d-e62c-4a86-809f-7f1a67541c2b/volumes"
Sep 15 22:43:13.133262 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.133219    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg"]
Sep 15 22:43:13.133716 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.133488    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg" podUID="885fa4e1-068e-4306-b350-ed16cc000595" containerName="maas-api" containerID="cri-o://47c2b2428278c08435902b28599052d91bea296eeb4b85bd31e653485705b432" gracePeriod=30
Sep 15 22:43:13.233950 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.233912    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-b08870-659658b6b6-hjhhz"]
Sep 15 22:43:13.234219 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.234186    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-b08870-659658b6b6-hjhhz" podUID="2032f779-1d90-415a-b785-f9662989cf34" containerName="payload-pre-processing" containerID="cri-o://c9c6ba3301e642cecdff98fddf2854772e5d30effa52543a47adb5c2debf43d4" gracePeriod=30
Sep 15 22:43:13.399346 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.399282    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg"
Sep 15 22:43:13.491730 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.491709    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-b08870-659658b6b6-hjhhz"
Sep 15 22:43:13.575257 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.575224    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/885fa4e1-068e-4306-b350-ed16cc000595-metrics-certs\") pod \"885fa4e1-068e-4306-b350-ed16cc000595\" (UID: \"885fa4e1-068e-4306-b350-ed16cc000595\") "
Sep 15 22:43:13.575443 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.575323    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2032f779-1d90-415a-b785-f9662989cf34-plugins-config-volume\") pod \"2032f779-1d90-415a-b785-f9662989cf34\" (UID: \"2032f779-1d90-415a-b785-f9662989cf34\") "
Sep 15 22:43:13.575443 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.575397    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/885fa4e1-068e-4306-b350-ed16cc000595-openshift-service-ca\") pod \"885fa4e1-068e-4306-b350-ed16cc000595\" (UID: \"885fa4e1-068e-4306-b350-ed16cc000595\") "
Sep 15 22:43:13.575567 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.575456    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8vrpx\" (UniqueName: \"kubernetes.io/projected/885fa4e1-068e-4306-b350-ed16cc000595-kube-api-access-8vrpx\") pod \"885fa4e1-068e-4306-b350-ed16cc000595\" (UID: \"885fa4e1-068e-4306-b350-ed16cc000595\") "
Sep 15 22:43:13.575626 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.575570    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dv4qq\" (UniqueName: \"kubernetes.io/projected/2032f779-1d90-415a-b785-f9662989cf34-kube-api-access-dv4qq\") pod \"2032f779-1d90-415a-b785-f9662989cf34\" (UID: \"2032f779-1d90-415a-b785-f9662989cf34\") "
Sep 15 22:43:13.575721 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.575700    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/885fa4e1-068e-4306-b350-ed16cc000595-maas-api-tls\") pod \"885fa4e1-068e-4306-b350-ed16cc000595\" (UID: \"885fa4e1-068e-4306-b350-ed16cc000595\") "
Sep 15 22:43:13.575797 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.575724    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/885fa4e1-068e-4306-b350-ed16cc000595-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "885fa4e1-068e-4306-b350-ed16cc000595" (UID: "885fa4e1-068e-4306-b350-ed16cc000595"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:43:13.575797 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.575743    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2032f779-1d90-415a-b785-f9662989cf34-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2032f779-1d90-415a-b785-f9662989cf34" (UID: "2032f779-1d90-415a-b785-f9662989cf34"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:43:13.576148 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.576126    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2032f779-1d90-415a-b785-f9662989cf34-plugins-config-volume\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:43:13.576321 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.576299    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/885fa4e1-068e-4306-b350-ed16cc000595-openshift-service-ca\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:43:13.577702 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.577676    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2032f779-1d90-415a-b785-f9662989cf34-kube-api-access-dv4qq" (OuterVolumeSpecName: "kube-api-access-dv4qq") pod "2032f779-1d90-415a-b785-f9662989cf34" (UID: "2032f779-1d90-415a-b785-f9662989cf34"). InnerVolumeSpecName "kube-api-access-dv4qq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:43:13.577796 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.577766    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/885fa4e1-068e-4306-b350-ed16cc000595-kube-api-access-8vrpx" (OuterVolumeSpecName: "kube-api-access-8vrpx") pod "885fa4e1-068e-4306-b350-ed16cc000595" (UID: "885fa4e1-068e-4306-b350-ed16cc000595"). InnerVolumeSpecName "kube-api-access-8vrpx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:43:13.577935 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.577912    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/885fa4e1-068e-4306-b350-ed16cc000595-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "885fa4e1-068e-4306-b350-ed16cc000595" (UID: "885fa4e1-068e-4306-b350-ed16cc000595"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:43:13.578073 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.578057    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/885fa4e1-068e-4306-b350-ed16cc000595-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "885fa4e1-068e-4306-b350-ed16cc000595" (UID: "885fa4e1-068e-4306-b350-ed16cc000595"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:43:13.582727 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.582702    2565 generic.go:356] "Generic (PLEG): container finished" podID="885fa4e1-068e-4306-b350-ed16cc000595" containerID="47c2b2428278c08435902b28599052d91bea296eeb4b85bd31e653485705b432" exitCode=0
Sep 15 22:43:13.582832 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.582745    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg"
Sep 15 22:43:13.582832 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.582783    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg" event={"ID":"885fa4e1-068e-4306-b350-ed16cc000595","Type":"ContainerDied","Data":"47c2b2428278c08435902b28599052d91bea296eeb4b85bd31e653485705b432"}
Sep 15 22:43:13.582923 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.582837    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg" event={"ID":"885fa4e1-068e-4306-b350-ed16cc000595","Type":"ContainerDied","Data":"868f92e4fab8058974ee469801558cb641435eebd15a2df9f52728bbd36d82d1"}
Sep 15 22:43:13.582923 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.582867    2565 scope.go:117] "RemoveContainer" containerID="47c2b2428278c08435902b28599052d91bea296eeb4b85bd31e653485705b432"
Sep 15 22:43:13.584239 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.584219    2565 generic.go:356] "Generic (PLEG): container finished" podID="2032f779-1d90-415a-b785-f9662989cf34" containerID="c9c6ba3301e642cecdff98fddf2854772e5d30effa52543a47adb5c2debf43d4" exitCode=0
Sep 15 22:43:13.584338 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.584268    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-b08870-659658b6b6-hjhhz"
Sep 15 22:43:13.584338 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.584274    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-b08870-659658b6b6-hjhhz" event={"ID":"2032f779-1d90-415a-b785-f9662989cf34","Type":"ContainerDied","Data":"c9c6ba3301e642cecdff98fddf2854772e5d30effa52543a47adb5c2debf43d4"}
Sep 15 22:43:13.584338 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.584309    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-b08870-659658b6b6-hjhhz" event={"ID":"2032f779-1d90-415a-b785-f9662989cf34","Type":"ContainerDied","Data":"07d9276c1ec48e3b17aeac910fa0557ecac09acb3e43db3a21d4029ae0ba93f9"}
Sep 15 22:43:13.591903 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.591888    2565 scope.go:117] "RemoveContainer" containerID="47c2b2428278c08435902b28599052d91bea296eeb4b85bd31e653485705b432"
Sep 15 22:43:13.592204 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:43:13.592176    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47c2b2428278c08435902b28599052d91bea296eeb4b85bd31e653485705b432\": container with ID starting with 47c2b2428278c08435902b28599052d91bea296eeb4b85bd31e653485705b432 not found: ID does not exist" containerID="47c2b2428278c08435902b28599052d91bea296eeb4b85bd31e653485705b432"
Sep 15 22:43:13.592297 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.592213    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47c2b2428278c08435902b28599052d91bea296eeb4b85bd31e653485705b432"} err="failed to get container status \"47c2b2428278c08435902b28599052d91bea296eeb4b85bd31e653485705b432\": rpc error: code = NotFound desc = could not find container \"47c2b2428278c08435902b28599052d91bea296eeb4b85bd31e653485705b432\": container with ID starting with 47c2b2428278c08435902b28599052d91bea296eeb4b85bd31e653485705b432 not found: ID does not exist"
Sep 15 22:43:13.592297 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.592235    2565 scope.go:117] "RemoveContainer" containerID="c9c6ba3301e642cecdff98fddf2854772e5d30effa52543a47adb5c2debf43d4"
Sep 15 22:43:13.600675 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.600658    2565 scope.go:117] "RemoveContainer" containerID="c9c6ba3301e642cecdff98fddf2854772e5d30effa52543a47adb5c2debf43d4"
Sep 15 22:43:13.600916 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:43:13.600901    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9c6ba3301e642cecdff98fddf2854772e5d30effa52543a47adb5c2debf43d4\": container with ID starting with c9c6ba3301e642cecdff98fddf2854772e5d30effa52543a47adb5c2debf43d4 not found: ID does not exist" containerID="c9c6ba3301e642cecdff98fddf2854772e5d30effa52543a47adb5c2debf43d4"
Sep 15 22:43:13.600962 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.600921    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9c6ba3301e642cecdff98fddf2854772e5d30effa52543a47adb5c2debf43d4"} err="failed to get container status \"c9c6ba3301e642cecdff98fddf2854772e5d30effa52543a47adb5c2debf43d4\": rpc error: code = NotFound desc = could not find container \"c9c6ba3301e642cecdff98fddf2854772e5d30effa52543a47adb5c2debf43d4\": container with ID starting with c9c6ba3301e642cecdff98fddf2854772e5d30effa52543a47adb5c2debf43d4 not found: ID does not exist"
Sep 15 22:43:13.610226 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.610201    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg"]
Sep 15 22:43:13.616111 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.616089    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-b08870-869cd8f5b5-jvjsg"]
Sep 15 22:43:13.623272 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.623248    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-b08870-659658b6b6-hjhhz"]
Sep 15 22:43:13.627750 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.627727    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-b08870-659658b6b6-hjhhz"]
Sep 15 22:43:13.677646 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.677596    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vrpx\" (UniqueName: \"kubernetes.io/projected/885fa4e1-068e-4306-b350-ed16cc000595-kube-api-access-8vrpx\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:43:13.677646 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.677618    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dv4qq\" (UniqueName: \"kubernetes.io/projected/2032f779-1d90-415a-b785-f9662989cf34-kube-api-access-dv4qq\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:43:13.677646 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.677628    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/885fa4e1-068e-4306-b350-ed16cc000595-maas-api-tls\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:43:13.677646 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:13.677636    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/885fa4e1-068e-4306-b350-ed16cc000595-metrics-certs\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:43:14.592269 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:14.592232    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2032f779-1d90-415a-b785-f9662989cf34" path="/var/lib/kubelet/pods/2032f779-1d90-415a-b785-f9662989cf34/volumes"
Sep 15 22:43:14.592807 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:43:14.592785    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="885fa4e1-068e-4306-b350-ed16cc000595" path="/var/lib/kubelet/pods/885fa4e1-068e-4306-b350-ed16cc000595/volumes"
Sep 15 22:44:03.336254 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.336223    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4"]
Sep 15 22:44:03.336708 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.336648    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2032f779-1d90-415a-b785-f9662989cf34" containerName="payload-pre-processing"
Sep 15 22:44:03.336708 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.336661    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2032f779-1d90-415a-b785-f9662989cf34" containerName="payload-pre-processing"
Sep 15 22:44:03.336708 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.336673    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="343536b9-0d5e-48e7-bac0-160c01578579" containerName="maas-api"
Sep 15 22:44:03.336708 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.336679    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="343536b9-0d5e-48e7-bac0-160c01578579" containerName="maas-api"
Sep 15 22:44:03.336708 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.336689    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="885fa4e1-068e-4306-b350-ed16cc000595" containerName="maas-api"
Sep 15 22:44:03.336708 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.336695    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="885fa4e1-068e-4306-b350-ed16cc000595" containerName="maas-api"
Sep 15 22:44:03.336928 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.336713    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e496a66d-e62c-4a86-809f-7f1a67541c2b" containerName="payload-processing"
Sep 15 22:44:03.336928 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.336719    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e496a66d-e62c-4a86-809f-7f1a67541c2b" containerName="payload-processing"
Sep 15 22:44:03.336928 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.336727    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf3c9028-4caa-441f-a47b-b7effd50bc33" containerName="payload-pre-processing"
Sep 15 22:44:03.336928 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.336732    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf3c9028-4caa-441f-a47b-b7effd50bc33" containerName="payload-pre-processing"
Sep 15 22:44:03.336928 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.336791    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2032f779-1d90-415a-b785-f9662989cf34" containerName="payload-pre-processing"
Sep 15 22:44:03.336928 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.336798    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="343536b9-0d5e-48e7-bac0-160c01578579" containerName="maas-api"
Sep 15 22:44:03.336928 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.336804    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="885fa4e1-068e-4306-b350-ed16cc000595" containerName="maas-api"
Sep 15 22:44:03.336928 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.336812    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e496a66d-e62c-4a86-809f-7f1a67541c2b" containerName="payload-processing"
Sep 15 22:44:03.336928 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.336820    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf3c9028-4caa-441f-a47b-b7effd50bc33" containerName="payload-pre-processing"
Sep 15 22:44:03.340005 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.339983    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4"
Sep 15 22:44:03.341885 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.341865    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-4ccd57-serving-cert\""
Sep 15 22:44:03.350571 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.350545    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4"]
Sep 15 22:44:03.445003 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.444969    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8498a6c8-a46e-4189-95a3-c611467f5fc4-maas-api-tls\") pod \"maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4\" (UID: \"8498a6c8-a46e-4189-95a3-c611467f5fc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4"
Sep 15 22:44:03.445203 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.445015    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8498a6c8-a46e-4189-95a3-c611467f5fc4-openshift-service-ca\") pod \"maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4\" (UID: \"8498a6c8-a46e-4189-95a3-c611467f5fc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4"
Sep 15 22:44:03.445203 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.445095    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8498a6c8-a46e-4189-95a3-c611467f5fc4-metrics-certs\") pod \"maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4\" (UID: \"8498a6c8-a46e-4189-95a3-c611467f5fc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4"
Sep 15 22:44:03.445203 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.445112    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzpx9\" (UniqueName: \"kubernetes.io/projected/8498a6c8-a46e-4189-95a3-c611467f5fc4-kube-api-access-mzpx9\") pod \"maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4\" (UID: \"8498a6c8-a46e-4189-95a3-c611467f5fc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4"
Sep 15 22:44:03.546504 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.546471    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8498a6c8-a46e-4189-95a3-c611467f5fc4-maas-api-tls\") pod \"maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4\" (UID: \"8498a6c8-a46e-4189-95a3-c611467f5fc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4"
Sep 15 22:44:03.546718 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.546517    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8498a6c8-a46e-4189-95a3-c611467f5fc4-openshift-service-ca\") pod \"maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4\" (UID: \"8498a6c8-a46e-4189-95a3-c611467f5fc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4"
Sep 15 22:44:03.546718 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.546563    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8498a6c8-a46e-4189-95a3-c611467f5fc4-metrics-certs\") pod \"maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4\" (UID: \"8498a6c8-a46e-4189-95a3-c611467f5fc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4"
Sep 15 22:44:03.546718 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.546590    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzpx9\" (UniqueName: \"kubernetes.io/projected/8498a6c8-a46e-4189-95a3-c611467f5fc4-kube-api-access-mzpx9\") pod \"maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4\" (UID: \"8498a6c8-a46e-4189-95a3-c611467f5fc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4"
Sep 15 22:44:03.547369 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.547345    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8498a6c8-a46e-4189-95a3-c611467f5fc4-openshift-service-ca\") pod \"maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4\" (UID: \"8498a6c8-a46e-4189-95a3-c611467f5fc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4"
Sep 15 22:44:03.549140 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.549113    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8498a6c8-a46e-4189-95a3-c611467f5fc4-metrics-certs\") pod \"maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4\" (UID: \"8498a6c8-a46e-4189-95a3-c611467f5fc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4"
Sep 15 22:44:03.549250 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.549144    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8498a6c8-a46e-4189-95a3-c611467f5fc4-maas-api-tls\") pod \"maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4\" (UID: \"8498a6c8-a46e-4189-95a3-c611467f5fc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4"
Sep 15 22:44:03.554688 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.554664    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzpx9\" (UniqueName: \"kubernetes.io/projected/8498a6c8-a46e-4189-95a3-c611467f5fc4-kube-api-access-mzpx9\") pod \"maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4\" (UID: \"8498a6c8-a46e-4189-95a3-c611467f5fc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4"
Sep 15 22:44:03.653278 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.653191    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4"
Sep 15 22:44:03.937698 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.937667    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4"]
Sep 15 22:44:03.938628 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:44:03.938594    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8498a6c8_a46e_4189_95a3_c611467f5fc4.slice/crio-4edc28e5952db34c1cc102ef65dc4bafcdfada2c4317ed6f7565286672e4882a WatchSource:0}: Error finding container 4edc28e5952db34c1cc102ef65dc4bafcdfada2c4317ed6f7565286672e4882a: Status 404 returned error can't find the container with id 4edc28e5952db34c1cc102ef65dc4bafcdfada2c4317ed6f7565286672e4882a
Sep 15 22:44:03.940599 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.940553    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 22:44:03.943686 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.943662    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-4ccd57-5bd46c7d8b-wh5c6"]
Sep 15 22:44:03.948777 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.948756    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-4ccd57-5bd46c7d8b-wh5c6"
Sep 15 22:44:03.950918 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.950898    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-4ccd57\""
Sep 15 22:44:03.951055 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.950946    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-4ccd57-dockercfg-6sjtm\""
Sep 15 22:44:03.958873 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.958849    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-4ccd57-5bd46c7d8b-wh5c6"]
Sep 15 22:44:03.973655 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.973625    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-4ccd57-546878d877-92d5p"]
Sep 15 22:44:03.977213 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.977196    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-4ccd57-546878d877-92d5p"
Sep 15 22:44:03.987898 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:03.987876    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-4ccd57-546878d877-92d5p"]
Sep 15 22:44:04.051462 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:04.051424    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drbdf\" (UniqueName: \"kubernetes.io/projected/0156bbf9-4aee-4630-87aa-ab4b26fa5795-kube-api-access-drbdf\") pod \"payload-pre-processing-e2e-explicit-4ccd57-5bd46c7d8b-wh5c6\" (UID: \"0156bbf9-4aee-4630-87aa-ab4b26fa5795\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-4ccd57-5bd46c7d8b-wh5c6"
Sep 15 22:44:04.051655 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:04.051496    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0156bbf9-4aee-4630-87aa-ab4b26fa5795-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-4ccd57-5bd46c7d8b-wh5c6\" (UID: \"0156bbf9-4aee-4630-87aa-ab4b26fa5795\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-4ccd57-5bd46c7d8b-wh5c6"
Sep 15 22:44:04.152833 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:04.152786    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drbdf\" (UniqueName: \"kubernetes.io/projected/0156bbf9-4aee-4630-87aa-ab4b26fa5795-kube-api-access-drbdf\") pod \"payload-pre-processing-e2e-explicit-4ccd57-5bd46c7d8b-wh5c6\" (UID: \"0156bbf9-4aee-4630-87aa-ab4b26fa5795\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-4ccd57-5bd46c7d8b-wh5c6"
Sep 15 22:44:04.153008 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:04.152841    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s95gn\" (UniqueName: \"kubernetes.io/projected/d08e86c0-a1d6-440b-9731-22ffc0608aa8-kube-api-access-s95gn\") pod \"payload-processing-e2e-explicit-4ccd57-546878d877-92d5p\" (UID: \"d08e86c0-a1d6-440b-9731-22ffc0608aa8\") " pod="openshift-ingress/payload-processing-e2e-explicit-4ccd57-546878d877-92d5p"
Sep 15 22:44:04.153008 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:04.152870    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d08e86c0-a1d6-440b-9731-22ffc0608aa8-plugins-config-volume\") pod \"payload-processing-e2e-explicit-4ccd57-546878d877-92d5p\" (UID: \"d08e86c0-a1d6-440b-9731-22ffc0608aa8\") " pod="openshift-ingress/payload-processing-e2e-explicit-4ccd57-546878d877-92d5p"
Sep 15 22:44:04.153008 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:04.152912    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0156bbf9-4aee-4630-87aa-ab4b26fa5795-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-4ccd57-5bd46c7d8b-wh5c6\" (UID: \"0156bbf9-4aee-4630-87aa-ab4b26fa5795\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-4ccd57-5bd46c7d8b-wh5c6"
Sep 15 22:44:04.153591 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:04.153565    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0156bbf9-4aee-4630-87aa-ab4b26fa5795-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-4ccd57-5bd46c7d8b-wh5c6\" (UID: \"0156bbf9-4aee-4630-87aa-ab4b26fa5795\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-4ccd57-5bd46c7d8b-wh5c6"
Sep 15 22:44:04.160567 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:04.160543    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drbdf\" (UniqueName: \"kubernetes.io/projected/0156bbf9-4aee-4630-87aa-ab4b26fa5795-kube-api-access-drbdf\") pod \"payload-pre-processing-e2e-explicit-4ccd57-5bd46c7d8b-wh5c6\" (UID: \"0156bbf9-4aee-4630-87aa-ab4b26fa5795\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-4ccd57-5bd46c7d8b-wh5c6"
Sep 15 22:44:04.253787 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:04.253705    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s95gn\" (UniqueName: \"kubernetes.io/projected/d08e86c0-a1d6-440b-9731-22ffc0608aa8-kube-api-access-s95gn\") pod \"payload-processing-e2e-explicit-4ccd57-546878d877-92d5p\" (UID: \"d08e86c0-a1d6-440b-9731-22ffc0608aa8\") " pod="openshift-ingress/payload-processing-e2e-explicit-4ccd57-546878d877-92d5p"
Sep 15 22:44:04.253787 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:04.253749    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d08e86c0-a1d6-440b-9731-22ffc0608aa8-plugins-config-volume\") pod \"payload-processing-e2e-explicit-4ccd57-546878d877-92d5p\" (UID: \"d08e86c0-a1d6-440b-9731-22ffc0608aa8\") " pod="openshift-ingress/payload-processing-e2e-explicit-4ccd57-546878d877-92d5p"
Sep 15 22:44:04.254465 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:04.254445    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d08e86c0-a1d6-440b-9731-22ffc0608aa8-plugins-config-volume\") pod \"payload-processing-e2e-explicit-4ccd57-546878d877-92d5p\" (UID: \"d08e86c0-a1d6-440b-9731-22ffc0608aa8\") " pod="openshift-ingress/payload-processing-e2e-explicit-4ccd57-546878d877-92d5p"
Sep 15 22:44:04.260621 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:04.260603    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s95gn\" (UniqueName: \"kubernetes.io/projected/d08e86c0-a1d6-440b-9731-22ffc0608aa8-kube-api-access-s95gn\") pod \"payload-processing-e2e-explicit-4ccd57-546878d877-92d5p\" (UID: \"d08e86c0-a1d6-440b-9731-22ffc0608aa8\") " pod="openshift-ingress/payload-processing-e2e-explicit-4ccd57-546878d877-92d5p"
Sep 15 22:44:04.261418 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:04.261388    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-4ccd57-5bd46c7d8b-wh5c6"
Sep 15 22:44:04.289466 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:04.289438    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-4ccd57-546878d877-92d5p"
Sep 15 22:44:04.538332 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:04.538304    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-4ccd57-5bd46c7d8b-wh5c6"]
Sep 15 22:44:04.539205 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:44:04.539174    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0156bbf9_4aee_4630_87aa_ab4b26fa5795.slice/crio-69d0d48d5a696b632357f8b1adcb143fa07821d12275d347535cc503f3cd2653 WatchSource:0}: Error finding container 69d0d48d5a696b632357f8b1adcb143fa07821d12275d347535cc503f3cd2653: Status 404 returned error can't find the container with id 69d0d48d5a696b632357f8b1adcb143fa07821d12275d347535cc503f3cd2653
Sep 15 22:44:04.571880 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:04.571857    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-4ccd57-546878d877-92d5p"]
Sep 15 22:44:04.574063 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:44:04.574020    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd08e86c0_a1d6_440b_9731_22ffc0608aa8.slice/crio-f16a1c9cf530d850cbedb30afd5051224ade7cf089ed4e89c754b2a76fd4cfa5 WatchSource:0}: Error finding container f16a1c9cf530d850cbedb30afd5051224ade7cf089ed4e89c754b2a76fd4cfa5: Status 404 returned error can't find the container with id f16a1c9cf530d850cbedb30afd5051224ade7cf089ed4e89c754b2a76fd4cfa5
Sep 15 22:44:04.892338 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:04.892245    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-4ccd57-546878d877-92d5p" event={"ID":"d08e86c0-a1d6-440b-9731-22ffc0608aa8","Type":"ContainerStarted","Data":"eb849bbfd35bae3c719aec252d19311fc334d307ba2f55d27d610f362565248e"}
Sep 15 22:44:04.892338 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:04.892308    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-4ccd57-546878d877-92d5p" event={"ID":"d08e86c0-a1d6-440b-9731-22ffc0608aa8","Type":"ContainerStarted","Data":"f16a1c9cf530d850cbedb30afd5051224ade7cf089ed4e89c754b2a76fd4cfa5"}
Sep 15 22:44:04.892569 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:04.892341    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-4ccd57-546878d877-92d5p"
Sep 15 22:44:04.893867 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:04.893842    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4" event={"ID":"8498a6c8-a46e-4189-95a3-c611467f5fc4","Type":"ContainerStarted","Data":"97f45e6d668bfceab373cba2afad4db00425fa1338199f773dc4fd17c47c75cf"}
Sep 15 22:44:04.894005 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:04.893889    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4" event={"ID":"8498a6c8-a46e-4189-95a3-c611467f5fc4","Type":"ContainerStarted","Data":"4edc28e5952db34c1cc102ef65dc4bafcdfada2c4317ed6f7565286672e4882a"}
Sep 15 22:44:04.894005 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:04.893919    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4"
Sep 15 22:44:04.895607 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:04.895583    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-4ccd57-5bd46c7d8b-wh5c6" event={"ID":"0156bbf9-4aee-4630-87aa-ab4b26fa5795","Type":"ContainerStarted","Data":"3909abe39b0c2ede6618d330e46b2520a53b7e09249859e91849b53aad40296a"}
Sep 15 22:44:04.895692 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:04.895624    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-4ccd57-5bd46c7d8b-wh5c6" event={"ID":"0156bbf9-4aee-4630-87aa-ab4b26fa5795","Type":"ContainerStarted","Data":"69d0d48d5a696b632357f8b1adcb143fa07821d12275d347535cc503f3cd2653"}
Sep 15 22:44:04.895692 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:04.895643    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-4ccd57-5bd46c7d8b-wh5c6"
Sep 15 22:44:04.908512 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:04.908456    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-4ccd57-546878d877-92d5p" podStartSLOduration=1.90844105 podStartE2EDuration="1.90844105s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:44:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:44:04.907261684 +0000 UTC m=+994.766972980" watchObservedRunningTime="2026-09-15 22:44:04.90844105 +0000 UTC m=+994.768152334"
Sep 15 22:44:04.924273 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:04.924222    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4" podStartSLOduration=1.478802243 podStartE2EDuration="1.924202861s" totalImagesPullingTime="445.400618ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:44:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:44:04.922403082 +0000 UTC m=+994.782114366" watchObservedRunningTime="2026-09-15 22:44:04.924202861 +0000 UTC m=+994.783914145"
Sep 15 22:44:04.940074 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:04.940006    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-4ccd57-5bd46c7d8b-wh5c6" podStartSLOduration=1.939991749 podStartE2EDuration="1.939991749s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:44:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:44:04.938879719 +0000 UTC m=+994.798591264" watchObservedRunningTime="2026-09-15 22:44:04.939991749 +0000 UTC m=+994.799703050"
Sep 15 22:44:10.909131 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:10.909098    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4"
Sep 15 22:44:15.906231 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:15.906199    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-4ccd57-5bd46c7d8b-wh5c6"
Sep 15 22:44:15.906704 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:15.906667    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-4ccd57-546878d877-92d5p"
Sep 15 22:44:53.843766 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:53.843729    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2bbfe736-657cfb7c54-hdnv8"]
Sep 15 22:44:53.848616 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:53.848598    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2bbfe736-657cfb7c54-hdnv8"
Sep 15 22:44:53.850642 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:53.850620    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-2bbfe736\""
Sep 15 22:44:53.857256 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:53.857235    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2bbfe736-657cfb7c54-hdnv8"]
Sep 15 22:44:53.950533 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:53.950499    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fg9rj\" (UniqueName: \"kubernetes.io/projected/65c9e7e8-3b4c-44c0-ba58-9fd6bb9693b0-kube-api-access-fg9rj\") pod \"payload-pre-processing-e2e-mt-2bbfe736-657cfb7c54-hdnv8\" (UID: \"65c9e7e8-3b4c-44c0-ba58-9fd6bb9693b0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2bbfe736-657cfb7c54-hdnv8"
Sep 15 22:44:53.950716 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:53.950545    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65c9e7e8-3b4c-44c0-ba58-9fd6bb9693b0-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-2bbfe736-657cfb7c54-hdnv8\" (UID: \"65c9e7e8-3b4c-44c0-ba58-9fd6bb9693b0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2bbfe736-657cfb7c54-hdnv8"
Sep 15 22:44:54.051236 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:54.051200    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fg9rj\" (UniqueName: \"kubernetes.io/projected/65c9e7e8-3b4c-44c0-ba58-9fd6bb9693b0-kube-api-access-fg9rj\") pod \"payload-pre-processing-e2e-mt-2bbfe736-657cfb7c54-hdnv8\" (UID: \"65c9e7e8-3b4c-44c0-ba58-9fd6bb9693b0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2bbfe736-657cfb7c54-hdnv8"
Sep 15 22:44:54.051415 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:54.051249    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65c9e7e8-3b4c-44c0-ba58-9fd6bb9693b0-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-2bbfe736-657cfb7c54-hdnv8\" (UID: \"65c9e7e8-3b4c-44c0-ba58-9fd6bb9693b0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2bbfe736-657cfb7c54-hdnv8"
Sep 15 22:44:54.051950 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:54.051930    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65c9e7e8-3b4c-44c0-ba58-9fd6bb9693b0-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-2bbfe736-657cfb7c54-hdnv8\" (UID: \"65c9e7e8-3b4c-44c0-ba58-9fd6bb9693b0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2bbfe736-657cfb7c54-hdnv8"
Sep 15 22:44:54.058968 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:54.058936    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fg9rj\" (UniqueName: \"kubernetes.io/projected/65c9e7e8-3b4c-44c0-ba58-9fd6bb9693b0-kube-api-access-fg9rj\") pod \"payload-pre-processing-e2e-mt-2bbfe736-657cfb7c54-hdnv8\" (UID: \"65c9e7e8-3b4c-44c0-ba58-9fd6bb9693b0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2bbfe736-657cfb7c54-hdnv8"
Sep 15 22:44:54.159766 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:54.159682    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2bbfe736-657cfb7c54-hdnv8"
Sep 15 22:44:54.434956 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:54.434925    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2bbfe736-657cfb7c54-hdnv8"]
Sep 15 22:44:54.436386 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:44:54.436360    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65c9e7e8_3b4c_44c0_ba58_9fd6bb9693b0.slice/crio-e21e1e9e7b9cb5224fd6758e8a934e8ecbaec39dea66c36b5bbfc8ab1c137da1 WatchSource:0}: Error finding container e21e1e9e7b9cb5224fd6758e8a934e8ecbaec39dea66c36b5bbfc8ab1c137da1: Status 404 returned error can't find the container with id e21e1e9e7b9cb5224fd6758e8a934e8ecbaec39dea66c36b5bbfc8ab1c137da1
Sep 15 22:44:55.208048 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:55.207983    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2bbfe736-657cfb7c54-hdnv8" event={"ID":"65c9e7e8-3b4c-44c0-ba58-9fd6bb9693b0","Type":"ContainerStarted","Data":"6d2558b11651fd5ab55fe74155a757e50dae03e3e3381774e49e9c1b42ebcb7c"}
Sep 15 22:44:55.208048 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:55.208051    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2bbfe736-657cfb7c54-hdnv8" event={"ID":"65c9e7e8-3b4c-44c0-ba58-9fd6bb9693b0","Type":"ContainerStarted","Data":"e21e1e9e7b9cb5224fd6758e8a934e8ecbaec39dea66c36b5bbfc8ab1c137da1"}
Sep 15 22:44:55.208599 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:55.208082    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-2bbfe736-657cfb7c54-hdnv8"
Sep 15 22:44:55.225589 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:44:55.225537    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-2bbfe736-657cfb7c54-hdnv8" podStartSLOduration=2.225519156 podStartE2EDuration="2.225519156s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:44:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:44:55.222570726 +0000 UTC m=+1045.082282010" watchObservedRunningTime="2026-09-15 22:44:55.225519156 +0000 UTC m=+1045.085230475"
Sep 15 22:45:00.163367 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:45:00.163337    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-765e6d-29825205-jlgnj"]
Sep 15 22:45:00.167117 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:45:00.167097    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-765e6d-29825205-jlgnj"
Sep 15 22:45:00.169182 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:45:00.169155    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hnj7x\""
Sep 15 22:45:00.177294 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:45:00.177267    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-765e6d-29825205-jlgnj"]
Sep 15 22:45:00.179635 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:45:00.179613    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-765e6d-29825205-jlgnj"
Sep 15 22:45:00.461572 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:45:00.461539    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-765e6d-29825205-jlgnj"]
Sep 15 22:45:00.463630 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:45:00.463604    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50725131_d1e7_4fd4_aaed_4f2933313bd7.slice/crio-900b868b4755dc068a55a1368b7460532dca75f6e5d1aa33903f607235962e18 WatchSource:0}: Error finding container 900b868b4755dc068a55a1368b7460532dca75f6e5d1aa33903f607235962e18: Status 404 returned error can't find the container with id 900b868b4755dc068a55a1368b7460532dca75f6e5d1aa33903f607235962e18
Sep 15 22:45:01.248290 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:45:01.248257    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-765e6d-29825205-jlgnj" event={"ID":"50725131-d1e7-4fd4-aaed-4f2933313bd7","Type":"ContainerStarted","Data":"900b868b4755dc068a55a1368b7460532dca75f6e5d1aa33903f607235962e18"}
Sep 15 22:45:03.263007 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:45:03.262963    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-765e6d-29825205-jlgnj" event={"ID":"50725131-d1e7-4fd4-aaed-4f2933313bd7","Type":"ContainerStarted","Data":"b36d44a130ce4f0c2fc04fa2a4c17e40007f3a401986fdd8de19078361111868"}
Sep 15 22:45:03.279546 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:45:03.279497    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-765e6d-29825205-jlgnj" podStartSLOduration=1.049951783 podStartE2EDuration="3.279482104s" totalImagesPullingTime="2.229530321s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:45:03.278180713 +0000 UTC m=+1053.137891997" watchObservedRunningTime="2026-09-15 22:45:03.279482104 +0000 UTC m=+1053.139193387"
Sep 15 22:45:04.272063 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:45:04.271997    2565 generic.go:356] "Generic (PLEG): container finished" podID="50725131-d1e7-4fd4-aaed-4f2933313bd7" containerID="b36d44a130ce4f0c2fc04fa2a4c17e40007f3a401986fdd8de19078361111868" exitCode=0
Sep 15 22:45:04.272469 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:45:04.272085    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-765e6d-29825205-jlgnj" event={"ID":"50725131-d1e7-4fd4-aaed-4f2933313bd7","Type":"ContainerDied","Data":"b36d44a130ce4f0c2fc04fa2a4c17e40007f3a401986fdd8de19078361111868"}
Sep 15 22:45:05.421870 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:45:05.421845    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-765e6d-29825205-jlgnj"
Sep 15 22:45:06.217607 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:45:06.217573    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-2bbfe736-657cfb7c54-hdnv8"
Sep 15 22:45:06.284809 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:45:06.284783    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-765e6d-29825205-jlgnj"
Sep 15 22:45:06.284967 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:45:06.284822    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-765e6d-29825205-jlgnj" event={"ID":"50725131-d1e7-4fd4-aaed-4f2933313bd7","Type":"ContainerDied","Data":"900b868b4755dc068a55a1368b7460532dca75f6e5d1aa33903f607235962e18"}
Sep 15 22:45:06.284967 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:45:06.284878    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="900b868b4755dc068a55a1368b7460532dca75f6e5d1aa33903f607235962e18"
Sep 15 22:45:37.039148 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:45:37.039063    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2bbfe736-657cfb7c54-hdnv8"]
Sep 15 22:45:37.039749 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:45:37.039362    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-2bbfe736-657cfb7c54-hdnv8" podUID="65c9e7e8-3b4c-44c0-ba58-9fd6bb9693b0" containerName="payload-pre-processing" containerID="cri-o://6d2558b11651fd5ab55fe74155a757e50dae03e3e3381774e49e9c1b42ebcb7c" gracePeriod=30
Sep 15 22:45:37.296780 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:45:37.296726    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2bbfe736-657cfb7c54-hdnv8"
Sep 15 22:45:37.385520 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:45:37.385486    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fg9rj\" (UniqueName: \"kubernetes.io/projected/65c9e7e8-3b4c-44c0-ba58-9fd6bb9693b0-kube-api-access-fg9rj\") pod \"65c9e7e8-3b4c-44c0-ba58-9fd6bb9693b0\" (UID: \"65c9e7e8-3b4c-44c0-ba58-9fd6bb9693b0\") "
Sep 15 22:45:37.385520 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:45:37.385525    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65c9e7e8-3b4c-44c0-ba58-9fd6bb9693b0-plugins-config-volume\") pod \"65c9e7e8-3b4c-44c0-ba58-9fd6bb9693b0\" (UID: \"65c9e7e8-3b4c-44c0-ba58-9fd6bb9693b0\") "
Sep 15 22:45:37.385921 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:45:37.385898    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65c9e7e8-3b4c-44c0-ba58-9fd6bb9693b0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "65c9e7e8-3b4c-44c0-ba58-9fd6bb9693b0" (UID: "65c9e7e8-3b4c-44c0-ba58-9fd6bb9693b0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:45:37.387573 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:45:37.387547    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65c9e7e8-3b4c-44c0-ba58-9fd6bb9693b0-kube-api-access-fg9rj" (OuterVolumeSpecName: "kube-api-access-fg9rj") pod "65c9e7e8-3b4c-44c0-ba58-9fd6bb9693b0" (UID: "65c9e7e8-3b4c-44c0-ba58-9fd6bb9693b0"). InnerVolumeSpecName "kube-api-access-fg9rj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:45:37.472605 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:45:37.472564    2565 generic.go:356] "Generic (PLEG): container finished" podID="65c9e7e8-3b4c-44c0-ba58-9fd6bb9693b0" containerID="6d2558b11651fd5ab55fe74155a757e50dae03e3e3381774e49e9c1b42ebcb7c" exitCode=0
Sep 15 22:45:37.472764 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:45:37.472610    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2bbfe736-657cfb7c54-hdnv8"
Sep 15 22:45:37.472764 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:45:37.472652    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2bbfe736-657cfb7c54-hdnv8" event={"ID":"65c9e7e8-3b4c-44c0-ba58-9fd6bb9693b0","Type":"ContainerDied","Data":"6d2558b11651fd5ab55fe74155a757e50dae03e3e3381774e49e9c1b42ebcb7c"}
Sep 15 22:45:37.472764 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:45:37.472710    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2bbfe736-657cfb7c54-hdnv8" event={"ID":"65c9e7e8-3b4c-44c0-ba58-9fd6bb9693b0","Type":"ContainerDied","Data":"e21e1e9e7b9cb5224fd6758e8a934e8ecbaec39dea66c36b5bbfc8ab1c137da1"}
Sep 15 22:45:37.472764 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:45:37.472736    2565 scope.go:117] "RemoveContainer" containerID="6d2558b11651fd5ab55fe74155a757e50dae03e3e3381774e49e9c1b42ebcb7c"
Sep 15 22:45:37.482564 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:45:37.482546    2565 scope.go:117] "RemoveContainer" containerID="6d2558b11651fd5ab55fe74155a757e50dae03e3e3381774e49e9c1b42ebcb7c"
Sep 15 22:45:37.482800 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:45:37.482782    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d2558b11651fd5ab55fe74155a757e50dae03e3e3381774e49e9c1b42ebcb7c\": container with ID starting with 6d2558b11651fd5ab55fe74155a757e50dae03e3e3381774e49e9c1b42ebcb7c not found: ID does not exist" containerID="6d2558b11651fd5ab55fe74155a757e50dae03e3e3381774e49e9c1b42ebcb7c"
Sep 15 22:45:37.482847 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:45:37.482809    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d2558b11651fd5ab55fe74155a757e50dae03e3e3381774e49e9c1b42ebcb7c"} err="failed to get container status \"6d2558b11651fd5ab55fe74155a757e50dae03e3e3381774e49e9c1b42ebcb7c\": rpc error: code = NotFound desc = could not find container \"6d2558b11651fd5ab55fe74155a757e50dae03e3e3381774e49e9c1b42ebcb7c\": container with ID starting with 6d2558b11651fd5ab55fe74155a757e50dae03e3e3381774e49e9c1b42ebcb7c not found: ID does not exist"
Sep 15 22:45:37.486472 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:45:37.486453    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fg9rj\" (UniqueName: \"kubernetes.io/projected/65c9e7e8-3b4c-44c0-ba58-9fd6bb9693b0-kube-api-access-fg9rj\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:45:37.486529 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:45:37.486501    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65c9e7e8-3b4c-44c0-ba58-9fd6bb9693b0-plugins-config-volume\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:45:37.507669 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:45:37.507642    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2bbfe736-657cfb7c54-hdnv8"]
Sep 15 22:45:37.513889 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:45:37.513870    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2bbfe736-657cfb7c54-hdnv8"]
Sep 15 22:45:38.590719 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:45:38.590686    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65c9e7e8-3b4c-44c0-ba58-9fd6bb9693b0" path="/var/lib/kubelet/pods/65c9e7e8-3b4c-44c0-ba58-9fd6bb9693b0/volumes"
Sep 15 22:46:04.920174 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:04.920132    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f"]
Sep 15 22:46:04.920651 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:04.920609    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65c9e7e8-3b4c-44c0-ba58-9fd6bb9693b0" containerName="payload-pre-processing"
Sep 15 22:46:04.920651 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:04.920622    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="65c9e7e8-3b4c-44c0-ba58-9fd6bb9693b0" containerName="payload-pre-processing"
Sep 15 22:46:04.920651 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:04.920630    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50725131-d1e7-4fd4-aaed-4f2933313bd7" containerName="cleanup"
Sep 15 22:46:04.920651 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:04.920636    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="50725131-d1e7-4fd4-aaed-4f2933313bd7" containerName="cleanup"
Sep 15 22:46:04.920887 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:04.920703    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="50725131-d1e7-4fd4-aaed-4f2933313bd7" containerName="cleanup"
Sep 15 22:46:04.920887 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:04.920716    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="65c9e7e8-3b4c-44c0-ba58-9fd6bb9693b0" containerName="payload-pre-processing"
Sep 15 22:46:04.924247 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:04.924226    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f"
Sep 15 22:46:04.926240 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:04.926217    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-f2241f-serving-cert\""
Sep 15 22:46:04.934060 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:04.934000    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f"]
Sep 15 22:46:05.048280 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:05.048246    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7n5r\" (UniqueName: \"kubernetes.io/projected/7092845b-136b-4c47-9d1a-706ea4dbad85-kube-api-access-z7n5r\") pod \"maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f\" (UID: \"7092845b-136b-4c47-9d1a-706ea4dbad85\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f"
Sep 15 22:46:05.048458 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:05.048364    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7092845b-136b-4c47-9d1a-706ea4dbad85-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f\" (UID: \"7092845b-136b-4c47-9d1a-706ea4dbad85\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f"
Sep 15 22:46:05.048458 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:05.048435    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7092845b-136b-4c47-9d1a-706ea4dbad85-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f\" (UID: \"7092845b-136b-4c47-9d1a-706ea4dbad85\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f"
Sep 15 22:46:05.048550 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:05.048469    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7092845b-136b-4c47-9d1a-706ea4dbad85-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f\" (UID: \"7092845b-136b-4c47-9d1a-706ea4dbad85\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f"
Sep 15 22:46:05.149689 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:05.149659    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7092845b-136b-4c47-9d1a-706ea4dbad85-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f\" (UID: \"7092845b-136b-4c47-9d1a-706ea4dbad85\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f"
Sep 15 22:46:05.149689 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:05.149693    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7092845b-136b-4c47-9d1a-706ea4dbad85-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f\" (UID: \"7092845b-136b-4c47-9d1a-706ea4dbad85\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f"
Sep 15 22:46:05.149883 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:05.149718    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7n5r\" (UniqueName: \"kubernetes.io/projected/7092845b-136b-4c47-9d1a-706ea4dbad85-kube-api-access-z7n5r\") pod \"maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f\" (UID: \"7092845b-136b-4c47-9d1a-706ea4dbad85\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f"
Sep 15 22:46:05.149883 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:05.149817    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7092845b-136b-4c47-9d1a-706ea4dbad85-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f\" (UID: \"7092845b-136b-4c47-9d1a-706ea4dbad85\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f"
Sep 15 22:46:05.150492 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:05.150467    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7092845b-136b-4c47-9d1a-706ea4dbad85-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f\" (UID: \"7092845b-136b-4c47-9d1a-706ea4dbad85\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f"
Sep 15 22:46:05.152053 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:05.152011    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7092845b-136b-4c47-9d1a-706ea4dbad85-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f\" (UID: \"7092845b-136b-4c47-9d1a-706ea4dbad85\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f"
Sep 15 22:46:05.152178 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:05.152160    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7092845b-136b-4c47-9d1a-706ea4dbad85-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f\" (UID: \"7092845b-136b-4c47-9d1a-706ea4dbad85\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f"
Sep 15 22:46:05.157544 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:05.157523    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7n5r\" (UniqueName: \"kubernetes.io/projected/7092845b-136b-4c47-9d1a-706ea4dbad85-kube-api-access-z7n5r\") pod \"maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f\" (UID: \"7092845b-136b-4c47-9d1a-706ea4dbad85\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f"
Sep 15 22:46:05.237072 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:05.236960    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f"
Sep 15 22:46:05.512895 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:05.512867    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f"]
Sep 15 22:46:05.514492 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:46:05.514463    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7092845b_136b_4c47_9d1a_706ea4dbad85.slice/crio-babc6bc467c8c5ccbe116b8cd96a43d9396cdf301dd0adc62b5cef9d83965a5e WatchSource:0}: Error finding container babc6bc467c8c5ccbe116b8cd96a43d9396cdf301dd0adc62b5cef9d83965a5e: Status 404 returned error can't find the container with id babc6bc467c8c5ccbe116b8cd96a43d9396cdf301dd0adc62b5cef9d83965a5e
Sep 15 22:46:05.553464 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:05.553432    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f2241f-586669d47d-lc898"]
Sep 15 22:46:05.558195 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:05.558178    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f2241f-586669d47d-lc898"
Sep 15 22:46:05.560491 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:05.560462    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-f2241f-dockercfg-dsvs6\""
Sep 15 22:46:05.560608 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:05.560490    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-f2241f\""
Sep 15 22:46:05.567853 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:05.567831    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f2241f-586669d47d-lc898"]
Sep 15 22:46:05.654348 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:05.654313    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjcnt\" (UniqueName: \"kubernetes.io/projected/c4d9b866-cf8b-49c6-9f4e-a46a4477af1e-kube-api-access-sjcnt\") pod \"payload-pre-processing-e2e-shared-a-w2-f2241f-586669d47d-lc898\" (UID: \"c4d9b866-cf8b-49c6-9f4e-a46a4477af1e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f2241f-586669d47d-lc898"
Sep 15 22:46:05.654348 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:05.654360    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4d9b866-cf8b-49c6-9f4e-a46a4477af1e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-f2241f-586669d47d-lc898\" (UID: \"c4d9b866-cf8b-49c6-9f4e-a46a4477af1e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f2241f-586669d47d-lc898"
Sep 15 22:46:05.662683 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:05.662653    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f" event={"ID":"7092845b-136b-4c47-9d1a-706ea4dbad85","Type":"ContainerStarted","Data":"babc6bc467c8c5ccbe116b8cd96a43d9396cdf301dd0adc62b5cef9d83965a5e"}
Sep 15 22:46:05.755652 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:05.755610    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjcnt\" (UniqueName: \"kubernetes.io/projected/c4d9b866-cf8b-49c6-9f4e-a46a4477af1e-kube-api-access-sjcnt\") pod \"payload-pre-processing-e2e-shared-a-w2-f2241f-586669d47d-lc898\" (UID: \"c4d9b866-cf8b-49c6-9f4e-a46a4477af1e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f2241f-586669d47d-lc898"
Sep 15 22:46:05.755839 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:05.755676    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4d9b866-cf8b-49c6-9f4e-a46a4477af1e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-f2241f-586669d47d-lc898\" (UID: \"c4d9b866-cf8b-49c6-9f4e-a46a4477af1e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f2241f-586669d47d-lc898"
Sep 15 22:46:05.756406 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:05.756385    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4d9b866-cf8b-49c6-9f4e-a46a4477af1e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-f2241f-586669d47d-lc898\" (UID: \"c4d9b866-cf8b-49c6-9f4e-a46a4477af1e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f2241f-586669d47d-lc898"
Sep 15 22:46:05.763508 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:05.763447    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjcnt\" (UniqueName: \"kubernetes.io/projected/c4d9b866-cf8b-49c6-9f4e-a46a4477af1e-kube-api-access-sjcnt\") pod \"payload-pre-processing-e2e-shared-a-w2-f2241f-586669d47d-lc898\" (UID: \"c4d9b866-cf8b-49c6-9f4e-a46a4477af1e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f2241f-586669d47d-lc898"
Sep 15 22:46:05.870155 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:05.870116    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f2241f-586669d47d-lc898"
Sep 15 22:46:06.145203 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:06.145171    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f2241f-586669d47d-lc898"]
Sep 15 22:46:06.146622 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:46:06.146596    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4d9b866_cf8b_49c6_9f4e_a46a4477af1e.slice/crio-c638efcc11e6d029524371af5160c4c1f0471a449c50fa058d78634d09e078dd WatchSource:0}: Error finding container c638efcc11e6d029524371af5160c4c1f0471a449c50fa058d78634d09e078dd: Status 404 returned error can't find the container with id c638efcc11e6d029524371af5160c4c1f0471a449c50fa058d78634d09e078dd
Sep 15 22:46:06.670667 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:06.670631    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f" event={"ID":"7092845b-136b-4c47-9d1a-706ea4dbad85","Type":"ContainerStarted","Data":"56c2cd5849168aa35e9f9a005ae4f431c938bf13fa4b52805d989bd301f95b42"}
Sep 15 22:46:06.670873 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:06.670718    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f"
Sep 15 22:46:06.672250 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:06.672224    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f2241f-586669d47d-lc898" event={"ID":"c4d9b866-cf8b-49c6-9f4e-a46a4477af1e","Type":"ContainerStarted","Data":"9886ef7c741a280c1a00cf6e4b0fb2d2a164cea3117f671a5ffbab96b4082c8c"}
Sep 15 22:46:06.672343 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:06.672265    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f2241f-586669d47d-lc898" event={"ID":"c4d9b866-cf8b-49c6-9f4e-a46a4477af1e","Type":"ContainerStarted","Data":"c638efcc11e6d029524371af5160c4c1f0471a449c50fa058d78634d09e078dd"}
Sep 15 22:46:06.672343 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:06.672298    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f2241f-586669d47d-lc898"
Sep 15 22:46:06.689266 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:06.689214    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f" podStartSLOduration=2.2152383860000002 podStartE2EDuration="2.689197978s" totalImagesPullingTime="473.959592ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:46:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:46:06.686765179 +0000 UTC m=+1116.546476464" watchObservedRunningTime="2026-09-15 22:46:06.689197978 +0000 UTC m=+1116.548909261"
Sep 15 22:46:06.703126 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:06.703079    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f2241f-586669d47d-lc898" podStartSLOduration=1.703059016 podStartE2EDuration="1.703059016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:46:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:46:06.701053328 +0000 UTC m=+1116.560764612" watchObservedRunningTime="2026-09-15 22:46:06.703059016 +0000 UTC m=+1116.562770300"
Sep 15 22:46:12.683268 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:12.683233    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f"
Sep 15 22:46:17.680648 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:17.680619    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f2241f-586669d47d-lc898"
Sep 15 22:46:22.255306 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.255263    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4"]
Sep 15 22:46:22.255740 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.255508    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4" podUID="8498a6c8-a46e-4189-95a3-c611467f5fc4" containerName="maas-api" containerID="cri-o://97f45e6d668bfceab373cba2afad4db00425fa1338199f773dc4fd17c47c75cf" gracePeriod=30
Sep 15 22:46:22.313199 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.313161    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-4ccd57-546878d877-92d5p"]
Sep 15 22:46:22.313671 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.313407    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-4ccd57-546878d877-92d5p" podUID="d08e86c0-a1d6-440b-9731-22ffc0608aa8" containerName="payload-processing" containerID="cri-o://eb849bbfd35bae3c719aec252d19311fc334d307ba2f55d27d610f362565248e" gracePeriod=30
Sep 15 22:46:22.326511 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.326483    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-4ccd57-5bd46c7d8b-wh5c6"]
Sep 15 22:46:22.326769 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.326746    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-4ccd57-5bd46c7d8b-wh5c6" podUID="0156bbf9-4aee-4630-87aa-ab4b26fa5795" containerName="payload-pre-processing" containerID="cri-o://3909abe39b0c2ede6618d330e46b2520a53b7e09249859e91849b53aad40296a" gracePeriod=30
Sep 15 22:46:22.605875 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.605788    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-4ccd57-546878d877-92d5p"
Sep 15 22:46:22.621464 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.621438    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4"
Sep 15 22:46:22.632218 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.632194    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-4ccd57-5bd46c7d8b-wh5c6"
Sep 15 22:46:22.717279 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.717246    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drbdf\" (UniqueName: \"kubernetes.io/projected/0156bbf9-4aee-4630-87aa-ab4b26fa5795-kube-api-access-drbdf\") pod \"0156bbf9-4aee-4630-87aa-ab4b26fa5795\" (UID: \"0156bbf9-4aee-4630-87aa-ab4b26fa5795\") "
Sep 15 22:46:22.717443 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.717288    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mzpx9\" (UniqueName: \"kubernetes.io/projected/8498a6c8-a46e-4189-95a3-c611467f5fc4-kube-api-access-mzpx9\") pod \"8498a6c8-a46e-4189-95a3-c611467f5fc4\" (UID: \"8498a6c8-a46e-4189-95a3-c611467f5fc4\") "
Sep 15 22:46:22.717443 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.717329    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8498a6c8-a46e-4189-95a3-c611467f5fc4-maas-api-tls\") pod \"8498a6c8-a46e-4189-95a3-c611467f5fc4\" (UID: \"8498a6c8-a46e-4189-95a3-c611467f5fc4\") "
Sep 15 22:46:22.717557 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.717431    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s95gn\" (UniqueName: \"kubernetes.io/projected/d08e86c0-a1d6-440b-9731-22ffc0608aa8-kube-api-access-s95gn\") pod \"d08e86c0-a1d6-440b-9731-22ffc0608aa8\" (UID: \"d08e86c0-a1d6-440b-9731-22ffc0608aa8\") "
Sep 15 22:46:22.717557 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.717520    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8498a6c8-a46e-4189-95a3-c611467f5fc4-openshift-service-ca\") pod \"8498a6c8-a46e-4189-95a3-c611467f5fc4\" (UID: \"8498a6c8-a46e-4189-95a3-c611467f5fc4\") "
Sep 15 22:46:22.717665 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.717561    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8498a6c8-a46e-4189-95a3-c611467f5fc4-metrics-certs\") pod \"8498a6c8-a46e-4189-95a3-c611467f5fc4\" (UID: \"8498a6c8-a46e-4189-95a3-c611467f5fc4\") "
Sep 15 22:46:22.717665 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.717608    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0156bbf9-4aee-4630-87aa-ab4b26fa5795-plugins-config-volume\") pod \"0156bbf9-4aee-4630-87aa-ab4b26fa5795\" (UID: \"0156bbf9-4aee-4630-87aa-ab4b26fa5795\") "
Sep 15 22:46:22.717665 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.717636    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d08e86c0-a1d6-440b-9731-22ffc0608aa8-plugins-config-volume\") pod \"d08e86c0-a1d6-440b-9731-22ffc0608aa8\" (UID: \"d08e86c0-a1d6-440b-9731-22ffc0608aa8\") "
Sep 15 22:46:22.717924 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.717894    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8498a6c8-a46e-4189-95a3-c611467f5fc4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8498a6c8-a46e-4189-95a3-c611467f5fc4" (UID: "8498a6c8-a46e-4189-95a3-c611467f5fc4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:46:22.718055 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.717982    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0156bbf9-4aee-4630-87aa-ab4b26fa5795-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0156bbf9-4aee-4630-87aa-ab4b26fa5795" (UID: "0156bbf9-4aee-4630-87aa-ab4b26fa5795"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:46:22.718165 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.718144    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8498a6c8-a46e-4189-95a3-c611467f5fc4-openshift-service-ca\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:46:22.718232 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.718171    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0156bbf9-4aee-4630-87aa-ab4b26fa5795-plugins-config-volume\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:46:22.718316 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.718292    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d08e86c0-a1d6-440b-9731-22ffc0608aa8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d08e86c0-a1d6-440b-9731-22ffc0608aa8" (UID: "d08e86c0-a1d6-440b-9731-22ffc0608aa8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:46:22.719610 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.719586    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8498a6c8-a46e-4189-95a3-c611467f5fc4-kube-api-access-mzpx9" (OuterVolumeSpecName: "kube-api-access-mzpx9") pod "8498a6c8-a46e-4189-95a3-c611467f5fc4" (UID: "8498a6c8-a46e-4189-95a3-c611467f5fc4"). InnerVolumeSpecName "kube-api-access-mzpx9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:46:22.719610 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.719602    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d08e86c0-a1d6-440b-9731-22ffc0608aa8-kube-api-access-s95gn" (OuterVolumeSpecName: "kube-api-access-s95gn") pod "d08e86c0-a1d6-440b-9731-22ffc0608aa8" (UID: "d08e86c0-a1d6-440b-9731-22ffc0608aa8"). InnerVolumeSpecName "kube-api-access-s95gn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:46:22.719764 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.719736    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8498a6c8-a46e-4189-95a3-c611467f5fc4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8498a6c8-a46e-4189-95a3-c611467f5fc4" (UID: "8498a6c8-a46e-4189-95a3-c611467f5fc4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:46:22.720015 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.719997    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8498a6c8-a46e-4189-95a3-c611467f5fc4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8498a6c8-a46e-4189-95a3-c611467f5fc4" (UID: "8498a6c8-a46e-4189-95a3-c611467f5fc4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:46:22.720163 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.720148    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0156bbf9-4aee-4630-87aa-ab4b26fa5795-kube-api-access-drbdf" (OuterVolumeSpecName: "kube-api-access-drbdf") pod "0156bbf9-4aee-4630-87aa-ab4b26fa5795" (UID: "0156bbf9-4aee-4630-87aa-ab4b26fa5795"). InnerVolumeSpecName "kube-api-access-drbdf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:46:22.779013 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.778944    2565 generic.go:356] "Generic (PLEG): container finished" podID="8498a6c8-a46e-4189-95a3-c611467f5fc4" containerID="97f45e6d668bfceab373cba2afad4db00425fa1338199f773dc4fd17c47c75cf" exitCode=0
Sep 15 22:46:22.779013 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.778992    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4"
Sep 15 22:46:22.779244 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.779043    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4" event={"ID":"8498a6c8-a46e-4189-95a3-c611467f5fc4","Type":"ContainerDied","Data":"97f45e6d668bfceab373cba2afad4db00425fa1338199f773dc4fd17c47c75cf"}
Sep 15 22:46:22.779244 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.779090    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4" event={"ID":"8498a6c8-a46e-4189-95a3-c611467f5fc4","Type":"ContainerDied","Data":"4edc28e5952db34c1cc102ef65dc4bafcdfada2c4317ed6f7565286672e4882a"}
Sep 15 22:46:22.779244 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.779115    2565 scope.go:117] "RemoveContainer" containerID="97f45e6d668bfceab373cba2afad4db00425fa1338199f773dc4fd17c47c75cf"
Sep 15 22:46:22.780629 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.780609    2565 generic.go:356] "Generic (PLEG): container finished" podID="0156bbf9-4aee-4630-87aa-ab4b26fa5795" containerID="3909abe39b0c2ede6618d330e46b2520a53b7e09249859e91849b53aad40296a" exitCode=0
Sep 15 22:46:22.780733 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.780637    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-4ccd57-5bd46c7d8b-wh5c6" event={"ID":"0156bbf9-4aee-4630-87aa-ab4b26fa5795","Type":"ContainerDied","Data":"3909abe39b0c2ede6618d330e46b2520a53b7e09249859e91849b53aad40296a"}
Sep 15 22:46:22.780733 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.780662    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-4ccd57-5bd46c7d8b-wh5c6"
Sep 15 22:46:22.780733 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.780698    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-4ccd57-5bd46c7d8b-wh5c6" event={"ID":"0156bbf9-4aee-4630-87aa-ab4b26fa5795","Type":"ContainerDied","Data":"69d0d48d5a696b632357f8b1adcb143fa07821d12275d347535cc503f3cd2653"}
Sep 15 22:46:22.782446 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.782369    2565 generic.go:356] "Generic (PLEG): container finished" podID="d08e86c0-a1d6-440b-9731-22ffc0608aa8" containerID="eb849bbfd35bae3c719aec252d19311fc334d307ba2f55d27d610f362565248e" exitCode=0
Sep 15 22:46:22.782446 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.782405    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-4ccd57-546878d877-92d5p" event={"ID":"d08e86c0-a1d6-440b-9731-22ffc0608aa8","Type":"ContainerDied","Data":"eb849bbfd35bae3c719aec252d19311fc334d307ba2f55d27d610f362565248e"}
Sep 15 22:46:22.782446 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.782445    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-4ccd57-546878d877-92d5p" event={"ID":"d08e86c0-a1d6-440b-9731-22ffc0608aa8","Type":"ContainerDied","Data":"f16a1c9cf530d850cbedb30afd5051224ade7cf089ed4e89c754b2a76fd4cfa5"}
Sep 15 22:46:22.782693 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.782543    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-4ccd57-546878d877-92d5p"
Sep 15 22:46:22.789472 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.789457    2565 scope.go:117] "RemoveContainer" containerID="97f45e6d668bfceab373cba2afad4db00425fa1338199f773dc4fd17c47c75cf"
Sep 15 22:46:22.790128 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:46:22.790103    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97f45e6d668bfceab373cba2afad4db00425fa1338199f773dc4fd17c47c75cf\": container with ID starting with 97f45e6d668bfceab373cba2afad4db00425fa1338199f773dc4fd17c47c75cf not found: ID does not exist" containerID="97f45e6d668bfceab373cba2afad4db00425fa1338199f773dc4fd17c47c75cf"
Sep 15 22:46:22.790207 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.790139    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97f45e6d668bfceab373cba2afad4db00425fa1338199f773dc4fd17c47c75cf"} err="failed to get container status \"97f45e6d668bfceab373cba2afad4db00425fa1338199f773dc4fd17c47c75cf\": rpc error: code = NotFound desc = could not find container \"97f45e6d668bfceab373cba2afad4db00425fa1338199f773dc4fd17c47c75cf\": container with ID starting with 97f45e6d668bfceab373cba2afad4db00425fa1338199f773dc4fd17c47c75cf not found: ID does not exist"
Sep 15 22:46:22.790207 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.790162    2565 scope.go:117] "RemoveContainer" containerID="3909abe39b0c2ede6618d330e46b2520a53b7e09249859e91849b53aad40296a"
Sep 15 22:46:22.800119 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.800100    2565 scope.go:117] "RemoveContainer" containerID="3909abe39b0c2ede6618d330e46b2520a53b7e09249859e91849b53aad40296a"
Sep 15 22:46:22.800368 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:46:22.800348    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3909abe39b0c2ede6618d330e46b2520a53b7e09249859e91849b53aad40296a\": container with ID starting with 3909abe39b0c2ede6618d330e46b2520a53b7e09249859e91849b53aad40296a not found: ID does not exist" containerID="3909abe39b0c2ede6618d330e46b2520a53b7e09249859e91849b53aad40296a"
Sep 15 22:46:22.800419 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.800374    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3909abe39b0c2ede6618d330e46b2520a53b7e09249859e91849b53aad40296a"} err="failed to get container status \"3909abe39b0c2ede6618d330e46b2520a53b7e09249859e91849b53aad40296a\": rpc error: code = NotFound desc = could not find container \"3909abe39b0c2ede6618d330e46b2520a53b7e09249859e91849b53aad40296a\": container with ID starting with 3909abe39b0c2ede6618d330e46b2520a53b7e09249859e91849b53aad40296a not found: ID does not exist"
Sep 15 22:46:22.800419 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.800389    2565 scope.go:117] "RemoveContainer" containerID="eb849bbfd35bae3c719aec252d19311fc334d307ba2f55d27d610f362565248e"
Sep 15 22:46:22.810788 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.810768    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-4ccd57-546878d877-92d5p"]
Sep 15 22:46:22.811058 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.811008    2565 scope.go:117] "RemoveContainer" containerID="eb849bbfd35bae3c719aec252d19311fc334d307ba2f55d27d610f362565248e"
Sep 15 22:46:22.811338 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:46:22.811319    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb849bbfd35bae3c719aec252d19311fc334d307ba2f55d27d610f362565248e\": container with ID starting with eb849bbfd35bae3c719aec252d19311fc334d307ba2f55d27d610f362565248e not found: ID does not exist" containerID="eb849bbfd35bae3c719aec252d19311fc334d307ba2f55d27d610f362565248e"
Sep 15 22:46:22.811423 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.811348    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb849bbfd35bae3c719aec252d19311fc334d307ba2f55d27d610f362565248e"} err="failed to get container status \"eb849bbfd35bae3c719aec252d19311fc334d307ba2f55d27d610f362565248e\": rpc error: code = NotFound desc = could not find container \"eb849bbfd35bae3c719aec252d19311fc334d307ba2f55d27d610f362565248e\": container with ID starting with eb849bbfd35bae3c719aec252d19311fc334d307ba2f55d27d610f362565248e not found: ID does not exist"
Sep 15 22:46:22.817974 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.817953    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-4ccd57-546878d877-92d5p"]
Sep 15 22:46:22.819169 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.819153    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drbdf\" (UniqueName: \"kubernetes.io/projected/0156bbf9-4aee-4630-87aa-ab4b26fa5795-kube-api-access-drbdf\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:46:22.819225 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.819172    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzpx9\" (UniqueName: \"kubernetes.io/projected/8498a6c8-a46e-4189-95a3-c611467f5fc4-kube-api-access-mzpx9\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:46:22.819225 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.819182    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8498a6c8-a46e-4189-95a3-c611467f5fc4-maas-api-tls\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:46:22.819225 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.819191    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s95gn\" (UniqueName: \"kubernetes.io/projected/d08e86c0-a1d6-440b-9731-22ffc0608aa8-kube-api-access-s95gn\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:46:22.819225 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.819200    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8498a6c8-a46e-4189-95a3-c611467f5fc4-metrics-certs\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:46:22.819225 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.819208    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d08e86c0-a1d6-440b-9731-22ffc0608aa8-plugins-config-volume\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:46:22.827810 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.827789    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4"]
Sep 15 22:46:22.833859 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.833834    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-4ccd57-77c5cf886c-dhkf4"]
Sep 15 22:46:22.843728 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.843705    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-4ccd57-5bd46c7d8b-wh5c6"]
Sep 15 22:46:22.847139 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:22.847120    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-4ccd57-5bd46c7d8b-wh5c6"]
Sep 15 22:46:24.591310 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:24.591274    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0156bbf9-4aee-4630-87aa-ab4b26fa5795" path="/var/lib/kubelet/pods/0156bbf9-4aee-4630-87aa-ab4b26fa5795/volumes"
Sep 15 22:46:24.591713 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:24.591698    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8498a6c8-a46e-4189-95a3-c611467f5fc4" path="/var/lib/kubelet/pods/8498a6c8-a46e-4189-95a3-c611467f5fc4/volumes"
Sep 15 22:46:24.592038 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:24.592015    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d08e86c0-a1d6-440b-9731-22ffc0608aa8" path="/var/lib/kubelet/pods/d08e86c0-a1d6-440b-9731-22ffc0608aa8/volumes"
Sep 15 22:46:27.938661 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:27.938625    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh"]
Sep 15 22:46:27.939379 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:27.939349    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d08e86c0-a1d6-440b-9731-22ffc0608aa8" containerName="payload-processing"
Sep 15 22:46:27.939379 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:27.939373    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d08e86c0-a1d6-440b-9731-22ffc0608aa8" containerName="payload-processing"
Sep 15 22:46:27.939528 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:27.939389    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0156bbf9-4aee-4630-87aa-ab4b26fa5795" containerName="payload-pre-processing"
Sep 15 22:46:27.939528 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:27.939398    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="0156bbf9-4aee-4630-87aa-ab4b26fa5795" containerName="payload-pre-processing"
Sep 15 22:46:27.939528 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:27.939415    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8498a6c8-a46e-4189-95a3-c611467f5fc4" containerName="maas-api"
Sep 15 22:46:27.939528 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:27.939426    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="8498a6c8-a46e-4189-95a3-c611467f5fc4" containerName="maas-api"
Sep 15 22:46:27.939698 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:27.939534    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="8498a6c8-a46e-4189-95a3-c611467f5fc4" containerName="maas-api"
Sep 15 22:46:27.939698 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:27.939546    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="0156bbf9-4aee-4630-87aa-ab4b26fa5795" containerName="payload-pre-processing"
Sep 15 22:46:27.939698 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:27.939560    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d08e86c0-a1d6-440b-9731-22ffc0608aa8" containerName="payload-processing"
Sep 15 22:46:27.944743 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:27.944721    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh"
Sep 15 22:46:27.946907 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:27.946885    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-4f70a9-serving-cert\""
Sep 15 22:46:27.952727 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:27.952698    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh"]
Sep 15 22:46:28.075514 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:28.075483    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wb5z\" (UniqueName: \"kubernetes.io/projected/42a59c38-9365-4f8a-bed2-e5ed68013089-kube-api-access-6wb5z\") pod \"maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh\" (UID: \"42a59c38-9365-4f8a-bed2-e5ed68013089\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh"
Sep 15 22:46:28.075673 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:28.075529    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42a59c38-9365-4f8a-bed2-e5ed68013089-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh\" (UID: \"42a59c38-9365-4f8a-bed2-e5ed68013089\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh"
Sep 15 22:46:28.075673 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:28.075564    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42a59c38-9365-4f8a-bed2-e5ed68013089-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh\" (UID: \"42a59c38-9365-4f8a-bed2-e5ed68013089\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh"
Sep 15 22:46:28.075673 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:28.075588    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42a59c38-9365-4f8a-bed2-e5ed68013089-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh\" (UID: \"42a59c38-9365-4f8a-bed2-e5ed68013089\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh"
Sep 15 22:46:28.176696 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:28.176662    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42a59c38-9365-4f8a-bed2-e5ed68013089-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh\" (UID: \"42a59c38-9365-4f8a-bed2-e5ed68013089\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh"
Sep 15 22:46:28.176869 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:28.176702    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42a59c38-9365-4f8a-bed2-e5ed68013089-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh\" (UID: \"42a59c38-9365-4f8a-bed2-e5ed68013089\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh"
Sep 15 22:46:28.176869 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:28.176742    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42a59c38-9365-4f8a-bed2-e5ed68013089-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh\" (UID: \"42a59c38-9365-4f8a-bed2-e5ed68013089\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh"
Sep 15 22:46:28.176869 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:28.176831    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wb5z\" (UniqueName: \"kubernetes.io/projected/42a59c38-9365-4f8a-bed2-e5ed68013089-kube-api-access-6wb5z\") pod \"maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh\" (UID: \"42a59c38-9365-4f8a-bed2-e5ed68013089\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh"
Sep 15 22:46:28.177522 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:28.177495    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42a59c38-9365-4f8a-bed2-e5ed68013089-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh\" (UID: \"42a59c38-9365-4f8a-bed2-e5ed68013089\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh"
Sep 15 22:46:28.179176 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:28.179146    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42a59c38-9365-4f8a-bed2-e5ed68013089-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh\" (UID: \"42a59c38-9365-4f8a-bed2-e5ed68013089\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh"
Sep 15 22:46:28.179285 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:28.179171    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42a59c38-9365-4f8a-bed2-e5ed68013089-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh\" (UID: \"42a59c38-9365-4f8a-bed2-e5ed68013089\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh"
Sep 15 22:46:28.184473 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:28.184444    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wb5z\" (UniqueName: \"kubernetes.io/projected/42a59c38-9365-4f8a-bed2-e5ed68013089-kube-api-access-6wb5z\") pod \"maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh\" (UID: \"42a59c38-9365-4f8a-bed2-e5ed68013089\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh"
Sep 15 22:46:28.259380 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:28.259310    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh"
Sep 15 22:46:28.538736 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:28.538705    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh"]
Sep 15 22:46:28.539846 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:46:28.539813    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42a59c38_9365_4f8a_bed2_e5ed68013089.slice/crio-493bc1b8d1e72e4d56c05ee5d7ae789e87e85699cef4feda5978b28d1eac5a84 WatchSource:0}: Error finding container 493bc1b8d1e72e4d56c05ee5d7ae789e87e85699cef4feda5978b28d1eac5a84: Status 404 returned error can't find the container with id 493bc1b8d1e72e4d56c05ee5d7ae789e87e85699cef4feda5978b28d1eac5a84
Sep 15 22:46:28.558482 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:28.558459    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f70a9-d4d9dcd-nxh4d"]
Sep 15 22:46:28.563156 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:28.563139    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f70a9-d4d9dcd-nxh4d"
Sep 15 22:46:28.566520 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:28.566501    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-4f70a9\""
Sep 15 22:46:28.566640 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:28.566621    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-4f70a9-dockercfg-9t4gb\""
Sep 15 22:46:28.572526 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:28.572504    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f70a9-d4d9dcd-nxh4d"]
Sep 15 22:46:28.606681 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:28.603639    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-4f70a9-ffbd9b96f-87qsw"]
Sep 15 22:46:28.609230 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:28.609205    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-4f70a9-ffbd9b96f-87qsw"]
Sep 15 22:46:28.609346 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:28.609319    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4f70a9-ffbd9b96f-87qsw"
Sep 15 22:46:28.682464 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:28.682435    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7kcf\" (UniqueName: \"kubernetes.io/projected/43d18bd7-ffb8-4bb3-a79e-6415f5ae80d7-kube-api-access-x7kcf\") pod \"payload-pre-processing-e2e-shared-b-w2-4f70a9-d4d9dcd-nxh4d\" (UID: \"43d18bd7-ffb8-4bb3-a79e-6415f5ae80d7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f70a9-d4d9dcd-nxh4d"
Sep 15 22:46:28.682653 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:28.682471    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gg9c7\" (UniqueName: \"kubernetes.io/projected/da98f27a-4090-410e-9203-c2be6b669e7c-kube-api-access-gg9c7\") pod \"payload-processing-e2e-shared-b-w2-4f70a9-ffbd9b96f-87qsw\" (UID: \"da98f27a-4090-410e-9203-c2be6b669e7c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4f70a9-ffbd9b96f-87qsw"
Sep 15 22:46:28.682734 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:28.682696    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43d18bd7-ffb8-4bb3-a79e-6415f5ae80d7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-4f70a9-d4d9dcd-nxh4d\" (UID: \"43d18bd7-ffb8-4bb3-a79e-6415f5ae80d7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f70a9-d4d9dcd-nxh4d"
Sep 15 22:46:28.682800 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:28.682745    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da98f27a-4090-410e-9203-c2be6b669e7c-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-4f70a9-ffbd9b96f-87qsw\" (UID: \"da98f27a-4090-410e-9203-c2be6b669e7c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4f70a9-ffbd9b96f-87qsw"
Sep 15 22:46:28.783697 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:28.783666    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7kcf\" (UniqueName: \"kubernetes.io/projected/43d18bd7-ffb8-4bb3-a79e-6415f5ae80d7-kube-api-access-x7kcf\") pod \"payload-pre-processing-e2e-shared-b-w2-4f70a9-d4d9dcd-nxh4d\" (UID: \"43d18bd7-ffb8-4bb3-a79e-6415f5ae80d7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f70a9-d4d9dcd-nxh4d"
Sep 15 22:46:28.783697 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:28.783703    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gg9c7\" (UniqueName: \"kubernetes.io/projected/da98f27a-4090-410e-9203-c2be6b669e7c-kube-api-access-gg9c7\") pod \"payload-processing-e2e-shared-b-w2-4f70a9-ffbd9b96f-87qsw\" (UID: \"da98f27a-4090-410e-9203-c2be6b669e7c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4f70a9-ffbd9b96f-87qsw"
Sep 15 22:46:28.783914 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:28.783789    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43d18bd7-ffb8-4bb3-a79e-6415f5ae80d7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-4f70a9-d4d9dcd-nxh4d\" (UID: \"43d18bd7-ffb8-4bb3-a79e-6415f5ae80d7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f70a9-d4d9dcd-nxh4d"
Sep 15 22:46:28.783914 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:28.783819    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da98f27a-4090-410e-9203-c2be6b669e7c-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-4f70a9-ffbd9b96f-87qsw\" (UID: \"da98f27a-4090-410e-9203-c2be6b669e7c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4f70a9-ffbd9b96f-87qsw"
Sep 15 22:46:28.784512 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:28.784487    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43d18bd7-ffb8-4bb3-a79e-6415f5ae80d7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-4f70a9-d4d9dcd-nxh4d\" (UID: \"43d18bd7-ffb8-4bb3-a79e-6415f5ae80d7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f70a9-d4d9dcd-nxh4d"
Sep 15 22:46:28.784613 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:28.784528    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da98f27a-4090-410e-9203-c2be6b669e7c-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-4f70a9-ffbd9b96f-87qsw\" (UID: \"da98f27a-4090-410e-9203-c2be6b669e7c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4f70a9-ffbd9b96f-87qsw"
Sep 15 22:46:28.791577 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:28.791524    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gg9c7\" (UniqueName: \"kubernetes.io/projected/da98f27a-4090-410e-9203-c2be6b669e7c-kube-api-access-gg9c7\") pod \"payload-processing-e2e-shared-b-w2-4f70a9-ffbd9b96f-87qsw\" (UID: \"da98f27a-4090-410e-9203-c2be6b669e7c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4f70a9-ffbd9b96f-87qsw"
Sep 15 22:46:28.791677 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:28.791661    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7kcf\" (UniqueName: \"kubernetes.io/projected/43d18bd7-ffb8-4bb3-a79e-6415f5ae80d7-kube-api-access-x7kcf\") pod \"payload-pre-processing-e2e-shared-b-w2-4f70a9-d4d9dcd-nxh4d\" (UID: \"43d18bd7-ffb8-4bb3-a79e-6415f5ae80d7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f70a9-d4d9dcd-nxh4d"
Sep 15 22:46:28.824467 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:28.824440    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh" event={"ID":"42a59c38-9365-4f8a-bed2-e5ed68013089","Type":"ContainerStarted","Data":"493bc1b8d1e72e4d56c05ee5d7ae789e87e85699cef4feda5978b28d1eac5a84"}
Sep 15 22:46:28.874830 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:28.874809    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f70a9-d4d9dcd-nxh4d"
Sep 15 22:46:28.923388 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:28.923364    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4f70a9-ffbd9b96f-87qsw"
Sep 15 22:46:29.166150 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:29.166122    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f70a9-d4d9dcd-nxh4d"]
Sep 15 22:46:29.167061 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:46:29.167014    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43d18bd7_ffb8_4bb3_a79e_6415f5ae80d7.slice/crio-95b54b103bb20f4b9b00ba2276e054f1751d4e3ce0653c3240924031f386e417 WatchSource:0}: Error finding container 95b54b103bb20f4b9b00ba2276e054f1751d4e3ce0653c3240924031f386e417: Status 404 returned error can't find the container with id 95b54b103bb20f4b9b00ba2276e054f1751d4e3ce0653c3240924031f386e417
Sep 15 22:46:29.213746 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:29.213725    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-4f70a9-ffbd9b96f-87qsw"]
Sep 15 22:46:29.219263 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:46:29.219226    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda98f27a_4090_410e_9203_c2be6b669e7c.slice/crio-abae8691a66e6497495b3b062a4342cd8b7f62c5e42ce4a0d2a443810b5035e0 WatchSource:0}: Error finding container abae8691a66e6497495b3b062a4342cd8b7f62c5e42ce4a0d2a443810b5035e0: Status 404 returned error can't find the container with id abae8691a66e6497495b3b062a4342cd8b7f62c5e42ce4a0d2a443810b5035e0
Sep 15 22:46:29.835674 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:29.835634    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh" event={"ID":"42a59c38-9365-4f8a-bed2-e5ed68013089","Type":"ContainerStarted","Data":"5055552aa0a953c18605109d201bec1986a417be576e2d597e49196cf6ab7b64"}
Sep 15 22:46:29.836196 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:29.836016    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh"
Sep 15 22:46:29.837573 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:29.837541    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4f70a9-ffbd9b96f-87qsw" event={"ID":"da98f27a-4090-410e-9203-c2be6b669e7c","Type":"ContainerStarted","Data":"b3de325fc3c4eeb8a3aaaab6c9a69352f244b171b357637a92fbc5c688a5dfb8"}
Sep 15 22:46:29.837733 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:29.837718    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4f70a9-ffbd9b96f-87qsw" event={"ID":"da98f27a-4090-410e-9203-c2be6b669e7c","Type":"ContainerStarted","Data":"abae8691a66e6497495b3b062a4342cd8b7f62c5e42ce4a0d2a443810b5035e0"}
Sep 15 22:46:29.837871 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:29.837849    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4f70a9-ffbd9b96f-87qsw"
Sep 15 22:46:29.839590 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:29.839568    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f70a9-d4d9dcd-nxh4d" event={"ID":"43d18bd7-ffb8-4bb3-a79e-6415f5ae80d7","Type":"ContainerStarted","Data":"f9125318bffd0c09f693afc8e3d1eb6bde976e9a415534a6787eeb0f752aa29d"}
Sep 15 22:46:29.839684 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:29.839611    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f70a9-d4d9dcd-nxh4d" event={"ID":"43d18bd7-ffb8-4bb3-a79e-6415f5ae80d7","Type":"ContainerStarted","Data":"95b54b103bb20f4b9b00ba2276e054f1751d4e3ce0653c3240924031f386e417"}
Sep 15 22:46:29.839684 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:29.839667    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f70a9-d4d9dcd-nxh4d"
Sep 15 22:46:29.853676 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:29.853630    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh" podStartSLOduration=2.501465859 podStartE2EDuration="2.853614839s" totalImagesPullingTime="352.14898ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:46:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:46:29.851192372 +0000 UTC m=+1139.710903668" watchObservedRunningTime="2026-09-15 22:46:29.853614839 +0000 UTC m=+1139.713326122"
Sep 15 22:46:29.867259 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:29.867213    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4f70a9-ffbd9b96f-87qsw" podStartSLOduration=1.867195484 podStartE2EDuration="1.867195484s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:46:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:46:29.86687636 +0000 UTC m=+1139.726587644" watchObservedRunningTime="2026-09-15 22:46:29.867195484 +0000 UTC m=+1139.726906770"
Sep 15 22:46:29.886473 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:29.886433    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f70a9-d4d9dcd-nxh4d" podStartSLOduration=1.886418887 podStartE2EDuration="1.886418887s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:46:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:46:29.885073628 +0000 UTC m=+1139.744784911" watchObservedRunningTime="2026-09-15 22:46:29.886418887 +0000 UTC m=+1139.746130178"
Sep 15 22:46:35.852269 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:35.852238    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh"
Sep 15 22:46:40.848646 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:40.848615    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4f70a9-ffbd9b96f-87qsw"
Sep 15 22:46:40.849134 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:40.848781    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f70a9-d4d9dcd-nxh4d"
Sep 15 22:46:54.756804 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:54.756730    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7651-5da475"]
Sep 15 22:46:54.761459 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:54.761439    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-5da475"
Sep 15 22:46:54.763771 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:54.763747    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-4fdg9\""
Sep 15 22:46:54.766803 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:54.766775    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7651-5da475"]
Sep 15 22:46:54.844048 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:54.843993    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjjrz\" (UniqueName: \"kubernetes.io/projected/4f54ac22-4528-465e-9a19-5ded752c29c1-kube-api-access-fjjrz\") pod \"test-curl-7651-5da475\" (UID: \"4f54ac22-4528-465e-9a19-5ded752c29c1\") " pod="opendatahub/test-curl-7651-5da475"
Sep 15 22:46:54.944968 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:54.944930    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjjrz\" (UniqueName: \"kubernetes.io/projected/4f54ac22-4528-465e-9a19-5ded752c29c1-kube-api-access-fjjrz\") pod \"test-curl-7651-5da475\" (UID: \"4f54ac22-4528-465e-9a19-5ded752c29c1\") " pod="opendatahub/test-curl-7651-5da475"
Sep 15 22:46:54.955120 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:54.955089    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjjrz\" (UniqueName: \"kubernetes.io/projected/4f54ac22-4528-465e-9a19-5ded752c29c1-kube-api-access-fjjrz\") pod \"test-curl-7651-5da475\" (UID: \"4f54ac22-4528-465e-9a19-5ded752c29c1\") " pod="opendatahub/test-curl-7651-5da475"
Sep 15 22:46:55.072413 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:55.072378    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-5da475"
Sep 15 22:46:55.351783 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:55.351756    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7651-5da475"]
Sep 15 22:46:55.353047 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:46:55.353006    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4f54ac22_4528_465e_9a19_5ded752c29c1.slice/crio-3547b9bdcc9a7825c2c3e127c4a7fdf34aafa7bc920d7425b0cb441f3fa8cf16 WatchSource:0}: Error finding container 3547b9bdcc9a7825c2c3e127c4a7fdf34aafa7bc920d7425b0cb441f3fa8cf16: Status 404 returned error can't find the container with id 3547b9bdcc9a7825c2c3e127c4a7fdf34aafa7bc920d7425b0cb441f3fa8cf16
Sep 15 22:46:56.006599 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:56.006563    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7651-5da475" event={"ID":"4f54ac22-4528-465e-9a19-5ded752c29c1","Type":"ContainerStarted","Data":"3547b9bdcc9a7825c2c3e127c4a7fdf34aafa7bc920d7425b0cb441f3fa8cf16"}
Sep 15 22:46:58.022478 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:58.022390    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7651-5da475" event={"ID":"4f54ac22-4528-465e-9a19-5ded752c29c1","Type":"ContainerStarted","Data":"d59990543f1e82b5d8d7816e0d4c3bb0a211ed6e84e6b82565beefcbe050a200"}
Sep 15 22:46:58.038281 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:58.038228    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7651-5da475" podStartSLOduration=1.8413573250000002 podStartE2EDuration="4.038212325s" totalImagesPullingTime="2.196855s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:46:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:46:58.035570272 +0000 UTC m=+1167.895281556" watchObservedRunningTime="2026-09-15 22:46:58.038212325 +0000 UTC m=+1167.897923609"
Sep 15 22:46:58.646297 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:58.646260    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7651-5da475"]
Sep 15 22:46:58.655386 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:58.655357    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7651-5da475"]
Sep 15 22:46:58.769747 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:58.769668    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7651-5a3a64"]
Sep 15 22:46:58.770152 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:58.770138    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f54ac22-4528-465e-9a19-5ded752c29c1" containerName="test-curl-7651-5da475"
Sep 15 22:46:58.770152 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:58.770154    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f54ac22-4528-465e-9a19-5ded752c29c1" containerName="test-curl-7651-5da475"
Sep 15 22:46:58.770257 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:58.770236    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f54ac22-4528-465e-9a19-5ded752c29c1" containerName="test-curl-7651-5da475"
Sep 15 22:46:58.774102 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:58.774078    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-5a3a64"
Sep 15 22:46:58.777589 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:58.777562    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7651-5a3a64"]
Sep 15 22:46:58.887465 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:58.887432    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nj79\" (UniqueName: \"kubernetes.io/projected/6f539b1b-7219-40d5-a3b9-d579243008a4-kube-api-access-6nj79\") pod \"test-curl-7651-5a3a64\" (UID: \"6f539b1b-7219-40d5-a3b9-d579243008a4\") " pod="opendatahub/test-curl-7651-5a3a64"
Sep 15 22:46:58.988589 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:58.988555    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nj79\" (UniqueName: \"kubernetes.io/projected/6f539b1b-7219-40d5-a3b9-d579243008a4-kube-api-access-6nj79\") pod \"test-curl-7651-5a3a64\" (UID: \"6f539b1b-7219-40d5-a3b9-d579243008a4\") " pod="opendatahub/test-curl-7651-5a3a64"
Sep 15 22:46:59.009207 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:59.009178    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nj79\" (UniqueName: \"kubernetes.io/projected/6f539b1b-7219-40d5-a3b9-d579243008a4-kube-api-access-6nj79\") pod \"test-curl-7651-5a3a64\" (UID: \"6f539b1b-7219-40d5-a3b9-d579243008a4\") " pod="opendatahub/test-curl-7651-5a3a64"
Sep 15 22:46:59.028521 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:46:59.028456    2565 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7651-5da475" container="test-curl-7651-5da475"
Sep 15 22:46:59.086303 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:59.086264    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-5a3a64"
Sep 15 22:46:59.362161 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:46:59.362132    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7651-5a3a64"]
Sep 15 22:46:59.363744 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:46:59.363711    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f539b1b_7219_40d5_a3b9_d579243008a4.slice/crio-e80695c59cb6521f75f7dcd9bd766d7489dbcf0a305c776a9c66033aa44ac91c WatchSource:0}: Error finding container e80695c59cb6521f75f7dcd9bd766d7489dbcf0a305c776a9c66033aa44ac91c: Status 404 returned error can't find the container with id e80695c59cb6521f75f7dcd9bd766d7489dbcf0a305c776a9c66033aa44ac91c
Sep 15 22:47:00.036585 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:00.036548    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7651-5a3a64" event={"ID":"6f539b1b-7219-40d5-a3b9-d579243008a4","Type":"ContainerStarted","Data":"ee5d65cced23480c109c8bdb54a407f8dcbe2c157adf8e1538b4350e9a1a0748"}
Sep 15 22:47:00.037057 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:00.036594    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7651-5a3a64" event={"ID":"6f539b1b-7219-40d5-a3b9-d579243008a4","Type":"ContainerStarted","Data":"e80695c59cb6521f75f7dcd9bd766d7489dbcf0a305c776a9c66033aa44ac91c"}
Sep 15 22:47:00.037057 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:00.036597    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7651-5da475" podUID="4f54ac22-4528-465e-9a19-5ded752c29c1" containerName="test-curl-7651-5da475" containerID="cri-o://d59990543f1e82b5d8d7816e0d4c3bb0a211ed6e84e6b82565beefcbe050a200" gracePeriod=2
Sep 15 22:47:00.050362 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:00.050313    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7651-5a3a64" podStartSLOduration=2.050298749 podStartE2EDuration="2.050298749s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:46:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:47:00.049107855 +0000 UTC m=+1169.908819132" watchObservedRunningTime="2026-09-15 22:47:00.050298749 +0000 UTC m=+1169.910010033"
Sep 15 22:47:00.050915 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:47:00.050892    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7651-5da475\" is forbidden: User \"system:node:ip-10-0-128-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-208.ec2.internal' and this object" podUID="4f54ac22-4528-465e-9a19-5ded752c29c1" pod="opendatahub/test-curl-7651-5da475"
Sep 15 22:47:00.052403 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:47:00.052383    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7651-5da475\" is forbidden: User \"system:node:ip-10-0-128-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-208.ec2.internal' and this object" podUID="4f54ac22-4528-465e-9a19-5ded752c29c1" pod="opendatahub/test-curl-7651-5da475"
Sep 15 22:47:00.589628 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:47:00.589592    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7651-5da475\" is forbidden: User \"system:node:ip-10-0-128-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-208.ec2.internal' and this object" podUID="4f54ac22-4528-465e-9a19-5ded752c29c1" pod="opendatahub/test-curl-7651-5da475"
Sep 15 22:47:00.621520 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:00.621485    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7651-5a3a64"]
Sep 15 22:47:00.626889 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:00.626855    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7651-5a3a64"]
Sep 15 22:47:00.737352 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:00.737280    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7651-b2e5a0"]
Sep 15 22:47:00.737775 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:00.737762    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f539b1b-7219-40d5-a3b9-d579243008a4" containerName="test-curl-7651-5a3a64"
Sep 15 22:47:00.737827 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:00.737777    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f539b1b-7219-40d5-a3b9-d579243008a4" containerName="test-curl-7651-5a3a64"
Sep 15 22:47:00.737863 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:00.737845    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f539b1b-7219-40d5-a3b9-d579243008a4" containerName="test-curl-7651-5a3a64"
Sep 15 22:47:00.748009 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:00.743141    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-b2e5a0"
Sep 15 22:47:00.750727 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:00.750697    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7651-b2e5a0"]
Sep 15 22:47:00.806045 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:00.805985    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5l9jv\" (UniqueName: \"kubernetes.io/projected/e8719cc6-db15-4b1c-8100-844cfec5690f-kube-api-access-5l9jv\") pod \"test-curl-7651-b2e5a0\" (UID: \"e8719cc6-db15-4b1c-8100-844cfec5690f\") " pod="opendatahub/test-curl-7651-b2e5a0"
Sep 15 22:47:00.906792 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:00.906757    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5l9jv\" (UniqueName: \"kubernetes.io/projected/e8719cc6-db15-4b1c-8100-844cfec5690f-kube-api-access-5l9jv\") pod \"test-curl-7651-b2e5a0\" (UID: \"e8719cc6-db15-4b1c-8100-844cfec5690f\") " pod="opendatahub/test-curl-7651-b2e5a0"
Sep 15 22:47:00.915369 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:00.915343    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l9jv\" (UniqueName: \"kubernetes.io/projected/e8719cc6-db15-4b1c-8100-844cfec5690f-kube-api-access-5l9jv\") pod \"test-curl-7651-b2e5a0\" (UID: \"e8719cc6-db15-4b1c-8100-844cfec5690f\") " pod="opendatahub/test-curl-7651-b2e5a0"
Sep 15 22:47:01.042988 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:47:01.042960    2565 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7651-5a3a64" container="test-curl-7651-5a3a64"
Sep 15 22:47:01.058205 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:01.058187    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-b2e5a0"
Sep 15 22:47:01.333231 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:01.333199    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7651-b2e5a0"]
Sep 15 22:47:01.333495 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:47:01.333471    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode8719cc6_db15_4b1c_8100_844cfec5690f.slice/crio-c45c903d20f44186da179a7f4490fa7c840bb88c98c21961f9ae62333dc1363f WatchSource:0}: Error finding container c45c903d20f44186da179a7f4490fa7c840bb88c98c21961f9ae62333dc1363f: Status 404 returned error can't find the container with id c45c903d20f44186da179a7f4490fa7c840bb88c98c21961f9ae62333dc1363f
Sep 15 22:47:02.050973 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:02.050941    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7651-b2e5a0" event={"ID":"e8719cc6-db15-4b1c-8100-844cfec5690f","Type":"ContainerStarted","Data":"eb4241ee3affa02ef68d53c9a25b991046710391f8f85353a0d77f9e7e700513"}
Sep 15 22:47:02.051356 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:02.050986    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7651-b2e5a0" event={"ID":"e8719cc6-db15-4b1c-8100-844cfec5690f","Type":"ContainerStarted","Data":"c45c903d20f44186da179a7f4490fa7c840bb88c98c21961f9ae62333dc1363f"}
Sep 15 22:47:02.051356 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:02.051093    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7651-5a3a64" podUID="6f539b1b-7219-40d5-a3b9-d579243008a4" containerName="test-curl-7651-5a3a64" containerID="cri-o://ee5d65cced23480c109c8bdb54a407f8dcbe2c157adf8e1538b4350e9a1a0748" gracePeriod=2
Sep 15 22:47:02.053061 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:47:02.053010    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7651-5a3a64\" is forbidden: User \"system:node:ip-10-0-128-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-208.ec2.internal' and this object" podUID="6f539b1b-7219-40d5-a3b9-d579243008a4" pod="opendatahub/test-curl-7651-5a3a64"
Sep 15 22:47:02.068683 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:47:02.068660    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7651-5a3a64\" is forbidden: User \"system:node:ip-10-0-128-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-208.ec2.internal' and this object" podUID="6f539b1b-7219-40d5-a3b9-d579243008a4" pod="opendatahub/test-curl-7651-5a3a64"
Sep 15 22:47:02.069175 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:02.069140    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7651-b2e5a0" podStartSLOduration=2.069125352 podStartE2EDuration="2.069125352s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:47:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:47:02.066561306 +0000 UTC m=+1171.926272592" watchObservedRunningTime="2026-09-15 22:47:02.069125352 +0000 UTC m=+1171.928836636"
Sep 15 22:47:02.210379 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:02.210355    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-5da475"
Sep 15 22:47:02.212208 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:47:02.212182    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7651-5a3a64\" is forbidden: User \"system:node:ip-10-0-128-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-208.ec2.internal' and this object" podUID="6f539b1b-7219-40d5-a3b9-d579243008a4" pod="opendatahub/test-curl-7651-5a3a64"
Sep 15 22:47:02.321859 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:02.321791    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fjjrz\" (UniqueName: \"kubernetes.io/projected/4f54ac22-4528-465e-9a19-5ded752c29c1-kube-api-access-fjjrz\") pod \"4f54ac22-4528-465e-9a19-5ded752c29c1\" (UID: \"4f54ac22-4528-465e-9a19-5ded752c29c1\") "
Sep 15 22:47:02.323634 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:02.323604    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f54ac22-4528-465e-9a19-5ded752c29c1-kube-api-access-fjjrz" (OuterVolumeSpecName: "kube-api-access-fjjrz") pod "4f54ac22-4528-465e-9a19-5ded752c29c1" (UID: "4f54ac22-4528-465e-9a19-5ded752c29c1"). InnerVolumeSpecName "kube-api-access-fjjrz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:47:02.422515 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:02.422482    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fjjrz\" (UniqueName: \"kubernetes.io/projected/4f54ac22-4528-465e-9a19-5ded752c29c1-kube-api-access-fjjrz\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:47:02.591068 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:02.590974    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f54ac22-4528-465e-9a19-5ded752c29c1" path="/var/lib/kubelet/pods/4f54ac22-4528-465e-9a19-5ded752c29c1/volumes"
Sep 15 22:47:02.638351 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:02.638317    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7651-b2e5a0"]
Sep 15 22:47:02.642515 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:02.642488    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7651-b2e5a0"]
Sep 15 22:47:02.753183 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:02.753156    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7651-faeea7"]
Sep 15 22:47:02.753627 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:02.753613    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8719cc6-db15-4b1c-8100-844cfec5690f" containerName="test-curl-7651-b2e5a0"
Sep 15 22:47:02.753672 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:02.753629    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8719cc6-db15-4b1c-8100-844cfec5690f" containerName="test-curl-7651-b2e5a0"
Sep 15 22:47:02.753704 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:02.753701    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8719cc6-db15-4b1c-8100-844cfec5690f" containerName="test-curl-7651-b2e5a0"
Sep 15 22:47:02.756973 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:02.756956    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-faeea7"
Sep 15 22:47:02.760874 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:02.760845    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7651-faeea7"]
Sep 15 22:47:02.827682 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:02.827647    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ps57n\" (UniqueName: \"kubernetes.io/projected/305278b5-188f-4667-8c91-d379ca81391e-kube-api-access-ps57n\") pod \"test-curl-7651-faeea7\" (UID: \"305278b5-188f-4667-8c91-d379ca81391e\") " pod="opendatahub/test-curl-7651-faeea7"
Sep 15 22:47:02.929014 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:02.928938    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ps57n\" (UniqueName: \"kubernetes.io/projected/305278b5-188f-4667-8c91-d379ca81391e-kube-api-access-ps57n\") pod \"test-curl-7651-faeea7\" (UID: \"305278b5-188f-4667-8c91-d379ca81391e\") " pod="opendatahub/test-curl-7651-faeea7"
Sep 15 22:47:02.936477 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:02.936453    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ps57n\" (UniqueName: \"kubernetes.io/projected/305278b5-188f-4667-8c91-d379ca81391e-kube-api-access-ps57n\") pod \"test-curl-7651-faeea7\" (UID: \"305278b5-188f-4667-8c91-d379ca81391e\") " pod="opendatahub/test-curl-7651-faeea7"
Sep 15 22:47:03.058287 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:03.058247    2565 generic.go:356] "Generic (PLEG): container finished" podID="4f54ac22-4528-465e-9a19-5ded752c29c1" containerID="d59990543f1e82b5d8d7816e0d4c3bb0a211ed6e84e6b82565beefcbe050a200" exitCode=137
Sep 15 22:47:03.058287 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:03.058291    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-5da475"
Sep 15 22:47:03.058774 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:03.058349    2565 scope.go:117] "RemoveContainer" containerID="d59990543f1e82b5d8d7816e0d4c3bb0a211ed6e84e6b82565beefcbe050a200"
Sep 15 22:47:03.058774 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:47:03.058612    2565 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7651-b2e5a0" container="test-curl-7651-b2e5a0"
Sep 15 22:47:03.067398 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:03.067279    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-faeea7"
Sep 15 22:47:03.067460 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:03.067407    2565 scope.go:117] "RemoveContainer" containerID="d59990543f1e82b5d8d7816e0d4c3bb0a211ed6e84e6b82565beefcbe050a200"
Sep 15 22:47:03.067684 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:47:03.067666    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d59990543f1e82b5d8d7816e0d4c3bb0a211ed6e84e6b82565beefcbe050a200\": container with ID starting with d59990543f1e82b5d8d7816e0d4c3bb0a211ed6e84e6b82565beefcbe050a200 not found: ID does not exist" containerID="d59990543f1e82b5d8d7816e0d4c3bb0a211ed6e84e6b82565beefcbe050a200"
Sep 15 22:47:03.067725 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:03.067693    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d59990543f1e82b5d8d7816e0d4c3bb0a211ed6e84e6b82565beefcbe050a200"} err="failed to get container status \"d59990543f1e82b5d8d7816e0d4c3bb0a211ed6e84e6b82565beefcbe050a200\": rpc error: code = NotFound desc = could not find container \"d59990543f1e82b5d8d7816e0d4c3bb0a211ed6e84e6b82565beefcbe050a200\": container with ID starting with d59990543f1e82b5d8d7816e0d4c3bb0a211ed6e84e6b82565beefcbe050a200 not found: ID does not exist"
Sep 15 22:47:03.338864 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:03.338837    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7651-faeea7"]
Sep 15 22:47:03.340083 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:47:03.340058    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod305278b5_188f_4667_8c91_d379ca81391e.slice/crio-4801470a9e205d78cbf1167aaa2289d0818c070ad5d8e09de94da5be568eab88 WatchSource:0}: Error finding container 4801470a9e205d78cbf1167aaa2289d0818c070ad5d8e09de94da5be568eab88: Status 404 returned error can't find the container with id 4801470a9e205d78cbf1167aaa2289d0818c070ad5d8e09de94da5be568eab88
Sep 15 22:47:04.068960 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:04.068928    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7651-faeea7" event={"ID":"305278b5-188f-4667-8c91-d379ca81391e","Type":"ContainerStarted","Data":"61a6c27530ebd09160a5d82a1a4d31d7f7a3079751273209f6a3b44578a85dbb"}
Sep 15 22:47:04.069325 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:04.068976    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7651-faeea7" event={"ID":"305278b5-188f-4667-8c91-d379ca81391e","Type":"ContainerStarted","Data":"4801470a9e205d78cbf1167aaa2289d0818c070ad5d8e09de94da5be568eab88"}
Sep 15 22:47:04.069325 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:04.068972    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7651-b2e5a0" podUID="e8719cc6-db15-4b1c-8100-844cfec5690f" containerName="test-curl-7651-b2e5a0" containerID="cri-o://eb4241ee3affa02ef68d53c9a25b991046710391f8f85353a0d77f9e7e700513" gracePeriod=2
Sep 15 22:47:04.071169 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:47:04.071143    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7651-b2e5a0\" is forbidden: User \"system:node:ip-10-0-128-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-208.ec2.internal' and this object" podUID="e8719cc6-db15-4b1c-8100-844cfec5690f" pod="opendatahub/test-curl-7651-b2e5a0"
Sep 15 22:47:04.084465 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:04.084424    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7651-faeea7" podStartSLOduration=2.084409296 podStartE2EDuration="2.084409296s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:47:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:47:04.083336764 +0000 UTC m=+1173.943048048" watchObservedRunningTime="2026-09-15 22:47:04.084409296 +0000 UTC m=+1173.944120580"
Sep 15 22:47:04.085006 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:47:04.084982    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7651-b2e5a0\" is forbidden: User \"system:node:ip-10-0-128-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-208.ec2.internal' and this object" podUID="e8719cc6-db15-4b1c-8100-844cfec5690f" pod="opendatahub/test-curl-7651-b2e5a0"
Sep 15 22:47:04.201311 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:04.201282    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-5a3a64"
Sep 15 22:47:04.203209 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:47:04.203182    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7651-b2e5a0\" is forbidden: User \"system:node:ip-10-0-128-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-208.ec2.internal' and this object" podUID="e8719cc6-db15-4b1c-8100-844cfec5690f" pod="opendatahub/test-curl-7651-b2e5a0"
Sep 15 22:47:04.343696 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:04.343626    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6nj79\" (UniqueName: \"kubernetes.io/projected/6f539b1b-7219-40d5-a3b9-d579243008a4-kube-api-access-6nj79\") pod \"6f539b1b-7219-40d5-a3b9-d579243008a4\" (UID: \"6f539b1b-7219-40d5-a3b9-d579243008a4\") "
Sep 15 22:47:04.345517 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:04.345497    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f539b1b-7219-40d5-a3b9-d579243008a4-kube-api-access-6nj79" (OuterVolumeSpecName: "kube-api-access-6nj79") pod "6f539b1b-7219-40d5-a3b9-d579243008a4" (UID: "6f539b1b-7219-40d5-a3b9-d579243008a4"). InnerVolumeSpecName "kube-api-access-6nj79". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:47:04.445294 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:04.445272    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nj79\" (UniqueName: \"kubernetes.io/projected/6f539b1b-7219-40d5-a3b9-d579243008a4-kube-api-access-6nj79\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:47:04.591471 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:04.591436    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f539b1b-7219-40d5-a3b9-d579243008a4" path="/var/lib/kubelet/pods/6f539b1b-7219-40d5-a3b9-d579243008a4/volumes"
Sep 15 22:47:04.653191 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:04.653162    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7651-faeea7"]
Sep 15 22:47:04.658304 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:04.658275    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7651-faeea7"]
Sep 15 22:47:05.077239 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:05.077203    2565 generic.go:356] "Generic (PLEG): container finished" podID="6f539b1b-7219-40d5-a3b9-d579243008a4" containerID="ee5d65cced23480c109c8bdb54a407f8dcbe2c157adf8e1538b4350e9a1a0748" exitCode=137
Sep 15 22:47:05.077669 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:05.077244    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-5a3a64"
Sep 15 22:47:05.077669 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:05.077299    2565 scope.go:117] "RemoveContainer" containerID="ee5d65cced23480c109c8bdb54a407f8dcbe2c157adf8e1538b4350e9a1a0748"
Sep 15 22:47:05.077669 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:47:05.077530    2565 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7651-faeea7" container="test-curl-7651-faeea7"
Sep 15 22:47:05.086379 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:05.086358    2565 scope.go:117] "RemoveContainer" containerID="ee5d65cced23480c109c8bdb54a407f8dcbe2c157adf8e1538b4350e9a1a0748"
Sep 15 22:47:05.086629 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:47:05.086613    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee5d65cced23480c109c8bdb54a407f8dcbe2c157adf8e1538b4350e9a1a0748\": container with ID starting with ee5d65cced23480c109c8bdb54a407f8dcbe2c157adf8e1538b4350e9a1a0748 not found: ID does not exist" containerID="ee5d65cced23480c109c8bdb54a407f8dcbe2c157adf8e1538b4350e9a1a0748"
Sep 15 22:47:05.086687 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:05.086637    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee5d65cced23480c109c8bdb54a407f8dcbe2c157adf8e1538b4350e9a1a0748"} err="failed to get container status \"ee5d65cced23480c109c8bdb54a407f8dcbe2c157adf8e1538b4350e9a1a0748\": rpc error: code = NotFound desc = could not find container \"ee5d65cced23480c109c8bdb54a407f8dcbe2c157adf8e1538b4350e9a1a0748\": container with ID starting with ee5d65cced23480c109c8bdb54a407f8dcbe2c157adf8e1538b4350e9a1a0748 not found: ID does not exist"
Sep 15 22:47:06.085633 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:06.085600    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7651-faeea7" podUID="305278b5-188f-4667-8c91-d379ca81391e" containerName="test-curl-7651-faeea7" containerID="cri-o://61a6c27530ebd09160a5d82a1a4d31d7f7a3079751273209f6a3b44578a85dbb" gracePeriod=2
Sep 15 22:47:06.087392 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:47:06.087363    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7651-faeea7\" is forbidden: User \"system:node:ip-10-0-128-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-208.ec2.internal' and this object" podUID="305278b5-188f-4667-8c91-d379ca81391e" pod="opendatahub/test-curl-7651-faeea7"
Sep 15 22:47:06.266090 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:06.266066    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-b2e5a0"
Sep 15 22:47:06.267966 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:47:06.267939    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7651-faeea7\" is forbidden: User \"system:node:ip-10-0-128-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-208.ec2.internal' and this object" podUID="305278b5-188f-4667-8c91-d379ca81391e" pod="opendatahub/test-curl-7651-faeea7"
Sep 15 22:47:06.362530 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:06.362452    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5l9jv\" (UniqueName: \"kubernetes.io/projected/e8719cc6-db15-4b1c-8100-844cfec5690f-kube-api-access-5l9jv\") pod \"e8719cc6-db15-4b1c-8100-844cfec5690f\" (UID: \"e8719cc6-db15-4b1c-8100-844cfec5690f\") "
Sep 15 22:47:06.364541 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:06.364504    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8719cc6-db15-4b1c-8100-844cfec5690f-kube-api-access-5l9jv" (OuterVolumeSpecName: "kube-api-access-5l9jv") pod "e8719cc6-db15-4b1c-8100-844cfec5690f" (UID: "e8719cc6-db15-4b1c-8100-844cfec5690f"). InnerVolumeSpecName "kube-api-access-5l9jv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:47:06.463844 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:06.463813    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5l9jv\" (UniqueName: \"kubernetes.io/projected/e8719cc6-db15-4b1c-8100-844cfec5690f-kube-api-access-5l9jv\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:47:06.591866 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:06.591834    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8719cc6-db15-4b1c-8100-844cfec5690f" path="/var/lib/kubelet/pods/e8719cc6-db15-4b1c-8100-844cfec5690f/volumes"
Sep 15 22:47:07.092983 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:07.092949    2565 generic.go:356] "Generic (PLEG): container finished" podID="e8719cc6-db15-4b1c-8100-844cfec5690f" containerID="eb4241ee3affa02ef68d53c9a25b991046710391f8f85353a0d77f9e7e700513" exitCode=137
Sep 15 22:47:07.093427 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:07.092989    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-b2e5a0"
Sep 15 22:47:07.093427 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:07.093070    2565 scope.go:117] "RemoveContainer" containerID="eb4241ee3affa02ef68d53c9a25b991046710391f8f85353a0d77f9e7e700513"
Sep 15 22:47:07.102340 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:07.102318    2565 scope.go:117] "RemoveContainer" containerID="eb4241ee3affa02ef68d53c9a25b991046710391f8f85353a0d77f9e7e700513"
Sep 15 22:47:07.102638 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:47:07.102605    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb4241ee3affa02ef68d53c9a25b991046710391f8f85353a0d77f9e7e700513\": container with ID starting with eb4241ee3affa02ef68d53c9a25b991046710391f8f85353a0d77f9e7e700513 not found: ID does not exist" containerID="eb4241ee3affa02ef68d53c9a25b991046710391f8f85353a0d77f9e7e700513"
Sep 15 22:47:07.102732 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:07.102651    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb4241ee3affa02ef68d53c9a25b991046710391f8f85353a0d77f9e7e700513"} err="failed to get container status \"eb4241ee3affa02ef68d53c9a25b991046710391f8f85353a0d77f9e7e700513\": rpc error: code = NotFound desc = could not find container \"eb4241ee3affa02ef68d53c9a25b991046710391f8f85353a0d77f9e7e700513\": container with ID starting with eb4241ee3affa02ef68d53c9a25b991046710391f8f85353a0d77f9e7e700513 not found: ID does not exist"
Sep 15 22:47:08.271119 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:08.271094    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-faeea7"
Sep 15 22:47:08.382853 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:08.382770    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ps57n\" (UniqueName: \"kubernetes.io/projected/305278b5-188f-4667-8c91-d379ca81391e-kube-api-access-ps57n\") pod \"305278b5-188f-4667-8c91-d379ca81391e\" (UID: \"305278b5-188f-4667-8c91-d379ca81391e\") "
Sep 15 22:47:08.384828 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:08.384805    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/305278b5-188f-4667-8c91-d379ca81391e-kube-api-access-ps57n" (OuterVolumeSpecName: "kube-api-access-ps57n") pod "305278b5-188f-4667-8c91-d379ca81391e" (UID: "305278b5-188f-4667-8c91-d379ca81391e"). InnerVolumeSpecName "kube-api-access-ps57n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:47:08.483977 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:08.483947    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ps57n\" (UniqueName: \"kubernetes.io/projected/305278b5-188f-4667-8c91-d379ca81391e-kube-api-access-ps57n\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:47:08.591144 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:08.591111    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="305278b5-188f-4667-8c91-d379ca81391e" path="/var/lib/kubelet/pods/305278b5-188f-4667-8c91-d379ca81391e/volumes"
Sep 15 22:47:09.109576 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:09.109542    2565 generic.go:356] "Generic (PLEG): container finished" podID="305278b5-188f-4667-8c91-d379ca81391e" containerID="61a6c27530ebd09160a5d82a1a4d31d7f7a3079751273209f6a3b44578a85dbb" exitCode=137
Sep 15 22:47:09.109754 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:09.109603    2565 scope.go:117] "RemoveContainer" containerID="61a6c27530ebd09160a5d82a1a4d31d7f7a3079751273209f6a3b44578a85dbb"
Sep 15 22:47:09.109754 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:09.109606    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7651-faeea7"
Sep 15 22:47:09.119194 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:09.119013    2565 scope.go:117] "RemoveContainer" containerID="61a6c27530ebd09160a5d82a1a4d31d7f7a3079751273209f6a3b44578a85dbb"
Sep 15 22:47:09.119350 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:47:09.119330    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61a6c27530ebd09160a5d82a1a4d31d7f7a3079751273209f6a3b44578a85dbb\": container with ID starting with 61a6c27530ebd09160a5d82a1a4d31d7f7a3079751273209f6a3b44578a85dbb not found: ID does not exist" containerID="61a6c27530ebd09160a5d82a1a4d31d7f7a3079751273209f6a3b44578a85dbb"
Sep 15 22:47:09.119396 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:09.119360    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61a6c27530ebd09160a5d82a1a4d31d7f7a3079751273209f6a3b44578a85dbb"} err="failed to get container status \"61a6c27530ebd09160a5d82a1a4d31d7f7a3079751273209f6a3b44578a85dbb\": rpc error: code = NotFound desc = could not find container \"61a6c27530ebd09160a5d82a1a4d31d7f7a3079751273209f6a3b44578a85dbb\": container with ID starting with 61a6c27530ebd09160a5d82a1a4d31d7f7a3079751273209f6a3b44578a85dbb not found: ID does not exist"
Sep 15 22:47:13.868008 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:13.867974    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf"]
Sep 15 22:47:13.868716 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:13.868698    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="305278b5-188f-4667-8c91-d379ca81391e" containerName="test-curl-7651-faeea7"
Sep 15 22:47:13.868770 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:13.868720    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="305278b5-188f-4667-8c91-d379ca81391e" containerName="test-curl-7651-faeea7"
Sep 15 22:47:13.868867 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:13.868856    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="305278b5-188f-4667-8c91-d379ca81391e" containerName="test-curl-7651-faeea7"
Sep 15 22:47:13.873738 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:13.873717    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf"
Sep 15 22:47:13.875986 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:13.875961    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-6cdcda22-serving-cert\""
Sep 15 22:47:13.881738 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:13.881717    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf"]
Sep 15 22:47:14.039376 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:14.039340    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c99c4271-25a8-4e71-a6ee-7d62b78378c9-maas-api-tls\") pod \"maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf\" (UID: \"c99c4271-25a8-4e71-a6ee-7d62b78378c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf"
Sep 15 22:47:14.039538 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:14.039471    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c99c4271-25a8-4e71-a6ee-7d62b78378c9-metrics-certs\") pod \"maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf\" (UID: \"c99c4271-25a8-4e71-a6ee-7d62b78378c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf"
Sep 15 22:47:14.039538 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:14.039522    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hzzw\" (UniqueName: \"kubernetes.io/projected/c99c4271-25a8-4e71-a6ee-7d62b78378c9-kube-api-access-9hzzw\") pod \"maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf\" (UID: \"c99c4271-25a8-4e71-a6ee-7d62b78378c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf"
Sep 15 22:47:14.039618 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:14.039548    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c99c4271-25a8-4e71-a6ee-7d62b78378c9-openshift-service-ca\") pod \"maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf\" (UID: \"c99c4271-25a8-4e71-a6ee-7d62b78378c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf"
Sep 15 22:47:14.140522 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:14.140441    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c99c4271-25a8-4e71-a6ee-7d62b78378c9-metrics-certs\") pod \"maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf\" (UID: \"c99c4271-25a8-4e71-a6ee-7d62b78378c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf"
Sep 15 22:47:14.140522 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:14.140489    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hzzw\" (UniqueName: \"kubernetes.io/projected/c99c4271-25a8-4e71-a6ee-7d62b78378c9-kube-api-access-9hzzw\") pod \"maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf\" (UID: \"c99c4271-25a8-4e71-a6ee-7d62b78378c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf"
Sep 15 22:47:14.140709 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:14.140618    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c99c4271-25a8-4e71-a6ee-7d62b78378c9-openshift-service-ca\") pod \"maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf\" (UID: \"c99c4271-25a8-4e71-a6ee-7d62b78378c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf"
Sep 15 22:47:14.140709 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:14.140681    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c99c4271-25a8-4e71-a6ee-7d62b78378c9-maas-api-tls\") pod \"maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf\" (UID: \"c99c4271-25a8-4e71-a6ee-7d62b78378c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf"
Sep 15 22:47:14.140861 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:47:14.140841    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-6cdcda22-serving-cert: secret "maas-api-e2e-mt-6cdcda22-serving-cert" not found
Sep 15 22:47:14.140951 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:47:14.140937    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c99c4271-25a8-4e71-a6ee-7d62b78378c9-maas-api-tls podName:c99c4271-25a8-4e71-a6ee-7d62b78378c9 nodeName:}" failed. No retries permitted until 2026-09-15 22:47:14.64091256 +0000 UTC m=+1184.500623839 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c99c4271-25a8-4e71-a6ee-7d62b78378c9-maas-api-tls") pod "maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf" (UID: "c99c4271-25a8-4e71-a6ee-7d62b78378c9") : secret "maas-api-e2e-mt-6cdcda22-serving-cert" not found
Sep 15 22:47:14.141297 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:14.141277    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c99c4271-25a8-4e71-a6ee-7d62b78378c9-openshift-service-ca\") pod \"maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf\" (UID: \"c99c4271-25a8-4e71-a6ee-7d62b78378c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf"
Sep 15 22:47:14.142856 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:14.142838    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c99c4271-25a8-4e71-a6ee-7d62b78378c9-metrics-certs\") pod \"maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf\" (UID: \"c99c4271-25a8-4e71-a6ee-7d62b78378c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf"
Sep 15 22:47:14.148439 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:14.148421    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hzzw\" (UniqueName: \"kubernetes.io/projected/c99c4271-25a8-4e71-a6ee-7d62b78378c9-kube-api-access-9hzzw\") pod \"maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf\" (UID: \"c99c4271-25a8-4e71-a6ee-7d62b78378c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf"
Sep 15 22:47:14.646279 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:14.646241    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c99c4271-25a8-4e71-a6ee-7d62b78378c9-maas-api-tls\") pod \"maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf\" (UID: \"c99c4271-25a8-4e71-a6ee-7d62b78378c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf"
Sep 15 22:47:14.648623 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:14.648602    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c99c4271-25a8-4e71-a6ee-7d62b78378c9-maas-api-tls\") pod \"maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf\" (UID: \"c99c4271-25a8-4e71-a6ee-7d62b78378c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf"
Sep 15 22:47:14.787132 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:14.787099    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf"
Sep 15 22:47:15.067417 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:15.067394    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf"]
Sep 15 22:47:15.068305 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:47:15.068276    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc99c4271_25a8_4e71_a6ee_7d62b78378c9.slice/crio-9dae4efe54ab6b393402491d17e5472cf12c5ce347d66ea47f7e2d1563896121 WatchSource:0}: Error finding container 9dae4efe54ab6b393402491d17e5472cf12c5ce347d66ea47f7e2d1563896121: Status 404 returned error can't find the container with id 9dae4efe54ab6b393402491d17e5472cf12c5ce347d66ea47f7e2d1563896121
Sep 15 22:47:15.150096 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:15.150065    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf" event={"ID":"c99c4271-25a8-4e71-a6ee-7d62b78378c9","Type":"ContainerStarted","Data":"9dae4efe54ab6b393402491d17e5472cf12c5ce347d66ea47f7e2d1563896121"}
Sep 15 22:47:16.157957 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:16.157922    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf" event={"ID":"c99c4271-25a8-4e71-a6ee-7d62b78378c9","Type":"ContainerStarted","Data":"7f23d247d40271a19d3cbcc286058e812d95d8598aa64a68c678cdaa5ca54d54"}
Sep 15 22:47:16.158340 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:16.158058    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf"
Sep 15 22:47:16.175446 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:16.175405    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf" podStartSLOduration=2.67177049 podStartE2EDuration="3.175391179s" totalImagesPullingTime="503.620689ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:47:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:47:16.174421688 +0000 UTC m=+1186.034132984" watchObservedRunningTime="2026-09-15 22:47:16.175391179 +0000 UTC m=+1186.035102475"
Sep 15 22:47:22.169889 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:47:22.169857    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf"
Sep 15 22:48:09.787716 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:09.787685    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4"]
Sep 15 22:48:09.788218 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:09.787884    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4" podUID="743d1809-ef85-4780-88df-646139103059" containerName="maas-api" containerID="cri-o://9af813dc1a955f4172451173cebeae63e0c63c14a1b61ffe38565672c9aadf1f" gracePeriod=30
Sep 15 22:48:09.854804 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:09.854750    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-fffdae-846d8cb8b-9bsts"]
Sep 15 22:48:09.855051 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:09.855008    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-fffdae-846d8cb8b-9bsts" podUID="f52bbe42-b0f6-4b1a-bfff-a8df7b5550ba" containerName="payload-pre-processing" containerID="cri-o://06975fd91b3d085301bc520018eb42107159f28347d416c79cd6f305f388757f" gracePeriod=30
Sep 15 22:48:10.137037 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.137007    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4"
Sep 15 22:48:10.151266 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.151238    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-fffdae-846d8cb8b-9bsts"
Sep 15 22:48:10.276698 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.276663    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/743d1809-ef85-4780-88df-646139103059-maas-api-tls\") pod \"743d1809-ef85-4780-88df-646139103059\" (UID: \"743d1809-ef85-4780-88df-646139103059\") "
Sep 15 22:48:10.276698 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.276705    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/743d1809-ef85-4780-88df-646139103059-metrics-certs\") pod \"743d1809-ef85-4780-88df-646139103059\" (UID: \"743d1809-ef85-4780-88df-646139103059\") "
Sep 15 22:48:10.276949 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.276905    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f52bbe42-b0f6-4b1a-bfff-a8df7b5550ba-plugins-config-volume\") pod \"f52bbe42-b0f6-4b1a-bfff-a8df7b5550ba\" (UID: \"f52bbe42-b0f6-4b1a-bfff-a8df7b5550ba\") "
Sep 15 22:48:10.277018 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.276975    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z68th\" (UniqueName: \"kubernetes.io/projected/743d1809-ef85-4780-88df-646139103059-kube-api-access-z68th\") pod \"743d1809-ef85-4780-88df-646139103059\" (UID: \"743d1809-ef85-4780-88df-646139103059\") "
Sep 15 22:48:10.277018 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.277012    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/743d1809-ef85-4780-88df-646139103059-openshift-service-ca\") pod \"743d1809-ef85-4780-88df-646139103059\" (UID: \"743d1809-ef85-4780-88df-646139103059\") "
Sep 15 22:48:10.277166 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.277150    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mksxd\" (UniqueName: \"kubernetes.io/projected/f52bbe42-b0f6-4b1a-bfff-a8df7b5550ba-kube-api-access-mksxd\") pod \"f52bbe42-b0f6-4b1a-bfff-a8df7b5550ba\" (UID: \"f52bbe42-b0f6-4b1a-bfff-a8df7b5550ba\") "
Sep 15 22:48:10.277354 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.277324    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f52bbe42-b0f6-4b1a-bfff-a8df7b5550ba-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f52bbe42-b0f6-4b1a-bfff-a8df7b5550ba" (UID: "f52bbe42-b0f6-4b1a-bfff-a8df7b5550ba"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:48:10.277497 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.277445    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/743d1809-ef85-4780-88df-646139103059-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "743d1809-ef85-4780-88df-646139103059" (UID: "743d1809-ef85-4780-88df-646139103059"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:48:10.277634 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.277563    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f52bbe42-b0f6-4b1a-bfff-a8df7b5550ba-plugins-config-volume\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:48:10.277634 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.277584    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/743d1809-ef85-4780-88df-646139103059-openshift-service-ca\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:48:10.278999 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.278966    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/743d1809-ef85-4780-88df-646139103059-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "743d1809-ef85-4780-88df-646139103059" (UID: "743d1809-ef85-4780-88df-646139103059"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:48:10.279196 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.279172    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f52bbe42-b0f6-4b1a-bfff-a8df7b5550ba-kube-api-access-mksxd" (OuterVolumeSpecName: "kube-api-access-mksxd") pod "f52bbe42-b0f6-4b1a-bfff-a8df7b5550ba" (UID: "f52bbe42-b0f6-4b1a-bfff-a8df7b5550ba"). InnerVolumeSpecName "kube-api-access-mksxd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:48:10.279365 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.279344    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/743d1809-ef85-4780-88df-646139103059-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "743d1809-ef85-4780-88df-646139103059" (UID: "743d1809-ef85-4780-88df-646139103059"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:48:10.279365 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.279356    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/743d1809-ef85-4780-88df-646139103059-kube-api-access-z68th" (OuterVolumeSpecName: "kube-api-access-z68th") pod "743d1809-ef85-4780-88df-646139103059" (UID: "743d1809-ef85-4780-88df-646139103059"). InnerVolumeSpecName "kube-api-access-z68th". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:48:10.378509 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.378438    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mksxd\" (UniqueName: \"kubernetes.io/projected/f52bbe42-b0f6-4b1a-bfff-a8df7b5550ba-kube-api-access-mksxd\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:48:10.378509 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.378466    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/743d1809-ef85-4780-88df-646139103059-maas-api-tls\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:48:10.378509 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.378476    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/743d1809-ef85-4780-88df-646139103059-metrics-certs\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:48:10.378509 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.378485    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z68th\" (UniqueName: \"kubernetes.io/projected/743d1809-ef85-4780-88df-646139103059-kube-api-access-z68th\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:48:10.499684 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.499650    2565 generic.go:356] "Generic (PLEG): container finished" podID="f52bbe42-b0f6-4b1a-bfff-a8df7b5550ba" containerID="06975fd91b3d085301bc520018eb42107159f28347d416c79cd6f305f388757f" exitCode=0
Sep 15 22:48:10.499870 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.499729    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-fffdae-846d8cb8b-9bsts"
Sep 15 22:48:10.499870 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.499737    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-fffdae-846d8cb8b-9bsts" event={"ID":"f52bbe42-b0f6-4b1a-bfff-a8df7b5550ba","Type":"ContainerDied","Data":"06975fd91b3d085301bc520018eb42107159f28347d416c79cd6f305f388757f"}
Sep 15 22:48:10.499870 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.499830    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-fffdae-846d8cb8b-9bsts" event={"ID":"f52bbe42-b0f6-4b1a-bfff-a8df7b5550ba","Type":"ContainerDied","Data":"7d01df4d943e1b47920be5a72633de49b12c74380d0fe249b9e21aff44493594"}
Sep 15 22:48:10.499870 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.499860    2565 scope.go:117] "RemoveContainer" containerID="06975fd91b3d085301bc520018eb42107159f28347d416c79cd6f305f388757f"
Sep 15 22:48:10.502065 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.501309    2565 generic.go:356] "Generic (PLEG): container finished" podID="743d1809-ef85-4780-88df-646139103059" containerID="9af813dc1a955f4172451173cebeae63e0c63c14a1b61ffe38565672c9aadf1f" exitCode=0
Sep 15 22:48:10.502065 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.501365    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4" event={"ID":"743d1809-ef85-4780-88df-646139103059","Type":"ContainerDied","Data":"9af813dc1a955f4172451173cebeae63e0c63c14a1b61ffe38565672c9aadf1f"}
Sep 15 22:48:10.502065 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.501405    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4" event={"ID":"743d1809-ef85-4780-88df-646139103059","Type":"ContainerDied","Data":"83d4a32bcf3c40b2580a66f33c24c7828ab3d732f33f444233c507ea96488163"}
Sep 15 22:48:10.502065 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.501482    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4"
Sep 15 22:48:10.511065 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.511044    2565 scope.go:117] "RemoveContainer" containerID="06975fd91b3d085301bc520018eb42107159f28347d416c79cd6f305f388757f"
Sep 15 22:48:10.511405 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:48:10.511348    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06975fd91b3d085301bc520018eb42107159f28347d416c79cd6f305f388757f\": container with ID starting with 06975fd91b3d085301bc520018eb42107159f28347d416c79cd6f305f388757f not found: ID does not exist" containerID="06975fd91b3d085301bc520018eb42107159f28347d416c79cd6f305f388757f"
Sep 15 22:48:10.511405 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.511373    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06975fd91b3d085301bc520018eb42107159f28347d416c79cd6f305f388757f"} err="failed to get container status \"06975fd91b3d085301bc520018eb42107159f28347d416c79cd6f305f388757f\": rpc error: code = NotFound desc = could not find container \"06975fd91b3d085301bc520018eb42107159f28347d416c79cd6f305f388757f\": container with ID starting with 06975fd91b3d085301bc520018eb42107159f28347d416c79cd6f305f388757f not found: ID does not exist"
Sep 15 22:48:10.511405 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.511388    2565 scope.go:117] "RemoveContainer" containerID="9af813dc1a955f4172451173cebeae63e0c63c14a1b61ffe38565672c9aadf1f"
Sep 15 22:48:10.519912 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.519896    2565 scope.go:117] "RemoveContainer" containerID="9af813dc1a955f4172451173cebeae63e0c63c14a1b61ffe38565672c9aadf1f"
Sep 15 22:48:10.520179 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:48:10.520161    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9af813dc1a955f4172451173cebeae63e0c63c14a1b61ffe38565672c9aadf1f\": container with ID starting with 9af813dc1a955f4172451173cebeae63e0c63c14a1b61ffe38565672c9aadf1f not found: ID does not exist" containerID="9af813dc1a955f4172451173cebeae63e0c63c14a1b61ffe38565672c9aadf1f"
Sep 15 22:48:10.520253 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.520185    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9af813dc1a955f4172451173cebeae63e0c63c14a1b61ffe38565672c9aadf1f"} err="failed to get container status \"9af813dc1a955f4172451173cebeae63e0c63c14a1b61ffe38565672c9aadf1f\": rpc error: code = NotFound desc = could not find container \"9af813dc1a955f4172451173cebeae63e0c63c14a1b61ffe38565672c9aadf1f\": container with ID starting with 9af813dc1a955f4172451173cebeae63e0c63c14a1b61ffe38565672c9aadf1f not found: ID does not exist"
Sep 15 22:48:10.526483 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.526454    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-fffdae-846d8cb8b-9bsts"]
Sep 15 22:48:10.529972 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.529953    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-fffdae-846d8cb8b-9bsts"]
Sep 15 22:48:10.543362 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.543342    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4"]
Sep 15 22:48:10.545827 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.545808    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fffdae-775f59f75b-rltd4"]
Sep 15 22:48:10.591728 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.591704    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="743d1809-ef85-4780-88df-646139103059" path="/var/lib/kubelet/pods/743d1809-ef85-4780-88df-646139103059/volumes"
Sep 15 22:48:10.592072 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:48:10.592058    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f52bbe42-b0f6-4b1a-bfff-a8df7b5550ba" path="/var/lib/kubelet/pods/f52bbe42-b0f6-4b1a-bfff-a8df7b5550ba/volumes"
Sep 15 22:49:10.374458 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:10.374424    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-765e6d-29825205-jlgnj"]
Sep 15 22:49:10.377289 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:10.377269    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-765e6d-29825205-jlgnj"]
Sep 15 22:49:10.409642 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:10.409610    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-765e6d-7cc87d5854-kdqmd"]
Sep 15 22:49:10.410133 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:10.410078    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-765e6d-7cc87d5854-kdqmd" podUID="ec30af64-6264-4f7a-9ff2-b3be5d20b42a" containerName="payload-pre-processing" containerID="cri-o://16d9782645be11bb0d4036c0f1f3c34ea5a30341e2e1aaaebd50fb09089e7e39" gracePeriod=30
Sep 15 22:49:10.594333 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:10.594085    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50725131-d1e7-4fd4-aaed-4f2933313bd7" path="/var/lib/kubelet/pods/50725131-d1e7-4fd4-aaed-4f2933313bd7/volumes"
Sep 15 22:49:10.655208 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:10.655181    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-765e6d-7cc87d5854-kdqmd"
Sep 15 22:49:10.775399 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:10.775370    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8bf74\" (UniqueName: \"kubernetes.io/projected/ec30af64-6264-4f7a-9ff2-b3be5d20b42a-kube-api-access-8bf74\") pod \"ec30af64-6264-4f7a-9ff2-b3be5d20b42a\" (UID: \"ec30af64-6264-4f7a-9ff2-b3be5d20b42a\") "
Sep 15 22:49:10.775569 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:10.775438    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec30af64-6264-4f7a-9ff2-b3be5d20b42a-plugins-config-volume\") pod \"ec30af64-6264-4f7a-9ff2-b3be5d20b42a\" (UID: \"ec30af64-6264-4f7a-9ff2-b3be5d20b42a\") "
Sep 15 22:49:10.775824 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:10.775802    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec30af64-6264-4f7a-9ff2-b3be5d20b42a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ec30af64-6264-4f7a-9ff2-b3be5d20b42a" (UID: "ec30af64-6264-4f7a-9ff2-b3be5d20b42a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:49:10.777543 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:10.777524    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec30af64-6264-4f7a-9ff2-b3be5d20b42a-kube-api-access-8bf74" (OuterVolumeSpecName: "kube-api-access-8bf74") pod "ec30af64-6264-4f7a-9ff2-b3be5d20b42a" (UID: "ec30af64-6264-4f7a-9ff2-b3be5d20b42a"). InnerVolumeSpecName "kube-api-access-8bf74". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:49:10.876195 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:10.876163    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8bf74\" (UniqueName: \"kubernetes.io/projected/ec30af64-6264-4f7a-9ff2-b3be5d20b42a-kube-api-access-8bf74\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:49:10.876195 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:10.876191    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec30af64-6264-4f7a-9ff2-b3be5d20b42a-plugins-config-volume\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:49:10.878635 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:10.878608    2565 generic.go:356] "Generic (PLEG): container finished" podID="ec30af64-6264-4f7a-9ff2-b3be5d20b42a" containerID="16d9782645be11bb0d4036c0f1f3c34ea5a30341e2e1aaaebd50fb09089e7e39" exitCode=0
Sep 15 22:49:10.878769 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:10.878653    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-765e6d-7cc87d5854-kdqmd"
Sep 15 22:49:10.878769 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:10.878675    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-765e6d-7cc87d5854-kdqmd" event={"ID":"ec30af64-6264-4f7a-9ff2-b3be5d20b42a","Type":"ContainerDied","Data":"16d9782645be11bb0d4036c0f1f3c34ea5a30341e2e1aaaebd50fb09089e7e39"}
Sep 15 22:49:10.878769 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:10.878716    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-765e6d-7cc87d5854-kdqmd" event={"ID":"ec30af64-6264-4f7a-9ff2-b3be5d20b42a","Type":"ContainerDied","Data":"38002ef4fd9638b6602a85579095b47b6d6b8d867e879d96593d2ecb07542666"}
Sep 15 22:49:10.878769 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:10.878736    2565 scope.go:117] "RemoveContainer" containerID="16d9782645be11bb0d4036c0f1f3c34ea5a30341e2e1aaaebd50fb09089e7e39"
Sep 15 22:49:10.888933 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:10.888917    2565 scope.go:117] "RemoveContainer" containerID="16d9782645be11bb0d4036c0f1f3c34ea5a30341e2e1aaaebd50fb09089e7e39"
Sep 15 22:49:10.889212 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:49:10.889193    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16d9782645be11bb0d4036c0f1f3c34ea5a30341e2e1aaaebd50fb09089e7e39\": container with ID starting with 16d9782645be11bb0d4036c0f1f3c34ea5a30341e2e1aaaebd50fb09089e7e39 not found: ID does not exist" containerID="16d9782645be11bb0d4036c0f1f3c34ea5a30341e2e1aaaebd50fb09089e7e39"
Sep 15 22:49:10.889280 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:10.889220    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16d9782645be11bb0d4036c0f1f3c34ea5a30341e2e1aaaebd50fb09089e7e39"} err="failed to get container status \"16d9782645be11bb0d4036c0f1f3c34ea5a30341e2e1aaaebd50fb09089e7e39\": rpc error: code = NotFound desc = could not find container \"16d9782645be11bb0d4036c0f1f3c34ea5a30341e2e1aaaebd50fb09089e7e39\": container with ID starting with 16d9782645be11bb0d4036c0f1f3c34ea5a30341e2e1aaaebd50fb09089e7e39 not found: ID does not exist"
Sep 15 22:49:10.901156 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:10.901130    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-765e6d-7cc87d5854-kdqmd"]
Sep 15 22:49:10.907369 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:10.907348    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-765e6d-7cc87d5854-kdqmd"]
Sep 15 22:49:12.590870 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:12.590836    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec30af64-6264-4f7a-9ff2-b3be5d20b42a" path="/var/lib/kubelet/pods/ec30af64-6264-4f7a-9ff2-b3be5d20b42a/volumes"
Sep 15 22:49:24.922279 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:24.922231    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr"]
Sep 15 22:49:24.922929 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:24.922903    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="743d1809-ef85-4780-88df-646139103059" containerName="maas-api"
Sep 15 22:49:24.922929 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:24.922931    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="743d1809-ef85-4780-88df-646139103059" containerName="maas-api"
Sep 15 22:49:24.923110 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:24.922957    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f52bbe42-b0f6-4b1a-bfff-a8df7b5550ba" containerName="payload-pre-processing"
Sep 15 22:49:24.923110 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:24.922966    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f52bbe42-b0f6-4b1a-bfff-a8df7b5550ba" containerName="payload-pre-processing"
Sep 15 22:49:24.923110 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:24.922997    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec30af64-6264-4f7a-9ff2-b3be5d20b42a" containerName="payload-pre-processing"
Sep 15 22:49:24.923110 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:24.923005    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec30af64-6264-4f7a-9ff2-b3be5d20b42a" containerName="payload-pre-processing"
Sep 15 22:49:24.923321 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:24.923135    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec30af64-6264-4f7a-9ff2-b3be5d20b42a" containerName="payload-pre-processing"
Sep 15 22:49:24.923321 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:24.923156    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f52bbe42-b0f6-4b1a-bfff-a8df7b5550ba" containerName="payload-pre-processing"
Sep 15 22:49:24.923321 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:24.923165    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="743d1809-ef85-4780-88df-646139103059" containerName="maas-api"
Sep 15 22:49:24.927922 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:24.927900    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr"
Sep 15 22:49:24.929834 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:24.929811    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-0016d4-serving-cert\""
Sep 15 22:49:24.950234 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:24.950188    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr"]
Sep 15 22:49:25.017277 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:25.017237    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8939264-e179-4809-b361-91388cf1dcce-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr\" (UID: \"a8939264-e179-4809-b361-91388cf1dcce\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr"
Sep 15 22:49:25.017456 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:25.017310    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8939264-e179-4809-b361-91388cf1dcce-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr\" (UID: \"a8939264-e179-4809-b361-91388cf1dcce\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr"
Sep 15 22:49:25.017456 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:25.017372    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8939264-e179-4809-b361-91388cf1dcce-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr\" (UID: \"a8939264-e179-4809-b361-91388cf1dcce\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr"
Sep 15 22:49:25.017594 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:25.017445    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvrmw\" (UniqueName: \"kubernetes.io/projected/a8939264-e179-4809-b361-91388cf1dcce-kube-api-access-jvrmw\") pod \"maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr\" (UID: \"a8939264-e179-4809-b361-91388cf1dcce\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr"
Sep 15 22:49:25.118048 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:25.117990    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvrmw\" (UniqueName: \"kubernetes.io/projected/a8939264-e179-4809-b361-91388cf1dcce-kube-api-access-jvrmw\") pod \"maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr\" (UID: \"a8939264-e179-4809-b361-91388cf1dcce\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr"
Sep 15 22:49:25.118246 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:25.118098    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8939264-e179-4809-b361-91388cf1dcce-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr\" (UID: \"a8939264-e179-4809-b361-91388cf1dcce\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr"
Sep 15 22:49:25.118246 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:25.118148    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8939264-e179-4809-b361-91388cf1dcce-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr\" (UID: \"a8939264-e179-4809-b361-91388cf1dcce\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr"
Sep 15 22:49:25.118246 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:25.118190    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8939264-e179-4809-b361-91388cf1dcce-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr\" (UID: \"a8939264-e179-4809-b361-91388cf1dcce\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr"
Sep 15 22:49:25.118781 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:25.118747    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8939264-e179-4809-b361-91388cf1dcce-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr\" (UID: \"a8939264-e179-4809-b361-91388cf1dcce\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr"
Sep 15 22:49:25.120703 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:25.120680    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8939264-e179-4809-b361-91388cf1dcce-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr\" (UID: \"a8939264-e179-4809-b361-91388cf1dcce\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr"
Sep 15 22:49:25.120800 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:25.120699    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8939264-e179-4809-b361-91388cf1dcce-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr\" (UID: \"a8939264-e179-4809-b361-91388cf1dcce\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr"
Sep 15 22:49:25.125621 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:25.125596    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvrmw\" (UniqueName: \"kubernetes.io/projected/a8939264-e179-4809-b361-91388cf1dcce-kube-api-access-jvrmw\") pod \"maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr\" (UID: \"a8939264-e179-4809-b361-91388cf1dcce\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr"
Sep 15 22:49:25.246856 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:25.246779    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr"
Sep 15 22:49:25.534443 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:25.534416    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr"]
Sep 15 22:49:25.536475 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:49:25.536450    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8939264_e179_4809_b361_91388cf1dcce.slice/crio-4a2fa48e832302df14978dfd87cf17d26918b708cd15f5561604f39c088b5a8c WatchSource:0}: Error finding container 4a2fa48e832302df14978dfd87cf17d26918b708cd15f5561604f39c088b5a8c: Status 404 returned error can't find the container with id 4a2fa48e832302df14978dfd87cf17d26918b708cd15f5561604f39c088b5a8c
Sep 15 22:49:25.537948 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:25.537927    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 22:49:25.972473 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:25.972445    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr" event={"ID":"a8939264-e179-4809-b361-91388cf1dcce","Type":"ContainerStarted","Data":"4a2fa48e832302df14978dfd87cf17d26918b708cd15f5561604f39c088b5a8c"}
Sep 15 22:49:26.980362 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:26.980323    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr" event={"ID":"a8939264-e179-4809-b361-91388cf1dcce","Type":"ContainerStarted","Data":"c2c260f0ffd826b92e7a13d523c742c934b6dbfe6303f97ef1157e8c6bc627b6"}
Sep 15 22:49:26.980738 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:26.980478    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr"
Sep 15 22:49:26.997374 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:26.997324    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr" podStartSLOduration=2.6211854470000002 podStartE2EDuration="2.997310436s" totalImagesPullingTime="376.124989ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:49:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:49:26.995114632 +0000 UTC m=+1316.854825913" watchObservedRunningTime="2026-09-15 22:49:26.997310436 +0000 UTC m=+1316.857021719"
Sep 15 22:49:32.992645 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:49:32.992615    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr"
Sep 15 22:50:00.011364 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:00.011314    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-92482b-774868dc8b-l7lwz"]
Sep 15 22:50:00.016395 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:00.016372    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-92482b-774868dc8b-l7lwz"
Sep 15 22:50:00.018761 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:00.018724    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-92482b\""
Sep 15 22:50:00.018881 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:00.018783    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-92482b-dockercfg-jpbm6\""
Sep 15 22:50:00.025795 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:00.025774    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-92482b-774868dc8b-l7lwz"]
Sep 15 22:50:00.046775 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:00.046746    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-92482b-bbfb58bf5-gf25f"]
Sep 15 22:50:00.050330 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:00.050311    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-92482b-bbfb58bf5-gf25f"
Sep 15 22:50:00.060064 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:00.060040    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-92482b-bbfb58bf5-gf25f"]
Sep 15 22:50:00.159708 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:00.159680    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f8cf34a-3ee1-4080-b58a-dc5f3122a114-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-92482b-774868dc8b-l7lwz\" (UID: \"9f8cf34a-3ee1-4080-b58a-dc5f3122a114\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-92482b-774868dc8b-l7lwz"
Sep 15 22:50:00.159878 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:00.159725    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f624109-7163-47bc-9d72-cf5204a41a10-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-92482b-bbfb58bf5-gf25f\" (UID: \"5f624109-7163-47bc-9d72-cf5204a41a10\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-92482b-bbfb58bf5-gf25f"
Sep 15 22:50:00.159878 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:00.159774    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-778rs\" (UniqueName: \"kubernetes.io/projected/9f8cf34a-3ee1-4080-b58a-dc5f3122a114-kube-api-access-778rs\") pod \"payload-pre-processing-e2e-shared-b-w3-92482b-774868dc8b-l7lwz\" (UID: \"9f8cf34a-3ee1-4080-b58a-dc5f3122a114\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-92482b-774868dc8b-l7lwz"
Sep 15 22:50:00.159878 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:00.159801    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s45h8\" (UniqueName: \"kubernetes.io/projected/5f624109-7163-47bc-9d72-cf5204a41a10-kube-api-access-s45h8\") pod \"payload-processing-e2e-shared-b-w3-92482b-bbfb58bf5-gf25f\" (UID: \"5f624109-7163-47bc-9d72-cf5204a41a10\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-92482b-bbfb58bf5-gf25f"
Sep 15 22:50:00.260593 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:00.260555    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f8cf34a-3ee1-4080-b58a-dc5f3122a114-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-92482b-774868dc8b-l7lwz\" (UID: \"9f8cf34a-3ee1-4080-b58a-dc5f3122a114\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-92482b-774868dc8b-l7lwz"
Sep 15 22:50:00.260759 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:00.260612    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f624109-7163-47bc-9d72-cf5204a41a10-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-92482b-bbfb58bf5-gf25f\" (UID: \"5f624109-7163-47bc-9d72-cf5204a41a10\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-92482b-bbfb58bf5-gf25f"
Sep 15 22:50:00.260759 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:00.260639    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-778rs\" (UniqueName: \"kubernetes.io/projected/9f8cf34a-3ee1-4080-b58a-dc5f3122a114-kube-api-access-778rs\") pod \"payload-pre-processing-e2e-shared-b-w3-92482b-774868dc8b-l7lwz\" (UID: \"9f8cf34a-3ee1-4080-b58a-dc5f3122a114\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-92482b-774868dc8b-l7lwz"
Sep 15 22:50:00.260759 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:00.260664    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s45h8\" (UniqueName: \"kubernetes.io/projected/5f624109-7163-47bc-9d72-cf5204a41a10-kube-api-access-s45h8\") pod \"payload-processing-e2e-shared-b-w3-92482b-bbfb58bf5-gf25f\" (UID: \"5f624109-7163-47bc-9d72-cf5204a41a10\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-92482b-bbfb58bf5-gf25f"
Sep 15 22:50:00.261320 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:00.261301    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f8cf34a-3ee1-4080-b58a-dc5f3122a114-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-92482b-774868dc8b-l7lwz\" (UID: \"9f8cf34a-3ee1-4080-b58a-dc5f3122a114\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-92482b-774868dc8b-l7lwz"
Sep 15 22:50:00.261401 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:00.261361    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f624109-7163-47bc-9d72-cf5204a41a10-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-92482b-bbfb58bf5-gf25f\" (UID: \"5f624109-7163-47bc-9d72-cf5204a41a10\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-92482b-bbfb58bf5-gf25f"
Sep 15 22:50:00.267892 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:00.267869    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s45h8\" (UniqueName: \"kubernetes.io/projected/5f624109-7163-47bc-9d72-cf5204a41a10-kube-api-access-s45h8\") pod \"payload-processing-e2e-shared-b-w3-92482b-bbfb58bf5-gf25f\" (UID: \"5f624109-7163-47bc-9d72-cf5204a41a10\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-92482b-bbfb58bf5-gf25f"
Sep 15 22:50:00.268088 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:00.268066    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-778rs\" (UniqueName: \"kubernetes.io/projected/9f8cf34a-3ee1-4080-b58a-dc5f3122a114-kube-api-access-778rs\") pod \"payload-pre-processing-e2e-shared-b-w3-92482b-774868dc8b-l7lwz\" (UID: \"9f8cf34a-3ee1-4080-b58a-dc5f3122a114\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-92482b-774868dc8b-l7lwz"
Sep 15 22:50:00.329742 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:00.329712    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-92482b-774868dc8b-l7lwz"
Sep 15 22:50:00.361791 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:00.361757    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-92482b-bbfb58bf5-gf25f"
Sep 15 22:50:00.603187 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:00.603160    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-92482b-774868dc8b-l7lwz"]
Sep 15 22:50:00.605488 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:50:00.605457    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f8cf34a_3ee1_4080_b58a_dc5f3122a114.slice/crio-c0086f5ecfe5d6c13e3576d19faa4cea10fdf95a22a0173130a6858bafd6ff01 WatchSource:0}: Error finding container c0086f5ecfe5d6c13e3576d19faa4cea10fdf95a22a0173130a6858bafd6ff01: Status 404 returned error can't find the container with id c0086f5ecfe5d6c13e3576d19faa4cea10fdf95a22a0173130a6858bafd6ff01
Sep 15 22:50:00.647007 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:00.646984    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-92482b-bbfb58bf5-gf25f"]
Sep 15 22:50:00.648778 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:50:00.648748    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f624109_7163_47bc_9d72_cf5204a41a10.slice/crio-948f090941e98e3a13b3fd9116e51a1f25113ab5be09d86e0bf687805ef33975 WatchSource:0}: Error finding container 948f090941e98e3a13b3fd9116e51a1f25113ab5be09d86e0bf687805ef33975: Status 404 returned error can't find the container with id 948f090941e98e3a13b3fd9116e51a1f25113ab5be09d86e0bf687805ef33975
Sep 15 22:50:01.195962 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:01.195916    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-92482b-bbfb58bf5-gf25f" event={"ID":"5f624109-7163-47bc-9d72-cf5204a41a10","Type":"ContainerStarted","Data":"ce7ce1af93abfbdd036c74087f7bd4521fd94cf183e3da23482885067e1cff1e"}
Sep 15 22:50:01.196429 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:01.196130    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-92482b-bbfb58bf5-gf25f"
Sep 15 22:50:01.196429 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:01.196178    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-92482b-bbfb58bf5-gf25f" event={"ID":"5f624109-7163-47bc-9d72-cf5204a41a10","Type":"ContainerStarted","Data":"948f090941e98e3a13b3fd9116e51a1f25113ab5be09d86e0bf687805ef33975"}
Sep 15 22:50:01.197598 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:01.197575    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-92482b-774868dc8b-l7lwz" event={"ID":"9f8cf34a-3ee1-4080-b58a-dc5f3122a114","Type":"ContainerStarted","Data":"61b73a55addc31c5e416dce2da949d5c15f14f8bb4dc6945b3073dbf5bc51bd6"}
Sep 15 22:50:01.197713 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:01.197612    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-92482b-774868dc8b-l7lwz" event={"ID":"9f8cf34a-3ee1-4080-b58a-dc5f3122a114","Type":"ContainerStarted","Data":"c0086f5ecfe5d6c13e3576d19faa4cea10fdf95a22a0173130a6858bafd6ff01"}
Sep 15 22:50:01.197713 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:01.197663    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-92482b-774868dc8b-l7lwz"
Sep 15 22:50:01.216477 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:01.216427    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-92482b-bbfb58bf5-gf25f" podStartSLOduration=1.216409988 podStartE2EDuration="1.216409988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:50:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:50:01.215497826 +0000 UTC m=+1351.075209110" watchObservedRunningTime="2026-09-15 22:50:01.216409988 +0000 UTC m=+1351.076121272"
Sep 15 22:50:01.233874 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:01.233830    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-92482b-774868dc8b-l7lwz" podStartSLOduration=2.23381669 podStartE2EDuration="2.23381669s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:49:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:50:01.231354267 +0000 UTC m=+1351.091065552" watchObservedRunningTime="2026-09-15 22:50:01.23381669 +0000 UTC m=+1351.093528005"
Sep 15 22:50:01.765157 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:01.765124    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf"]
Sep 15 22:50:01.765353 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:01.765326    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf" podUID="c99c4271-25a8-4e71-a6ee-7d62b78378c9" containerName="maas-api" containerID="cri-o://7f23d247d40271a19d3cbcc286058e812d95d8598aa64a68c678cdaa5ca54d54" gracePeriod=30
Sep 15 22:50:02.032235 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:02.032207    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf"
Sep 15 22:50:02.181313 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:02.181270    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c99c4271-25a8-4e71-a6ee-7d62b78378c9-maas-api-tls\") pod \"c99c4271-25a8-4e71-a6ee-7d62b78378c9\" (UID: \"c99c4271-25a8-4e71-a6ee-7d62b78378c9\") "
Sep 15 22:50:02.181313 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:02.181315    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c99c4271-25a8-4e71-a6ee-7d62b78378c9-metrics-certs\") pod \"c99c4271-25a8-4e71-a6ee-7d62b78378c9\" (UID: \"c99c4271-25a8-4e71-a6ee-7d62b78378c9\") "
Sep 15 22:50:02.181527 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:02.181410    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9hzzw\" (UniqueName: \"kubernetes.io/projected/c99c4271-25a8-4e71-a6ee-7d62b78378c9-kube-api-access-9hzzw\") pod \"c99c4271-25a8-4e71-a6ee-7d62b78378c9\" (UID: \"c99c4271-25a8-4e71-a6ee-7d62b78378c9\") "
Sep 15 22:50:02.181527 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:02.181446    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c99c4271-25a8-4e71-a6ee-7d62b78378c9-openshift-service-ca\") pod \"c99c4271-25a8-4e71-a6ee-7d62b78378c9\" (UID: \"c99c4271-25a8-4e71-a6ee-7d62b78378c9\") "
Sep 15 22:50:02.181897 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:02.181870    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c99c4271-25a8-4e71-a6ee-7d62b78378c9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c99c4271-25a8-4e71-a6ee-7d62b78378c9" (UID: "c99c4271-25a8-4e71-a6ee-7d62b78378c9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:50:02.183367 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:02.183345    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c99c4271-25a8-4e71-a6ee-7d62b78378c9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c99c4271-25a8-4e71-a6ee-7d62b78378c9" (UID: "c99c4271-25a8-4e71-a6ee-7d62b78378c9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:50:02.183516 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:02.183494    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c99c4271-25a8-4e71-a6ee-7d62b78378c9-kube-api-access-9hzzw" (OuterVolumeSpecName: "kube-api-access-9hzzw") pod "c99c4271-25a8-4e71-a6ee-7d62b78378c9" (UID: "c99c4271-25a8-4e71-a6ee-7d62b78378c9"). InnerVolumeSpecName "kube-api-access-9hzzw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:50:02.183569 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:02.183533    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c99c4271-25a8-4e71-a6ee-7d62b78378c9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c99c4271-25a8-4e71-a6ee-7d62b78378c9" (UID: "c99c4271-25a8-4e71-a6ee-7d62b78378c9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:50:02.208248 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:02.208217    2565 generic.go:356] "Generic (PLEG): container finished" podID="c99c4271-25a8-4e71-a6ee-7d62b78378c9" containerID="7f23d247d40271a19d3cbcc286058e812d95d8598aa64a68c678cdaa5ca54d54" exitCode=0
Sep 15 22:50:02.208577 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:02.208261    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf"
Sep 15 22:50:02.208577 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:02.208300    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf" event={"ID":"c99c4271-25a8-4e71-a6ee-7d62b78378c9","Type":"ContainerDied","Data":"7f23d247d40271a19d3cbcc286058e812d95d8598aa64a68c678cdaa5ca54d54"}
Sep 15 22:50:02.208577 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:02.208370    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf" event={"ID":"c99c4271-25a8-4e71-a6ee-7d62b78378c9","Type":"ContainerDied","Data":"9dae4efe54ab6b393402491d17e5472cf12c5ce347d66ea47f7e2d1563896121"}
Sep 15 22:50:02.208577 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:02.208401    2565 scope.go:117] "RemoveContainer" containerID="7f23d247d40271a19d3cbcc286058e812d95d8598aa64a68c678cdaa5ca54d54"
Sep 15 22:50:02.218412 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:02.218398    2565 scope.go:117] "RemoveContainer" containerID="7f23d247d40271a19d3cbcc286058e812d95d8598aa64a68c678cdaa5ca54d54"
Sep 15 22:50:02.218683 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:50:02.218664    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f23d247d40271a19d3cbcc286058e812d95d8598aa64a68c678cdaa5ca54d54\": container with ID starting with 7f23d247d40271a19d3cbcc286058e812d95d8598aa64a68c678cdaa5ca54d54 not found: ID does not exist" containerID="7f23d247d40271a19d3cbcc286058e812d95d8598aa64a68c678cdaa5ca54d54"
Sep 15 22:50:02.218729 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:02.218691    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f23d247d40271a19d3cbcc286058e812d95d8598aa64a68c678cdaa5ca54d54"} err="failed to get container status \"7f23d247d40271a19d3cbcc286058e812d95d8598aa64a68c678cdaa5ca54d54\": rpc error: code = NotFound desc = could not find container \"7f23d247d40271a19d3cbcc286058e812d95d8598aa64a68c678cdaa5ca54d54\": container with ID starting with 7f23d247d40271a19d3cbcc286058e812d95d8598aa64a68c678cdaa5ca54d54 not found: ID does not exist"
Sep 15 22:50:02.232957 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:02.232934    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf"]
Sep 15 22:50:02.236746 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:02.236726    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6cdcda22-fc54c44ff-8khwf"]
Sep 15 22:50:02.282793 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:02.282742    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hzzw\" (UniqueName: \"kubernetes.io/projected/c99c4271-25a8-4e71-a6ee-7d62b78378c9-kube-api-access-9hzzw\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:50:02.282793 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:02.282764    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c99c4271-25a8-4e71-a6ee-7d62b78378c9-openshift-service-ca\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:50:02.282793 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:02.282774    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c99c4271-25a8-4e71-a6ee-7d62b78378c9-maas-api-tls\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:50:02.282793 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:02.282782    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c99c4271-25a8-4e71-a6ee-7d62b78378c9-metrics-certs\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:50:02.592742 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:02.592666    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c99c4271-25a8-4e71-a6ee-7d62b78378c9" path="/var/lib/kubelet/pods/c99c4271-25a8-4e71-a6ee-7d62b78378c9/volumes"
Sep 15 22:50:12.210376 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:12.210344    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-92482b-bbfb58bf5-gf25f"
Sep 15 22:50:12.210994 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:50:12.210966    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-92482b-774868dc8b-l7lwz"
Sep 15 22:51:30.864899 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:51:30.864826    2565 scope.go:117] "RemoveContainer" containerID="b36d44a130ce4f0c2fc04fa2a4c17e40007f3a401986fdd8de19078361111868"
Sep 15 22:52:09.758521 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:09.758487    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc"]
Sep 15 22:52:09.759002 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:09.758988    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c99c4271-25a8-4e71-a6ee-7d62b78378c9" containerName="maas-api"
Sep 15 22:52:09.759077 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:09.759004    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c99c4271-25a8-4e71-a6ee-7d62b78378c9" containerName="maas-api"
Sep 15 22:52:09.759116 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:09.759108    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c99c4271-25a8-4e71-a6ee-7d62b78378c9" containerName="maas-api"
Sep 15 22:52:09.762540 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:09.762520    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc"
Sep 15 22:52:09.764337 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:09.764318    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-54db18-serving-cert\""
Sep 15 22:52:09.773827 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:09.773804    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc"]
Sep 15 22:52:09.835336 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:09.835300    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2dac7197-131b-4040-82c4-feb64d4665a6-metrics-certs\") pod \"maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc\" (UID: \"2dac7197-131b-4040-82c4-feb64d4665a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc"
Sep 15 22:52:09.835508 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:09.835354    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2dac7197-131b-4040-82c4-feb64d4665a6-maas-api-tls\") pod \"maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc\" (UID: \"2dac7197-131b-4040-82c4-feb64d4665a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc"
Sep 15 22:52:09.835508 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:09.835401    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dac7197-131b-4040-82c4-feb64d4665a6-openshift-service-ca\") pod \"maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc\" (UID: \"2dac7197-131b-4040-82c4-feb64d4665a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc"
Sep 15 22:52:09.835508 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:09.835423    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfmqp\" (UniqueName: \"kubernetes.io/projected/2dac7197-131b-4040-82c4-feb64d4665a6-kube-api-access-tfmqp\") pod \"maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc\" (UID: \"2dac7197-131b-4040-82c4-feb64d4665a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc"
Sep 15 22:52:09.936119 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:09.936088    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dac7197-131b-4040-82c4-feb64d4665a6-openshift-service-ca\") pod \"maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc\" (UID: \"2dac7197-131b-4040-82c4-feb64d4665a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc"
Sep 15 22:52:09.936119 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:09.936124    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfmqp\" (UniqueName: \"kubernetes.io/projected/2dac7197-131b-4040-82c4-feb64d4665a6-kube-api-access-tfmqp\") pod \"maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc\" (UID: \"2dac7197-131b-4040-82c4-feb64d4665a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc"
Sep 15 22:52:09.936332 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:09.936217    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2dac7197-131b-4040-82c4-feb64d4665a6-metrics-certs\") pod \"maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc\" (UID: \"2dac7197-131b-4040-82c4-feb64d4665a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc"
Sep 15 22:52:09.936332 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:09.936258    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2dac7197-131b-4040-82c4-feb64d4665a6-maas-api-tls\") pod \"maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc\" (UID: \"2dac7197-131b-4040-82c4-feb64d4665a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc"
Sep 15 22:52:09.936836 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:09.936809    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dac7197-131b-4040-82c4-feb64d4665a6-openshift-service-ca\") pod \"maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc\" (UID: \"2dac7197-131b-4040-82c4-feb64d4665a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc"
Sep 15 22:52:09.938578 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:09.938556    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2dac7197-131b-4040-82c4-feb64d4665a6-metrics-certs\") pod \"maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc\" (UID: \"2dac7197-131b-4040-82c4-feb64d4665a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc"
Sep 15 22:52:09.938676 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:09.938640    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2dac7197-131b-4040-82c4-feb64d4665a6-maas-api-tls\") pod \"maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc\" (UID: \"2dac7197-131b-4040-82c4-feb64d4665a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc"
Sep 15 22:52:09.945528 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:09.945506    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfmqp\" (UniqueName: \"kubernetes.io/projected/2dac7197-131b-4040-82c4-feb64d4665a6-kube-api-access-tfmqp\") pod \"maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc\" (UID: \"2dac7197-131b-4040-82c4-feb64d4665a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc"
Sep 15 22:52:10.076089 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:10.076050    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc"
Sep 15 22:52:10.354743 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:10.354710    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc"]
Sep 15 22:52:10.356319 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:52:10.356298    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dac7197_131b_4040_82c4_feb64d4665a6.slice/crio-eeca37e9a0db2b06a18f133b501e7f832dd236369d127f4501f1b7a43562e820 WatchSource:0}: Error finding container eeca37e9a0db2b06a18f133b501e7f832dd236369d127f4501f1b7a43562e820: Status 404 returned error can't find the container with id eeca37e9a0db2b06a18f133b501e7f832dd236369d127f4501f1b7a43562e820
Sep 15 22:52:10.450791 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:10.450726    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-54db18-5b69bfcff5-9qwdq"]
Sep 15 22:52:10.455618 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:10.455601    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-54db18-5b69bfcff5-9qwdq"
Sep 15 22:52:10.457599 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:10.457575    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-54db18\""
Sep 15 22:52:10.457709 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:10.457618    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-54db18-dockercfg-hkt49\""
Sep 15 22:52:10.466418 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:10.466393    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-54db18-5b69bfcff5-9qwdq"]
Sep 15 22:52:10.542810 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:10.542777    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hl2bm\" (UniqueName: \"kubernetes.io/projected/4ffd92a3-8f8f-48a2-84eb-379cb320f48d-kube-api-access-hl2bm\") pod \"payload-processing-e2e-api-a-54db18-5b69bfcff5-9qwdq\" (UID: \"4ffd92a3-8f8f-48a2-84eb-379cb320f48d\") " pod="openshift-ingress/payload-processing-e2e-api-a-54db18-5b69bfcff5-9qwdq"
Sep 15 22:52:10.542965 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:10.542896    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ffd92a3-8f8f-48a2-84eb-379cb320f48d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-54db18-5b69bfcff5-9qwdq\" (UID: \"4ffd92a3-8f8f-48a2-84eb-379cb320f48d\") " pod="openshift-ingress/payload-processing-e2e-api-a-54db18-5b69bfcff5-9qwdq"
Sep 15 22:52:10.644351 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:10.644257    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ffd92a3-8f8f-48a2-84eb-379cb320f48d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-54db18-5b69bfcff5-9qwdq\" (UID: \"4ffd92a3-8f8f-48a2-84eb-379cb320f48d\") " pod="openshift-ingress/payload-processing-e2e-api-a-54db18-5b69bfcff5-9qwdq"
Sep 15 22:52:10.644525 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:10.644406    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl2bm\" (UniqueName: \"kubernetes.io/projected/4ffd92a3-8f8f-48a2-84eb-379cb320f48d-kube-api-access-hl2bm\") pod \"payload-processing-e2e-api-a-54db18-5b69bfcff5-9qwdq\" (UID: \"4ffd92a3-8f8f-48a2-84eb-379cb320f48d\") " pod="openshift-ingress/payload-processing-e2e-api-a-54db18-5b69bfcff5-9qwdq"
Sep 15 22:52:10.645236 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:10.645216    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ffd92a3-8f8f-48a2-84eb-379cb320f48d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-54db18-5b69bfcff5-9qwdq\" (UID: \"4ffd92a3-8f8f-48a2-84eb-379cb320f48d\") " pod="openshift-ingress/payload-processing-e2e-api-a-54db18-5b69bfcff5-9qwdq"
Sep 15 22:52:10.655684 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:10.655661    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hl2bm\" (UniqueName: \"kubernetes.io/projected/4ffd92a3-8f8f-48a2-84eb-379cb320f48d-kube-api-access-hl2bm\") pod \"payload-processing-e2e-api-a-54db18-5b69bfcff5-9qwdq\" (UID: \"4ffd92a3-8f8f-48a2-84eb-379cb320f48d\") " pod="openshift-ingress/payload-processing-e2e-api-a-54db18-5b69bfcff5-9qwdq"
Sep 15 22:52:10.768603 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:10.768575    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-54db18-5b69bfcff5-9qwdq"
Sep 15 22:52:11.016412 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:11.016310    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc" event={"ID":"2dac7197-131b-4040-82c4-feb64d4665a6","Type":"ContainerStarted","Data":"dc9f647d115a06b25a689c1f22fcff20c9606353705d88e38f48088b791ad584"}
Sep 15 22:52:11.016412 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:11.016372    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc" event={"ID":"2dac7197-131b-4040-82c4-feb64d4665a6","Type":"ContainerStarted","Data":"eeca37e9a0db2b06a18f133b501e7f832dd236369d127f4501f1b7a43562e820"}
Sep 15 22:52:11.016412 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:11.016402    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc"
Sep 15 22:52:11.035488 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:11.035439    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc" podStartSLOduration=1.627792812 podStartE2EDuration="2.035422163s" totalImagesPullingTime="407.629351ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:52:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:52:11.032674226 +0000 UTC m=+1480.892385523" watchObservedRunningTime="2026-09-15 22:52:11.035422163 +0000 UTC m=+1480.895133446"
Sep 15 22:52:11.049928 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:11.049904    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-54db18-5b69bfcff5-9qwdq"]
Sep 15 22:52:11.051222 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:52:11.051199    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ffd92a3_8f8f_48a2_84eb_379cb320f48d.slice/crio-24c31403c42f0e889adf28426d537cdb28ba32a6399c0714c23f103244e23634 WatchSource:0}: Error finding container 24c31403c42f0e889adf28426d537cdb28ba32a6399c0714c23f103244e23634: Status 404 returned error can't find the container with id 24c31403c42f0e889adf28426d537cdb28ba32a6399c0714c23f103244e23634
Sep 15 22:52:12.024779 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:12.024744    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-54db18-5b69bfcff5-9qwdq" event={"ID":"4ffd92a3-8f8f-48a2-84eb-379cb320f48d","Type":"ContainerStarted","Data":"b988e1fb0fc14d8364878e88cb025b56234fc2be7409796207b3ae2447a57ee5"}
Sep 15 22:52:12.025195 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:12.024798    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-54db18-5b69bfcff5-9qwdq" event={"ID":"4ffd92a3-8f8f-48a2-84eb-379cb320f48d","Type":"ContainerStarted","Data":"24c31403c42f0e889adf28426d537cdb28ba32a6399c0714c23f103244e23634"}
Sep 15 22:52:12.025195 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:12.024845    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-54db18-5b69bfcff5-9qwdq"
Sep 15 22:52:12.040703 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:12.040649    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-54db18-5b69bfcff5-9qwdq" podStartSLOduration=2.040631731 podStartE2EDuration="2.040631731s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:52:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:52:12.040070968 +0000 UTC m=+1481.899782249" watchObservedRunningTime="2026-09-15 22:52:12.040631731 +0000 UTC m=+1481.900343017"
Sep 15 22:52:17.031197 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:17.031167    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc"
Sep 15 22:52:23.033957 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:23.033928    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-54db18-5b69bfcff5-9qwdq"
Sep 15 22:52:46.733441 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:46.733401    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f667ee-76c8c66b4b-gkbjl"]
Sep 15 22:52:46.737761 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:46.737735    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f667ee-76c8c66b4b-gkbjl"
Sep 15 22:52:46.739940 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:46.739912    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-f667ee\""
Sep 15 22:52:46.749778 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:46.749750    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f667ee-76c8c66b4b-gkbjl"]
Sep 15 22:52:46.808775 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:46.808737    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjpw5\" (UniqueName: \"kubernetes.io/projected/8fbcc540-a2af-4f10-b9bd-08f82930b8b3-kube-api-access-pjpw5\") pod \"payload-pre-processing-e2e-api-b-f667ee-76c8c66b4b-gkbjl\" (UID: \"8fbcc540-a2af-4f10-b9bd-08f82930b8b3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f667ee-76c8c66b4b-gkbjl"
Sep 15 22:52:46.808982 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:46.808890    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fbcc540-a2af-4f10-b9bd-08f82930b8b3-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-f667ee-76c8c66b4b-gkbjl\" (UID: \"8fbcc540-a2af-4f10-b9bd-08f82930b8b3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f667ee-76c8c66b4b-gkbjl"
Sep 15 22:52:46.909733 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:46.909697    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fbcc540-a2af-4f10-b9bd-08f82930b8b3-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-f667ee-76c8c66b4b-gkbjl\" (UID: \"8fbcc540-a2af-4f10-b9bd-08f82930b8b3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f667ee-76c8c66b4b-gkbjl"
Sep 15 22:52:46.909899 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:46.909819    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjpw5\" (UniqueName: \"kubernetes.io/projected/8fbcc540-a2af-4f10-b9bd-08f82930b8b3-kube-api-access-pjpw5\") pod \"payload-pre-processing-e2e-api-b-f667ee-76c8c66b4b-gkbjl\" (UID: \"8fbcc540-a2af-4f10-b9bd-08f82930b8b3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f667ee-76c8c66b4b-gkbjl"
Sep 15 22:52:46.910537 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:46.910517    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fbcc540-a2af-4f10-b9bd-08f82930b8b3-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-f667ee-76c8c66b4b-gkbjl\" (UID: \"8fbcc540-a2af-4f10-b9bd-08f82930b8b3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f667ee-76c8c66b4b-gkbjl"
Sep 15 22:52:46.918759 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:46.918733    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjpw5\" (UniqueName: \"kubernetes.io/projected/8fbcc540-a2af-4f10-b9bd-08f82930b8b3-kube-api-access-pjpw5\") pod \"payload-pre-processing-e2e-api-b-f667ee-76c8c66b4b-gkbjl\" (UID: \"8fbcc540-a2af-4f10-b9bd-08f82930b8b3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f667ee-76c8c66b4b-gkbjl"
Sep 15 22:52:47.051516 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:47.051483    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f667ee-76c8c66b4b-gkbjl"
Sep 15 22:52:47.332679 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:47.332649    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f667ee-76c8c66b4b-gkbjl"]
Sep 15 22:52:47.335591 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:52:47.335561    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fbcc540_a2af_4f10_b9bd_08f82930b8b3.slice/crio-95f655e5bcb28e35426d82eeaa6e3f5a5c654f5213eea7ce275ee2154ab00d9a WatchSource:0}: Error finding container 95f655e5bcb28e35426d82eeaa6e3f5a5c654f5213eea7ce275ee2154ab00d9a: Status 404 returned error can't find the container with id 95f655e5bcb28e35426d82eeaa6e3f5a5c654f5213eea7ce275ee2154ab00d9a
Sep 15 22:52:48.255778 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:48.255733    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f667ee-76c8c66b4b-gkbjl" event={"ID":"8fbcc540-a2af-4f10-b9bd-08f82930b8b3","Type":"ContainerStarted","Data":"4d6126a621bf55325d57bfb34e8e7302fd640b8033c33f37630c92651b3ee491"}
Sep 15 22:52:48.256284 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:48.255802    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f667ee-76c8c66b4b-gkbjl" event={"ID":"8fbcc540-a2af-4f10-b9bd-08f82930b8b3","Type":"ContainerStarted","Data":"95f655e5bcb28e35426d82eeaa6e3f5a5c654f5213eea7ce275ee2154ab00d9a"}
Sep 15 22:52:48.256284 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:48.255864    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f667ee-76c8c66b4b-gkbjl"
Sep 15 22:52:48.273722 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:48.273673    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f667ee-76c8c66b4b-gkbjl" podStartSLOduration=2.273658396 podStartE2EDuration="2.273658396s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:52:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:52:48.270493042 +0000 UTC m=+1518.130204326" watchObservedRunningTime="2026-09-15 22:52:48.273658396 +0000 UTC m=+1518.133369679"
Sep 15 22:52:59.265018 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:52:59.264982    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f667ee-76c8c66b4b-gkbjl"
Sep 15 22:53:27.821624 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:27.821594    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc"]
Sep 15 22:53:27.822354 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:27.821812    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc" podUID="2dac7197-131b-4040-82c4-feb64d4665a6" containerName="maas-api" containerID="cri-o://dc9f647d115a06b25a689c1f22fcff20c9606353705d88e38f48088b791ad584" gracePeriod=30
Sep 15 22:53:27.876464 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:27.876432    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-54db18-5b69bfcff5-9qwdq"]
Sep 15 22:53:27.876721 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:27.876693    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-54db18-5b69bfcff5-9qwdq" podUID="4ffd92a3-8f8f-48a2-84eb-379cb320f48d" containerName="payload-processing" containerID="cri-o://b988e1fb0fc14d8364878e88cb025b56234fc2be7409796207b3ae2447a57ee5" gracePeriod=30
Sep 15 22:53:28.125387 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.125362    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc"
Sep 15 22:53:28.165912 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.165891    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-54db18-5b69bfcff5-9qwdq"
Sep 15 22:53:28.249642 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.249608    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2dac7197-131b-4040-82c4-feb64d4665a6-maas-api-tls\") pod \"2dac7197-131b-4040-82c4-feb64d4665a6\" (UID: \"2dac7197-131b-4040-82c4-feb64d4665a6\") "
Sep 15 22:53:28.249816 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.249696    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ffd92a3-8f8f-48a2-84eb-379cb320f48d-plugins-config-volume\") pod \"4ffd92a3-8f8f-48a2-84eb-379cb320f48d\" (UID: \"4ffd92a3-8f8f-48a2-84eb-379cb320f48d\") "
Sep 15 22:53:28.249816 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.249748    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tfmqp\" (UniqueName: \"kubernetes.io/projected/2dac7197-131b-4040-82c4-feb64d4665a6-kube-api-access-tfmqp\") pod \"2dac7197-131b-4040-82c4-feb64d4665a6\" (UID: \"2dac7197-131b-4040-82c4-feb64d4665a6\") "
Sep 15 22:53:28.249816 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.249766    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hl2bm\" (UniqueName: \"kubernetes.io/projected/4ffd92a3-8f8f-48a2-84eb-379cb320f48d-kube-api-access-hl2bm\") pod \"4ffd92a3-8f8f-48a2-84eb-379cb320f48d\" (UID: \"4ffd92a3-8f8f-48a2-84eb-379cb320f48d\") "
Sep 15 22:53:28.249993 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.249818    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2dac7197-131b-4040-82c4-feb64d4665a6-metrics-certs\") pod \"2dac7197-131b-4040-82c4-feb64d4665a6\" (UID: \"2dac7197-131b-4040-82c4-feb64d4665a6\") "
Sep 15 22:53:28.249993 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.249842    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dac7197-131b-4040-82c4-feb64d4665a6-openshift-service-ca\") pod \"2dac7197-131b-4040-82c4-feb64d4665a6\" (UID: \"2dac7197-131b-4040-82c4-feb64d4665a6\") "
Sep 15 22:53:28.250327 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.250295    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ffd92a3-8f8f-48a2-84eb-379cb320f48d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4ffd92a3-8f8f-48a2-84eb-379cb320f48d" (UID: "4ffd92a3-8f8f-48a2-84eb-379cb320f48d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:53:28.250434 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.250337    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2dac7197-131b-4040-82c4-feb64d4665a6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2dac7197-131b-4040-82c4-feb64d4665a6" (UID: "2dac7197-131b-4040-82c4-feb64d4665a6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:53:28.251860 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.251826    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2dac7197-131b-4040-82c4-feb64d4665a6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2dac7197-131b-4040-82c4-feb64d4665a6" (UID: "2dac7197-131b-4040-82c4-feb64d4665a6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:53:28.251957 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.251879    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2dac7197-131b-4040-82c4-feb64d4665a6-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2dac7197-131b-4040-82c4-feb64d4665a6" (UID: "2dac7197-131b-4040-82c4-feb64d4665a6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:53:28.251957 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.251904    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ffd92a3-8f8f-48a2-84eb-379cb320f48d-kube-api-access-hl2bm" (OuterVolumeSpecName: "kube-api-access-hl2bm") pod "4ffd92a3-8f8f-48a2-84eb-379cb320f48d" (UID: "4ffd92a3-8f8f-48a2-84eb-379cb320f48d"). InnerVolumeSpecName "kube-api-access-hl2bm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:53:28.251957 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.251931    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2dac7197-131b-4040-82c4-feb64d4665a6-kube-api-access-tfmqp" (OuterVolumeSpecName: "kube-api-access-tfmqp") pod "2dac7197-131b-4040-82c4-feb64d4665a6" (UID: "2dac7197-131b-4040-82c4-feb64d4665a6"). InnerVolumeSpecName "kube-api-access-tfmqp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:53:28.351006 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.350910    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ffd92a3-8f8f-48a2-84eb-379cb320f48d-plugins-config-volume\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:53:28.351006 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.350949    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tfmqp\" (UniqueName: \"kubernetes.io/projected/2dac7197-131b-4040-82c4-feb64d4665a6-kube-api-access-tfmqp\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:53:28.351006 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.350962    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hl2bm\" (UniqueName: \"kubernetes.io/projected/4ffd92a3-8f8f-48a2-84eb-379cb320f48d-kube-api-access-hl2bm\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:53:28.351006 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.350975    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2dac7197-131b-4040-82c4-feb64d4665a6-metrics-certs\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:53:28.351006 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.350987    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dac7197-131b-4040-82c4-feb64d4665a6-openshift-service-ca\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:53:28.351006 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.350999    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2dac7197-131b-4040-82c4-feb64d4665a6-maas-api-tls\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:53:28.510151 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.510113    2565 generic.go:356] "Generic (PLEG): container finished" podID="4ffd92a3-8f8f-48a2-84eb-379cb320f48d" containerID="b988e1fb0fc14d8364878e88cb025b56234fc2be7409796207b3ae2447a57ee5" exitCode=0
Sep 15 22:53:28.510326 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.510164    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-54db18-5b69bfcff5-9qwdq"
Sep 15 22:53:28.510326 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.510208    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-54db18-5b69bfcff5-9qwdq" event={"ID":"4ffd92a3-8f8f-48a2-84eb-379cb320f48d","Type":"ContainerDied","Data":"b988e1fb0fc14d8364878e88cb025b56234fc2be7409796207b3ae2447a57ee5"}
Sep 15 22:53:28.510326 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.510304    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-54db18-5b69bfcff5-9qwdq" event={"ID":"4ffd92a3-8f8f-48a2-84eb-379cb320f48d","Type":"ContainerDied","Data":"24c31403c42f0e889adf28426d537cdb28ba32a6399c0714c23f103244e23634"}
Sep 15 22:53:28.510506 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.510335    2565 scope.go:117] "RemoveContainer" containerID="b988e1fb0fc14d8364878e88cb025b56234fc2be7409796207b3ae2447a57ee5"
Sep 15 22:53:28.511828 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.511807    2565 generic.go:356] "Generic (PLEG): container finished" podID="2dac7197-131b-4040-82c4-feb64d4665a6" containerID="dc9f647d115a06b25a689c1f22fcff20c9606353705d88e38f48088b791ad584" exitCode=0
Sep 15 22:53:28.511948 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.511857    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc"
Sep 15 22:53:28.511948 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.511898    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc" event={"ID":"2dac7197-131b-4040-82c4-feb64d4665a6","Type":"ContainerDied","Data":"dc9f647d115a06b25a689c1f22fcff20c9606353705d88e38f48088b791ad584"}
Sep 15 22:53:28.511948 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.511937    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc" event={"ID":"2dac7197-131b-4040-82c4-feb64d4665a6","Type":"ContainerDied","Data":"eeca37e9a0db2b06a18f133b501e7f832dd236369d127f4501f1b7a43562e820"}
Sep 15 22:53:28.521657 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.521640    2565 scope.go:117] "RemoveContainer" containerID="b988e1fb0fc14d8364878e88cb025b56234fc2be7409796207b3ae2447a57ee5"
Sep 15 22:53:28.522073 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:53:28.521937    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b988e1fb0fc14d8364878e88cb025b56234fc2be7409796207b3ae2447a57ee5\": container with ID starting with b988e1fb0fc14d8364878e88cb025b56234fc2be7409796207b3ae2447a57ee5 not found: ID does not exist" containerID="b988e1fb0fc14d8364878e88cb025b56234fc2be7409796207b3ae2447a57ee5"
Sep 15 22:53:28.522073 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.521962    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b988e1fb0fc14d8364878e88cb025b56234fc2be7409796207b3ae2447a57ee5"} err="failed to get container status \"b988e1fb0fc14d8364878e88cb025b56234fc2be7409796207b3ae2447a57ee5\": rpc error: code = NotFound desc = could not find container \"b988e1fb0fc14d8364878e88cb025b56234fc2be7409796207b3ae2447a57ee5\": container with ID starting with b988e1fb0fc14d8364878e88cb025b56234fc2be7409796207b3ae2447a57ee5 not found: ID does not exist"
Sep 15 22:53:28.522073 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.521981    2565 scope.go:117] "RemoveContainer" containerID="dc9f647d115a06b25a689c1f22fcff20c9606353705d88e38f48088b791ad584"
Sep 15 22:53:28.531267 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.531245    2565 scope.go:117] "RemoveContainer" containerID="dc9f647d115a06b25a689c1f22fcff20c9606353705d88e38f48088b791ad584"
Sep 15 22:53:28.531580 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:53:28.531560    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc9f647d115a06b25a689c1f22fcff20c9606353705d88e38f48088b791ad584\": container with ID starting with dc9f647d115a06b25a689c1f22fcff20c9606353705d88e38f48088b791ad584 not found: ID does not exist" containerID="dc9f647d115a06b25a689c1f22fcff20c9606353705d88e38f48088b791ad584"
Sep 15 22:53:28.531623 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.531587    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc9f647d115a06b25a689c1f22fcff20c9606353705d88e38f48088b791ad584"} err="failed to get container status \"dc9f647d115a06b25a689c1f22fcff20c9606353705d88e38f48088b791ad584\": rpc error: code = NotFound desc = could not find container \"dc9f647d115a06b25a689c1f22fcff20c9606353705d88e38f48088b791ad584\": container with ID starting with dc9f647d115a06b25a689c1f22fcff20c9606353705d88e38f48088b791ad584 not found: ID does not exist"
Sep 15 22:53:28.543201 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.543172    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-54db18-5b69bfcff5-9qwdq"]
Sep 15 22:53:28.550189 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.550167    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-54db18-5b69bfcff5-9qwdq"]
Sep 15 22:53:28.560239 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.560209    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc"]
Sep 15 22:53:28.567697 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.567674    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-54db18-5c4f9b9657-jtdpc"]
Sep 15 22:53:28.591248 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.591216    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2dac7197-131b-4040-82c4-feb64d4665a6" path="/var/lib/kubelet/pods/2dac7197-131b-4040-82c4-feb64d4665a6/volumes"
Sep 15 22:53:28.591620 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:28.591606    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ffd92a3-8f8f-48a2-84eb-379cb320f48d" path="/var/lib/kubelet/pods/4ffd92a3-8f8f-48a2-84eb-379cb320f48d/volumes"
Sep 15 22:53:50.595200 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:50.595158    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f667ee-76c8c66b4b-gkbjl"]
Sep 15 22:53:50.595803 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:50.595358    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f667ee-76c8c66b4b-gkbjl" podUID="8fbcc540-a2af-4f10-b9bd-08f82930b8b3" containerName="payload-pre-processing" containerID="cri-o://4d6126a621bf55325d57bfb34e8e7302fd640b8033c33f37630c92651b3ee491" gracePeriod=30
Sep 15 22:53:50.859916 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:50.859855    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f667ee-76c8c66b4b-gkbjl"
Sep 15 22:53:50.981935 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:50.981893    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fbcc540-a2af-4f10-b9bd-08f82930b8b3-plugins-config-volume\") pod \"8fbcc540-a2af-4f10-b9bd-08f82930b8b3\" (UID: \"8fbcc540-a2af-4f10-b9bd-08f82930b8b3\") "
Sep 15 22:53:50.982140 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:50.982067    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pjpw5\" (UniqueName: \"kubernetes.io/projected/8fbcc540-a2af-4f10-b9bd-08f82930b8b3-kube-api-access-pjpw5\") pod \"8fbcc540-a2af-4f10-b9bd-08f82930b8b3\" (UID: \"8fbcc540-a2af-4f10-b9bd-08f82930b8b3\") "
Sep 15 22:53:50.982316 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:50.982291    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fbcc540-a2af-4f10-b9bd-08f82930b8b3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8fbcc540-a2af-4f10-b9bd-08f82930b8b3" (UID: "8fbcc540-a2af-4f10-b9bd-08f82930b8b3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:53:50.984158 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:50.984127    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fbcc540-a2af-4f10-b9bd-08f82930b8b3-kube-api-access-pjpw5" (OuterVolumeSpecName: "kube-api-access-pjpw5") pod "8fbcc540-a2af-4f10-b9bd-08f82930b8b3" (UID: "8fbcc540-a2af-4f10-b9bd-08f82930b8b3"). InnerVolumeSpecName "kube-api-access-pjpw5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:53:51.083224 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:51.083195    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjpw5\" (UniqueName: \"kubernetes.io/projected/8fbcc540-a2af-4f10-b9bd-08f82930b8b3-kube-api-access-pjpw5\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:53:51.083224 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:51.083220    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fbcc540-a2af-4f10-b9bd-08f82930b8b3-plugins-config-volume\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:53:51.651886 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:51.651845    2565 generic.go:356] "Generic (PLEG): container finished" podID="8fbcc540-a2af-4f10-b9bd-08f82930b8b3" containerID="4d6126a621bf55325d57bfb34e8e7302fd640b8033c33f37630c92651b3ee491" exitCode=0
Sep 15 22:53:51.651886 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:51.651889    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f667ee-76c8c66b4b-gkbjl"
Sep 15 22:53:51.652320 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:51.651925    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f667ee-76c8c66b4b-gkbjl" event={"ID":"8fbcc540-a2af-4f10-b9bd-08f82930b8b3","Type":"ContainerDied","Data":"4d6126a621bf55325d57bfb34e8e7302fd640b8033c33f37630c92651b3ee491"}
Sep 15 22:53:51.652320 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:51.651988    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f667ee-76c8c66b4b-gkbjl" event={"ID":"8fbcc540-a2af-4f10-b9bd-08f82930b8b3","Type":"ContainerDied","Data":"95f655e5bcb28e35426d82eeaa6e3f5a5c654f5213eea7ce275ee2154ab00d9a"}
Sep 15 22:53:51.652320 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:51.652011    2565 scope.go:117] "RemoveContainer" containerID="4d6126a621bf55325d57bfb34e8e7302fd640b8033c33f37630c92651b3ee491"
Sep 15 22:53:51.661846 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:51.661829    2565 scope.go:117] "RemoveContainer" containerID="4d6126a621bf55325d57bfb34e8e7302fd640b8033c33f37630c92651b3ee491"
Sep 15 22:53:51.662111 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:53:51.662094    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d6126a621bf55325d57bfb34e8e7302fd640b8033c33f37630c92651b3ee491\": container with ID starting with 4d6126a621bf55325d57bfb34e8e7302fd640b8033c33f37630c92651b3ee491 not found: ID does not exist" containerID="4d6126a621bf55325d57bfb34e8e7302fd640b8033c33f37630c92651b3ee491"
Sep 15 22:53:51.662177 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:51.662118    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d6126a621bf55325d57bfb34e8e7302fd640b8033c33f37630c92651b3ee491"} err="failed to get container status \"4d6126a621bf55325d57bfb34e8e7302fd640b8033c33f37630c92651b3ee491\": rpc error: code = NotFound desc = could not find container \"4d6126a621bf55325d57bfb34e8e7302fd640b8033c33f37630c92651b3ee491\": container with ID starting with 4d6126a621bf55325d57bfb34e8e7302fd640b8033c33f37630c92651b3ee491 not found: ID does not exist"
Sep 15 22:53:51.676528 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:51.676508    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f667ee-76c8c66b4b-gkbjl"]
Sep 15 22:53:51.682993 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:51.682971    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f667ee-76c8c66b4b-gkbjl"]
Sep 15 22:53:52.590833 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:53:52.590801    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fbcc540-a2af-4f10-b9bd-08f82930b8b3" path="/var/lib/kubelet/pods/8fbcc540-a2af-4f10-b9bd-08f82930b8b3/volumes"
Sep 15 22:54:08.144222 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.144187    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f"]
Sep 15 22:54:08.144810 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.144441    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f" podUID="7092845b-136b-4c47-9d1a-706ea4dbad85" containerName="maas-api" containerID="cri-o://56c2cd5849168aa35e9f9a005ae4f431c938bf13fa4b52805d989bd301f95b42" gracePeriod=30
Sep 15 22:54:08.217988 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.217946    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f2241f-586669d47d-lc898"]
Sep 15 22:54:08.218746 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.218707    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f2241f-586669d47d-lc898" podUID="c4d9b866-cf8b-49c6-9f4e-a46a4477af1e" containerName="payload-pre-processing" containerID="cri-o://9886ef7c741a280c1a00cf6e4b0fb2d2a164cea3117f671a5ffbab96b4082c8c" gracePeriod=30
Sep 15 22:54:08.476325 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.476303    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f"
Sep 15 22:54:08.511628 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.511600    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f2241f-586669d47d-lc898"
Sep 15 22:54:08.560924 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.560891    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7092845b-136b-4c47-9d1a-706ea4dbad85-openshift-service-ca\") pod \"7092845b-136b-4c47-9d1a-706ea4dbad85\" (UID: \"7092845b-136b-4c47-9d1a-706ea4dbad85\") "
Sep 15 22:54:08.561136 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.561001    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7092845b-136b-4c47-9d1a-706ea4dbad85-metrics-certs\") pod \"7092845b-136b-4c47-9d1a-706ea4dbad85\" (UID: \"7092845b-136b-4c47-9d1a-706ea4dbad85\") "
Sep 15 22:54:08.561136 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.561081    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7092845b-136b-4c47-9d1a-706ea4dbad85-maas-api-tls\") pod \"7092845b-136b-4c47-9d1a-706ea4dbad85\" (UID: \"7092845b-136b-4c47-9d1a-706ea4dbad85\") "
Sep 15 22:54:08.561136 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.561118    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7n5r\" (UniqueName: \"kubernetes.io/projected/7092845b-136b-4c47-9d1a-706ea4dbad85-kube-api-access-z7n5r\") pod \"7092845b-136b-4c47-9d1a-706ea4dbad85\" (UID: \"7092845b-136b-4c47-9d1a-706ea4dbad85\") "
Sep 15 22:54:08.561339 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.561316    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7092845b-136b-4c47-9d1a-706ea4dbad85-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7092845b-136b-4c47-9d1a-706ea4dbad85" (UID: "7092845b-136b-4c47-9d1a-706ea4dbad85"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:54:08.561457 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.561442    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7092845b-136b-4c47-9d1a-706ea4dbad85-openshift-service-ca\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:54:08.563242 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.563207    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7092845b-136b-4c47-9d1a-706ea4dbad85-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7092845b-136b-4c47-9d1a-706ea4dbad85" (UID: "7092845b-136b-4c47-9d1a-706ea4dbad85"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:54:08.563348 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.563257    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7092845b-136b-4c47-9d1a-706ea4dbad85-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7092845b-136b-4c47-9d1a-706ea4dbad85" (UID: "7092845b-136b-4c47-9d1a-706ea4dbad85"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:54:08.563348 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.563280    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7092845b-136b-4c47-9d1a-706ea4dbad85-kube-api-access-z7n5r" (OuterVolumeSpecName: "kube-api-access-z7n5r") pod "7092845b-136b-4c47-9d1a-706ea4dbad85" (UID: "7092845b-136b-4c47-9d1a-706ea4dbad85"). InnerVolumeSpecName "kube-api-access-z7n5r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:54:08.662348 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.662317    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4d9b866-cf8b-49c6-9f4e-a46a4477af1e-plugins-config-volume\") pod \"c4d9b866-cf8b-49c6-9f4e-a46a4477af1e\" (UID: \"c4d9b866-cf8b-49c6-9f4e-a46a4477af1e\") "
Sep 15 22:54:08.662772 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.662466    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sjcnt\" (UniqueName: \"kubernetes.io/projected/c4d9b866-cf8b-49c6-9f4e-a46a4477af1e-kube-api-access-sjcnt\") pod \"c4d9b866-cf8b-49c6-9f4e-a46a4477af1e\" (UID: \"c4d9b866-cf8b-49c6-9f4e-a46a4477af1e\") "
Sep 15 22:54:08.662854 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.662809    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7092845b-136b-4c47-9d1a-706ea4dbad85-metrics-certs\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:54:08.662854 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.662808    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4d9b866-cf8b-49c6-9f4e-a46a4477af1e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c4d9b866-cf8b-49c6-9f4e-a46a4477af1e" (UID: "c4d9b866-cf8b-49c6-9f4e-a46a4477af1e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:54:08.662854 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.662823    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7092845b-136b-4c47-9d1a-706ea4dbad85-maas-api-tls\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:54:08.662961 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.662870    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7n5r\" (UniqueName: \"kubernetes.io/projected/7092845b-136b-4c47-9d1a-706ea4dbad85-kube-api-access-z7n5r\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:54:08.664695 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.664675    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4d9b866-cf8b-49c6-9f4e-a46a4477af1e-kube-api-access-sjcnt" (OuterVolumeSpecName: "kube-api-access-sjcnt") pod "c4d9b866-cf8b-49c6-9f4e-a46a4477af1e" (UID: "c4d9b866-cf8b-49c6-9f4e-a46a4477af1e"). InnerVolumeSpecName "kube-api-access-sjcnt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:54:08.757697 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.757605    2565 generic.go:356] "Generic (PLEG): container finished" podID="c4d9b866-cf8b-49c6-9f4e-a46a4477af1e" containerID="9886ef7c741a280c1a00cf6e4b0fb2d2a164cea3117f671a5ffbab96b4082c8c" exitCode=0
Sep 15 22:54:08.757697 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.757651    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f2241f-586669d47d-lc898"
Sep 15 22:54:08.757907 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.757697    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f2241f-586669d47d-lc898" event={"ID":"c4d9b866-cf8b-49c6-9f4e-a46a4477af1e","Type":"ContainerDied","Data":"9886ef7c741a280c1a00cf6e4b0fb2d2a164cea3117f671a5ffbab96b4082c8c"}
Sep 15 22:54:08.757907 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.757748    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f2241f-586669d47d-lc898" event={"ID":"c4d9b866-cf8b-49c6-9f4e-a46a4477af1e","Type":"ContainerDied","Data":"c638efcc11e6d029524371af5160c4c1f0471a449c50fa058d78634d09e078dd"}
Sep 15 22:54:08.757907 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.757770    2565 scope.go:117] "RemoveContainer" containerID="9886ef7c741a280c1a00cf6e4b0fb2d2a164cea3117f671a5ffbab96b4082c8c"
Sep 15 22:54:08.759317 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.759284    2565 generic.go:356] "Generic (PLEG): container finished" podID="7092845b-136b-4c47-9d1a-706ea4dbad85" containerID="56c2cd5849168aa35e9f9a005ae4f431c938bf13fa4b52805d989bd301f95b42" exitCode=0
Sep 15 22:54:08.759428 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.759335    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f" event={"ID":"7092845b-136b-4c47-9d1a-706ea4dbad85","Type":"ContainerDied","Data":"56c2cd5849168aa35e9f9a005ae4f431c938bf13fa4b52805d989bd301f95b42"}
Sep 15 22:54:08.759428 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.759388    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f" event={"ID":"7092845b-136b-4c47-9d1a-706ea4dbad85","Type":"ContainerDied","Data":"babc6bc467c8c5ccbe116b8cd96a43d9396cdf301dd0adc62b5cef9d83965a5e"}
Sep 15 22:54:08.759428 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.759341    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f"
Sep 15 22:54:08.763438 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.763417    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjcnt\" (UniqueName: \"kubernetes.io/projected/c4d9b866-cf8b-49c6-9f4e-a46a4477af1e-kube-api-access-sjcnt\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:54:08.763438 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.763438    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4d9b866-cf8b-49c6-9f4e-a46a4477af1e-plugins-config-volume\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:54:08.767849 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.767832    2565 scope.go:117] "RemoveContainer" containerID="9886ef7c741a280c1a00cf6e4b0fb2d2a164cea3117f671a5ffbab96b4082c8c"
Sep 15 22:54:08.768160 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:54:08.768132    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9886ef7c741a280c1a00cf6e4b0fb2d2a164cea3117f671a5ffbab96b4082c8c\": container with ID starting with 9886ef7c741a280c1a00cf6e4b0fb2d2a164cea3117f671a5ffbab96b4082c8c not found: ID does not exist" containerID="9886ef7c741a280c1a00cf6e4b0fb2d2a164cea3117f671a5ffbab96b4082c8c"
Sep 15 22:54:08.768230 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.768167    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9886ef7c741a280c1a00cf6e4b0fb2d2a164cea3117f671a5ffbab96b4082c8c"} err="failed to get container status \"9886ef7c741a280c1a00cf6e4b0fb2d2a164cea3117f671a5ffbab96b4082c8c\": rpc error: code = NotFound desc = could not find container \"9886ef7c741a280c1a00cf6e4b0fb2d2a164cea3117f671a5ffbab96b4082c8c\": container with ID starting with 9886ef7c741a280c1a00cf6e4b0fb2d2a164cea3117f671a5ffbab96b4082c8c not found: ID does not exist"
Sep 15 22:54:08.768230 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.768184    2565 scope.go:117] "RemoveContainer" containerID="56c2cd5849168aa35e9f9a005ae4f431c938bf13fa4b52805d989bd301f95b42"
Sep 15 22:54:08.775856 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.775830    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f"]
Sep 15 22:54:08.778752 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.778728    2565 scope.go:117] "RemoveContainer" containerID="56c2cd5849168aa35e9f9a005ae4f431c938bf13fa4b52805d989bd301f95b42"
Sep 15 22:54:08.779147 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:54:08.779119    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56c2cd5849168aa35e9f9a005ae4f431c938bf13fa4b52805d989bd301f95b42\": container with ID starting with 56c2cd5849168aa35e9f9a005ae4f431c938bf13fa4b52805d989bd301f95b42 not found: ID does not exist" containerID="56c2cd5849168aa35e9f9a005ae4f431c938bf13fa4b52805d989bd301f95b42"
Sep 15 22:54:08.779217 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.779158    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56c2cd5849168aa35e9f9a005ae4f431c938bf13fa4b52805d989bd301f95b42"} err="failed to get container status \"56c2cd5849168aa35e9f9a005ae4f431c938bf13fa4b52805d989bd301f95b42\": rpc error: code = NotFound desc = could not find container \"56c2cd5849168aa35e9f9a005ae4f431c938bf13fa4b52805d989bd301f95b42\": container with ID starting with 56c2cd5849168aa35e9f9a005ae4f431c938bf13fa4b52805d989bd301f95b42 not found: ID does not exist"
Sep 15 22:54:08.781332 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.781311    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f2241f-669bf664fc-mm98f"]
Sep 15 22:54:08.791491 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.791465    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f2241f-586669d47d-lc898"]
Sep 15 22:54:08.797715 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:08.797550    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f2241f-586669d47d-lc898"]
Sep 15 22:54:09.495449 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:09.495361    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-fe020d-6dfcbdd58f-2vjb6"]
Sep 15 22:54:09.495992 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:09.495948    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-fe020d-6dfcbdd58f-2vjb6" podUID="caaa5987-ca96-4c13-937d-fa83342f056c" containerName="payload-processing" containerID="cri-o://12dcb2e13132c33f0f6adad25a30293b5af8de9168916e4f096037320c15dad3" gracePeriod=30
Sep 15 22:54:09.754823 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:09.754762    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-fe020d-6dfcbdd58f-2vjb6"
Sep 15 22:54:09.766873 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:09.766845    2565 generic.go:356] "Generic (PLEG): container finished" podID="caaa5987-ca96-4c13-937d-fa83342f056c" containerID="12dcb2e13132c33f0f6adad25a30293b5af8de9168916e4f096037320c15dad3" exitCode=0
Sep 15 22:54:09.767050 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:09.766883    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-fe020d-6dfcbdd58f-2vjb6"
Sep 15 22:54:09.767050 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:09.766886    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-fe020d-6dfcbdd58f-2vjb6" event={"ID":"caaa5987-ca96-4c13-937d-fa83342f056c","Type":"ContainerDied","Data":"12dcb2e13132c33f0f6adad25a30293b5af8de9168916e4f096037320c15dad3"}
Sep 15 22:54:09.767050 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:09.766944    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-fe020d-6dfcbdd58f-2vjb6" event={"ID":"caaa5987-ca96-4c13-937d-fa83342f056c","Type":"ContainerDied","Data":"51892ff2208df6a290aa572541c808ff1ad3ac6e0f9265c772a2e45c5bdb6367"}
Sep 15 22:54:09.767050 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:09.766963    2565 scope.go:117] "RemoveContainer" containerID="12dcb2e13132c33f0f6adad25a30293b5af8de9168916e4f096037320c15dad3"
Sep 15 22:54:09.778648 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:09.778627    2565 scope.go:117] "RemoveContainer" containerID="12dcb2e13132c33f0f6adad25a30293b5af8de9168916e4f096037320c15dad3"
Sep 15 22:54:09.778962 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:54:09.778936    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12dcb2e13132c33f0f6adad25a30293b5af8de9168916e4f096037320c15dad3\": container with ID starting with 12dcb2e13132c33f0f6adad25a30293b5af8de9168916e4f096037320c15dad3 not found: ID does not exist" containerID="12dcb2e13132c33f0f6adad25a30293b5af8de9168916e4f096037320c15dad3"
Sep 15 22:54:09.779046 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:09.778972    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12dcb2e13132c33f0f6adad25a30293b5af8de9168916e4f096037320c15dad3"} err="failed to get container status \"12dcb2e13132c33f0f6adad25a30293b5af8de9168916e4f096037320c15dad3\": rpc error: code = NotFound desc = could not find container \"12dcb2e13132c33f0f6adad25a30293b5af8de9168916e4f096037320c15dad3\": container with ID starting with 12dcb2e13132c33f0f6adad25a30293b5af8de9168916e4f096037320c15dad3 not found: ID does not exist"
Sep 15 22:54:09.874968 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:09.874935    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6mc4d\" (UniqueName: \"kubernetes.io/projected/caaa5987-ca96-4c13-937d-fa83342f056c-kube-api-access-6mc4d\") pod \"caaa5987-ca96-4c13-937d-fa83342f056c\" (UID: \"caaa5987-ca96-4c13-937d-fa83342f056c\") "
Sep 15 22:54:09.875151 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:09.875089    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/caaa5987-ca96-4c13-937d-fa83342f056c-plugins-config-volume\") pod \"caaa5987-ca96-4c13-937d-fa83342f056c\" (UID: \"caaa5987-ca96-4c13-937d-fa83342f056c\") "
Sep 15 22:54:09.875458 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:09.875432    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/caaa5987-ca96-4c13-937d-fa83342f056c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "caaa5987-ca96-4c13-937d-fa83342f056c" (UID: "caaa5987-ca96-4c13-937d-fa83342f056c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:54:09.877109 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:09.877085    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/caaa5987-ca96-4c13-937d-fa83342f056c-kube-api-access-6mc4d" (OuterVolumeSpecName: "kube-api-access-6mc4d") pod "caaa5987-ca96-4c13-937d-fa83342f056c" (UID: "caaa5987-ca96-4c13-937d-fa83342f056c"). InnerVolumeSpecName "kube-api-access-6mc4d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:54:09.976529 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:09.976499    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mc4d\" (UniqueName: \"kubernetes.io/projected/caaa5987-ca96-4c13-937d-fa83342f056c-kube-api-access-6mc4d\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:54:09.976529 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:09.976527    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/caaa5987-ca96-4c13-937d-fa83342f056c-plugins-config-volume\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:54:10.097221 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:10.097189    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-fe020d-6dfcbdd58f-2vjb6"]
Sep 15 22:54:10.121420 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:10.121392    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-fe020d-6dfcbdd58f-2vjb6"]
Sep 15 22:54:10.591834 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:10.591802    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7092845b-136b-4c47-9d1a-706ea4dbad85" path="/var/lib/kubelet/pods/7092845b-136b-4c47-9d1a-706ea4dbad85/volumes"
Sep 15 22:54:10.592203 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:10.592193    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4d9b866-cf8b-49c6-9f4e-a46a4477af1e" path="/var/lib/kubelet/pods/c4d9b866-cf8b-49c6-9f4e-a46a4477af1e/volumes"
Sep 15 22:54:10.592512 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:10.592499    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="caaa5987-ca96-4c13-937d-fa83342f056c" path="/var/lib/kubelet/pods/caaa5987-ca96-4c13-937d-fa83342f056c/volumes"
Sep 15 22:54:26.027186 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.027099    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh"]
Sep 15 22:54:26.027706 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.027419    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh" podUID="42a59c38-9365-4f8a-bed2-e5ed68013089" containerName="maas-api" containerID="cri-o://5055552aa0a953c18605109d201bec1986a417be576e2d597e49196cf6ab7b64" gracePeriod=30
Sep 15 22:54:26.077766 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.077734    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-4f70a9-ffbd9b96f-87qsw"]
Sep 15 22:54:26.077997 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.077950    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4f70a9-ffbd9b96f-87qsw" podUID="da98f27a-4090-410e-9203-c2be6b669e7c" containerName="payload-processing" containerID="cri-o://b3de325fc3c4eeb8a3aaaab6c9a69352f244b171b357637a92fbc5c688a5dfb8" gracePeriod=30
Sep 15 22:54:26.087251 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.087219    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f70a9-d4d9dcd-nxh4d"]
Sep 15 22:54:26.087515 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.087467    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f70a9-d4d9dcd-nxh4d" podUID="43d18bd7-ffb8-4bb3-a79e-6415f5ae80d7" containerName="payload-pre-processing" containerID="cri-o://f9125318bffd0c09f693afc8e3d1eb6bde976e9a415534a6787eeb0f752aa29d" gracePeriod=30
Sep 15 22:54:26.403775 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.403742    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4f70a9-ffbd9b96f-87qsw"
Sep 15 22:54:26.407963 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.407885    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh"
Sep 15 22:54:26.412530 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.412509    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f70a9-d4d9dcd-nxh4d"
Sep 15 22:54:26.437714 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.437652    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gg9c7\" (UniqueName: \"kubernetes.io/projected/da98f27a-4090-410e-9203-c2be6b669e7c-kube-api-access-gg9c7\") pod \"da98f27a-4090-410e-9203-c2be6b669e7c\" (UID: \"da98f27a-4090-410e-9203-c2be6b669e7c\") "
Sep 15 22:54:26.437714 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.437695    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7kcf\" (UniqueName: \"kubernetes.io/projected/43d18bd7-ffb8-4bb3-a79e-6415f5ae80d7-kube-api-access-x7kcf\") pod \"43d18bd7-ffb8-4bb3-a79e-6415f5ae80d7\" (UID: \"43d18bd7-ffb8-4bb3-a79e-6415f5ae80d7\") "
Sep 15 22:54:26.437714 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.437720    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da98f27a-4090-410e-9203-c2be6b669e7c-plugins-config-volume\") pod \"da98f27a-4090-410e-9203-c2be6b669e7c\" (UID: \"da98f27a-4090-410e-9203-c2be6b669e7c\") "
Sep 15 22:54:26.438074 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.437749    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42a59c38-9365-4f8a-bed2-e5ed68013089-maas-api-tls\") pod \"42a59c38-9365-4f8a-bed2-e5ed68013089\" (UID: \"42a59c38-9365-4f8a-bed2-e5ed68013089\") "
Sep 15 22:54:26.438074 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.437775    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6wb5z\" (UniqueName: \"kubernetes.io/projected/42a59c38-9365-4f8a-bed2-e5ed68013089-kube-api-access-6wb5z\") pod \"42a59c38-9365-4f8a-bed2-e5ed68013089\" (UID: \"42a59c38-9365-4f8a-bed2-e5ed68013089\") "
Sep 15 22:54:26.438074 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.437827    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43d18bd7-ffb8-4bb3-a79e-6415f5ae80d7-plugins-config-volume\") pod \"43d18bd7-ffb8-4bb3-a79e-6415f5ae80d7\" (UID: \"43d18bd7-ffb8-4bb3-a79e-6415f5ae80d7\") "
Sep 15 22:54:26.438074 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.437858    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42a59c38-9365-4f8a-bed2-e5ed68013089-openshift-service-ca\") pod \"42a59c38-9365-4f8a-bed2-e5ed68013089\" (UID: \"42a59c38-9365-4f8a-bed2-e5ed68013089\") "
Sep 15 22:54:26.438074 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.437932    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42a59c38-9365-4f8a-bed2-e5ed68013089-metrics-certs\") pod \"42a59c38-9365-4f8a-bed2-e5ed68013089\" (UID: \"42a59c38-9365-4f8a-bed2-e5ed68013089\") "
Sep 15 22:54:26.438704 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.438459    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43d18bd7-ffb8-4bb3-a79e-6415f5ae80d7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "43d18bd7-ffb8-4bb3-a79e-6415f5ae80d7" (UID: "43d18bd7-ffb8-4bb3-a79e-6415f5ae80d7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:54:26.439007 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.438955    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42a59c38-9365-4f8a-bed2-e5ed68013089-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "42a59c38-9365-4f8a-bed2-e5ed68013089" (UID: "42a59c38-9365-4f8a-bed2-e5ed68013089"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:54:26.439007 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.438975    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da98f27a-4090-410e-9203-c2be6b669e7c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "da98f27a-4090-410e-9203-c2be6b669e7c" (UID: "da98f27a-4090-410e-9203-c2be6b669e7c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:54:26.442207 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.442162    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da98f27a-4090-410e-9203-c2be6b669e7c-kube-api-access-gg9c7" (OuterVolumeSpecName: "kube-api-access-gg9c7") pod "da98f27a-4090-410e-9203-c2be6b669e7c" (UID: "da98f27a-4090-410e-9203-c2be6b669e7c"). InnerVolumeSpecName "kube-api-access-gg9c7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:54:26.442207 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.442176    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42a59c38-9365-4f8a-bed2-e5ed68013089-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "42a59c38-9365-4f8a-bed2-e5ed68013089" (UID: "42a59c38-9365-4f8a-bed2-e5ed68013089"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:54:26.442207 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.442193    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42a59c38-9365-4f8a-bed2-e5ed68013089-kube-api-access-6wb5z" (OuterVolumeSpecName: "kube-api-access-6wb5z") pod "42a59c38-9365-4f8a-bed2-e5ed68013089" (UID: "42a59c38-9365-4f8a-bed2-e5ed68013089"). InnerVolumeSpecName "kube-api-access-6wb5z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:54:26.442525 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.442316    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43d18bd7-ffb8-4bb3-a79e-6415f5ae80d7-kube-api-access-x7kcf" (OuterVolumeSpecName: "kube-api-access-x7kcf") pod "43d18bd7-ffb8-4bb3-a79e-6415f5ae80d7" (UID: "43d18bd7-ffb8-4bb3-a79e-6415f5ae80d7"). InnerVolumeSpecName "kube-api-access-x7kcf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:54:26.443949 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.443925    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42a59c38-9365-4f8a-bed2-e5ed68013089-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "42a59c38-9365-4f8a-bed2-e5ed68013089" (UID: "42a59c38-9365-4f8a-bed2-e5ed68013089"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:54:26.539447 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.539408    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42a59c38-9365-4f8a-bed2-e5ed68013089-metrics-certs\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:54:26.539447 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.539444    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gg9c7\" (UniqueName: \"kubernetes.io/projected/da98f27a-4090-410e-9203-c2be6b669e7c-kube-api-access-gg9c7\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:54:26.539669 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.539462    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7kcf\" (UniqueName: \"kubernetes.io/projected/43d18bd7-ffb8-4bb3-a79e-6415f5ae80d7-kube-api-access-x7kcf\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:54:26.539669 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.539477    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da98f27a-4090-410e-9203-c2be6b669e7c-plugins-config-volume\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:54:26.539669 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.539486    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42a59c38-9365-4f8a-bed2-e5ed68013089-maas-api-tls\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:54:26.539669 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.539494    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6wb5z\" (UniqueName: \"kubernetes.io/projected/42a59c38-9365-4f8a-bed2-e5ed68013089-kube-api-access-6wb5z\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:54:26.539669 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.539502    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43d18bd7-ffb8-4bb3-a79e-6415f5ae80d7-plugins-config-volume\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:54:26.539669 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.539511    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42a59c38-9365-4f8a-bed2-e5ed68013089-openshift-service-ca\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:54:26.886427 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.886336    2565 generic.go:356] "Generic (PLEG): container finished" podID="da98f27a-4090-410e-9203-c2be6b669e7c" containerID="b3de325fc3c4eeb8a3aaaab6c9a69352f244b171b357637a92fbc5c688a5dfb8" exitCode=0
Sep 15 22:54:26.886427 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.886389    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4f70a9-ffbd9b96f-87qsw"
Sep 15 22:54:26.886642 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.886423    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4f70a9-ffbd9b96f-87qsw" event={"ID":"da98f27a-4090-410e-9203-c2be6b669e7c","Type":"ContainerDied","Data":"b3de325fc3c4eeb8a3aaaab6c9a69352f244b171b357637a92fbc5c688a5dfb8"}
Sep 15 22:54:26.886642 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.886479    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4f70a9-ffbd9b96f-87qsw" event={"ID":"da98f27a-4090-410e-9203-c2be6b669e7c","Type":"ContainerDied","Data":"abae8691a66e6497495b3b062a4342cd8b7f62c5e42ce4a0d2a443810b5035e0"}
Sep 15 22:54:26.886642 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.886510    2565 scope.go:117] "RemoveContainer" containerID="b3de325fc3c4eeb8a3aaaab6c9a69352f244b171b357637a92fbc5c688a5dfb8"
Sep 15 22:54:26.888066 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.888019    2565 generic.go:356] "Generic (PLEG): container finished" podID="43d18bd7-ffb8-4bb3-a79e-6415f5ae80d7" containerID="f9125318bffd0c09f693afc8e3d1eb6bde976e9a415534a6787eeb0f752aa29d" exitCode=0
Sep 15 22:54:26.888173 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.888080    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f70a9-d4d9dcd-nxh4d"
Sep 15 22:54:26.888236 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.888193    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f70a9-d4d9dcd-nxh4d" event={"ID":"43d18bd7-ffb8-4bb3-a79e-6415f5ae80d7","Type":"ContainerDied","Data":"f9125318bffd0c09f693afc8e3d1eb6bde976e9a415534a6787eeb0f752aa29d"}
Sep 15 22:54:26.888285 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.888239    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f70a9-d4d9dcd-nxh4d" event={"ID":"43d18bd7-ffb8-4bb3-a79e-6415f5ae80d7","Type":"ContainerDied","Data":"95b54b103bb20f4b9b00ba2276e054f1751d4e3ce0653c3240924031f386e417"}
Sep 15 22:54:26.889986 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.889963    2565 generic.go:356] "Generic (PLEG): container finished" podID="42a59c38-9365-4f8a-bed2-e5ed68013089" containerID="5055552aa0a953c18605109d201bec1986a417be576e2d597e49196cf6ab7b64" exitCode=0
Sep 15 22:54:26.890110 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.890015    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh"
Sep 15 22:54:26.890110 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.890053    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh" event={"ID":"42a59c38-9365-4f8a-bed2-e5ed68013089","Type":"ContainerDied","Data":"5055552aa0a953c18605109d201bec1986a417be576e2d597e49196cf6ab7b64"}
Sep 15 22:54:26.890456 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.890120    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh" event={"ID":"42a59c38-9365-4f8a-bed2-e5ed68013089","Type":"ContainerDied","Data":"493bc1b8d1e72e4d56c05ee5d7ae789e87e85699cef4feda5978b28d1eac5a84"}
Sep 15 22:54:26.897067 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.897054    2565 scope.go:117] "RemoveContainer" containerID="b3de325fc3c4eeb8a3aaaab6c9a69352f244b171b357637a92fbc5c688a5dfb8"
Sep 15 22:54:26.897353 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:54:26.897335    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3de325fc3c4eeb8a3aaaab6c9a69352f244b171b357637a92fbc5c688a5dfb8\": container with ID starting with b3de325fc3c4eeb8a3aaaab6c9a69352f244b171b357637a92fbc5c688a5dfb8 not found: ID does not exist" containerID="b3de325fc3c4eeb8a3aaaab6c9a69352f244b171b357637a92fbc5c688a5dfb8"
Sep 15 22:54:26.897403 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.897360    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3de325fc3c4eeb8a3aaaab6c9a69352f244b171b357637a92fbc5c688a5dfb8"} err="failed to get container status \"b3de325fc3c4eeb8a3aaaab6c9a69352f244b171b357637a92fbc5c688a5dfb8\": rpc error: code = NotFound desc = could not find container \"b3de325fc3c4eeb8a3aaaab6c9a69352f244b171b357637a92fbc5c688a5dfb8\": container with ID starting with b3de325fc3c4eeb8a3aaaab6c9a69352f244b171b357637a92fbc5c688a5dfb8 not found: ID does not exist"
Sep 15 22:54:26.897403 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.897378    2565 scope.go:117] "RemoveContainer" containerID="f9125318bffd0c09f693afc8e3d1eb6bde976e9a415534a6787eeb0f752aa29d"
Sep 15 22:54:26.906878 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.906851    2565 scope.go:117] "RemoveContainer" containerID="f9125318bffd0c09f693afc8e3d1eb6bde976e9a415534a6787eeb0f752aa29d"
Sep 15 22:54:26.907183 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:54:26.907159    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9125318bffd0c09f693afc8e3d1eb6bde976e9a415534a6787eeb0f752aa29d\": container with ID starting with f9125318bffd0c09f693afc8e3d1eb6bde976e9a415534a6787eeb0f752aa29d not found: ID does not exist" containerID="f9125318bffd0c09f693afc8e3d1eb6bde976e9a415534a6787eeb0f752aa29d"
Sep 15 22:54:26.907267 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.907192    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9125318bffd0c09f693afc8e3d1eb6bde976e9a415534a6787eeb0f752aa29d"} err="failed to get container status \"f9125318bffd0c09f693afc8e3d1eb6bde976e9a415534a6787eeb0f752aa29d\": rpc error: code = NotFound desc = could not find container \"f9125318bffd0c09f693afc8e3d1eb6bde976e9a415534a6787eeb0f752aa29d\": container with ID starting with f9125318bffd0c09f693afc8e3d1eb6bde976e9a415534a6787eeb0f752aa29d not found: ID does not exist"
Sep 15 22:54:26.907267 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.907216    2565 scope.go:117] "RemoveContainer" containerID="5055552aa0a953c18605109d201bec1986a417be576e2d597e49196cf6ab7b64"
Sep 15 22:54:26.908693 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.908671    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-4f70a9-ffbd9b96f-87qsw"]
Sep 15 22:54:26.915826 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.915805    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-4f70a9-ffbd9b96f-87qsw"]
Sep 15 22:54:26.916628 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.916615    2565 scope.go:117] "RemoveContainer" containerID="5055552aa0a953c18605109d201bec1986a417be576e2d597e49196cf6ab7b64"
Sep 15 22:54:26.916888 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:54:26.916868    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5055552aa0a953c18605109d201bec1986a417be576e2d597e49196cf6ab7b64\": container with ID starting with 5055552aa0a953c18605109d201bec1986a417be576e2d597e49196cf6ab7b64 not found: ID does not exist" containerID="5055552aa0a953c18605109d201bec1986a417be576e2d597e49196cf6ab7b64"
Sep 15 22:54:26.916952 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.916898    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5055552aa0a953c18605109d201bec1986a417be576e2d597e49196cf6ab7b64"} err="failed to get container status \"5055552aa0a953c18605109d201bec1986a417be576e2d597e49196cf6ab7b64\": rpc error: code = NotFound desc = could not find container \"5055552aa0a953c18605109d201bec1986a417be576e2d597e49196cf6ab7b64\": container with ID starting with 5055552aa0a953c18605109d201bec1986a417be576e2d597e49196cf6ab7b64 not found: ID does not exist"
Sep 15 22:54:26.925582 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.925557    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh"]
Sep 15 22:54:26.931862 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.931838    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4f70a9-6b8c6c684-cvnzh"]
Sep 15 22:54:26.941006 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.940986    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f70a9-d4d9dcd-nxh4d"]
Sep 15 22:54:26.944817 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:26.944792    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4f70a9-d4d9dcd-nxh4d"]
Sep 15 22:54:28.592050 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:28.591993    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42a59c38-9365-4f8a-bed2-e5ed68013089" path="/var/lib/kubelet/pods/42a59c38-9365-4f8a-bed2-e5ed68013089/volumes"
Sep 15 22:54:28.592432 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:28.592384    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43d18bd7-ffb8-4bb3-a79e-6415f5ae80d7" path="/var/lib/kubelet/pods/43d18bd7-ffb8-4bb3-a79e-6415f5ae80d7/volumes"
Sep 15 22:54:28.592696 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:28.592683    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da98f27a-4090-410e-9203-c2be6b669e7c" path="/var/lib/kubelet/pods/da98f27a-4090-410e-9203-c2be6b669e7c/volumes"
Sep 15 22:54:48.800982 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:48.800949    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx"]
Sep 15 22:54:48.801494 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:48.801236    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx" podUID="ad7fe46d-f88b-434d-ad47-92372e7c1ba8" containerName="maas-api" containerID="cri-o://52f45ad3405551add58a4e76312f8798edbb8736ad1c169dd70c38a286b5fa1d" gracePeriod=30
Sep 15 22:54:49.036665 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:49.036629    2565 generic.go:356] "Generic (PLEG): container finished" podID="ad7fe46d-f88b-434d-ad47-92372e7c1ba8" containerID="52f45ad3405551add58a4e76312f8798edbb8736ad1c169dd70c38a286b5fa1d" exitCode=0
Sep 15 22:54:49.036822 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:49.036704    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx" event={"ID":"ad7fe46d-f88b-434d-ad47-92372e7c1ba8","Type":"ContainerDied","Data":"52f45ad3405551add58a4e76312f8798edbb8736ad1c169dd70c38a286b5fa1d"}
Sep 15 22:54:49.068876 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:49.068848    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx"
Sep 15 22:54:49.155338 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:49.155305    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad7fe46d-f88b-434d-ad47-92372e7c1ba8-openshift-service-ca\") pod \"ad7fe46d-f88b-434d-ad47-92372e7c1ba8\" (UID: \"ad7fe46d-f88b-434d-ad47-92372e7c1ba8\") "
Sep 15 22:54:49.155497 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:49.155418    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad7fe46d-f88b-434d-ad47-92372e7c1ba8-metrics-certs\") pod \"ad7fe46d-f88b-434d-ad47-92372e7c1ba8\" (UID: \"ad7fe46d-f88b-434d-ad47-92372e7c1ba8\") "
Sep 15 22:54:49.155497 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:49.155465    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gqj59\" (UniqueName: \"kubernetes.io/projected/ad7fe46d-f88b-434d-ad47-92372e7c1ba8-kube-api-access-gqj59\") pod \"ad7fe46d-f88b-434d-ad47-92372e7c1ba8\" (UID: \"ad7fe46d-f88b-434d-ad47-92372e7c1ba8\") "
Sep 15 22:54:49.155646 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:49.155542    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad7fe46d-f88b-434d-ad47-92372e7c1ba8-maas-api-tls\") pod \"ad7fe46d-f88b-434d-ad47-92372e7c1ba8\" (UID: \"ad7fe46d-f88b-434d-ad47-92372e7c1ba8\") "
Sep 15 22:54:49.155736 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:49.155711    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad7fe46d-f88b-434d-ad47-92372e7c1ba8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ad7fe46d-f88b-434d-ad47-92372e7c1ba8" (UID: "ad7fe46d-f88b-434d-ad47-92372e7c1ba8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 22:54:49.155869 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:49.155851    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad7fe46d-f88b-434d-ad47-92372e7c1ba8-openshift-service-ca\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:54:49.157591 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:49.157556    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad7fe46d-f88b-434d-ad47-92372e7c1ba8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ad7fe46d-f88b-434d-ad47-92372e7c1ba8" (UID: "ad7fe46d-f88b-434d-ad47-92372e7c1ba8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:54:49.157703 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:49.157600    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad7fe46d-f88b-434d-ad47-92372e7c1ba8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ad7fe46d-f88b-434d-ad47-92372e7c1ba8" (UID: "ad7fe46d-f88b-434d-ad47-92372e7c1ba8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:54:49.157703 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:49.157626    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad7fe46d-f88b-434d-ad47-92372e7c1ba8-kube-api-access-gqj59" (OuterVolumeSpecName: "kube-api-access-gqj59") pod "ad7fe46d-f88b-434d-ad47-92372e7c1ba8" (UID: "ad7fe46d-f88b-434d-ad47-92372e7c1ba8"). InnerVolumeSpecName "kube-api-access-gqj59". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:54:49.256742 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:49.256702    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad7fe46d-f88b-434d-ad47-92372e7c1ba8-metrics-certs\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:54:49.256742 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:49.256736    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gqj59\" (UniqueName: \"kubernetes.io/projected/ad7fe46d-f88b-434d-ad47-92372e7c1ba8-kube-api-access-gqj59\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:54:49.256742 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:49.256751    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad7fe46d-f88b-434d-ad47-92372e7c1ba8-maas-api-tls\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:54:50.043951 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:50.043917    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx"
Sep 15 22:54:50.043951 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:50.043928    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx" event={"ID":"ad7fe46d-f88b-434d-ad47-92372e7c1ba8","Type":"ContainerDied","Data":"7864b10a0f4763f7fc86d4ef56c1c5bc1ee98cdfecf31bd0dd4e25873bfea88b"}
Sep 15 22:54:50.044491 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:50.043994    2565 scope.go:117] "RemoveContainer" containerID="52f45ad3405551add58a4e76312f8798edbb8736ad1c169dd70c38a286b5fa1d"
Sep 15 22:54:50.069255 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:50.069225    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx"]
Sep 15 22:54:50.071924 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:50.071902    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-03f258-6b4b8c76cf-77tfx"]
Sep 15 22:54:50.591531 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:54:50.591501    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad7fe46d-f88b-434d-ad47-92372e7c1ba8" path="/var/lib/kubelet/pods/ad7fe46d-f88b-434d-ad47-92372e7c1ba8/volumes"
Sep 15 22:57:16.666496 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.666456    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-0016d4/rate-test-model-0016d4-kserve-5686f99bbc-t4fbf"]
Sep 15 22:57:16.666937 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.666896    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da98f27a-4090-410e-9203-c2be6b669e7c" containerName="payload-processing"
Sep 15 22:57:16.666937 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.666909    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="da98f27a-4090-410e-9203-c2be6b669e7c" containerName="payload-processing"
Sep 15 22:57:16.667013 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.666948    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4d9b866-cf8b-49c6-9f4e-a46a4477af1e" containerName="payload-pre-processing"
Sep 15 22:57:16.667013 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.666955    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4d9b866-cf8b-49c6-9f4e-a46a4477af1e" containerName="payload-pre-processing"
Sep 15 22:57:16.667013 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.666962    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2dac7197-131b-4040-82c4-feb64d4665a6" containerName="maas-api"
Sep 15 22:57:16.667013 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.666968    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dac7197-131b-4040-82c4-feb64d4665a6" containerName="maas-api"
Sep 15 22:57:16.667013 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.666978    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43d18bd7-ffb8-4bb3-a79e-6415f5ae80d7" containerName="payload-pre-processing"
Sep 15 22:57:16.667013 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.666984    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="43d18bd7-ffb8-4bb3-a79e-6415f5ae80d7" containerName="payload-pre-processing"
Sep 15 22:57:16.667013 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.666991    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad7fe46d-f88b-434d-ad47-92372e7c1ba8" containerName="maas-api"
Sep 15 22:57:16.667013 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.666996    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad7fe46d-f88b-434d-ad47-92372e7c1ba8" containerName="maas-api"
Sep 15 22:57:16.667013 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.667008    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="caaa5987-ca96-4c13-937d-fa83342f056c" containerName="payload-processing"
Sep 15 22:57:16.667013 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.667016    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="caaa5987-ca96-4c13-937d-fa83342f056c" containerName="payload-processing"
Sep 15 22:57:16.667354 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.667036    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7092845b-136b-4c47-9d1a-706ea4dbad85" containerName="maas-api"
Sep 15 22:57:16.667354 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.667045    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7092845b-136b-4c47-9d1a-706ea4dbad85" containerName="maas-api"
Sep 15 22:57:16.667354 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.667058    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ffd92a3-8f8f-48a2-84eb-379cb320f48d" containerName="payload-processing"
Sep 15 22:57:16.667354 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.667064    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ffd92a3-8f8f-48a2-84eb-379cb320f48d" containerName="payload-processing"
Sep 15 22:57:16.667354 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.667071    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fbcc540-a2af-4f10-b9bd-08f82930b8b3" containerName="payload-pre-processing"
Sep 15 22:57:16.667354 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.667076    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fbcc540-a2af-4f10-b9bd-08f82930b8b3" containerName="payload-pre-processing"
Sep 15 22:57:16.667354 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.667092    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42a59c38-9365-4f8a-bed2-e5ed68013089" containerName="maas-api"
Sep 15 22:57:16.667354 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.667098    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="42a59c38-9365-4f8a-bed2-e5ed68013089" containerName="maas-api"
Sep 15 22:57:16.667354 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.667174    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ffd92a3-8f8f-48a2-84eb-379cb320f48d" containerName="payload-processing"
Sep 15 22:57:16.667354 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.667183    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2dac7197-131b-4040-82c4-feb64d4665a6" containerName="maas-api"
Sep 15 22:57:16.667354 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.667188    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="42a59c38-9365-4f8a-bed2-e5ed68013089" containerName="maas-api"
Sep 15 22:57:16.667354 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.667195    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad7fe46d-f88b-434d-ad47-92372e7c1ba8" containerName="maas-api"
Sep 15 22:57:16.667354 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.667201    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="caaa5987-ca96-4c13-937d-fa83342f056c" containerName="payload-processing"
Sep 15 22:57:16.667354 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.667206    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7092845b-136b-4c47-9d1a-706ea4dbad85" containerName="maas-api"
Sep 15 22:57:16.667354 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.667214    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="43d18bd7-ffb8-4bb3-a79e-6415f5ae80d7" containerName="payload-pre-processing"
Sep 15 22:57:16.667354 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.667220    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="da98f27a-4090-410e-9203-c2be6b669e7c" containerName="payload-processing"
Sep 15 22:57:16.667354 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.667226    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fbcc540-a2af-4f10-b9bd-08f82930b8b3" containerName="payload-pre-processing"
Sep 15 22:57:16.667354 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.667232    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4d9b866-cf8b-49c6-9f4e-a46a4477af1e" containerName="payload-pre-processing"
Sep 15 22:57:16.670187 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.670169    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-0016d4/rate-test-model-0016d4-kserve-5686f99bbc-t4fbf"
Sep 15 22:57:16.672422 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.672400    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-0016d4\"/\"kube-root-ca.crt\""
Sep 15 22:57:16.673080 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.673055    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-0016d4\"/\"openshift-service-ca.crt\""
Sep 15 22:57:16.673201 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.673061    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-0016d4\"/\"default-dockercfg-nhjlf\""
Sep 15 22:57:16.673201 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.673092    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-0016d4\"/\"rate-test-model-0016d4-kserve-self-signed-certs\""
Sep 15 22:57:16.681687 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.681660    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-0016d4/rate-test-model-0016d4-kserve-5686f99bbc-t4fbf"]
Sep 15 22:57:16.759352 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.759314    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcrgc\" (UniqueName: \"kubernetes.io/projected/c72137db-a8d9-4f2c-aee8-bcce95ad8cf4-kube-api-access-dcrgc\") pod \"rate-test-model-0016d4-kserve-5686f99bbc-t4fbf\" (UID: \"c72137db-a8d9-4f2c-aee8-bcce95ad8cf4\") " pod="ai-tenant-e2e-shared-a-w3-0016d4/rate-test-model-0016d4-kserve-5686f99bbc-t4fbf"
Sep 15 22:57:16.759543 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.759395    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c72137db-a8d9-4f2c-aee8-bcce95ad8cf4-tmp-dir\") pod \"rate-test-model-0016d4-kserve-5686f99bbc-t4fbf\" (UID: \"c72137db-a8d9-4f2c-aee8-bcce95ad8cf4\") " pod="ai-tenant-e2e-shared-a-w3-0016d4/rate-test-model-0016d4-kserve-5686f99bbc-t4fbf"
Sep 15 22:57:16.759543 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.759417    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c72137db-a8d9-4f2c-aee8-bcce95ad8cf4-tls-certs\") pod \"rate-test-model-0016d4-kserve-5686f99bbc-t4fbf\" (UID: \"c72137db-a8d9-4f2c-aee8-bcce95ad8cf4\") " pod="ai-tenant-e2e-shared-a-w3-0016d4/rate-test-model-0016d4-kserve-5686f99bbc-t4fbf"
Sep 15 22:57:16.759543 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.759437    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c72137db-a8d9-4f2c-aee8-bcce95ad8cf4-home\") pod \"rate-test-model-0016d4-kserve-5686f99bbc-t4fbf\" (UID: \"c72137db-a8d9-4f2c-aee8-bcce95ad8cf4\") " pod="ai-tenant-e2e-shared-a-w3-0016d4/rate-test-model-0016d4-kserve-5686f99bbc-t4fbf"
Sep 15 22:57:16.759543 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.759463    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c72137db-a8d9-4f2c-aee8-bcce95ad8cf4-dshm\") pod \"rate-test-model-0016d4-kserve-5686f99bbc-t4fbf\" (UID: \"c72137db-a8d9-4f2c-aee8-bcce95ad8cf4\") " pod="ai-tenant-e2e-shared-a-w3-0016d4/rate-test-model-0016d4-kserve-5686f99bbc-t4fbf"
Sep 15 22:57:16.759695 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.759590    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c72137db-a8d9-4f2c-aee8-bcce95ad8cf4-model-cache\") pod \"rate-test-model-0016d4-kserve-5686f99bbc-t4fbf\" (UID: \"c72137db-a8d9-4f2c-aee8-bcce95ad8cf4\") " pod="ai-tenant-e2e-shared-a-w3-0016d4/rate-test-model-0016d4-kserve-5686f99bbc-t4fbf"
Sep 15 22:57:16.860899 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.860867    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c72137db-a8d9-4f2c-aee8-bcce95ad8cf4-model-cache\") pod \"rate-test-model-0016d4-kserve-5686f99bbc-t4fbf\" (UID: \"c72137db-a8d9-4f2c-aee8-bcce95ad8cf4\") " pod="ai-tenant-e2e-shared-a-w3-0016d4/rate-test-model-0016d4-kserve-5686f99bbc-t4fbf"
Sep 15 22:57:16.861117 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.860909    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcrgc\" (UniqueName: \"kubernetes.io/projected/c72137db-a8d9-4f2c-aee8-bcce95ad8cf4-kube-api-access-dcrgc\") pod \"rate-test-model-0016d4-kserve-5686f99bbc-t4fbf\" (UID: \"c72137db-a8d9-4f2c-aee8-bcce95ad8cf4\") " pod="ai-tenant-e2e-shared-a-w3-0016d4/rate-test-model-0016d4-kserve-5686f99bbc-t4fbf"
Sep 15 22:57:16.861117 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.860963    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c72137db-a8d9-4f2c-aee8-bcce95ad8cf4-tmp-dir\") pod \"rate-test-model-0016d4-kserve-5686f99bbc-t4fbf\" (UID: \"c72137db-a8d9-4f2c-aee8-bcce95ad8cf4\") " pod="ai-tenant-e2e-shared-a-w3-0016d4/rate-test-model-0016d4-kserve-5686f99bbc-t4fbf"
Sep 15 22:57:16.861117 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.861000    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c72137db-a8d9-4f2c-aee8-bcce95ad8cf4-tls-certs\") pod \"rate-test-model-0016d4-kserve-5686f99bbc-t4fbf\" (UID: \"c72137db-a8d9-4f2c-aee8-bcce95ad8cf4\") " pod="ai-tenant-e2e-shared-a-w3-0016d4/rate-test-model-0016d4-kserve-5686f99bbc-t4fbf"
Sep 15 22:57:16.861117 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.861059    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c72137db-a8d9-4f2c-aee8-bcce95ad8cf4-home\") pod \"rate-test-model-0016d4-kserve-5686f99bbc-t4fbf\" (UID: \"c72137db-a8d9-4f2c-aee8-bcce95ad8cf4\") " pod="ai-tenant-e2e-shared-a-w3-0016d4/rate-test-model-0016d4-kserve-5686f99bbc-t4fbf"
Sep 15 22:57:16.861117 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.861088    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c72137db-a8d9-4f2c-aee8-bcce95ad8cf4-dshm\") pod \"rate-test-model-0016d4-kserve-5686f99bbc-t4fbf\" (UID: \"c72137db-a8d9-4f2c-aee8-bcce95ad8cf4\") " pod="ai-tenant-e2e-shared-a-w3-0016d4/rate-test-model-0016d4-kserve-5686f99bbc-t4fbf"
Sep 15 22:57:16.861423 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.861310    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c72137db-a8d9-4f2c-aee8-bcce95ad8cf4-model-cache\") pod \"rate-test-model-0016d4-kserve-5686f99bbc-t4fbf\" (UID: \"c72137db-a8d9-4f2c-aee8-bcce95ad8cf4\") " pod="ai-tenant-e2e-shared-a-w3-0016d4/rate-test-model-0016d4-kserve-5686f99bbc-t4fbf"
Sep 15 22:57:16.861423 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.861375    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c72137db-a8d9-4f2c-aee8-bcce95ad8cf4-tmp-dir\") pod \"rate-test-model-0016d4-kserve-5686f99bbc-t4fbf\" (UID: \"c72137db-a8d9-4f2c-aee8-bcce95ad8cf4\") " pod="ai-tenant-e2e-shared-a-w3-0016d4/rate-test-model-0016d4-kserve-5686f99bbc-t4fbf"
Sep 15 22:57:16.861501 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.861427    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c72137db-a8d9-4f2c-aee8-bcce95ad8cf4-home\") pod \"rate-test-model-0016d4-kserve-5686f99bbc-t4fbf\" (UID: \"c72137db-a8d9-4f2c-aee8-bcce95ad8cf4\") " pod="ai-tenant-e2e-shared-a-w3-0016d4/rate-test-model-0016d4-kserve-5686f99bbc-t4fbf"
Sep 15 22:57:16.863327 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.863308    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c72137db-a8d9-4f2c-aee8-bcce95ad8cf4-dshm\") pod \"rate-test-model-0016d4-kserve-5686f99bbc-t4fbf\" (UID: \"c72137db-a8d9-4f2c-aee8-bcce95ad8cf4\") " pod="ai-tenant-e2e-shared-a-w3-0016d4/rate-test-model-0016d4-kserve-5686f99bbc-t4fbf"
Sep 15 22:57:16.863607 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.863586    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c72137db-a8d9-4f2c-aee8-bcce95ad8cf4-tls-certs\") pod \"rate-test-model-0016d4-kserve-5686f99bbc-t4fbf\" (UID: \"c72137db-a8d9-4f2c-aee8-bcce95ad8cf4\") " pod="ai-tenant-e2e-shared-a-w3-0016d4/rate-test-model-0016d4-kserve-5686f99bbc-t4fbf"
Sep 15 22:57:16.868972 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.868948    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcrgc\" (UniqueName: \"kubernetes.io/projected/c72137db-a8d9-4f2c-aee8-bcce95ad8cf4-kube-api-access-dcrgc\") pod \"rate-test-model-0016d4-kserve-5686f99bbc-t4fbf\" (UID: \"c72137db-a8d9-4f2c-aee8-bcce95ad8cf4\") " pod="ai-tenant-e2e-shared-a-w3-0016d4/rate-test-model-0016d4-kserve-5686f99bbc-t4fbf"
Sep 15 22:57:16.982339 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:16.982247    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-0016d4/rate-test-model-0016d4-kserve-5686f99bbc-t4fbf"
Sep 15 22:57:17.258529 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:17.258501    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-0016d4/rate-test-model-0016d4-kserve-5686f99bbc-t4fbf"]
Sep 15 22:57:17.259855 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:57:17.259829    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc72137db_a8d9_4f2c_aee8_bcce95ad8cf4.slice/crio-f54705d42cd690da90898f169954cdd5d091f0a5c80c57b39abdf00f28ee76bf WatchSource:0}: Error finding container f54705d42cd690da90898f169954cdd5d091f0a5c80c57b39abdf00f28ee76bf: Status 404 returned error can't find the container with id f54705d42cd690da90898f169954cdd5d091f0a5c80c57b39abdf00f28ee76bf
Sep 15 22:57:17.911255 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:17.911223    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-0016d4/rate-test-model-0016d4-kserve-5686f99bbc-t4fbf" event={"ID":"c72137db-a8d9-4f2c-aee8-bcce95ad8cf4","Type":"ContainerStarted","Data":"9f2586ed1a281fd986adaaf6dbaf735368fce7714f6b098817245844ac48eb5c"}
Sep 15 22:57:17.911635 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:17.911263    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-0016d4/rate-test-model-0016d4-kserve-5686f99bbc-t4fbf" event={"ID":"c72137db-a8d9-4f2c-aee8-bcce95ad8cf4","Type":"ContainerStarted","Data":"f54705d42cd690da90898f169954cdd5d091f0a5c80c57b39abdf00f28ee76bf"}
Sep 15 22:57:17.926647 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:17.926582    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-0016d4/rate-test-model-0016d4-kserve-5686f99bbc-t4fbf" podStartSLOduration=1.92656133 podStartE2EDuration="1.92656133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:57:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:57:17.926277743 +0000 UTC m=+1787.785989027" watchObservedRunningTime="2026-09-15 22:57:17.92656133 +0000 UTC m=+1787.786272617"
Sep 15 22:57:26.982732 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:26.982602    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-0016d4/rate-test-model-0016d4-kserve-5686f99bbc-t4fbf"
Sep 15 22:57:26.982732 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:26.982707    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-0016d4/rate-test-model-0016d4-kserve-5686f99bbc-t4fbf"
Sep 15 22:57:26.995316 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:26.995283    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-0016d4/rate-test-model-0016d4-kserve-5686f99bbc-t4fbf"
Sep 15 22:57:27.984443 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:57:27.984414    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-0016d4/rate-test-model-0016d4-kserve-5686f99bbc-t4fbf"
Sep 15 22:58:13.843392 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:13.843348    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-0016d4/rate-test-model-0016d4-kserve-5686f99bbc-t4fbf"]
Sep 15 22:58:28.862914 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:28.862851    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-0016d4/rate-test-model-0016d4-kserve-5686f99bbc-t4fbf" podUID="c72137db-a8d9-4f2c-aee8-bcce95ad8cf4" containerName="main" containerID="cri-o://9f2586ed1a281fd986adaaf6dbaf735368fce7714f6b098817245844ac48eb5c" gracePeriod=45
Sep 15 22:58:29.121486 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:29.121419    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-0016d4/rate-test-model-0016d4-kserve-5686f99bbc-t4fbf"
Sep 15 22:58:29.269327 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:29.269291    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c72137db-a8d9-4f2c-aee8-bcce95ad8cf4-dshm\") pod \"c72137db-a8d9-4f2c-aee8-bcce95ad8cf4\" (UID: \"c72137db-a8d9-4f2c-aee8-bcce95ad8cf4\") "
Sep 15 22:58:29.269327 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:29.269330    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c72137db-a8d9-4f2c-aee8-bcce95ad8cf4-home\") pod \"c72137db-a8d9-4f2c-aee8-bcce95ad8cf4\" (UID: \"c72137db-a8d9-4f2c-aee8-bcce95ad8cf4\") "
Sep 15 22:58:29.269591 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:29.269354    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c72137db-a8d9-4f2c-aee8-bcce95ad8cf4-tls-certs\") pod \"c72137db-a8d9-4f2c-aee8-bcce95ad8cf4\" (UID: \"c72137db-a8d9-4f2c-aee8-bcce95ad8cf4\") "
Sep 15 22:58:29.269591 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:29.269376    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c72137db-a8d9-4f2c-aee8-bcce95ad8cf4-tmp-dir\") pod \"c72137db-a8d9-4f2c-aee8-bcce95ad8cf4\" (UID: \"c72137db-a8d9-4f2c-aee8-bcce95ad8cf4\") "
Sep 15 22:58:29.269591 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:29.269471    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c72137db-a8d9-4f2c-aee8-bcce95ad8cf4-model-cache\") pod \"c72137db-a8d9-4f2c-aee8-bcce95ad8cf4\" (UID: \"c72137db-a8d9-4f2c-aee8-bcce95ad8cf4\") "
Sep 15 22:58:29.269591 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:29.269501    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dcrgc\" (UniqueName: \"kubernetes.io/projected/c72137db-a8d9-4f2c-aee8-bcce95ad8cf4-kube-api-access-dcrgc\") pod \"c72137db-a8d9-4f2c-aee8-bcce95ad8cf4\" (UID: \"c72137db-a8d9-4f2c-aee8-bcce95ad8cf4\") "
Sep 15 22:58:29.269855 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:29.269656    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c72137db-a8d9-4f2c-aee8-bcce95ad8cf4-home" (OuterVolumeSpecName: "home") pod "c72137db-a8d9-4f2c-aee8-bcce95ad8cf4" (UID: "c72137db-a8d9-4f2c-aee8-bcce95ad8cf4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:58:29.269855 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:29.269703    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c72137db-a8d9-4f2c-aee8-bcce95ad8cf4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c72137db-a8d9-4f2c-aee8-bcce95ad8cf4" (UID: "c72137db-a8d9-4f2c-aee8-bcce95ad8cf4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:58:29.269855 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:29.269750    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c72137db-a8d9-4f2c-aee8-bcce95ad8cf4-model-cache" (OuterVolumeSpecName: "model-cache") pod "c72137db-a8d9-4f2c-aee8-bcce95ad8cf4" (UID: "c72137db-a8d9-4f2c-aee8-bcce95ad8cf4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:58:29.269968 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:29.269900    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c72137db-a8d9-4f2c-aee8-bcce95ad8cf4-model-cache\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:58:29.269968 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:29.269919    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c72137db-a8d9-4f2c-aee8-bcce95ad8cf4-home\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:58:29.269968 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:29.269946    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c72137db-a8d9-4f2c-aee8-bcce95ad8cf4-tmp-dir\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:58:29.271607 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:29.271587    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c72137db-a8d9-4f2c-aee8-bcce95ad8cf4-dshm" (OuterVolumeSpecName: "dshm") pod "c72137db-a8d9-4f2c-aee8-bcce95ad8cf4" (UID: "c72137db-a8d9-4f2c-aee8-bcce95ad8cf4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 22:58:29.271692 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:29.271671    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c72137db-a8d9-4f2c-aee8-bcce95ad8cf4-kube-api-access-dcrgc" (OuterVolumeSpecName: "kube-api-access-dcrgc") pod "c72137db-a8d9-4f2c-aee8-bcce95ad8cf4" (UID: "c72137db-a8d9-4f2c-aee8-bcce95ad8cf4"). InnerVolumeSpecName "kube-api-access-dcrgc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 22:58:29.271769 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:29.271731    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c72137db-a8d9-4f2c-aee8-bcce95ad8cf4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c72137db-a8d9-4f2c-aee8-bcce95ad8cf4" (UID: "c72137db-a8d9-4f2c-aee8-bcce95ad8cf4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 22:58:29.345646 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:29.345605    2565 generic.go:356] "Generic (PLEG): container finished" podID="c72137db-a8d9-4f2c-aee8-bcce95ad8cf4" containerID="9f2586ed1a281fd986adaaf6dbaf735368fce7714f6b098817245844ac48eb5c" exitCode=0
Sep 15 22:58:29.345850 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:29.345650    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-0016d4/rate-test-model-0016d4-kserve-5686f99bbc-t4fbf"
Sep 15 22:58:29.345850 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:29.345696    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-0016d4/rate-test-model-0016d4-kserve-5686f99bbc-t4fbf" event={"ID":"c72137db-a8d9-4f2c-aee8-bcce95ad8cf4","Type":"ContainerDied","Data":"9f2586ed1a281fd986adaaf6dbaf735368fce7714f6b098817245844ac48eb5c"}
Sep 15 22:58:29.345850 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:29.345743    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-0016d4/rate-test-model-0016d4-kserve-5686f99bbc-t4fbf" event={"ID":"c72137db-a8d9-4f2c-aee8-bcce95ad8cf4","Type":"ContainerDied","Data":"f54705d42cd690da90898f169954cdd5d091f0a5c80c57b39abdf00f28ee76bf"}
Sep 15 22:58:29.345850 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:29.345763    2565 scope.go:117] "RemoveContainer" containerID="9f2586ed1a281fd986adaaf6dbaf735368fce7714f6b098817245844ac48eb5c"
Sep 15 22:58:29.355964 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:29.355944    2565 scope.go:117] "RemoveContainer" containerID="9f2586ed1a281fd986adaaf6dbaf735368fce7714f6b098817245844ac48eb5c"
Sep 15 22:58:29.356267 ip-10-0-128-208 kubenswrapper[2565]: E0915 22:58:29.356245    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f2586ed1a281fd986adaaf6dbaf735368fce7714f6b098817245844ac48eb5c\": container with ID starting with 9f2586ed1a281fd986adaaf6dbaf735368fce7714f6b098817245844ac48eb5c not found: ID does not exist" containerID="9f2586ed1a281fd986adaaf6dbaf735368fce7714f6b098817245844ac48eb5c"
Sep 15 22:58:29.356322 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:29.356275    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f2586ed1a281fd986adaaf6dbaf735368fce7714f6b098817245844ac48eb5c"} err="failed to get container status \"9f2586ed1a281fd986adaaf6dbaf735368fce7714f6b098817245844ac48eb5c\": rpc error: code = NotFound desc = could not find container \"9f2586ed1a281fd986adaaf6dbaf735368fce7714f6b098817245844ac48eb5c\": container with ID starting with 9f2586ed1a281fd986adaaf6dbaf735368fce7714f6b098817245844ac48eb5c not found: ID does not exist"
Sep 15 22:58:29.367931 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:29.367908    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-0016d4/rate-test-model-0016d4-kserve-5686f99bbc-t4fbf"]
Sep 15 22:58:29.370944 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:29.370917    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dcrgc\" (UniqueName: \"kubernetes.io/projected/c72137db-a8d9-4f2c-aee8-bcce95ad8cf4-kube-api-access-dcrgc\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:58:29.370944 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:29.370941    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c72137db-a8d9-4f2c-aee8-bcce95ad8cf4-dshm\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:58:29.371152 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:29.370956    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c72137db-a8d9-4f2c-aee8-bcce95ad8cf4-tls-certs\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 22:58:29.371365 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:29.371344    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-0016d4/rate-test-model-0016d4-kserve-5686f99bbc-t4fbf"]
Sep 15 22:58:30.590598 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:30.590567    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c72137db-a8d9-4f2c-aee8-bcce95ad8cf4" path="/var/lib/kubelet/pods/c72137db-a8d9-4f2c-aee8-bcce95ad8cf4/volumes"
Sep 15 22:58:51.051800 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.051765    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf"]
Sep 15 22:58:51.052339 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.052214    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c72137db-a8d9-4f2c-aee8-bcce95ad8cf4" containerName="main"
Sep 15 22:58:51.052339 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.052228    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c72137db-a8d9-4f2c-aee8-bcce95ad8cf4" containerName="main"
Sep 15 22:58:51.052339 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.052314    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c72137db-a8d9-4f2c-aee8-bcce95ad8cf4" containerName="main"
Sep 15 22:58:51.055383 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.055359    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf"
Sep 15 22:58:51.057262 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.057236    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-668e8d-serving-cert\""
Sep 15 22:58:51.067047 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.067005    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf"]
Sep 15 22:58:51.184186 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.184153    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64f7b48b-c18a-4f5d-8ad6-dffcadacf163-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf\" (UID: \"64f7b48b-c18a-4f5d-8ad6-dffcadacf163\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf"
Sep 15 22:58:51.184362 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.184213    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64f7b48b-c18a-4f5d-8ad6-dffcadacf163-maas-api-tls\") pod \"maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf\" (UID: \"64f7b48b-c18a-4f5d-8ad6-dffcadacf163\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf"
Sep 15 22:58:51.184362 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.184230    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64f7b48b-c18a-4f5d-8ad6-dffcadacf163-metrics-certs\") pod \"maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf\" (UID: \"64f7b48b-c18a-4f5d-8ad6-dffcadacf163\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf"
Sep 15 22:58:51.184362 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.184256    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shd2b\" (UniqueName: \"kubernetes.io/projected/64f7b48b-c18a-4f5d-8ad6-dffcadacf163-kube-api-access-shd2b\") pod \"maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf\" (UID: \"64f7b48b-c18a-4f5d-8ad6-dffcadacf163\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf"
Sep 15 22:58:51.284831 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.284800    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64f7b48b-c18a-4f5d-8ad6-dffcadacf163-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf\" (UID: \"64f7b48b-c18a-4f5d-8ad6-dffcadacf163\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf"
Sep 15 22:58:51.285080 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.284850    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64f7b48b-c18a-4f5d-8ad6-dffcadacf163-maas-api-tls\") pod \"maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf\" (UID: \"64f7b48b-c18a-4f5d-8ad6-dffcadacf163\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf"
Sep 15 22:58:51.285080 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.284867    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64f7b48b-c18a-4f5d-8ad6-dffcadacf163-metrics-certs\") pod \"maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf\" (UID: \"64f7b48b-c18a-4f5d-8ad6-dffcadacf163\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf"
Sep 15 22:58:51.285080 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.284895    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shd2b\" (UniqueName: \"kubernetes.io/projected/64f7b48b-c18a-4f5d-8ad6-dffcadacf163-kube-api-access-shd2b\") pod \"maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf\" (UID: \"64f7b48b-c18a-4f5d-8ad6-dffcadacf163\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf"
Sep 15 22:58:51.285539 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.285502    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64f7b48b-c18a-4f5d-8ad6-dffcadacf163-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf\" (UID: \"64f7b48b-c18a-4f5d-8ad6-dffcadacf163\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf"
Sep 15 22:58:51.287451 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.287428    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64f7b48b-c18a-4f5d-8ad6-dffcadacf163-maas-api-tls\") pod \"maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf\" (UID: \"64f7b48b-c18a-4f5d-8ad6-dffcadacf163\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf"
Sep 15 22:58:51.287555 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.287535    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64f7b48b-c18a-4f5d-8ad6-dffcadacf163-metrics-certs\") pod \"maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf\" (UID: \"64f7b48b-c18a-4f5d-8ad6-dffcadacf163\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf"
Sep 15 22:58:51.292860 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.292829    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shd2b\" (UniqueName: \"kubernetes.io/projected/64f7b48b-c18a-4f5d-8ad6-dffcadacf163-kube-api-access-shd2b\") pod \"maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf\" (UID: \"64f7b48b-c18a-4f5d-8ad6-dffcadacf163\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf"
Sep 15 22:58:51.367786 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.367700    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf"
Sep 15 22:58:51.647492 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.647451    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf"]
Sep 15 22:58:51.648573 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:58:51.648547    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64f7b48b_c18a_4f5d_8ad6_dffcadacf163.slice/crio-eb0348bf9e2161319ba71b192ef29d685c9cc69e496b43cbce308eb1f35a7e31 WatchSource:0}: Error finding container eb0348bf9e2161319ba71b192ef29d685c9cc69e496b43cbce308eb1f35a7e31: Status 404 returned error can't find the container with id eb0348bf9e2161319ba71b192ef29d685c9cc69e496b43cbce308eb1f35a7e31
Sep 15 22:58:51.649851 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.649831    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 22:58:51.676004 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.675976    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-668e8d-5bff4bc59d-67nhg"]
Sep 15 22:58:51.680872 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.680856    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-668e8d-5bff4bc59d-67nhg"
Sep 15 22:58:51.682799 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.682779    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-668e8d-dockercfg-xg7ln\""
Sep 15 22:58:51.683202 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.683187    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-668e8d\""
Sep 15 22:58:51.688444 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.688420    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-668e8d-5bff4bc59d-67nhg"]
Sep 15 22:58:51.705134 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.705104    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-668e8d-7f86956789-47rrh"]
Sep 15 22:58:51.709436 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.709413    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-668e8d-7f86956789-47rrh"
Sep 15 22:58:51.718725 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.718698    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-668e8d-7f86956789-47rrh"]
Sep 15 22:58:51.790207 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.790175    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w92dk\" (UniqueName: \"kubernetes.io/projected/a1a4a04e-3093-4314-bd73-826876c973d7-kube-api-access-w92dk\") pod \"payload-processing-e2e-ipp-b-668e8d-7f86956789-47rrh\" (UID: \"a1a4a04e-3093-4314-bd73-826876c973d7\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-668e8d-7f86956789-47rrh"
Sep 15 22:58:51.790207 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.790211    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8975b00a-2fc5-4002-82c0-b92207df26fa-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-668e8d-5bff4bc59d-67nhg\" (UID: \"8975b00a-2fc5-4002-82c0-b92207df26fa\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-668e8d-5bff4bc59d-67nhg"
Sep 15 22:58:51.790410 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.790314    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1a4a04e-3093-4314-bd73-826876c973d7-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-668e8d-7f86956789-47rrh\" (UID: \"a1a4a04e-3093-4314-bd73-826876c973d7\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-668e8d-7f86956789-47rrh"
Sep 15 22:58:51.790470 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.790446    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cv6vf\" (UniqueName: \"kubernetes.io/projected/8975b00a-2fc5-4002-82c0-b92207df26fa-kube-api-access-cv6vf\") pod \"payload-pre-processing-e2e-ipp-b-668e8d-5bff4bc59d-67nhg\" (UID: \"8975b00a-2fc5-4002-82c0-b92207df26fa\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-668e8d-5bff4bc59d-67nhg"
Sep 15 22:58:51.891239 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.891200    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1a4a04e-3093-4314-bd73-826876c973d7-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-668e8d-7f86956789-47rrh\" (UID: \"a1a4a04e-3093-4314-bd73-826876c973d7\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-668e8d-7f86956789-47rrh"
Sep 15 22:58:51.891411 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.891303    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cv6vf\" (UniqueName: \"kubernetes.io/projected/8975b00a-2fc5-4002-82c0-b92207df26fa-kube-api-access-cv6vf\") pod \"payload-pre-processing-e2e-ipp-b-668e8d-5bff4bc59d-67nhg\" (UID: \"8975b00a-2fc5-4002-82c0-b92207df26fa\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-668e8d-5bff4bc59d-67nhg"
Sep 15 22:58:51.891411 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.891367    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w92dk\" (UniqueName: \"kubernetes.io/projected/a1a4a04e-3093-4314-bd73-826876c973d7-kube-api-access-w92dk\") pod \"payload-processing-e2e-ipp-b-668e8d-7f86956789-47rrh\" (UID: \"a1a4a04e-3093-4314-bd73-826876c973d7\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-668e8d-7f86956789-47rrh"
Sep 15 22:58:51.891411 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.891397    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8975b00a-2fc5-4002-82c0-b92207df26fa-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-668e8d-5bff4bc59d-67nhg\" (UID: \"8975b00a-2fc5-4002-82c0-b92207df26fa\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-668e8d-5bff4bc59d-67nhg"
Sep 15 22:58:51.892176 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.892155    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1a4a04e-3093-4314-bd73-826876c973d7-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-668e8d-7f86956789-47rrh\" (UID: \"a1a4a04e-3093-4314-bd73-826876c973d7\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-668e8d-7f86956789-47rrh"
Sep 15 22:58:51.892223 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.892191    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8975b00a-2fc5-4002-82c0-b92207df26fa-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-668e8d-5bff4bc59d-67nhg\" (UID: \"8975b00a-2fc5-4002-82c0-b92207df26fa\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-668e8d-5bff4bc59d-67nhg"
Sep 15 22:58:51.899168 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.899108    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w92dk\" (UniqueName: \"kubernetes.io/projected/a1a4a04e-3093-4314-bd73-826876c973d7-kube-api-access-w92dk\") pod \"payload-processing-e2e-ipp-b-668e8d-7f86956789-47rrh\" (UID: \"a1a4a04e-3093-4314-bd73-826876c973d7\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-668e8d-7f86956789-47rrh"
Sep 15 22:58:51.899424 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.899407    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cv6vf\" (UniqueName: \"kubernetes.io/projected/8975b00a-2fc5-4002-82c0-b92207df26fa-kube-api-access-cv6vf\") pod \"payload-pre-processing-e2e-ipp-b-668e8d-5bff4bc59d-67nhg\" (UID: \"8975b00a-2fc5-4002-82c0-b92207df26fa\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-668e8d-5bff4bc59d-67nhg"
Sep 15 22:58:51.994012 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:51.993990    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-668e8d-5bff4bc59d-67nhg"
Sep 15 22:58:52.022366 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:52.022330    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-668e8d-7f86956789-47rrh"
Sep 15 22:58:52.277858 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:52.277832    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-668e8d-5bff4bc59d-67nhg"]
Sep 15 22:58:52.278641 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:58:52.278610    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8975b00a_2fc5_4002_82c0_b92207df26fa.slice/crio-616c95d2e771e73a278814cffe5262c592493043eb73a94444ee586cca48aa0f WatchSource:0}: Error finding container 616c95d2e771e73a278814cffe5262c592493043eb73a94444ee586cca48aa0f: Status 404 returned error can't find the container with id 616c95d2e771e73a278814cffe5262c592493043eb73a94444ee586cca48aa0f
Sep 15 22:58:52.318728 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:52.318703    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-668e8d-7f86956789-47rrh"]
Sep 15 22:58:52.328436 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:58:52.328404    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1a4a04e_3093_4314_bd73_826876c973d7.slice/crio-68a5b621bddddb3b26088e1faf7d3c29ccd50eddae3868644f1854192cbdbdbb WatchSource:0}: Error finding container 68a5b621bddddb3b26088e1faf7d3c29ccd50eddae3868644f1854192cbdbdbb: Status 404 returned error can't find the container with id 68a5b621bddddb3b26088e1faf7d3c29ccd50eddae3868644f1854192cbdbdbb
Sep 15 22:58:52.490683 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:52.490644    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf" event={"ID":"64f7b48b-c18a-4f5d-8ad6-dffcadacf163","Type":"ContainerStarted","Data":"eba5d73962df40563b51a532f6c273d98341ff099918649b9fd1e205c3727618"}
Sep 15 22:58:52.490880 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:52.490707    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf" event={"ID":"64f7b48b-c18a-4f5d-8ad6-dffcadacf163","Type":"ContainerStarted","Data":"eb0348bf9e2161319ba71b192ef29d685c9cc69e496b43cbce308eb1f35a7e31"}
Sep 15 22:58:52.490880 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:52.490873    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf"
Sep 15 22:58:52.492402 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:52.492377    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-668e8d-7f86956789-47rrh" event={"ID":"a1a4a04e-3093-4314-bd73-826876c973d7","Type":"ContainerStarted","Data":"1ca24a59315ef492dbce0394582359a507f73f7e033ed743954dd113bdbd9459"}
Sep 15 22:58:52.492534 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:52.492423    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-668e8d-7f86956789-47rrh" event={"ID":"a1a4a04e-3093-4314-bd73-826876c973d7","Type":"ContainerStarted","Data":"68a5b621bddddb3b26088e1faf7d3c29ccd50eddae3868644f1854192cbdbdbb"}
Sep 15 22:58:52.492534 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:52.492451    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-668e8d-7f86956789-47rrh"
Sep 15 22:58:52.494120 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:52.494097    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-668e8d-5bff4bc59d-67nhg" event={"ID":"8975b00a-2fc5-4002-82c0-b92207df26fa","Type":"ContainerStarted","Data":"2ca417ce43ecbff70e15290c9c438b2fe71ac6e45386f70280a22cab55c04ede"}
Sep 15 22:58:52.494220 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:52.494137    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-668e8d-5bff4bc59d-67nhg" event={"ID":"8975b00a-2fc5-4002-82c0-b92207df26fa","Type":"ContainerStarted","Data":"616c95d2e771e73a278814cffe5262c592493043eb73a94444ee586cca48aa0f"}
Sep 15 22:58:52.494220 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:52.494191    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-668e8d-5bff4bc59d-67nhg"
Sep 15 22:58:52.512439 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:52.512380    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf" podStartSLOduration=1.186927131 podStartE2EDuration="1.512361223s" totalImagesPullingTime="325.434092ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:58:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:58:52.506571318 +0000 UTC m=+1882.366282600" watchObservedRunningTime="2026-09-15 22:58:52.512361223 +0000 UTC m=+1882.372072507"
Sep 15 22:58:52.527975 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:52.527887    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-668e8d-5bff4bc59d-67nhg" podStartSLOduration=1.52787349 podStartE2EDuration="1.52787349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:58:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:58:52.526111136 +0000 UTC m=+1882.385822411" watchObservedRunningTime="2026-09-15 22:58:52.52787349 +0000 UTC m=+1882.387584774"
Sep 15 22:58:52.543432 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:52.543374    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-668e8d-7f86956789-47rrh" podStartSLOduration=1.54335609 podStartE2EDuration="1.54335609s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:58:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:58:52.5401503 +0000 UTC m=+1882.399861587" watchObservedRunningTime="2026-09-15 22:58:52.54335609 +0000 UTC m=+1882.403067377"
Sep 15 22:58:58.506311 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:58:58.506277    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf"
Sep 15 22:59:03.503561 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:59:03.503529    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-668e8d-7f86956789-47rrh"
Sep 15 22:59:03.504020 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:59:03.503607    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-668e8d-5bff4bc59d-67nhg"
Sep 15 22:59:18.381984 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:59:18.381949    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-668e8d-7f86956789-47rrh_a1a4a04e-3093-4314-bd73-826876c973d7/payload-processing/0.log"
Sep 15 22:59:49.416473 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:59:49.416443    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-668e8d-7f86956789-47rrh_a1a4a04e-3093-4314-bd73-826876c973d7/payload-processing/0.log"
Sep 15 22:59:56.139872 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:59:56.139833    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx"]
Sep 15 22:59:56.143863 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:59:56.143838    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx"
Sep 15 22:59:56.145992 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:59:56.145972    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-10ec22-serving-cert\""
Sep 15 22:59:56.153848 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:59:56.153826    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx"]
Sep 15 22:59:56.213479 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:59:56.213447    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pn7nx\" (UniqueName: \"kubernetes.io/projected/7d7c02f7-6693-4cbb-898d-e5cdfdef37f1-kube-api-access-pn7nx\") pod \"maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx\" (UID: \"7d7c02f7-6693-4cbb-898d-e5cdfdef37f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx"
Sep 15 22:59:56.213658 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:59:56.213487    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d7c02f7-6693-4cbb-898d-e5cdfdef37f1-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx\" (UID: \"7d7c02f7-6693-4cbb-898d-e5cdfdef37f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx"
Sep 15 22:59:56.213658 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:59:56.213575    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d7c02f7-6693-4cbb-898d-e5cdfdef37f1-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx\" (UID: \"7d7c02f7-6693-4cbb-898d-e5cdfdef37f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx"
Sep 15 22:59:56.213658 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:59:56.213634    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d7c02f7-6693-4cbb-898d-e5cdfdef37f1-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx\" (UID: \"7d7c02f7-6693-4cbb-898d-e5cdfdef37f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx"
Sep 15 22:59:56.314708 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:59:56.314666    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pn7nx\" (UniqueName: \"kubernetes.io/projected/7d7c02f7-6693-4cbb-898d-e5cdfdef37f1-kube-api-access-pn7nx\") pod \"maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx\" (UID: \"7d7c02f7-6693-4cbb-898d-e5cdfdef37f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx"
Sep 15 22:59:56.314708 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:59:56.314709    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d7c02f7-6693-4cbb-898d-e5cdfdef37f1-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx\" (UID: \"7d7c02f7-6693-4cbb-898d-e5cdfdef37f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx"
Sep 15 22:59:56.314945 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:59:56.314748    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d7c02f7-6693-4cbb-898d-e5cdfdef37f1-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx\" (UID: \"7d7c02f7-6693-4cbb-898d-e5cdfdef37f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx"
Sep 15 22:59:56.314945 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:59:56.314776    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d7c02f7-6693-4cbb-898d-e5cdfdef37f1-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx\" (UID: \"7d7c02f7-6693-4cbb-898d-e5cdfdef37f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx"
Sep 15 22:59:56.315603 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:59:56.315580    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d7c02f7-6693-4cbb-898d-e5cdfdef37f1-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx\" (UID: \"7d7c02f7-6693-4cbb-898d-e5cdfdef37f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx"
Sep 15 22:59:56.317320 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:59:56.317296    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d7c02f7-6693-4cbb-898d-e5cdfdef37f1-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx\" (UID: \"7d7c02f7-6693-4cbb-898d-e5cdfdef37f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx"
Sep 15 22:59:56.317417 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:59:56.317356    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d7c02f7-6693-4cbb-898d-e5cdfdef37f1-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx\" (UID: \"7d7c02f7-6693-4cbb-898d-e5cdfdef37f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx"
Sep 15 22:59:56.321735 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:59:56.321713    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pn7nx\" (UniqueName: \"kubernetes.io/projected/7d7c02f7-6693-4cbb-898d-e5cdfdef37f1-kube-api-access-pn7nx\") pod \"maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx\" (UID: \"7d7c02f7-6693-4cbb-898d-e5cdfdef37f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx"
Sep 15 22:59:56.456457 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:59:56.456364    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx"
Sep 15 22:59:56.731642 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:59:56.731604    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx"]
Sep 15 22:59:56.733249 ip-10-0-128-208 kubenswrapper[2565]: W0915 22:59:56.733224    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d7c02f7_6693_4cbb_898d_e5cdfdef37f1.slice/crio-4dca03b73492f7198891ad32bed739ba6088ace41de8e20b3c48b06d043498a7 WatchSource:0}: Error finding container 4dca03b73492f7198891ad32bed739ba6088ace41de8e20b3c48b06d043498a7: Status 404 returned error can't find the container with id 4dca03b73492f7198891ad32bed739ba6088ace41de8e20b3c48b06d043498a7
Sep 15 22:59:56.874039 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:59:56.874001    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx" event={"ID":"7d7c02f7-6693-4cbb-898d-e5cdfdef37f1","Type":"ContainerStarted","Data":"4dca03b73492f7198891ad32bed739ba6088ace41de8e20b3c48b06d043498a7"}
Sep 15 22:59:57.881130 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:59:57.881089    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx" event={"ID":"7d7c02f7-6693-4cbb-898d-e5cdfdef37f1","Type":"ContainerStarted","Data":"9457b0d7e9ae6982a708556de62d7297d1516fa4ccadb40853117cf1b2289082"}
Sep 15 22:59:57.881558 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:59:57.881218    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx"
Sep 15 22:59:57.900735 ip-10-0-128-208 kubenswrapper[2565]: I0915 22:59:57.900683    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx" podStartSLOduration=1.524663459 podStartE2EDuration="1.900668748s" totalImagesPullingTime="376.005289ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 22:59:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 22:59:57.89764557 +0000 UTC m=+1947.757356853" watchObservedRunningTime="2026-09-15 22:59:57.900668748 +0000 UTC m=+1947.760380041"
Sep 15 23:00:03.892350 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:00:03.892321    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx"
Sep 15 23:00:25.252086 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:00:25.252001    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx"]
Sep 15 23:00:25.252703 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:00:25.252237    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx" podUID="7d7c02f7-6693-4cbb-898d-e5cdfdef37f1" containerName="maas-api" containerID="cri-o://9457b0d7e9ae6982a708556de62d7297d1516fa4ccadb40853117cf1b2289082" gracePeriod=30
Sep 15 23:00:25.511662 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:00:25.511592    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx"
Sep 15 23:00:25.704527 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:00:25.704489    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d7c02f7-6693-4cbb-898d-e5cdfdef37f1-metrics-certs\") pod \"7d7c02f7-6693-4cbb-898d-e5cdfdef37f1\" (UID: \"7d7c02f7-6693-4cbb-898d-e5cdfdef37f1\") "
Sep 15 23:00:25.704724 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:00:25.704536    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pn7nx\" (UniqueName: \"kubernetes.io/projected/7d7c02f7-6693-4cbb-898d-e5cdfdef37f1-kube-api-access-pn7nx\") pod \"7d7c02f7-6693-4cbb-898d-e5cdfdef37f1\" (UID: \"7d7c02f7-6693-4cbb-898d-e5cdfdef37f1\") "
Sep 15 23:00:25.704724 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:00:25.704559    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d7c02f7-6693-4cbb-898d-e5cdfdef37f1-openshift-service-ca\") pod \"7d7c02f7-6693-4cbb-898d-e5cdfdef37f1\" (UID: \"7d7c02f7-6693-4cbb-898d-e5cdfdef37f1\") "
Sep 15 23:00:25.704724 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:00:25.704601    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d7c02f7-6693-4cbb-898d-e5cdfdef37f1-maas-api-tls\") pod \"7d7c02f7-6693-4cbb-898d-e5cdfdef37f1\" (UID: \"7d7c02f7-6693-4cbb-898d-e5cdfdef37f1\") "
Sep 15 23:00:25.704991 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:00:25.704957    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d7c02f7-6693-4cbb-898d-e5cdfdef37f1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7d7c02f7-6693-4cbb-898d-e5cdfdef37f1" (UID: "7d7c02f7-6693-4cbb-898d-e5cdfdef37f1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 23:00:25.706840 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:00:25.706810    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d7c02f7-6693-4cbb-898d-e5cdfdef37f1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7d7c02f7-6693-4cbb-898d-e5cdfdef37f1" (UID: "7d7c02f7-6693-4cbb-898d-e5cdfdef37f1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 23:00:25.706968 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:00:25.706847    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d7c02f7-6693-4cbb-898d-e5cdfdef37f1-kube-api-access-pn7nx" (OuterVolumeSpecName: "kube-api-access-pn7nx") pod "7d7c02f7-6693-4cbb-898d-e5cdfdef37f1" (UID: "7d7c02f7-6693-4cbb-898d-e5cdfdef37f1"). InnerVolumeSpecName "kube-api-access-pn7nx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 23:00:25.707201 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:00:25.707174    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d7c02f7-6693-4cbb-898d-e5cdfdef37f1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7d7c02f7-6693-4cbb-898d-e5cdfdef37f1" (UID: "7d7c02f7-6693-4cbb-898d-e5cdfdef37f1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 23:00:25.805897 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:00:25.805849    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d7c02f7-6693-4cbb-898d-e5cdfdef37f1-metrics-certs\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 23:00:25.805897 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:00:25.805891    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pn7nx\" (UniqueName: \"kubernetes.io/projected/7d7c02f7-6693-4cbb-898d-e5cdfdef37f1-kube-api-access-pn7nx\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 23:00:25.806202 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:00:25.805909    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d7c02f7-6693-4cbb-898d-e5cdfdef37f1-openshift-service-ca\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 23:00:25.806202 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:00:25.805923    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d7c02f7-6693-4cbb-898d-e5cdfdef37f1-maas-api-tls\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 23:00:26.047691 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:00:26.047654    2565 generic.go:356] "Generic (PLEG): container finished" podID="7d7c02f7-6693-4cbb-898d-e5cdfdef37f1" containerID="9457b0d7e9ae6982a708556de62d7297d1516fa4ccadb40853117cf1b2289082" exitCode=0
Sep 15 23:00:26.047857 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:00:26.047703    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx"
Sep 15 23:00:26.047857 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:00:26.047743    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx" event={"ID":"7d7c02f7-6693-4cbb-898d-e5cdfdef37f1","Type":"ContainerDied","Data":"9457b0d7e9ae6982a708556de62d7297d1516fa4ccadb40853117cf1b2289082"}
Sep 15 23:00:26.047857 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:00:26.047798    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx" event={"ID":"7d7c02f7-6693-4cbb-898d-e5cdfdef37f1","Type":"ContainerDied","Data":"4dca03b73492f7198891ad32bed739ba6088ace41de8e20b3c48b06d043498a7"}
Sep 15 23:00:26.047857 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:00:26.047828    2565 scope.go:117] "RemoveContainer" containerID="9457b0d7e9ae6982a708556de62d7297d1516fa4ccadb40853117cf1b2289082"
Sep 15 23:00:26.058512 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:00:26.058489    2565 scope.go:117] "RemoveContainer" containerID="9457b0d7e9ae6982a708556de62d7297d1516fa4ccadb40853117cf1b2289082"
Sep 15 23:00:26.058799 ip-10-0-128-208 kubenswrapper[2565]: E0915 23:00:26.058781    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9457b0d7e9ae6982a708556de62d7297d1516fa4ccadb40853117cf1b2289082\": container with ID starting with 9457b0d7e9ae6982a708556de62d7297d1516fa4ccadb40853117cf1b2289082 not found: ID does not exist" containerID="9457b0d7e9ae6982a708556de62d7297d1516fa4ccadb40853117cf1b2289082"
Sep 15 23:00:26.058873 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:00:26.058809    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9457b0d7e9ae6982a708556de62d7297d1516fa4ccadb40853117cf1b2289082"} err="failed to get container status \"9457b0d7e9ae6982a708556de62d7297d1516fa4ccadb40853117cf1b2289082\": rpc error: code = NotFound desc = could not find container \"9457b0d7e9ae6982a708556de62d7297d1516fa4ccadb40853117cf1b2289082\": container with ID starting with 9457b0d7e9ae6982a708556de62d7297d1516fa4ccadb40853117cf1b2289082 not found: ID does not exist"
Sep 15 23:00:26.069821 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:00:26.069792    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx"]
Sep 15 23:00:26.072538 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:00:26.072515    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-10ec22-56d876c87b-r9mzx"]
Sep 15 23:00:26.591388 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:00:26.591356    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d7c02f7-6693-4cbb-898d-e5cdfdef37f1" path="/var/lib/kubelet/pods/7d7c02f7-6693-4cbb-898d-e5cdfdef37f1/volumes"
Sep 15 23:01:56.063823 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.063729    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf"]
Sep 15 23:01:56.064348 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.063956    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf" podUID="64f7b48b-c18a-4f5d-8ad6-dffcadacf163" containerName="maas-api" containerID="cri-o://eba5d73962df40563b51a532f6c273d98341ff099918649b9fd1e205c3727618" gracePeriod=30
Sep 15 23:01:56.116236 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.116198    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-668e8d-7f86956789-47rrh"]
Sep 15 23:01:56.116496 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.116468    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-668e8d-7f86956789-47rrh" podUID="a1a4a04e-3093-4314-bd73-826876c973d7" containerName="payload-processing" containerID="cri-o://1ca24a59315ef492dbce0394582359a507f73f7e033ed743954dd113bdbd9459" gracePeriod=30
Sep 15 23:01:56.121706 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.121668    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-668e8d-5bff4bc59d-67nhg"]
Sep 15 23:01:56.122091 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.122061    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-668e8d-5bff4bc59d-67nhg" podUID="8975b00a-2fc5-4002-82c0-b92207df26fa" containerName="payload-pre-processing" containerID="cri-o://2ca417ce43ecbff70e15290c9c438b2fe71ac6e45386f70280a22cab55c04ede" gracePeriod=30
Sep 15 23:01:56.384763 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.384726    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf"
Sep 15 23:01:56.429862 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.429838    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-668e8d-7f86956789-47rrh"
Sep 15 23:01:56.433780 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.433760    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-668e8d-5bff4bc59d-67nhg"
Sep 15 23:01:56.499754 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.499720    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1a4a04e-3093-4314-bd73-826876c973d7-plugins-config-volume\") pod \"a1a4a04e-3093-4314-bd73-826876c973d7\" (UID: \"a1a4a04e-3093-4314-bd73-826876c973d7\") "
Sep 15 23:01:56.499910 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.499772    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w92dk\" (UniqueName: \"kubernetes.io/projected/a1a4a04e-3093-4314-bd73-826876c973d7-kube-api-access-w92dk\") pod \"a1a4a04e-3093-4314-bd73-826876c973d7\" (UID: \"a1a4a04e-3093-4314-bd73-826876c973d7\") "
Sep 15 23:01:56.499910 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.499800    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cv6vf\" (UniqueName: \"kubernetes.io/projected/8975b00a-2fc5-4002-82c0-b92207df26fa-kube-api-access-cv6vf\") pod \"8975b00a-2fc5-4002-82c0-b92207df26fa\" (UID: \"8975b00a-2fc5-4002-82c0-b92207df26fa\") "
Sep 15 23:01:56.499910 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.499854    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-shd2b\" (UniqueName: \"kubernetes.io/projected/64f7b48b-c18a-4f5d-8ad6-dffcadacf163-kube-api-access-shd2b\") pod \"64f7b48b-c18a-4f5d-8ad6-dffcadacf163\" (UID: \"64f7b48b-c18a-4f5d-8ad6-dffcadacf163\") "
Sep 15 23:01:56.499910 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.499892    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8975b00a-2fc5-4002-82c0-b92207df26fa-plugins-config-volume\") pod \"8975b00a-2fc5-4002-82c0-b92207df26fa\" (UID: \"8975b00a-2fc5-4002-82c0-b92207df26fa\") "
Sep 15 23:01:56.500172 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.499927    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64f7b48b-c18a-4f5d-8ad6-dffcadacf163-openshift-service-ca\") pod \"64f7b48b-c18a-4f5d-8ad6-dffcadacf163\" (UID: \"64f7b48b-c18a-4f5d-8ad6-dffcadacf163\") "
Sep 15 23:01:56.500172 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.499954    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64f7b48b-c18a-4f5d-8ad6-dffcadacf163-metrics-certs\") pod \"64f7b48b-c18a-4f5d-8ad6-dffcadacf163\" (UID: \"64f7b48b-c18a-4f5d-8ad6-dffcadacf163\") "
Sep 15 23:01:56.500172 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.499984    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64f7b48b-c18a-4f5d-8ad6-dffcadacf163-maas-api-tls\") pod \"64f7b48b-c18a-4f5d-8ad6-dffcadacf163\" (UID: \"64f7b48b-c18a-4f5d-8ad6-dffcadacf163\") "
Sep 15 23:01:56.500348 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.500159    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1a4a04e-3093-4314-bd73-826876c973d7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a1a4a04e-3093-4314-bd73-826876c973d7" (UID: "a1a4a04e-3093-4314-bd73-826876c973d7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 23:01:56.500537 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.500505    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64f7b48b-c18a-4f5d-8ad6-dffcadacf163-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "64f7b48b-c18a-4f5d-8ad6-dffcadacf163" (UID: "64f7b48b-c18a-4f5d-8ad6-dffcadacf163"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 23:01:56.500681 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.500658    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8975b00a-2fc5-4002-82c0-b92207df26fa-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8975b00a-2fc5-4002-82c0-b92207df26fa" (UID: "8975b00a-2fc5-4002-82c0-b92207df26fa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 23:01:56.502581 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.502501    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1a4a04e-3093-4314-bd73-826876c973d7-kube-api-access-w92dk" (OuterVolumeSpecName: "kube-api-access-w92dk") pod "a1a4a04e-3093-4314-bd73-826876c973d7" (UID: "a1a4a04e-3093-4314-bd73-826876c973d7"). InnerVolumeSpecName "kube-api-access-w92dk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 23:01:56.502581 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.502538    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64f7b48b-c18a-4f5d-8ad6-dffcadacf163-kube-api-access-shd2b" (OuterVolumeSpecName: "kube-api-access-shd2b") pod "64f7b48b-c18a-4f5d-8ad6-dffcadacf163" (UID: "64f7b48b-c18a-4f5d-8ad6-dffcadacf163"). InnerVolumeSpecName "kube-api-access-shd2b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 23:01:56.502581 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.502572    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64f7b48b-c18a-4f5d-8ad6-dffcadacf163-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "64f7b48b-c18a-4f5d-8ad6-dffcadacf163" (UID: "64f7b48b-c18a-4f5d-8ad6-dffcadacf163"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 23:01:56.502770 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.502587    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8975b00a-2fc5-4002-82c0-b92207df26fa-kube-api-access-cv6vf" (OuterVolumeSpecName: "kube-api-access-cv6vf") pod "8975b00a-2fc5-4002-82c0-b92207df26fa" (UID: "8975b00a-2fc5-4002-82c0-b92207df26fa"). InnerVolumeSpecName "kube-api-access-cv6vf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 23:01:56.502770 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.502619    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64f7b48b-c18a-4f5d-8ad6-dffcadacf163-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "64f7b48b-c18a-4f5d-8ad6-dffcadacf163" (UID: "64f7b48b-c18a-4f5d-8ad6-dffcadacf163"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 23:01:56.600888 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.600855    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shd2b\" (UniqueName: \"kubernetes.io/projected/64f7b48b-c18a-4f5d-8ad6-dffcadacf163-kube-api-access-shd2b\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 23:01:56.601019 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.600891    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8975b00a-2fc5-4002-82c0-b92207df26fa-plugins-config-volume\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 23:01:56.601019 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.600906    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64f7b48b-c18a-4f5d-8ad6-dffcadacf163-openshift-service-ca\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 23:01:56.601019 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.600920    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64f7b48b-c18a-4f5d-8ad6-dffcadacf163-metrics-certs\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 23:01:56.601019 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.600932    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64f7b48b-c18a-4f5d-8ad6-dffcadacf163-maas-api-tls\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 23:01:56.601019 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.600944    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1a4a04e-3093-4314-bd73-826876c973d7-plugins-config-volume\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 23:01:56.601019 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.600957    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w92dk\" (UniqueName: \"kubernetes.io/projected/a1a4a04e-3093-4314-bd73-826876c973d7-kube-api-access-w92dk\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 23:01:56.601019 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.600971    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cv6vf\" (UniqueName: \"kubernetes.io/projected/8975b00a-2fc5-4002-82c0-b92207df26fa-kube-api-access-cv6vf\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 23:01:56.606628 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.606603    2565 generic.go:356] "Generic (PLEG): container finished" podID="64f7b48b-c18a-4f5d-8ad6-dffcadacf163" containerID="eba5d73962df40563b51a532f6c273d98341ff099918649b9fd1e205c3727618" exitCode=0
Sep 15 23:01:56.606752 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.606644    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf"
Sep 15 23:01:56.606752 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.606684    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf" event={"ID":"64f7b48b-c18a-4f5d-8ad6-dffcadacf163","Type":"ContainerDied","Data":"eba5d73962df40563b51a532f6c273d98341ff099918649b9fd1e205c3727618"}
Sep 15 23:01:56.606870 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.606749    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf" event={"ID":"64f7b48b-c18a-4f5d-8ad6-dffcadacf163","Type":"ContainerDied","Data":"eb0348bf9e2161319ba71b192ef29d685c9cc69e496b43cbce308eb1f35a7e31"}
Sep 15 23:01:56.606870 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.606779    2565 scope.go:117] "RemoveContainer" containerID="eba5d73962df40563b51a532f6c273d98341ff099918649b9fd1e205c3727618"
Sep 15 23:01:56.608248 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.608221    2565 generic.go:356] "Generic (PLEG): container finished" podID="a1a4a04e-3093-4314-bd73-826876c973d7" containerID="1ca24a59315ef492dbce0394582359a507f73f7e033ed743954dd113bdbd9459" exitCode=0
Sep 15 23:01:56.608369 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.608264    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-668e8d-7f86956789-47rrh"
Sep 15 23:01:56.608369 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.608311    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-668e8d-7f86956789-47rrh" event={"ID":"a1a4a04e-3093-4314-bd73-826876c973d7","Type":"ContainerDied","Data":"1ca24a59315ef492dbce0394582359a507f73f7e033ed743954dd113bdbd9459"}
Sep 15 23:01:56.608369 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.608361    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-668e8d-7f86956789-47rrh" event={"ID":"a1a4a04e-3093-4314-bd73-826876c973d7","Type":"ContainerDied","Data":"68a5b621bddddb3b26088e1faf7d3c29ccd50eddae3868644f1854192cbdbdbb"}
Sep 15 23:01:56.610196 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.610177    2565 generic.go:356] "Generic (PLEG): container finished" podID="8975b00a-2fc5-4002-82c0-b92207df26fa" containerID="2ca417ce43ecbff70e15290c9c438b2fe71ac6e45386f70280a22cab55c04ede" exitCode=0
Sep 15 23:01:56.610402 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.610215    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-668e8d-5bff4bc59d-67nhg"
Sep 15 23:01:56.610402 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.610237    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-668e8d-5bff4bc59d-67nhg" event={"ID":"8975b00a-2fc5-4002-82c0-b92207df26fa","Type":"ContainerDied","Data":"2ca417ce43ecbff70e15290c9c438b2fe71ac6e45386f70280a22cab55c04ede"}
Sep 15 23:01:56.610402 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.610282    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-668e8d-5bff4bc59d-67nhg" event={"ID":"8975b00a-2fc5-4002-82c0-b92207df26fa","Type":"ContainerDied","Data":"616c95d2e771e73a278814cffe5262c592493043eb73a94444ee586cca48aa0f"}
Sep 15 23:01:56.617901 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.617886    2565 scope.go:117] "RemoveContainer" containerID="eba5d73962df40563b51a532f6c273d98341ff099918649b9fd1e205c3727618"
Sep 15 23:01:56.618197 ip-10-0-128-208 kubenswrapper[2565]: E0915 23:01:56.618178    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eba5d73962df40563b51a532f6c273d98341ff099918649b9fd1e205c3727618\": container with ID starting with eba5d73962df40563b51a532f6c273d98341ff099918649b9fd1e205c3727618 not found: ID does not exist" containerID="eba5d73962df40563b51a532f6c273d98341ff099918649b9fd1e205c3727618"
Sep 15 23:01:56.618260 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.618207    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eba5d73962df40563b51a532f6c273d98341ff099918649b9fd1e205c3727618"} err="failed to get container status \"eba5d73962df40563b51a532f6c273d98341ff099918649b9fd1e205c3727618\": rpc error: code = NotFound desc = could not find container \"eba5d73962df40563b51a532f6c273d98341ff099918649b9fd1e205c3727618\": container with ID starting with eba5d73962df40563b51a532f6c273d98341ff099918649b9fd1e205c3727618 not found: ID does not exist"
Sep 15 23:01:56.618260 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.618223    2565 scope.go:117] "RemoveContainer" containerID="1ca24a59315ef492dbce0394582359a507f73f7e033ed743954dd113bdbd9459"
Sep 15 23:01:56.626520 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.626500    2565 scope.go:117] "RemoveContainer" containerID="1ca24a59315ef492dbce0394582359a507f73f7e033ed743954dd113bdbd9459"
Sep 15 23:01:56.626782 ip-10-0-128-208 kubenswrapper[2565]: E0915 23:01:56.626763    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ca24a59315ef492dbce0394582359a507f73f7e033ed743954dd113bdbd9459\": container with ID starting with 1ca24a59315ef492dbce0394582359a507f73f7e033ed743954dd113bdbd9459 not found: ID does not exist" containerID="1ca24a59315ef492dbce0394582359a507f73f7e033ed743954dd113bdbd9459"
Sep 15 23:01:56.626849 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.626789    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ca24a59315ef492dbce0394582359a507f73f7e033ed743954dd113bdbd9459"} err="failed to get container status \"1ca24a59315ef492dbce0394582359a507f73f7e033ed743954dd113bdbd9459\": rpc error: code = NotFound desc = could not find container \"1ca24a59315ef492dbce0394582359a507f73f7e033ed743954dd113bdbd9459\": container with ID starting with 1ca24a59315ef492dbce0394582359a507f73f7e033ed743954dd113bdbd9459 not found: ID does not exist"
Sep 15 23:01:56.626849 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.626806    2565 scope.go:117] "RemoveContainer" containerID="2ca417ce43ecbff70e15290c9c438b2fe71ac6e45386f70280a22cab55c04ede"
Sep 15 23:01:56.628169 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.628148    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf"]
Sep 15 23:01:56.635222 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.635204    2565 scope.go:117] "RemoveContainer" containerID="2ca417ce43ecbff70e15290c9c438b2fe71ac6e45386f70280a22cab55c04ede"
Sep 15 23:01:56.635501 ip-10-0-128-208 kubenswrapper[2565]: E0915 23:01:56.635482    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ca417ce43ecbff70e15290c9c438b2fe71ac6e45386f70280a22cab55c04ede\": container with ID starting with 2ca417ce43ecbff70e15290c9c438b2fe71ac6e45386f70280a22cab55c04ede not found: ID does not exist" containerID="2ca417ce43ecbff70e15290c9c438b2fe71ac6e45386f70280a22cab55c04ede"
Sep 15 23:01:56.635562 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.635511    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ca417ce43ecbff70e15290c9c438b2fe71ac6e45386f70280a22cab55c04ede"} err="failed to get container status \"2ca417ce43ecbff70e15290c9c438b2fe71ac6e45386f70280a22cab55c04ede\": rpc error: code = NotFound desc = could not find container \"2ca417ce43ecbff70e15290c9c438b2fe71ac6e45386f70280a22cab55c04ede\": container with ID starting with 2ca417ce43ecbff70e15290c9c438b2fe71ac6e45386f70280a22cab55c04ede not found: ID does not exist"
Sep 15 23:01:56.639647 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.639623    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-668e8d-846ddf6db4-5vtpf"]
Sep 15 23:01:56.653731 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.653709    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-668e8d-5bff4bc59d-67nhg"]
Sep 15 23:01:56.656009 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.655993    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-668e8d-5bff4bc59d-67nhg"]
Sep 15 23:01:56.665174 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.665153    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-668e8d-7f86956789-47rrh"]
Sep 15 23:01:56.670131 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:56.670113    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-668e8d-7f86956789-47rrh"]
Sep 15 23:01:58.591673 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:58.591642    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64f7b48b-c18a-4f5d-8ad6-dffcadacf163" path="/var/lib/kubelet/pods/64f7b48b-c18a-4f5d-8ad6-dffcadacf163/volumes"
Sep 15 23:01:58.592063 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:58.592000    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8975b00a-2fc5-4002-82c0-b92207df26fa" path="/var/lib/kubelet/pods/8975b00a-2fc5-4002-82c0-b92207df26fa/volumes"
Sep 15 23:01:58.592359 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:01:58.592345    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1a4a04e-3093-4314-bd73-826876c973d7" path="/var/lib/kubelet/pods/a1a4a04e-3093-4314-bd73-826876c973d7/volumes"
Sep 15 23:02:13.480096 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:13.480060    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr"]
Sep 15 23:02:13.480955 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:13.480892    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr" podUID="a8939264-e179-4809-b361-91388cf1dcce" containerName="maas-api" containerID="cri-o://c2c260f0ffd826b92e7a13d523c742c934b6dbfe6303f97ef1157e8c6bc627b6" gracePeriod=30
Sep 15 23:02:13.712047 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:13.711996    2565 generic.go:356] "Generic (PLEG): container finished" podID="a8939264-e179-4809-b361-91388cf1dcce" containerID="c2c260f0ffd826b92e7a13d523c742c934b6dbfe6303f97ef1157e8c6bc627b6" exitCode=0
Sep 15 23:02:13.712213 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:13.712066    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr" event={"ID":"a8939264-e179-4809-b361-91388cf1dcce","Type":"ContainerDied","Data":"c2c260f0ffd826b92e7a13d523c742c934b6dbfe6303f97ef1157e8c6bc627b6"}
Sep 15 23:02:13.742649 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:13.742577    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr"
Sep 15 23:02:13.771636 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:13.771608    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jvrmw\" (UniqueName: \"kubernetes.io/projected/a8939264-e179-4809-b361-91388cf1dcce-kube-api-access-jvrmw\") pod \"a8939264-e179-4809-b361-91388cf1dcce\" (UID: \"a8939264-e179-4809-b361-91388cf1dcce\") "
Sep 15 23:02:13.771808 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:13.771673    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8939264-e179-4809-b361-91388cf1dcce-maas-api-tls\") pod \"a8939264-e179-4809-b361-91388cf1dcce\" (UID: \"a8939264-e179-4809-b361-91388cf1dcce\") "
Sep 15 23:02:13.771808 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:13.771727    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8939264-e179-4809-b361-91388cf1dcce-metrics-certs\") pod \"a8939264-e179-4809-b361-91388cf1dcce\" (UID: \"a8939264-e179-4809-b361-91388cf1dcce\") "
Sep 15 23:02:13.771808 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:13.771752    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8939264-e179-4809-b361-91388cf1dcce-openshift-service-ca\") pod \"a8939264-e179-4809-b361-91388cf1dcce\" (UID: \"a8939264-e179-4809-b361-91388cf1dcce\") "
Sep 15 23:02:13.772154 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:13.772121    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8939264-e179-4809-b361-91388cf1dcce-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a8939264-e179-4809-b361-91388cf1dcce" (UID: "a8939264-e179-4809-b361-91388cf1dcce"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 23:02:13.773742 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:13.773713    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8939264-e179-4809-b361-91388cf1dcce-kube-api-access-jvrmw" (OuterVolumeSpecName: "kube-api-access-jvrmw") pod "a8939264-e179-4809-b361-91388cf1dcce" (UID: "a8939264-e179-4809-b361-91388cf1dcce"). InnerVolumeSpecName "kube-api-access-jvrmw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 23:02:13.774178 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:13.774157    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8939264-e179-4809-b361-91388cf1dcce-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a8939264-e179-4809-b361-91388cf1dcce" (UID: "a8939264-e179-4809-b361-91388cf1dcce"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 23:02:13.774242 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:13.774185    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8939264-e179-4809-b361-91388cf1dcce-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a8939264-e179-4809-b361-91388cf1dcce" (UID: "a8939264-e179-4809-b361-91388cf1dcce"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 23:02:13.873277 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:13.873241    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8939264-e179-4809-b361-91388cf1dcce-maas-api-tls\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 23:02:13.873277 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:13.873269    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8939264-e179-4809-b361-91388cf1dcce-metrics-certs\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 23:02:13.873277 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:13.873281    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8939264-e179-4809-b361-91388cf1dcce-openshift-service-ca\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 23:02:13.873503 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:13.873290    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvrmw\" (UniqueName: \"kubernetes.io/projected/a8939264-e179-4809-b361-91388cf1dcce-kube-api-access-jvrmw\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 23:02:14.719900 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:14.719873    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr"
Sep 15 23:02:14.720413 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:14.719865    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr" event={"ID":"a8939264-e179-4809-b361-91388cf1dcce","Type":"ContainerDied","Data":"4a2fa48e832302df14978dfd87cf17d26918b708cd15f5561604f39c088b5a8c"}
Sep 15 23:02:14.720413 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:14.720037    2565 scope.go:117] "RemoveContainer" containerID="c2c260f0ffd826b92e7a13d523c742c934b6dbfe6303f97ef1157e8c6bc627b6"
Sep 15 23:02:14.736232 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:14.736207    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr"]
Sep 15 23:02:14.740331 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:14.740309    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0016d4-6497d9bd76-vt6fr"]
Sep 15 23:02:16.590757 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:16.590724    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8939264-e179-4809-b361-91388cf1dcce" path="/var/lib/kubelet/pods/a8939264-e179-4809-b361-91388cf1dcce/volumes"
Sep 15 23:02:31.986524 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:31.986485    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-92482b-bbfb58bf5-gf25f"]
Sep 15 23:02:31.986966 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:31.986767    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-92482b-bbfb58bf5-gf25f" podUID="5f624109-7163-47bc-9d72-cf5204a41a10" containerName="payload-processing" containerID="cri-o://ce7ce1af93abfbdd036c74087f7bd4521fd94cf183e3da23482885067e1cff1e" gracePeriod=30
Sep 15 23:02:31.996197 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:31.996169    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-92482b-774868dc8b-l7lwz"]
Sep 15 23:02:31.996450 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:31.996407    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-92482b-774868dc8b-l7lwz" podUID="9f8cf34a-3ee1-4080-b58a-dc5f3122a114" containerName="payload-pre-processing" containerID="cri-o://61b73a55addc31c5e416dce2da949d5c15f14f8bb4dc6945b3073dbf5bc51bd6" gracePeriod=30
Sep 15 23:02:32.268319 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:32.268286    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-92482b-bbfb58bf5-gf25f"
Sep 15 23:02:32.289526 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:32.289497    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-92482b-774868dc8b-l7lwz"
Sep 15 23:02:32.345210 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:32.345170    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-778rs\" (UniqueName: \"kubernetes.io/projected/9f8cf34a-3ee1-4080-b58a-dc5f3122a114-kube-api-access-778rs\") pod \"9f8cf34a-3ee1-4080-b58a-dc5f3122a114\" (UID: \"9f8cf34a-3ee1-4080-b58a-dc5f3122a114\") "
Sep 15 23:02:32.345210 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:32.345218    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f8cf34a-3ee1-4080-b58a-dc5f3122a114-plugins-config-volume\") pod \"9f8cf34a-3ee1-4080-b58a-dc5f3122a114\" (UID: \"9f8cf34a-3ee1-4080-b58a-dc5f3122a114\") "
Sep 15 23:02:32.345473 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:32.345306    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s45h8\" (UniqueName: \"kubernetes.io/projected/5f624109-7163-47bc-9d72-cf5204a41a10-kube-api-access-s45h8\") pod \"5f624109-7163-47bc-9d72-cf5204a41a10\" (UID: \"5f624109-7163-47bc-9d72-cf5204a41a10\") "
Sep 15 23:02:32.345473 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:32.345337    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f624109-7163-47bc-9d72-cf5204a41a10-plugins-config-volume\") pod \"5f624109-7163-47bc-9d72-cf5204a41a10\" (UID: \"5f624109-7163-47bc-9d72-cf5204a41a10\") "
Sep 15 23:02:32.345786 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:32.345747    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f8cf34a-3ee1-4080-b58a-dc5f3122a114-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9f8cf34a-3ee1-4080-b58a-dc5f3122a114" (UID: "9f8cf34a-3ee1-4080-b58a-dc5f3122a114"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 23:02:32.345786 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:32.345767    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f624109-7163-47bc-9d72-cf5204a41a10-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5f624109-7163-47bc-9d72-cf5204a41a10" (UID: "5f624109-7163-47bc-9d72-cf5204a41a10"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 23:02:32.347430 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:32.347406    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f8cf34a-3ee1-4080-b58a-dc5f3122a114-kube-api-access-778rs" (OuterVolumeSpecName: "kube-api-access-778rs") pod "9f8cf34a-3ee1-4080-b58a-dc5f3122a114" (UID: "9f8cf34a-3ee1-4080-b58a-dc5f3122a114"). InnerVolumeSpecName "kube-api-access-778rs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 23:02:32.347522 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:32.347474    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f624109-7163-47bc-9d72-cf5204a41a10-kube-api-access-s45h8" (OuterVolumeSpecName: "kube-api-access-s45h8") pod "5f624109-7163-47bc-9d72-cf5204a41a10" (UID: "5f624109-7163-47bc-9d72-cf5204a41a10"). InnerVolumeSpecName "kube-api-access-s45h8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 23:02:32.446394 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:32.446355    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-778rs\" (UniqueName: \"kubernetes.io/projected/9f8cf34a-3ee1-4080-b58a-dc5f3122a114-kube-api-access-778rs\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 23:02:32.446394 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:32.446383    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f8cf34a-3ee1-4080-b58a-dc5f3122a114-plugins-config-volume\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 23:02:32.446394 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:32.446393    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s45h8\" (UniqueName: \"kubernetes.io/projected/5f624109-7163-47bc-9d72-cf5204a41a10-kube-api-access-s45h8\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 23:02:32.446394 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:32.446402    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f624109-7163-47bc-9d72-cf5204a41a10-plugins-config-volume\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 23:02:32.827259 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:32.827215    2565 generic.go:356] "Generic (PLEG): container finished" podID="9f8cf34a-3ee1-4080-b58a-dc5f3122a114" containerID="61b73a55addc31c5e416dce2da949d5c15f14f8bb4dc6945b3073dbf5bc51bd6" exitCode=0
Sep 15 23:02:32.827426 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:32.827283    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-92482b-774868dc8b-l7lwz"
Sep 15 23:02:32.827426 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:32.827293    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-92482b-774868dc8b-l7lwz" event={"ID":"9f8cf34a-3ee1-4080-b58a-dc5f3122a114","Type":"ContainerDied","Data":"61b73a55addc31c5e416dce2da949d5c15f14f8bb4dc6945b3073dbf5bc51bd6"}
Sep 15 23:02:32.827426 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:32.827355    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-92482b-774868dc8b-l7lwz" event={"ID":"9f8cf34a-3ee1-4080-b58a-dc5f3122a114","Type":"ContainerDied","Data":"c0086f5ecfe5d6c13e3576d19faa4cea10fdf95a22a0173130a6858bafd6ff01"}
Sep 15 23:02:32.827426 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:32.827386    2565 scope.go:117] "RemoveContainer" containerID="61b73a55addc31c5e416dce2da949d5c15f14f8bb4dc6945b3073dbf5bc51bd6"
Sep 15 23:02:32.828699 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:32.828680    2565 generic.go:356] "Generic (PLEG): container finished" podID="5f624109-7163-47bc-9d72-cf5204a41a10" containerID="ce7ce1af93abfbdd036c74087f7bd4521fd94cf183e3da23482885067e1cff1e" exitCode=0
Sep 15 23:02:32.828789 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:32.828707    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-92482b-bbfb58bf5-gf25f" event={"ID":"5f624109-7163-47bc-9d72-cf5204a41a10","Type":"ContainerDied","Data":"ce7ce1af93abfbdd036c74087f7bd4521fd94cf183e3da23482885067e1cff1e"}
Sep 15 23:02:32.828789 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:32.828753    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-92482b-bbfb58bf5-gf25f"
Sep 15 23:02:32.828883 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:32.828760    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-92482b-bbfb58bf5-gf25f" event={"ID":"5f624109-7163-47bc-9d72-cf5204a41a10","Type":"ContainerDied","Data":"948f090941e98e3a13b3fd9116e51a1f25113ab5be09d86e0bf687805ef33975"}
Sep 15 23:02:32.836714 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:32.836695    2565 scope.go:117] "RemoveContainer" containerID="61b73a55addc31c5e416dce2da949d5c15f14f8bb4dc6945b3073dbf5bc51bd6"
Sep 15 23:02:32.836998 ip-10-0-128-208 kubenswrapper[2565]: E0915 23:02:32.836980    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61b73a55addc31c5e416dce2da949d5c15f14f8bb4dc6945b3073dbf5bc51bd6\": container with ID starting with 61b73a55addc31c5e416dce2da949d5c15f14f8bb4dc6945b3073dbf5bc51bd6 not found: ID does not exist" containerID="61b73a55addc31c5e416dce2da949d5c15f14f8bb4dc6945b3073dbf5bc51bd6"
Sep 15 23:02:32.837094 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:32.837004    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61b73a55addc31c5e416dce2da949d5c15f14f8bb4dc6945b3073dbf5bc51bd6"} err="failed to get container status \"61b73a55addc31c5e416dce2da949d5c15f14f8bb4dc6945b3073dbf5bc51bd6\": rpc error: code = NotFound desc = could not find container \"61b73a55addc31c5e416dce2da949d5c15f14f8bb4dc6945b3073dbf5bc51bd6\": container with ID starting with 61b73a55addc31c5e416dce2da949d5c15f14f8bb4dc6945b3073dbf5bc51bd6 not found: ID does not exist"
Sep 15 23:02:32.837094 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:32.837020    2565 scope.go:117] "RemoveContainer" containerID="ce7ce1af93abfbdd036c74087f7bd4521fd94cf183e3da23482885067e1cff1e"
Sep 15 23:02:32.845129 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:32.845108    2565 scope.go:117] "RemoveContainer" containerID="ce7ce1af93abfbdd036c74087f7bd4521fd94cf183e3da23482885067e1cff1e"
Sep 15 23:02:32.845388 ip-10-0-128-208 kubenswrapper[2565]: E0915 23:02:32.845370    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce7ce1af93abfbdd036c74087f7bd4521fd94cf183e3da23482885067e1cff1e\": container with ID starting with ce7ce1af93abfbdd036c74087f7bd4521fd94cf183e3da23482885067e1cff1e not found: ID does not exist" containerID="ce7ce1af93abfbdd036c74087f7bd4521fd94cf183e3da23482885067e1cff1e"
Sep 15 23:02:32.845437 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:32.845396    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce7ce1af93abfbdd036c74087f7bd4521fd94cf183e3da23482885067e1cff1e"} err="failed to get container status \"ce7ce1af93abfbdd036c74087f7bd4521fd94cf183e3da23482885067e1cff1e\": rpc error: code = NotFound desc = could not find container \"ce7ce1af93abfbdd036c74087f7bd4521fd94cf183e3da23482885067e1cff1e\": container with ID starting with ce7ce1af93abfbdd036c74087f7bd4521fd94cf183e3da23482885067e1cff1e not found: ID does not exist"
Sep 15 23:02:32.851122 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:32.851066    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-92482b-774868dc8b-l7lwz"]
Sep 15 23:02:32.856480 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:32.856455    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-92482b-774868dc8b-l7lwz"]
Sep 15 23:02:32.868762 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:32.868739    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-92482b-bbfb58bf5-gf25f"]
Sep 15 23:02:32.872309 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:32.872288    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-92482b-bbfb58bf5-gf25f"]
Sep 15 23:02:33.209538 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:33.209431    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-92482b-774868dc8b-l7lwz" podUID="9f8cf34a-3ee1-4080-b58a-dc5f3122a114" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.87:9004: i/o timeout"
Sep 15 23:02:33.209538 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:33.209451    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-92482b-bbfb58bf5-gf25f" podUID="5f624109-7163-47bc-9d72-cf5204a41a10" containerName="payload-processing" probeResult="failure" output="dial tcp 10.133.0.88:9004: i/o timeout"
Sep 15 23:02:34.590416 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:34.590383    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f624109-7163-47bc-9d72-cf5204a41a10" path="/var/lib/kubelet/pods/5f624109-7163-47bc-9d72-cf5204a41a10/volumes"
Sep 15 23:02:34.590791 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:02:34.590724    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f8cf34a-3ee1-4080-b58a-dc5f3122a114" path="/var/lib/kubelet/pods/9f8cf34a-3ee1-4080-b58a-dc5f3122a114/volumes"
Sep 15 23:05:24.072809 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:05:24.072766    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfdwh"]
Sep 15 23:05:24.073380 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:05:24.072961    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfdwh" podUID="f2565baa-d682-4dce-90d3-c486dd114112" containerName="manager" containerID="cri-o://e090b86d29010bc75297f1a4ef68ea7e76351bc18813b10c61ff2bdbb47f94d9" gracePeriod=10
Sep 15 23:05:24.319303 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:05:24.319276    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfdwh"
Sep 15 23:05:24.434282 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:05:24.434203    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6jdjg\" (UniqueName: \"kubernetes.io/projected/f2565baa-d682-4dce-90d3-c486dd114112-kube-api-access-6jdjg\") pod \"f2565baa-d682-4dce-90d3-c486dd114112\" (UID: \"f2565baa-d682-4dce-90d3-c486dd114112\") "
Sep 15 23:05:24.434426 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:05:24.434330    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2565baa-d682-4dce-90d3-c486dd114112-extensions-socket-volume\") pod \"f2565baa-d682-4dce-90d3-c486dd114112\" (UID: \"f2565baa-d682-4dce-90d3-c486dd114112\") "
Sep 15 23:05:24.434726 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:05:24.434693    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2565baa-d682-4dce-90d3-c486dd114112-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f2565baa-d682-4dce-90d3-c486dd114112" (UID: "f2565baa-d682-4dce-90d3-c486dd114112"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 23:05:24.436335 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:05:24.436314    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2565baa-d682-4dce-90d3-c486dd114112-kube-api-access-6jdjg" (OuterVolumeSpecName: "kube-api-access-6jdjg") pod "f2565baa-d682-4dce-90d3-c486dd114112" (UID: "f2565baa-d682-4dce-90d3-c486dd114112"). InnerVolumeSpecName "kube-api-access-6jdjg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 23:05:24.535686 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:05:24.535652    2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2565baa-d682-4dce-90d3-c486dd114112-extensions-socket-volume\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 23:05:24.535686 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:05:24.535679    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jdjg\" (UniqueName: \"kubernetes.io/projected/f2565baa-d682-4dce-90d3-c486dd114112-kube-api-access-6jdjg\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 23:05:24.805750 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:05:24.805712    2565 generic.go:356] "Generic (PLEG): container finished" podID="f2565baa-d682-4dce-90d3-c486dd114112" containerID="e090b86d29010bc75297f1a4ef68ea7e76351bc18813b10c61ff2bdbb47f94d9" exitCode=0
Sep 15 23:05:24.805921 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:05:24.805755    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfdwh"
Sep 15 23:05:24.805921 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:05:24.805794    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfdwh" event={"ID":"f2565baa-d682-4dce-90d3-c486dd114112","Type":"ContainerDied","Data":"e090b86d29010bc75297f1a4ef68ea7e76351bc18813b10c61ff2bdbb47f94d9"}
Sep 15 23:05:24.805921 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:05:24.805847    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfdwh" event={"ID":"f2565baa-d682-4dce-90d3-c486dd114112","Type":"ContainerDied","Data":"5aa512b49b27276feb7500dbb6f0d9fd31ef1d5373063408b39b4ca885fe061a"}
Sep 15 23:05:24.805921 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:05:24.805870    2565 scope.go:117] "RemoveContainer" containerID="e090b86d29010bc75297f1a4ef68ea7e76351bc18813b10c61ff2bdbb47f94d9"
Sep 15 23:05:24.821773 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:05:24.821751    2565 scope.go:117] "RemoveContainer" containerID="e090b86d29010bc75297f1a4ef68ea7e76351bc18813b10c61ff2bdbb47f94d9"
Sep 15 23:05:24.822165 ip-10-0-128-208 kubenswrapper[2565]: E0915 23:05:24.822137    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e090b86d29010bc75297f1a4ef68ea7e76351bc18813b10c61ff2bdbb47f94d9\": container with ID starting with e090b86d29010bc75297f1a4ef68ea7e76351bc18813b10c61ff2bdbb47f94d9 not found: ID does not exist" containerID="e090b86d29010bc75297f1a4ef68ea7e76351bc18813b10c61ff2bdbb47f94d9"
Sep 15 23:05:24.822288 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:05:24.822172    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e090b86d29010bc75297f1a4ef68ea7e76351bc18813b10c61ff2bdbb47f94d9"} err="failed to get container status \"e090b86d29010bc75297f1a4ef68ea7e76351bc18813b10c61ff2bdbb47f94d9\": rpc error: code = NotFound desc = could not find container \"e090b86d29010bc75297f1a4ef68ea7e76351bc18813b10c61ff2bdbb47f94d9\": container with ID starting with e090b86d29010bc75297f1a4ef68ea7e76351bc18813b10c61ff2bdbb47f94d9 not found: ID does not exist"
Sep 15 23:05:24.824189 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:05:24.824160    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfdwh"]
Sep 15 23:05:24.828205 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:05:24.828186    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wfdwh"]
Sep 15 23:05:26.591695 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:05:26.591662    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2565baa-d682-4dce-90d3-c486dd114112" path="/var/lib/kubelet/pods/f2565baa-d682-4dce-90d3-c486dd114112/volumes"
Sep 15 23:06:24.755242 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:24.755161    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rhrmk"]
Sep 15 23:06:24.755685 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:24.755558    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8975b00a-2fc5-4002-82c0-b92207df26fa" containerName="payload-pre-processing"
Sep 15 23:06:24.755685 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:24.755569    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="8975b00a-2fc5-4002-82c0-b92207df26fa" containerName="payload-pre-processing"
Sep 15 23:06:24.755685 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:24.755579    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1a4a04e-3093-4314-bd73-826876c973d7" containerName="payload-processing"
Sep 15 23:06:24.755685 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:24.755584    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1a4a04e-3093-4314-bd73-826876c973d7" containerName="payload-processing"
Sep 15 23:06:24.755685 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:24.755590    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8939264-e179-4809-b361-91388cf1dcce" containerName="maas-api"
Sep 15 23:06:24.755685 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:24.755597    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8939264-e179-4809-b361-91388cf1dcce" containerName="maas-api"
Sep 15 23:06:24.755685 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:24.755605    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f8cf34a-3ee1-4080-b58a-dc5f3122a114" containerName="payload-pre-processing"
Sep 15 23:06:24.755685 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:24.755610    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f8cf34a-3ee1-4080-b58a-dc5f3122a114" containerName="payload-pre-processing"
Sep 15 23:06:24.755685 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:24.755622    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f624109-7163-47bc-9d72-cf5204a41a10" containerName="payload-processing"
Sep 15 23:06:24.755685 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:24.755627    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f624109-7163-47bc-9d72-cf5204a41a10" containerName="payload-processing"
Sep 15 23:06:24.755685 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:24.755637    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64f7b48b-c18a-4f5d-8ad6-dffcadacf163" containerName="maas-api"
Sep 15 23:06:24.755685 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:24.755642    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="64f7b48b-c18a-4f5d-8ad6-dffcadacf163" containerName="maas-api"
Sep 15 23:06:24.755685 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:24.755653    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2565baa-d682-4dce-90d3-c486dd114112" containerName="manager"
Sep 15 23:06:24.755685 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:24.755658    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2565baa-d682-4dce-90d3-c486dd114112" containerName="manager"
Sep 15 23:06:24.755685 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:24.755664    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d7c02f7-6693-4cbb-898d-e5cdfdef37f1" containerName="maas-api"
Sep 15 23:06:24.755685 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:24.755669    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d7c02f7-6693-4cbb-898d-e5cdfdef37f1" containerName="maas-api"
Sep 15 23:06:24.756206 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:24.755725    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f8cf34a-3ee1-4080-b58a-dc5f3122a114" containerName="payload-pre-processing"
Sep 15 23:06:24.756206 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:24.755735    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f624109-7163-47bc-9d72-cf5204a41a10" containerName="payload-processing"
Sep 15 23:06:24.756206 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:24.755742    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1a4a04e-3093-4314-bd73-826876c973d7" containerName="payload-processing"
Sep 15 23:06:24.756206 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:24.755748    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8939264-e179-4809-b361-91388cf1dcce" containerName="maas-api"
Sep 15 23:06:24.756206 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:24.755754    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d7c02f7-6693-4cbb-898d-e5cdfdef37f1" containerName="maas-api"
Sep 15 23:06:24.756206 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:24.755760    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2565baa-d682-4dce-90d3-c486dd114112" containerName="manager"
Sep 15 23:06:24.756206 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:24.755767    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="64f7b48b-c18a-4f5d-8ad6-dffcadacf163" containerName="maas-api"
Sep 15 23:06:24.756206 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:24.755771    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="8975b00a-2fc5-4002-82c0-b92207df26fa" containerName="payload-pre-processing"
Sep 15 23:06:24.758764 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:24.758747    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rhrmk"
Sep 15 23:06:24.760979 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:24.760961    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-d7f88\""
Sep 15 23:06:24.770968 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:24.770942    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rhrmk"]
Sep 15 23:06:24.879353 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:24.879319    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/08b23842-c3ef-4f3a-bd47-3587c4011d35-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-rhrmk\" (UID: \"08b23842-c3ef-4f3a-bd47-3587c4011d35\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rhrmk"
Sep 15 23:06:24.879527 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:24.879398    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxscj\" (UniqueName: \"kubernetes.io/projected/08b23842-c3ef-4f3a-bd47-3587c4011d35-kube-api-access-xxscj\") pod \"kuadrant-operator-controller-manager-5769f99d9f-rhrmk\" (UID: \"08b23842-c3ef-4f3a-bd47-3587c4011d35\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rhrmk"
Sep 15 23:06:24.980198 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:24.980155    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/08b23842-c3ef-4f3a-bd47-3587c4011d35-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-rhrmk\" (UID: \"08b23842-c3ef-4f3a-bd47-3587c4011d35\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rhrmk"
Sep 15 23:06:24.980373 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:24.980220    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxscj\" (UniqueName: \"kubernetes.io/projected/08b23842-c3ef-4f3a-bd47-3587c4011d35-kube-api-access-xxscj\") pod \"kuadrant-operator-controller-manager-5769f99d9f-rhrmk\" (UID: \"08b23842-c3ef-4f3a-bd47-3587c4011d35\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rhrmk"
Sep 15 23:06:24.980552 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:24.980532    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/08b23842-c3ef-4f3a-bd47-3587c4011d35-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-rhrmk\" (UID: \"08b23842-c3ef-4f3a-bd47-3587c4011d35\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rhrmk"
Sep 15 23:06:24.988633 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:24.988596    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxscj\" (UniqueName: \"kubernetes.io/projected/08b23842-c3ef-4f3a-bd47-3587c4011d35-kube-api-access-xxscj\") pod \"kuadrant-operator-controller-manager-5769f99d9f-rhrmk\" (UID: \"08b23842-c3ef-4f3a-bd47-3587c4011d35\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rhrmk"
Sep 15 23:06:25.069557 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:25.069514    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rhrmk"
Sep 15 23:06:25.348689 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:25.348664    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rhrmk"]
Sep 15 23:06:25.350056 ip-10-0-128-208 kubenswrapper[2565]: W0915 23:06:25.350015    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08b23842_c3ef_4f3a_bd47_3587c4011d35.slice/crio-d45f1319c8d9dca43f12cdea8e75acfc1858fa61695a5e758bb9bb3b334f76a8 WatchSource:0}: Error finding container d45f1319c8d9dca43f12cdea8e75acfc1858fa61695a5e758bb9bb3b334f76a8: Status 404 returned error can't find the container with id d45f1319c8d9dca43f12cdea8e75acfc1858fa61695a5e758bb9bb3b334f76a8
Sep 15 23:06:26.153606 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:26.153571    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rhrmk" event={"ID":"08b23842-c3ef-4f3a-bd47-3587c4011d35","Type":"ContainerStarted","Data":"43443df7bf88b757cf9676e34e7de5632577630457abe8a7fe1e98d421fcabf7"}
Sep 15 23:06:26.154006 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:26.153625    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rhrmk" event={"ID":"08b23842-c3ef-4f3a-bd47-3587c4011d35","Type":"ContainerStarted","Data":"d45f1319c8d9dca43f12cdea8e75acfc1858fa61695a5e758bb9bb3b334f76a8"}
Sep 15 23:06:26.154006 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:26.153652    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rhrmk"
Sep 15 23:06:26.173334 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:26.173282    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rhrmk" podStartSLOduration=2.173266547 podStartE2EDuration="2.173266547s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 23:06:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 23:06:26.17190643 +0000 UTC m=+2336.031617715" watchObservedRunningTime="2026-09-15 23:06:26.173266547 +0000 UTC m=+2336.032977830"
Sep 15 23:06:37.161893 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:06:37.161859    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rhrmk"
Sep 15 23:08:38.696536 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:38.696500    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-79d555d56b-77gfz"]
Sep 15 23:08:38.699888 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:38.699868    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-79d555d56b-77gfz"
Sep 15 23:08:38.711169 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:38.711139    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-79d555d56b-77gfz"]
Sep 15 23:08:38.824637 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:38.824606    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57618e18-658c-4827-862a-2de0c19e4d24-metrics-certs\") pod \"maas-api-79d555d56b-77gfz\" (UID: \"57618e18-658c-4827-862a-2de0c19e4d24\") " pod="odh-ai-gateway-infra/maas-api-79d555d56b-77gfz"
Sep 15 23:08:38.824809 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:38.824643    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6m9dq\" (UniqueName: \"kubernetes.io/projected/57618e18-658c-4827-862a-2de0c19e4d24-kube-api-access-6m9dq\") pod \"maas-api-79d555d56b-77gfz\" (UID: \"57618e18-658c-4827-862a-2de0c19e4d24\") " pod="odh-ai-gateway-infra/maas-api-79d555d56b-77gfz"
Sep 15 23:08:38.824809 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:38.824667    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57618e18-658c-4827-862a-2de0c19e4d24-openshift-service-ca\") pod \"maas-api-79d555d56b-77gfz\" (UID: \"57618e18-658c-4827-862a-2de0c19e4d24\") " pod="odh-ai-gateway-infra/maas-api-79d555d56b-77gfz"
Sep 15 23:08:38.824809 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:38.824795    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57618e18-658c-4827-862a-2de0c19e4d24-maas-api-tls\") pod \"maas-api-79d555d56b-77gfz\" (UID: \"57618e18-658c-4827-862a-2de0c19e4d24\") " pod="odh-ai-gateway-infra/maas-api-79d555d56b-77gfz"
Sep 15 23:08:38.925796 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:38.925746    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57618e18-658c-4827-862a-2de0c19e4d24-maas-api-tls\") pod \"maas-api-79d555d56b-77gfz\" (UID: \"57618e18-658c-4827-862a-2de0c19e4d24\") " pod="odh-ai-gateway-infra/maas-api-79d555d56b-77gfz"
Sep 15 23:08:38.925989 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:38.925827    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57618e18-658c-4827-862a-2de0c19e4d24-metrics-certs\") pod \"maas-api-79d555d56b-77gfz\" (UID: \"57618e18-658c-4827-862a-2de0c19e4d24\") " pod="odh-ai-gateway-infra/maas-api-79d555d56b-77gfz"
Sep 15 23:08:38.925989 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:38.925859    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6m9dq\" (UniqueName: \"kubernetes.io/projected/57618e18-658c-4827-862a-2de0c19e4d24-kube-api-access-6m9dq\") pod \"maas-api-79d555d56b-77gfz\" (UID: \"57618e18-658c-4827-862a-2de0c19e4d24\") " pod="odh-ai-gateway-infra/maas-api-79d555d56b-77gfz"
Sep 15 23:08:38.925989 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:38.925891    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57618e18-658c-4827-862a-2de0c19e4d24-openshift-service-ca\") pod \"maas-api-79d555d56b-77gfz\" (UID: \"57618e18-658c-4827-862a-2de0c19e4d24\") " pod="odh-ai-gateway-infra/maas-api-79d555d56b-77gfz"
Sep 15 23:08:38.926754 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:38.926727    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57618e18-658c-4827-862a-2de0c19e4d24-openshift-service-ca\") pod \"maas-api-79d555d56b-77gfz\" (UID: \"57618e18-658c-4827-862a-2de0c19e4d24\") " pod="odh-ai-gateway-infra/maas-api-79d555d56b-77gfz"
Sep 15 23:08:38.928771 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:38.928745    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57618e18-658c-4827-862a-2de0c19e4d24-metrics-certs\") pod \"maas-api-79d555d56b-77gfz\" (UID: \"57618e18-658c-4827-862a-2de0c19e4d24\") " pod="odh-ai-gateway-infra/maas-api-79d555d56b-77gfz"
Sep 15 23:08:38.928880 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:38.928773    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57618e18-658c-4827-862a-2de0c19e4d24-maas-api-tls\") pod \"maas-api-79d555d56b-77gfz\" (UID: \"57618e18-658c-4827-862a-2de0c19e4d24\") " pod="odh-ai-gateway-infra/maas-api-79d555d56b-77gfz"
Sep 15 23:08:38.935505 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:38.935475    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6m9dq\" (UniqueName: \"kubernetes.io/projected/57618e18-658c-4827-862a-2de0c19e4d24-kube-api-access-6m9dq\") pod \"maas-api-79d555d56b-77gfz\" (UID: \"57618e18-658c-4827-862a-2de0c19e4d24\") " pod="odh-ai-gateway-infra/maas-api-79d555d56b-77gfz"
Sep 15 23:08:39.013565 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:39.013480    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-79d555d56b-77gfz"
Sep 15 23:08:39.285859 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:39.285834    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-79d555d56b-77gfz"]
Sep 15 23:08:39.287443 ip-10-0-128-208 kubenswrapper[2565]: W0915 23:08:39.287414    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57618e18_658c_4827_862a_2de0c19e4d24.slice/crio-29d508d56de1aafefec901ad0a65298e059ce92e3b37bb7779c11f2d97d17b05 WatchSource:0}: Error finding container 29d508d56de1aafefec901ad0a65298e059ce92e3b37bb7779c11f2d97d17b05: Status 404 returned error can't find the container with id 29d508d56de1aafefec901ad0a65298e059ce92e3b37bb7779c11f2d97d17b05
Sep 15 23:08:39.288810 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:39.288790    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 23:08:39.912077 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:39.911983    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-79d555d56b-77gfz" event={"ID":"57618e18-658c-4827-862a-2de0c19e4d24","Type":"ContainerStarted","Data":"e15d93601ed1d78ef45282602192e58f338a9f996b877035f65a92cc1172ac0a"}
Sep 15 23:08:39.912077 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:39.912047    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-79d555d56b-77gfz" event={"ID":"57618e18-658c-4827-862a-2de0c19e4d24","Type":"ContainerStarted","Data":"29d508d56de1aafefec901ad0a65298e059ce92e3b37bb7779c11f2d97d17b05"}
Sep 15 23:08:39.912531 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:39.912143    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-79d555d56b-77gfz"
Sep 15 23:08:39.929007 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:39.928945    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-79d555d56b-77gfz" podStartSLOduration=1.616573887 podStartE2EDuration="1.928906112s" totalImagesPullingTime="312.332225ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 23:08:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 23:08:39.927280818 +0000 UTC m=+2469.786992105" watchObservedRunningTime="2026-09-15 23:08:39.928906112 +0000 UTC m=+2469.788617397"
Sep 15 23:08:45.923891 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:45.923863    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-79d555d56b-77gfz"
Sep 15 23:08:45.970667 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:45.970627    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-98857cbf4-hpw2w"]
Sep 15 23:08:45.970955 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:45.970904    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-98857cbf4-hpw2w" podUID="34b70b32-cf8b-4b40-9882-dfa432ee13d4" containerName="maas-api" containerID="cri-o://e20cacf4f7c79a3c5f77ddc94e1ee01bb27c15ee29738e04990e61878e84dccd" gracePeriod=30
Sep 15 23:08:47.221733 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:47.221710    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-98857cbf4-hpw2w"
Sep 15 23:08:47.293199 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:47.293166    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tv8gb\" (UniqueName: \"kubernetes.io/projected/34b70b32-cf8b-4b40-9882-dfa432ee13d4-kube-api-access-tv8gb\") pod \"34b70b32-cf8b-4b40-9882-dfa432ee13d4\" (UID: \"34b70b32-cf8b-4b40-9882-dfa432ee13d4\") "
Sep 15 23:08:47.293392 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:47.293219    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34b70b32-cf8b-4b40-9882-dfa432ee13d4-openshift-service-ca\") pod \"34b70b32-cf8b-4b40-9882-dfa432ee13d4\" (UID: \"34b70b32-cf8b-4b40-9882-dfa432ee13d4\") "
Sep 15 23:08:47.293392 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:47.293244    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34b70b32-cf8b-4b40-9882-dfa432ee13d4-metrics-certs\") pod \"34b70b32-cf8b-4b40-9882-dfa432ee13d4\" (UID: \"34b70b32-cf8b-4b40-9882-dfa432ee13d4\") "
Sep 15 23:08:47.293392 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:47.293316    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34b70b32-cf8b-4b40-9882-dfa432ee13d4-maas-api-tls\") pod \"34b70b32-cf8b-4b40-9882-dfa432ee13d4\" (UID: \"34b70b32-cf8b-4b40-9882-dfa432ee13d4\") "
Sep 15 23:08:47.293669 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:47.293642    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34b70b32-cf8b-4b40-9882-dfa432ee13d4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "34b70b32-cf8b-4b40-9882-dfa432ee13d4" (UID: "34b70b32-cf8b-4b40-9882-dfa432ee13d4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 23:08:47.295349 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:47.295324    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34b70b32-cf8b-4b40-9882-dfa432ee13d4-kube-api-access-tv8gb" (OuterVolumeSpecName: "kube-api-access-tv8gb") pod "34b70b32-cf8b-4b40-9882-dfa432ee13d4" (UID: "34b70b32-cf8b-4b40-9882-dfa432ee13d4"). InnerVolumeSpecName "kube-api-access-tv8gb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 23:08:47.295435 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:47.295409    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/34b70b32-cf8b-4b40-9882-dfa432ee13d4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "34b70b32-cf8b-4b40-9882-dfa432ee13d4" (UID: "34b70b32-cf8b-4b40-9882-dfa432ee13d4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 23:08:47.295473 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:47.295435    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/34b70b32-cf8b-4b40-9882-dfa432ee13d4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "34b70b32-cf8b-4b40-9882-dfa432ee13d4" (UID: "34b70b32-cf8b-4b40-9882-dfa432ee13d4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 23:08:47.394098 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:47.393983    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tv8gb\" (UniqueName: \"kubernetes.io/projected/34b70b32-cf8b-4b40-9882-dfa432ee13d4-kube-api-access-tv8gb\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 23:08:47.394098 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:47.394015    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34b70b32-cf8b-4b40-9882-dfa432ee13d4-openshift-service-ca\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 23:08:47.394098 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:47.394053    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34b70b32-cf8b-4b40-9882-dfa432ee13d4-metrics-certs\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 23:08:47.394098 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:47.394061    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34b70b32-cf8b-4b40-9882-dfa432ee13d4-maas-api-tls\") on node \"ip-10-0-128-208.ec2.internal\" DevicePath \"\""
Sep 15 23:08:47.959904 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:47.959864    2565 generic.go:356] "Generic (PLEG): container finished" podID="34b70b32-cf8b-4b40-9882-dfa432ee13d4" containerID="e20cacf4f7c79a3c5f77ddc94e1ee01bb27c15ee29738e04990e61878e84dccd" exitCode=0
Sep 15 23:08:47.960118 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:47.959931    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-98857cbf4-hpw2w"
Sep 15 23:08:47.960118 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:47.959952    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-98857cbf4-hpw2w" event={"ID":"34b70b32-cf8b-4b40-9882-dfa432ee13d4","Type":"ContainerDied","Data":"e20cacf4f7c79a3c5f77ddc94e1ee01bb27c15ee29738e04990e61878e84dccd"}
Sep 15 23:08:47.960118 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:47.960008    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-98857cbf4-hpw2w" event={"ID":"34b70b32-cf8b-4b40-9882-dfa432ee13d4","Type":"ContainerDied","Data":"92c8d1d6e0b7310ac4764beed36c81e3030285bf6889d25081312648fb641079"}
Sep 15 23:08:47.960118 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:47.960050    2565 scope.go:117] "RemoveContainer" containerID="e20cacf4f7c79a3c5f77ddc94e1ee01bb27c15ee29738e04990e61878e84dccd"
Sep 15 23:08:47.970042 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:47.970006    2565 scope.go:117] "RemoveContainer" containerID="e20cacf4f7c79a3c5f77ddc94e1ee01bb27c15ee29738e04990e61878e84dccd"
Sep 15 23:08:47.970334 ip-10-0-128-208 kubenswrapper[2565]: E0915 23:08:47.970316    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e20cacf4f7c79a3c5f77ddc94e1ee01bb27c15ee29738e04990e61878e84dccd\": container with ID starting with e20cacf4f7c79a3c5f77ddc94e1ee01bb27c15ee29738e04990e61878e84dccd not found: ID does not exist" containerID="e20cacf4f7c79a3c5f77ddc94e1ee01bb27c15ee29738e04990e61878e84dccd"
Sep 15 23:08:47.970392 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:47.970342    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e20cacf4f7c79a3c5f77ddc94e1ee01bb27c15ee29738e04990e61878e84dccd"} err="failed to get container status \"e20cacf4f7c79a3c5f77ddc94e1ee01bb27c15ee29738e04990e61878e84dccd\": rpc error: code = NotFound desc = could not find container \"e20cacf4f7c79a3c5f77ddc94e1ee01bb27c15ee29738e04990e61878e84dccd\": container with ID starting with e20cacf4f7c79a3c5f77ddc94e1ee01bb27c15ee29738e04990e61878e84dccd not found: ID does not exist"
Sep 15 23:08:47.980863 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:47.980838    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-98857cbf4-hpw2w"]
Sep 15 23:08:47.984936 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:47.984915    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-98857cbf4-hpw2w"]
Sep 15 23:08:48.590728 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:08:48.590694    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34b70b32-cf8b-4b40-9882-dfa432ee13d4" path="/var/lib/kubelet/pods/34b70b32-cf8b-4b40-9882-dfa432ee13d4/volumes"
Sep 15 23:09:02.072735 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:02.072702    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-c4b5f44c7-25fnj_59d5908c-cac0-4888-9de9-a1c390b68e55/authorino/0.log"
Sep 15 23:09:07.333817 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:07.333784    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-t2pzq_c10794ba-ad48-471c-a78f-c2552e0d402f/manager/0.log"
Sep 15 23:09:07.448681 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:07.448651    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-45x2s_3dc56556-dd6c-42c1-9f68-816e78f5aa65/manager/0.log"
Sep 15 23:09:08.042858 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:08.042832    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-749d6986f8-lhz48_0b727e71-070d-493c-9a92-cbf87f151291/copy-manifests/0.log"
Sep 15 23:09:08.048349 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:08.048325    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-749d6986f8-lhz48_0b727e71-070d-493c-9a92-cbf87f151291/manager/0.log"
Sep 15 23:09:08.505656 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:08.505577    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-79d555d56b-77gfz_57618e18-658c-4827-862a-2de0c19e4d24/maas-api/0.log"
Sep 15 23:09:08.940849 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:08.940820    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-2drtx_2ce1a26d-3701-4c0a-96f8-463a1abb4384/postgres/0.log"
Sep 15 23:09:09.710654 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:09.710620    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-c4b5f44c7-25fnj_59d5908c-cac0-4888-9de9-a1c390b68e55/authorino/0.log"
Sep 15 23:09:09.835714 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:09.835682    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-fhn94_1f5dcd8e-b7f2-4f38-a96a-090ecad96a4f/manager/0.log"
Sep 15 23:09:10.049067 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:10.049009    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-f6c7t_0d0c07b6-bfae-499a-abdd-e5220cb59c37/kuadrant-console-plugin/0.log"
Sep 15 23:09:10.167360 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:10.167332    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-rhrmk_08b23842-c3ef-4f3a-bd47-3587c4011d35/manager/0.log"
Sep 15 23:09:10.421098 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:10.421013    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-gc8cm_6c109d0f-885d-407b-b630-12265bc44be9/manager/0.log"
Sep 15 23:09:11.105870 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:11.105840    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7f98985fb-xjpv7_f177deb3-c155-477c-91fa-b189eac69537/kube-auth-proxy/0.log"
Sep 15 23:09:11.327331 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:11.327299    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-fgpqt_dd29c53d-5d08-47be-ba67-0e959c0ee312/payload-pre-processing/0.log"
Sep 15 23:09:11.988617 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:11.988551    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-k4ln9_4c4d06c9-88c6-421a-ba3b-97bcc86e667e/main/0.log"
Sep 15 23:09:12.206888 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:12.206864    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-r5shq_0b49ca52-8bb2-4446-9479-21c426c881cc/main/0.log"
Sep 15 23:09:12.319938 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:12.319904    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzb6m7_eabf69af-5154-4d83-8f0e-9a072852b1ad/main/0.log"
Sep 15 23:09:12.426463 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:12.426436    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-cwtbq_91a9a4da-7b05-4220-b3c0-61364b9ef198/main/0.log"
Sep 15 23:09:20.308983 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:20.308952    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4d5lp_b22214d5-9c42-4710-bfff-08651d7942a3/global-pull-secret-syncer/0.log"
Sep 15 23:09:20.463042 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:20.463001    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-l9m9m_93274f69-efc2-48a0-8223-f3a37aca3947/konnectivity-agent/0.log"
Sep 15 23:09:20.509765 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:20.509733    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-208.ec2.internal_d23b474b380f11cca08f39b02958fed1/haproxy/0.log"
Sep 15 23:09:25.359186 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:25.359097    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-c4b5f44c7-25fnj_59d5908c-cac0-4888-9de9-a1c390b68e55/authorino/0.log"
Sep 15 23:09:25.396207 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:25.396175    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-fhn94_1f5dcd8e-b7f2-4f38-a96a-090ecad96a4f/manager/0.log"
Sep 15 23:09:25.457743 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:25.457713    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-f6c7t_0d0c07b6-bfae-499a-abdd-e5220cb59c37/kuadrant-console-plugin/0.log"
Sep 15 23:09:25.502284 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:25.502255    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-rhrmk_08b23842-c3ef-4f3a-bd47-3587c4011d35/manager/0.log"
Sep 15 23:09:25.585004 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:25.584967    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-gc8cm_6c109d0f-885d-407b-b630-12265bc44be9/manager/0.log"
Sep 15 23:09:27.009793 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:27.009764    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6543f8f8-8aee-4448-8534-fec54d43db35/alertmanager/0.log"
Sep 15 23:09:27.031454 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:27.031425    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6543f8f8-8aee-4448-8534-fec54d43db35/config-reloader/0.log"
Sep 15 23:09:27.052853 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:27.052828    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6543f8f8-8aee-4448-8534-fec54d43db35/kube-rbac-proxy-web/0.log"
Sep 15 23:09:27.072832 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:27.072805    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6543f8f8-8aee-4448-8534-fec54d43db35/kube-rbac-proxy/0.log"
Sep 15 23:09:27.091316 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:27.091287    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6543f8f8-8aee-4448-8534-fec54d43db35/kube-rbac-proxy-metric/0.log"
Sep 15 23:09:27.111881 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:27.111855    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6543f8f8-8aee-4448-8534-fec54d43db35/prom-label-proxy/0.log"
Sep 15 23:09:27.135988 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:27.135959    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6543f8f8-8aee-4448-8534-fec54d43db35/init-config-reloader/0.log"
Sep 15 23:09:27.211760 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:27.211731    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-4s6mz_ee812c64-1a26-483b-b4c8-d7abc9bcbca7/kube-state-metrics/0.log"
Sep 15 23:09:27.228034 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:27.228009    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-4s6mz_ee812c64-1a26-483b-b4c8-d7abc9bcbca7/kube-rbac-proxy-main/0.log"
Sep 15 23:09:27.247913 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:27.247890    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-4s6mz_ee812c64-1a26-483b-b4c8-d7abc9bcbca7/kube-rbac-proxy-self/0.log"
Sep 15 23:09:27.487630 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:27.487599    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wjrjh_448978b1-d7a1-4152-81b9-37aaacf532d5/node-exporter/0.log"
Sep 15 23:09:27.506119 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:27.506094    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wjrjh_448978b1-d7a1-4152-81b9-37aaacf532d5/kube-rbac-proxy/0.log"
Sep 15 23:09:27.526747 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:27.526718    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wjrjh_448978b1-d7a1-4152-81b9-37aaacf532d5/init-textfile/0.log"
Sep 15 23:09:27.555432 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:27.555404    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-2bv8w_1899c63e-fd24-4dae-8c3a-0059a53237ba/kube-rbac-proxy-main/0.log"
Sep 15 23:09:27.578363 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:27.578334    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-2bv8w_1899c63e-fd24-4dae-8c3a-0059a53237ba/kube-rbac-proxy-self/0.log"
Sep 15 23:09:27.601631 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:27.601605    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-2bv8w_1899c63e-fd24-4dae-8c3a-0059a53237ba/openshift-state-metrics/0.log"
Sep 15 23:09:27.926681 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:27.926651    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9778f96f8-tr52c_fb96bba3-1956-4f1f-8b67-3c7a06fcff76/telemeter-client/0.log"
Sep 15 23:09:27.964213 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:27.964187    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9778f96f8-tr52c_fb96bba3-1956-4f1f-8b67-3c7a06fcff76/reload/0.log"
Sep 15 23:09:27.988395 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:27.988369    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9778f96f8-tr52c_fb96bba3-1956-4f1f-8b67-3c7a06fcff76/kube-rbac-proxy/0.log"
Sep 15 23:09:30.355068 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:30.355019    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ctgsf/perf-node-gather-daemonset-jffzc"]
Sep 15 23:09:30.355505 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:30.355490    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34b70b32-cf8b-4b40-9882-dfa432ee13d4" containerName="maas-api"
Sep 15 23:09:30.355589 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:30.355505    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="34b70b32-cf8b-4b40-9882-dfa432ee13d4" containerName="maas-api"
Sep 15 23:09:30.355677 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:30.355595    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="34b70b32-cf8b-4b40-9882-dfa432ee13d4" containerName="maas-api"
Sep 15 23:09:30.359192 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:30.359171    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-jffzc"
Sep 15 23:09:30.361162 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:30.361130    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ctgsf\"/\"openshift-service-ca.crt\""
Sep 15 23:09:30.361285 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:30.361130    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ctgsf\"/\"default-dockercfg-drr56\""
Sep 15 23:09:30.361285 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:30.361134    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ctgsf\"/\"kube-root-ca.crt\""
Sep 15 23:09:30.368213 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:30.368186    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ctgsf/perf-node-gather-daemonset-jffzc"]
Sep 15 23:09:30.397420 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:30.397383    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8e212ebb-f7c8-41ee-8fe1-ca7928169608-podres\") pod \"perf-node-gather-daemonset-jffzc\" (UID: \"8e212ebb-f7c8-41ee-8fe1-ca7928169608\") " pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-jffzc"
Sep 15 23:09:30.397569 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:30.397424    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8e212ebb-f7c8-41ee-8fe1-ca7928169608-proc\") pod \"perf-node-gather-daemonset-jffzc\" (UID: \"8e212ebb-f7c8-41ee-8fe1-ca7928169608\") " pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-jffzc"
Sep 15 23:09:30.397569 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:30.397539    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8e212ebb-f7c8-41ee-8fe1-ca7928169608-lib-modules\") pod \"perf-node-gather-daemonset-jffzc\" (UID: \"8e212ebb-f7c8-41ee-8fe1-ca7928169608\") " pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-jffzc"
Sep 15 23:09:30.397663 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:30.397574    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcf8m\" (UniqueName: \"kubernetes.io/projected/8e212ebb-f7c8-41ee-8fe1-ca7928169608-kube-api-access-lcf8m\") pod \"perf-node-gather-daemonset-jffzc\" (UID: \"8e212ebb-f7c8-41ee-8fe1-ca7928169608\") " pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-jffzc"
Sep 15 23:09:30.397663 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:30.397597    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e212ebb-f7c8-41ee-8fe1-ca7928169608-sys\") pod \"perf-node-gather-daemonset-jffzc\" (UID: \"8e212ebb-f7c8-41ee-8fe1-ca7928169608\") " pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-jffzc"
Sep 15 23:09:30.498154 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:30.498117    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8e212ebb-f7c8-41ee-8fe1-ca7928169608-proc\") pod \"perf-node-gather-daemonset-jffzc\" (UID: \"8e212ebb-f7c8-41ee-8fe1-ca7928169608\") " pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-jffzc"
Sep 15 23:09:30.498357 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:30.498202    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8e212ebb-f7c8-41ee-8fe1-ca7928169608-lib-modules\") pod \"perf-node-gather-daemonset-jffzc\" (UID: \"8e212ebb-f7c8-41ee-8fe1-ca7928169608\") " pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-jffzc"
Sep 15 23:09:30.498357 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:30.498238    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcf8m\" (UniqueName: \"kubernetes.io/projected/8e212ebb-f7c8-41ee-8fe1-ca7928169608-kube-api-access-lcf8m\") pod \"perf-node-gather-daemonset-jffzc\" (UID: \"8e212ebb-f7c8-41ee-8fe1-ca7928169608\") " pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-jffzc"
Sep 15 23:09:30.498357 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:30.498239    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8e212ebb-f7c8-41ee-8fe1-ca7928169608-proc\") pod \"perf-node-gather-daemonset-jffzc\" (UID: \"8e212ebb-f7c8-41ee-8fe1-ca7928169608\") " pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-jffzc"
Sep 15 23:09:30.498357 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:30.498266    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e212ebb-f7c8-41ee-8fe1-ca7928169608-sys\") pod \"perf-node-gather-daemonset-jffzc\" (UID: \"8e212ebb-f7c8-41ee-8fe1-ca7928169608\") " pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-jffzc"
Sep 15 23:09:30.498357 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:30.498335    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8e212ebb-f7c8-41ee-8fe1-ca7928169608-podres\") pod \"perf-node-gather-daemonset-jffzc\" (UID: \"8e212ebb-f7c8-41ee-8fe1-ca7928169608\") " pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-jffzc"
Sep 15 23:09:30.498585 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:30.498376    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8e212ebb-f7c8-41ee-8fe1-ca7928169608-lib-modules\") pod \"perf-node-gather-daemonset-jffzc\" (UID: \"8e212ebb-f7c8-41ee-8fe1-ca7928169608\") " pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-jffzc"
Sep 15 23:09:30.498585 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:30.498396    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e212ebb-f7c8-41ee-8fe1-ca7928169608-sys\") pod \"perf-node-gather-daemonset-jffzc\" (UID: \"8e212ebb-f7c8-41ee-8fe1-ca7928169608\") " pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-jffzc"
Sep 15 23:09:30.498585 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:30.498463    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8e212ebb-f7c8-41ee-8fe1-ca7928169608-podres\") pod \"perf-node-gather-daemonset-jffzc\" (UID: \"8e212ebb-f7c8-41ee-8fe1-ca7928169608\") " pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-jffzc"
Sep 15 23:09:30.506236 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:30.506215    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcf8m\" (UniqueName: \"kubernetes.io/projected/8e212ebb-f7c8-41ee-8fe1-ca7928169608-kube-api-access-lcf8m\") pod \"perf-node-gather-daemonset-jffzc\" (UID: \"8e212ebb-f7c8-41ee-8fe1-ca7928169608\") " pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-jffzc"
Sep 15 23:09:30.671406 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:30.671331    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ctgsf\"/\"default-dockercfg-drr56\""
Sep 15 23:09:30.679867 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:30.679845    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-jffzc"
Sep 15 23:09:30.687089 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:30.687067    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-588b46b67f-jdbnq_b90747ad-4dbe-4fb1-84b4-a627e8083222/console/0.log"
Sep 15 23:09:30.954055 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:30.954015    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ctgsf/perf-node-gather-daemonset-jffzc"]
Sep 15 23:09:30.956199 ip-10-0-128-208 kubenswrapper[2565]: W0915 23:09:30.956169    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8e212ebb_f7c8_41ee_8fe1_ca7928169608.slice/crio-d398f176d56fc4401ab21b0a54e5c1c4e676984730b2e82a3d7b9f15f0ed5429 WatchSource:0}: Error finding container d398f176d56fc4401ab21b0a54e5c1c4e676984730b2e82a3d7b9f15f0ed5429: Status 404 returned error can't find the container with id d398f176d56fc4401ab21b0a54e5c1c4e676984730b2e82a3d7b9f15f0ed5429
Sep 15 23:09:31.199134 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:31.199101    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-jffzc" event={"ID":"8e212ebb-f7c8-41ee-8fe1-ca7928169608","Type":"ContainerStarted","Data":"e0c616e8e3d9aab46a19d71bf8501ecc62c3c18bb8d8d072b14c7a2269ed7945"}
Sep 15 23:09:31.199280 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:31.199154    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-jffzc"
Sep 15 23:09:31.199280 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:31.199173    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-jffzc" event={"ID":"8e212ebb-f7c8-41ee-8fe1-ca7928169608","Type":"ContainerStarted","Data":"d398f176d56fc4401ab21b0a54e5c1c4e676984730b2e82a3d7b9f15f0ed5429"}
Sep 15 23:09:31.215496 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:31.215407    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-jffzc" podStartSLOduration=1.215392868 podStartE2EDuration="1.215392868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 23:09:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 23:09:31.212839285 +0000 UTC m=+2521.072550568" watchObservedRunningTime="2026-09-15 23:09:31.215392868 +0000 UTC m=+2521.075104151"
Sep 15 23:09:32.582441 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:32.582410    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gk8zw_d08ce97f-50da-4ee1-9e09-b8dfa23cefb3/dns/0.log"
Sep 15 23:09:32.603075 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:32.603045    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gk8zw_d08ce97f-50da-4ee1-9e09-b8dfa23cefb3/kube-rbac-proxy/0.log"
Sep 15 23:09:32.717205 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:32.717175    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-m4w7h_1bd5f596-e129-4139-97d6-b8f7696c26e4/dns-node-resolver/0.log"
Sep 15 23:09:33.237092 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:33.237061    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5cddfccf5c-5zq8q_4f8a5789-3f4c-4671-b9b7-91374ea81f86/registry/0.log"
Sep 15 23:09:33.297103 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:33.297067    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8tjs6_d6f6b883-5742-4ebd-bde1-5c7295563940/node-ca/0.log"
Sep 15 23:09:34.689253 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:34.689226    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7f98985fb-xjpv7_f177deb3-c155-477c-91fa-b189eac69537/kube-auth-proxy/0.log"
Sep 15 23:09:34.747129 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:34.747100    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-fgpqt_dd29c53d-5d08-47be-ba67-0e959c0ee312/payload-pre-processing/0.log"
Sep 15 23:09:35.706844 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:35.706818    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-m6xfw_bf3ece62-14b8-4d64-b799-3b9796eb2ea4/serve-healthcheck-canary/0.log"
Sep 15 23:09:36.335489 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:36.335459    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q8hxs_f618f110-7684-44c9-85ec-cb5e0802bed0/kube-rbac-proxy/0.log"
Sep 15 23:09:36.353098 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:36.353074    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q8hxs_f618f110-7684-44c9-85ec-cb5e0802bed0/exporter/0.log"
Sep 15 23:09:36.370632 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:36.370610    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q8hxs_f618f110-7684-44c9-85ec-cb5e0802bed0/extractor/0.log"
Sep 15 23:09:37.215638 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:37.215606    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-ctgsf/perf-node-gather-daemonset-jffzc"
Sep 15 23:09:38.534519 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:38.534490    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-t2pzq_c10794ba-ad48-471c-a78f-c2552e0d402f/manager/0.log"
Sep 15 23:09:38.806284 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:38.806199    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-45x2s_3dc56556-dd6c-42c1-9f68-816e78f5aa65/manager/0.log"
Sep 15 23:09:39.100489 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:39.100409    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-749d6986f8-lhz48_0b727e71-070d-493c-9a92-cbf87f151291/manager/0.log"
Sep 15 23:09:39.119763 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:39.119738    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-749d6986f8-lhz48_0b727e71-070d-493c-9a92-cbf87f151291/copy-manifests/0.log"
Sep 15 23:09:40.936326 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:40.936298    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7f98985fb-xjpv7_f177deb3-c155-477c-91fa-b189eac69537/kube-auth-proxy/0.log"
Sep 15 23:09:40.985065 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:40.985019    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-fgpqt_dd29c53d-5d08-47be-ba67-0e959c0ee312/payload-pre-processing/0.log"
Sep 15 23:09:42.970116 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:42.970085    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7f98985fb-xjpv7_f177deb3-c155-477c-91fa-b189eac69537/kube-auth-proxy/0.log"
Sep 15 23:09:43.019245 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:43.019215    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-fgpqt_dd29c53d-5d08-47be-ba67-0e959c0ee312/payload-pre-processing/0.log"
Sep 15 23:09:47.186689 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:47.186658    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5xcfb_e096afa4-418d-41a6-95ee-a181ccd16731/kube-multus-additional-cni-plugins/0.log"
Sep 15 23:09:47.206720 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:47.206687    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5xcfb_e096afa4-418d-41a6-95ee-a181ccd16731/egress-router-binary-copy/0.log"
Sep 15 23:09:47.226193 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:47.226169    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5xcfb_e096afa4-418d-41a6-95ee-a181ccd16731/cni-plugins/0.log"
Sep 15 23:09:47.244708 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:47.244686    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5xcfb_e096afa4-418d-41a6-95ee-a181ccd16731/bond-cni-plugin/0.log"
Sep 15 23:09:47.263348 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:47.263327    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5xcfb_e096afa4-418d-41a6-95ee-a181ccd16731/routeoverride-cni/0.log"
Sep 15 23:09:47.282541 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:47.282523    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5xcfb_e096afa4-418d-41a6-95ee-a181ccd16731/whereabouts-cni-bincopy/0.log"
Sep 15 23:09:47.301527 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:47.301501    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5xcfb_e096afa4-418d-41a6-95ee-a181ccd16731/whereabouts-cni/0.log"
Sep 15 23:09:47.543316 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:47.543286    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-nmmxx_f132ba8d-e9aa-4a73-8713-82fe09532325/kube-multus/0.log"
Sep 15 23:09:47.598899 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:47.598867    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-j4tsb_909a094b-77e5-4f6e-9d69-6f4214156810/network-metrics-daemon/0.log"
Sep 15 23:09:47.614171 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:47.614148    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-j4tsb_909a094b-77e5-4f6e-9d69-6f4214156810/kube-rbac-proxy/0.log"
Sep 15 23:09:48.495781 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:48.495755    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74hmh_bcb145e6-76ea-43fd-98c0-2c30c59dc2d0/ovn-controller/0.log"
Sep 15 23:09:48.511046 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:48.511010    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74hmh_bcb145e6-76ea-43fd-98c0-2c30c59dc2d0/ovn-acl-logging/0.log"
Sep 15 23:09:48.531652 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:48.531626    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74hmh_bcb145e6-76ea-43fd-98c0-2c30c59dc2d0/kube-rbac-proxy-node/0.log"
Sep 15 23:09:48.551338 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:48.551314    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74hmh_bcb145e6-76ea-43fd-98c0-2c30c59dc2d0/kube-rbac-proxy-ovn-metrics/0.log"
Sep 15 23:09:48.570134 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:48.570115    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74hmh_bcb145e6-76ea-43fd-98c0-2c30c59dc2d0/northd/0.log"
Sep 15 23:09:48.590552 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:48.590532    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74hmh_bcb145e6-76ea-43fd-98c0-2c30c59dc2d0/nbdb/0.log"
Sep 15 23:09:48.610942 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:48.610922    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74hmh_bcb145e6-76ea-43fd-98c0-2c30c59dc2d0/sbdb/0.log"
Sep 15 23:09:48.672409 ip-10-0-128-208 kubenswrapper[2565]: I0915 23:09:48.672378    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-74hmh_bcb145e6-76ea-43fd-98c0-2c30c59dc2d0/ovnkube-controller/0.log"