Sep 09 03:56:10.559353 ip-10-0-141-71 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 09 03:56:10.559364 ip-10-0-141-71 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 09 03:56:10.559371 ip-10-0-141-71 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 09 03:56:10.559570 ip-10-0-141-71 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 09 03:56:20.729549 ip-10-0-141-71 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 09 03:56:20.729566 ip-10-0-141-71 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot bb1862971f764b979b9c7237fc297405 --
Sep 09 03:57:54.989504 ip-10-0-141-71 systemd[1]: Starting Kubernetes Kubelet...
Sep 09 03:57:55.383031 ip-10-0-141-71 kubenswrapper[2557]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 03:57:55.383031 ip-10-0-141-71 kubenswrapper[2557]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 09 03:57:55.383031 ip-10-0-141-71 kubenswrapper[2557]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 03:57:55.383031 ip-10-0-141-71 kubenswrapper[2557]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 09 03:57:55.383031 ip-10-0-141-71 kubenswrapper[2557]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 03:57:55.386922 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.386700    2557 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 09 03:57:55.392979 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.392956    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 03:57:55.392979 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.392975    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 03:57:55.392979 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.392979    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 03:57:55.392979 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.392983    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 03:57:55.392979 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.392986    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 03:57:55.393189 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.392990    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 03:57:55.393189 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.392993    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 03:57:55.393189 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.392996    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 03:57:55.393189 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.392998    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 03:57:55.393189 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393001    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 03:57:55.393189 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393004    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 03:57:55.393189 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393007    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 03:57:55.393189 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393009    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 03:57:55.393189 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393012    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 03:57:55.393189 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393016    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 03:57:55.393189 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393019    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 03:57:55.393189 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393022    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 03:57:55.393189 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393025    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 03:57:55.393189 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393027    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 03:57:55.393189 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393030    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 03:57:55.393189 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393032    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 03:57:55.393189 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393035    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 03:57:55.393189 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393038    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 03:57:55.393189 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393040    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 03:57:55.393189 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393043    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 03:57:55.393687 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393045    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 03:57:55.393687 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393048    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 03:57:55.393687 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393050    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 03:57:55.393687 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393053    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 03:57:55.393687 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393055    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 03:57:55.393687 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393057    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 03:57:55.393687 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393061    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 03:57:55.393687 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393063    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 03:57:55.393687 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393066    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 03:57:55.393687 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393068    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 03:57:55.393687 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393071    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 03:57:55.393687 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393073    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 03:57:55.393687 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393077    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 03:57:55.393687 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393080    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 03:57:55.393687 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393082    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 03:57:55.393687 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393085    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 03:57:55.393687 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393087    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 03:57:55.393687 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393091    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 03:57:55.393687 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393096    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 03:57:55.393687 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393099    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 03:57:55.394177 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393103    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 03:57:55.394177 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393106    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 03:57:55.394177 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393122    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 03:57:55.394177 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393125    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 03:57:55.394177 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393128    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 03:57:55.394177 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393131    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 03:57:55.394177 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393133    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 03:57:55.394177 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393136    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 03:57:55.394177 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393138    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 03:57:55.394177 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393141    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 03:57:55.394177 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393144    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 03:57:55.394177 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393146    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 03:57:55.394177 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393149    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 03:57:55.394177 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393152    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 03:57:55.394177 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393154    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 03:57:55.394177 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393157    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 03:57:55.394177 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393160    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 03:57:55.394177 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393162    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 03:57:55.394177 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393164    2557 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 03:57:55.394631 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393167    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 03:57:55.394631 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393169    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 03:57:55.394631 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393172    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 03:57:55.394631 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393174    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 03:57:55.394631 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393177    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 03:57:55.394631 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393181    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 03:57:55.394631 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393184    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 03:57:55.394631 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393186    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 03:57:55.394631 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393189    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 03:57:55.394631 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393192    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 03:57:55.394631 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393194    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 03:57:55.394631 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393197    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 03:57:55.394631 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393200    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 03:57:55.394631 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393202    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 03:57:55.394631 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393205    2557 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 03:57:55.394631 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393207    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 03:57:55.394631 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393210    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 03:57:55.394631 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393212    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 03:57:55.394631 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393215    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 03:57:55.395083 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393220    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 03:57:55.395083 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393224    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 03:57:55.395083 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393227    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 03:57:55.395083 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393230    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 03:57:55.395083 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393232    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 03:57:55.395083 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393235    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 03:57:55.395083 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393238    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 03:57:55.395083 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393240    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 03:57:55.395083 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393244    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 03:57:55.395083 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393246    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 03:57:55.395083 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393249    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 03:57:55.395083 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393251    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 03:57:55.395083 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393254    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 03:57:55.395083 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393256    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 03:57:55.395083 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393259    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 03:57:55.395083 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.393261    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 03:57:55.395083 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395014    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 03:57:55.395083 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395021    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 03:57:55.395083 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395025    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 03:57:55.395539 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395028    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 03:57:55.395539 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395030    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 03:57:55.395539 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395033    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 03:57:55.395539 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395036    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 03:57:55.395539 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395038    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 03:57:55.395539 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395041    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 03:57:55.395539 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395043    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 03:57:55.395539 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395046    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 03:57:55.395539 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395048    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 03:57:55.395539 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395051    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 03:57:55.395539 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395053    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 03:57:55.395539 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395055    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 03:57:55.395539 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395058    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 03:57:55.395539 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395060    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 03:57:55.395539 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395065    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 03:57:55.395539 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395067    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 03:57:55.395539 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395070    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 03:57:55.395539 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395073    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 03:57:55.395539 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395075    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 03:57:55.395539 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395078    2557 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 03:57:55.396033 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395081    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 03:57:55.396033 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395083    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 03:57:55.396033 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395086    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 03:57:55.396033 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395088    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 03:57:55.396033 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395091    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 03:57:55.396033 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395094    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 03:57:55.396033 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395097    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 03:57:55.396033 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395099    2557 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 03:57:55.396033 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395102    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 03:57:55.396033 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395104    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 03:57:55.396033 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395106    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 03:57:55.396033 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395124    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 03:57:55.396033 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395127    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 03:57:55.396033 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395130    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 03:57:55.396033 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395133    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 03:57:55.396033 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395136    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 03:57:55.396033 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395138    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 03:57:55.396033 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395140    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 03:57:55.396033 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395143    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 03:57:55.396514 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395145    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 03:57:55.396514 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395148    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 03:57:55.396514 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395150    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 03:57:55.396514 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395153    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 03:57:55.396514 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395155    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 03:57:55.396514 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395158    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 03:57:55.396514 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395160    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 03:57:55.396514 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395163    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 03:57:55.396514 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395166    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 03:57:55.396514 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395169    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 03:57:55.396514 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395172    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 03:57:55.396514 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395174    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 03:57:55.396514 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395177    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 03:57:55.396514 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395180    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 03:57:55.396514 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395183    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 03:57:55.396514 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395185    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 03:57:55.396514 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395188    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 03:57:55.396514 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395190    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 03:57:55.396514 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395193    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 03:57:55.396514 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395195    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 03:57:55.396981 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395197    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 03:57:55.396981 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395201    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 03:57:55.396981 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395205    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 03:57:55.396981 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395207    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 03:57:55.396981 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395210    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 03:57:55.396981 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395212    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 03:57:55.396981 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395215    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 03:57:55.396981 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395217    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 03:57:55.396981 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395220    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 03:57:55.396981 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395223    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 03:57:55.396981 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395225    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 03:57:55.396981 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395228    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 03:57:55.396981 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395230    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 03:57:55.396981 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395233    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 03:57:55.396981 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395235    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 03:57:55.396981 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395238    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 03:57:55.396981 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395240    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 03:57:55.396981 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395243    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 03:57:55.396981 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395245    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 03:57:55.396981 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395248    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 03:57:55.397496 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395251    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 03:57:55.397496 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395254    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 03:57:55.397496 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395257    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 03:57:55.397496 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395259    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 03:57:55.397496 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395261    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 03:57:55.397496 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395264    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 03:57:55.397496 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395267    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 03:57:55.397496 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395271    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 03:57:55.397496 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395273    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 03:57:55.397496 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395276    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 03:57:55.397496 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395280    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 03:57:55.397496 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395283    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 03:57:55.397496 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395287    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 03:57:55.397496 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395291    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 03:57:55.397496 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395294    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 03:57:55.397496 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395297    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 03:57:55.397496 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395299    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 03:57:55.397496 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395670    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 03:57:55.397496 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395675    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 03:57:55.397962 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395679    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 03:57:55.397962 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395682    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 03:57:55.397962 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395684    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 03:57:55.397962 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395687    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 03:57:55.397962 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395690    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 03:57:55.397962 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395693    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 03:57:55.397962 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395696    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 03:57:55.397962 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395699    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 03:57:55.397962 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395701    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 03:57:55.397962 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395704    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 03:57:55.397962 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395706    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 03:57:55.397962 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395709    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 03:57:55.397962 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395713    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 03:57:55.397962 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395715    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 03:57:55.397962 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395720    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 03:57:55.397962 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395724    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 03:57:55.397962 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395727    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 03:57:55.397962 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395730    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 03:57:55.397962 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395732    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 03:57:55.398442 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395735    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 03:57:55.398442 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395738    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 03:57:55.398442 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395740    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 03:57:55.398442 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395743    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 03:57:55.398442 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395746    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 03:57:55.398442 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395750    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 03:57:55.398442 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395753    2557 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 03:57:55.398442 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395756    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 03:57:55.398442 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395758    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 03:57:55.398442 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395761    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 03:57:55.398442 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395763    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 03:57:55.398442 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395766    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 03:57:55.398442 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395768    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 03:57:55.398442 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395770    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 03:57:55.398442 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395773    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 03:57:55.398442 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395776    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 03:57:55.398442 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395778    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 03:57:55.398442 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395781    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 03:57:55.398442 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395783    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 03:57:55.398442 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395786    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 03:57:55.398946 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395788    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 03:57:55.398946 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395791    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 03:57:55.398946 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395793    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 03:57:55.398946 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395796    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 03:57:55.398946 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395799    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 03:57:55.398946 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395801    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 03:57:55.398946 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395804    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 03:57:55.398946 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395807    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 03:57:55.398946 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395810    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 03:57:55.398946 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395812    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 03:57:55.398946 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395815    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 03:57:55.398946 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395817    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 03:57:55.398946 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395820    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 03:57:55.398946 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395822    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 03:57:55.398946 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395825    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 03:57:55.398946 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395827    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 03:57:55.398946 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395830    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 03:57:55.398946 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395832    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 03:57:55.398946 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395835    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 03:57:55.398946 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395837    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 03:57:55.399436 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395839    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 03:57:55.399436 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395842    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 03:57:55.399436 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395844    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 03:57:55.399436 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395847    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 03:57:55.399436 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395852    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 03:57:55.399436 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395855    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 03:57:55.399436 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395858    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 03:57:55.399436 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395861    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 03:57:55.399436 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395864    2557 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 03:57:55.399436 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395867    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 03:57:55.399436 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395869    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 03:57:55.399436 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395872    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 03:57:55.399436 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395874    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 03:57:55.399436 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395877    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 03:57:55.399436 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395879    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 03:57:55.399436 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395882    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 03:57:55.399436 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395884    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 03:57:55.399436 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395887    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 03:57:55.399436 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395889    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 03:57:55.399436 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395892    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 03:57:55.399912 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395894    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 03:57:55.399912 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395897    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 03:57:55.399912 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395900    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 03:57:55.399912 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395902    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 03:57:55.399912 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395905    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 03:57:55.399912 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395907    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 03:57:55.399912 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395910    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 03:57:55.399912 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395912    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 03:57:55.399912 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395914    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 03:57:55.399912 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395917    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 03:57:55.399912 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395919    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 03:57:55.399912 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395921    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 03:57:55.399912 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395924    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 03:57:55.399912 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395926    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 03:57:55.399912 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395929    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 03:57:55.399912 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395931    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 03:57:55.399912 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395933    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 03:57:55.399912 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.395936    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 03:57:55.399912 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396011    2557 flags.go:64] FLAG: --address="0.0.0.0"
Sep 09 03:57:55.399912 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396044    2557 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 09 03:57:55.399912 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396051    2557 flags.go:64] FLAG: --anonymous-auth="true"
Sep 09 03:57:55.400452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396056    2557 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 09 03:57:55.400452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396060    2557 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 09 03:57:55.400452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396063    2557 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 09 03:57:55.400452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396067    2557 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 09 03:57:55.400452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396072    2557 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 09 03:57:55.400452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396075    2557 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 09 03:57:55.400452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396078    2557 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 09 03:57:55.400452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396082    2557 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 09 03:57:55.400452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396085    2557 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 09 03:57:55.400452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396088    2557 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 09 03:57:55.400452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396090    2557 flags.go:64] FLAG: --cgroup-root=""
Sep 09 03:57:55.400452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396093    2557 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 09 03:57:55.400452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396096    2557 flags.go:64] FLAG: --client-ca-file=""
Sep 09 03:57:55.400452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396099    2557 flags.go:64] FLAG: --cloud-provider="external"
Sep 09 03:57:55.400452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396102    2557 flags.go:64] FLAG: --cluster-dns="[]"
Sep 09 03:57:55.400452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396106    2557 flags.go:64] FLAG: --cluster-domain=""
Sep 09 03:57:55.400452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396122    2557 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 09 03:57:55.400452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396125    2557 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 09 03:57:55.400452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396128    2557 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 09 03:57:55.400452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396131    2557 flags.go:64] FLAG: --container-log-max-files="5"
Sep 09 03:57:55.400452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396135    2557 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 09 03:57:55.400452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396138    2557 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 09 03:57:55.400452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396141    2557 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 09 03:57:55.400994 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396144    2557 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 09 03:57:55.400994 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396147    2557 flags.go:64] FLAG: --contention-profiling="false"
Sep 09 03:57:55.400994 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396150    2557 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 09 03:57:55.400994 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396153    2557 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 09 03:57:55.400994 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396155    2557 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 09 03:57:55.400994 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396161    2557 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 09 03:57:55.400994 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396165    2557 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 09 03:57:55.400994 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396168    2557 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 09 03:57:55.400994 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396171    2557 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 09 03:57:55.400994 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396174    2557 flags.go:64] FLAG: --enable-load-reader="false"
Sep 09 03:57:55.400994 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396177    2557 flags.go:64] FLAG: --enable-server="true"
Sep 09 03:57:55.400994 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396180    2557 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 09 03:57:55.400994 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396184    2557 flags.go:64] FLAG: --event-burst="100"
Sep 09 03:57:55.400994 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396187    2557 flags.go:64] FLAG: --event-qps="50"
Sep 09 03:57:55.400994 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396190    2557 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 09 03:57:55.400994 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396193    2557 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 09 03:57:55.400994 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396196    2557 flags.go:64] FLAG: --eviction-hard=""
Sep 09 03:57:55.400994 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396199    2557 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 09 03:57:55.400994 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396202    2557 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 09 03:57:55.400994 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396205    2557 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 09 03:57:55.400994 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396207    2557 flags.go:64] FLAG: --eviction-soft=""
Sep 09 03:57:55.400994 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396210    2557 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 09 03:57:55.400994 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396213    2557 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 09 03:57:55.400994 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396221    2557 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 09 03:57:55.400994 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396224    2557 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 09 03:57:55.401633 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396227    2557 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 09 03:57:55.401633 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396229    2557 flags.go:64] FLAG: --fail-swap-on="true"
Sep 09 03:57:55.401633 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396232    2557 flags.go:64] FLAG: --feature-gates=""
Sep 09 03:57:55.401633 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396236    2557 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 09 03:57:55.401633 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396239    2557 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 09 03:57:55.401633 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396242    2557 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 09 03:57:55.401633 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396245    2557 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 09 03:57:55.401633 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396248    2557 flags.go:64] FLAG: --healthz-port="10248"
Sep 09 03:57:55.401633 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396251    2557 flags.go:64] FLAG: --help="false"
Sep 09 03:57:55.401633 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396253    2557 flags.go:64] FLAG: --hostname-override="ip-10-0-141-71.ec2.internal"
Sep 09 03:57:55.401633 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396256    2557 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 09 03:57:55.401633 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396259    2557 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 09 03:57:55.401633 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396263    2557 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 09 03:57:55.401633 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396266    2557 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 09 03:57:55.401633 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396269    2557 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 09 03:57:55.401633 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396272    2557 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 09 03:57:55.401633 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396275    2557 flags.go:64] FLAG: --image-service-endpoint=""
Sep 09 03:57:55.401633 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396278    2557 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 09 03:57:55.401633 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396280    2557 flags.go:64] FLAG: --kube-api-burst="100"
Sep 09 03:57:55.401633 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396283    2557 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 09 03:57:55.401633 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396286    2557 flags.go:64] FLAG: --kube-api-qps="50"
Sep 09 03:57:55.401633 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396290    2557 flags.go:64] FLAG: --kube-reserved=""
Sep 09 03:57:55.401633 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396293    2557 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 09 03:57:55.401633 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396296    2557 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 09 03:57:55.402236 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396299    2557 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 09 03:57:55.402236 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396302    2557 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 09 03:57:55.402236 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396304    2557 flags.go:64] FLAG: --lock-file=""
Sep 09 03:57:55.402236 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396307    2557 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 09 03:57:55.402236 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396310    2557 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 09 03:57:55.402236 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396313    2557 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 09 03:57:55.402236 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396318    2557 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 09 03:57:55.402236 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396322    2557 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 09 03:57:55.402236 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396324    2557 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 09 03:57:55.402236 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396327    2557 flags.go:64] FLAG: --logging-format="text"
Sep 09 03:57:55.402236 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396330    2557 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 09 03:57:55.402236 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396333    2557 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 09 03:57:55.402236 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396336    2557 flags.go:64] FLAG: --manifest-url=""
Sep 09 03:57:55.402236 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396338    2557 flags.go:64] FLAG: --manifest-url-header=""
Sep 09 03:57:55.402236 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396343    2557 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 09 03:57:55.402236 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396345    2557 flags.go:64] FLAG: --max-open-files="1000000"
Sep 09 03:57:55.402236 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396349    2557 flags.go:64] FLAG: --max-pods="110"
Sep 09 03:57:55.402236 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396352    2557 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 09 03:57:55.402236 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396355    2557 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 09 03:57:55.402236 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396357    2557 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 09 03:57:55.402236 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396361    2557 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 09 03:57:55.402236 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396364    2557 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 09 03:57:55.402236 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396367    2557 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 09 03:57:55.402236 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396370    2557 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 09 03:57:55.402236 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396377    2557 flags.go:64] FLAG: --node-status-max-images="50"
Sep 09 03:57:55.402826 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396380    2557 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 09 03:57:55.402826 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396383    2557 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 09 03:57:55.402826 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396386    2557 flags.go:64] FLAG: --pod-cidr=""
Sep 09 03:57:55.402826 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396389    2557 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 09 03:57:55.402826 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396395    2557 flags.go:64] FLAG: --pod-manifest-path=""
Sep 09 03:57:55.402826 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396398    2557 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 09 03:57:55.402826 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396401    2557 flags.go:64] FLAG: --pods-per-core="0"
Sep 09 03:57:55.402826 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396404    2557 flags.go:64] FLAG: --port="10250"
Sep 09 03:57:55.402826 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396407    2557 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 09 03:57:55.402826 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396410    2557 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-098d85baf92661093"
Sep 09 03:57:55.402826 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396413    2557 flags.go:64] FLAG: --qos-reserved=""
Sep 09 03:57:55.402826 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396416    2557 flags.go:64] FLAG: --read-only-port="10255"
Sep 09 03:57:55.402826 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396419    2557 flags.go:64] FLAG: --register-node="true"
Sep 09 03:57:55.402826 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396421    2557 flags.go:64] FLAG: --register-with-taints=""
Sep 09 03:57:55.402826 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396425    2557 flags.go:64] FLAG: --registry-burst="10"
Sep 09 03:57:55.402826 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396428    2557 flags.go:64] FLAG: --registry-qps="5"
Sep 09 03:57:55.402826 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396431    2557 flags.go:64] FLAG: --reserved-cpus=""
Sep 09 03:57:55.402826 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396433    2557 flags.go:64] FLAG: --reserved-memory=""
Sep 09 03:57:55.402826 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396437    2557 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 09 03:57:55.402826 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396440    2557 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 09 03:57:55.402826 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396443    2557 flags.go:64] FLAG: --rotate-certificates="false"
Sep 09 03:57:55.402826 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396445    2557 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 09 03:57:55.402826 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396448    2557 flags.go:64] FLAG: --runonce="false"
Sep 09 03:57:55.402826 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396450    2557 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 09 03:57:55.402826 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396453    2557 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 09 03:57:55.403452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396456    2557 flags.go:64] FLAG: --seccomp-default="false"
Sep 09 03:57:55.403452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396459    2557 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 09 03:57:55.403452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396462    2557 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 09 03:57:55.403452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396466    2557 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 09 03:57:55.403452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396469    2557 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 09 03:57:55.403452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396472    2557 flags.go:64] FLAG: --storage-driver-password="root"
Sep 09 03:57:55.403452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396474    2557 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 09 03:57:55.403452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396477    2557 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 09 03:57:55.403452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396480    2557 flags.go:64] FLAG: --storage-driver-user="root"
Sep 09 03:57:55.403452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396482    2557 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 09 03:57:55.403452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396485    2557 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 09 03:57:55.403452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396489    2557 flags.go:64] FLAG: --system-cgroups=""
Sep 09 03:57:55.403452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396492    2557 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 09 03:57:55.403452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396497    2557 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 09 03:57:55.403452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396499    2557 flags.go:64] FLAG: --tls-cert-file=""
Sep 09 03:57:55.403452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396502    2557 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 09 03:57:55.403452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396506    2557 flags.go:64] FLAG: --tls-min-version=""
Sep 09 03:57:55.403452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396508    2557 flags.go:64] FLAG: --tls-private-key-file=""
Sep 09 03:57:55.403452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396511    2557 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 09 03:57:55.403452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396515    2557 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 09 03:57:55.403452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396518    2557 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 09 03:57:55.403452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396521    2557 flags.go:64] FLAG: --v="2"
Sep 09 03:57:55.403452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396525    2557 flags.go:64] FLAG: --version="false"
Sep 09 03:57:55.403452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396530    2557 flags.go:64] FLAG: --vmodule=""
Sep 09 03:57:55.403452 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396538    2557 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 09 03:57:55.404102 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.396541    2557 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 09 03:57:55.404102 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396645    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 03:57:55.404102 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396648    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 03:57:55.404102 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396651    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 03:57:55.404102 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396654    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 03:57:55.404102 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396656    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 03:57:55.404102 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396659    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 03:57:55.404102 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396662    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 03:57:55.404102 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396664    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 03:57:55.404102 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396667    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 03:57:55.404102 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396670    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 03:57:55.404102 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396673    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 03:57:55.404102 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396676    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 03:57:55.404102 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396678    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 03:57:55.404102 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396680    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 03:57:55.404102 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396683    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 03:57:55.404102 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396685    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 03:57:55.404102 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396688    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 03:57:55.404102 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396692    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 03:57:55.404102 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396694    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 03:57:55.404611 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396697    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 03:57:55.404611 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396699    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 03:57:55.404611 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396702    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 03:57:55.404611 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396704    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 03:57:55.404611 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396706    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 03:57:55.404611 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396709    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 03:57:55.404611 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396712    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 03:57:55.404611 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396714    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 03:57:55.404611 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396718    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 03:57:55.404611 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396722    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 03:57:55.404611 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396724    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 03:57:55.404611 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396727    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 03:57:55.404611 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396730    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 03:57:55.404611 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396732    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 03:57:55.404611 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396735    2557 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 03:57:55.404611 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396737    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 03:57:55.404611 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396741    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 03:57:55.404611 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396745    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 03:57:55.404611 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396748    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 03:57:55.405070 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396751    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 03:57:55.405070 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396753    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 03:57:55.405070 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396756    2557 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 03:57:55.405070 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396760    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 03:57:55.405070 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396762    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 03:57:55.405070 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396765    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 03:57:55.405070 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396767    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 03:57:55.405070 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396770    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 03:57:55.405070 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396772    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 03:57:55.405070 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396775    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 03:57:55.405070 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396777    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 03:57:55.405070 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396781    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 03:57:55.405070 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396784    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 03:57:55.405070 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396786    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 03:57:55.405070 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396788    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 03:57:55.405070 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396791    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 03:57:55.405070 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396794    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 03:57:55.405070 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396796    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 03:57:55.405070 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396799    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 03:57:55.405070 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396801    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 03:57:55.405559 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396804    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 03:57:55.405559 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396806    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 03:57:55.405559 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396809    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 03:57:55.405559 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396811    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 03:57:55.405559 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396814    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 03:57:55.405559 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396820    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 03:57:55.405559 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396823    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 03:57:55.405559 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396827    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 03:57:55.405559 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396829    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 03:57:55.405559 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396832    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 03:57:55.405559 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396835    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 03:57:55.405559 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396838    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 03:57:55.405559 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396840    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 03:57:55.405559 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396843    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 03:57:55.405559 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396845    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 03:57:55.405559 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396849    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 03:57:55.405559 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396852    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 03:57:55.405559 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396854    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 03:57:55.405559 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396857    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 03:57:55.406088 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396859    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 03:57:55.406088 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396862    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 03:57:55.406088 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396864    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 03:57:55.406088 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396867    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 03:57:55.406088 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396870    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 03:57:55.406088 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396873    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 03:57:55.406088 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396876    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 03:57:55.406088 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396878    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 03:57:55.406088 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396881    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 03:57:55.406088 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396883    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 03:57:55.406088 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396886    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 03:57:55.406088 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396888    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 03:57:55.406088 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396891    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 03:57:55.406088 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396893    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 03:57:55.406088 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396896    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 03:57:55.406088 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396898    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 03:57:55.406088 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396901    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 03:57:55.406088 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396903    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 03:57:55.406088 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396906    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 03:57:55.406088 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396909    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 03:57:55.406605 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396911    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 03:57:55.406605 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.396914    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 03:57:55.406605 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.397847    2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 03:57:55.406605 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.405412    2557 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 09 03:57:55.406605 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.405428    2557 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 09 03:57:55.406605 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.406339    2557 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 09 03:57:55.406605 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.406346    2557 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 09 03:57:55.406605 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406409    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 03:57:55.406605 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406414    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 03:57:55.406605 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406418    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 03:57:55.406605 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406421    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 03:57:55.406605 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406424    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 03:57:55.406605 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406427    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 03:57:55.406605 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406430    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 03:57:55.406605 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406433    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 03:57:55.406979 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406435    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 03:57:55.406979 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406438    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 03:57:55.406979 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406441    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 03:57:55.406979 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406443    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 03:57:55.406979 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406446    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 03:57:55.406979 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406449    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 03:57:55.406979 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406453    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 03:57:55.406979 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406457    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 03:57:55.406979 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406459    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 03:57:55.406979 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406462    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 03:57:55.406979 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406465    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 03:57:55.406979 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406468    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 03:57:55.406979 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406471    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 03:57:55.406979 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406474    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 03:57:55.406979 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406476    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 03:57:55.406979 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406479    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 03:57:55.406979 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406482    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 03:57:55.406979 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406485    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 03:57:55.406979 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406487    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 03:57:55.407500 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406490    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 03:57:55.407500 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406493    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 03:57:55.407500 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406495    2557 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 03:57:55.407500 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406497    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 03:57:55.407500 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406500    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 03:57:55.407500 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406503    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 03:57:55.407500 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406505    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 03:57:55.407500 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406508    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 03:57:55.407500 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406510    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 03:57:55.407500 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406513    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 03:57:55.407500 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406515    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 03:57:55.407500 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406518    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 03:57:55.407500 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406521    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 03:57:55.407500 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406523    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 03:57:55.407500 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406526    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 03:57:55.407500 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406528    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 03:57:55.407500 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406531    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 03:57:55.407500 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406533    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 03:57:55.407500 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406536    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 03:57:55.407500 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406538    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 03:57:55.407997 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406541    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 03:57:55.407997 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406543    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 03:57:55.407997 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406545    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 03:57:55.407997 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406548    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 03:57:55.407997 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406551    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 03:57:55.407997 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406553    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 03:57:55.407997 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406557    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 03:57:55.407997 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406559    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 03:57:55.407997 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406562    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 03:57:55.407997 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406564    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 03:57:55.407997 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406567    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 03:57:55.407997 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406570    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 03:57:55.407997 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406572    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 03:57:55.407997 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406575    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 03:57:55.407997 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406577    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 03:57:55.407997 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406579    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 03:57:55.407997 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406582    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 03:57:55.407997 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406584    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 03:57:55.407997 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406587    2557 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 03:57:55.407997 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406589    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 03:57:55.408485 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406592    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 03:57:55.408485 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406594    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 03:57:55.408485 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406596    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 03:57:55.408485 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406599    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 03:57:55.408485 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406601    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 03:57:55.408485 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406604    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 03:57:55.408485 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406606    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 03:57:55.408485 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406609    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 03:57:55.408485 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406613    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 03:57:55.408485 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406617    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 03:57:55.408485 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406620    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 03:57:55.408485 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406623    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 03:57:55.408485 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406626    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 03:57:55.408485 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406628    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 03:57:55.408485 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406631    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 03:57:55.408485 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406633    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 03:57:55.408485 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406636    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 03:57:55.408485 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406638    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 03:57:55.408485 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406641    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 03:57:55.408485 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406644    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 03:57:55.408957 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406646    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 03:57:55.408957 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406648    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 03:57:55.408957 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406652    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 03:57:55.408957 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406656    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 03:57:55.408957 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406658    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 03:57:55.408957 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406661    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 03:57:55.408957 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406664    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 03:57:55.408957 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406667    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 03:57:55.408957 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406670    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 03:57:55.408957 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406673    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 03:57:55.408957 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406675    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 03:57:55.408957 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406678    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 03:57:55.408957 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.406683    2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 03:57:55.408957 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406787    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 03:57:55.409307 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406792    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 03:57:55.409307 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406800    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 03:57:55.409307 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406803    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 03:57:55.409307 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406806    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 03:57:55.409307 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406809    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 03:57:55.409307 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406812    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 03:57:55.409307 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406815    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 03:57:55.409307 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406818    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 03:57:55.409307 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406820    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 03:57:55.409307 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406824    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 03:57:55.409307 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406826    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 03:57:55.409307 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406829    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 03:57:55.409307 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406831    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 03:57:55.409307 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406834    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 03:57:55.409307 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406836    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 03:57:55.409307 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406839    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 03:57:55.409307 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406841    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 03:57:55.409307 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406844    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 03:57:55.409307 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406847    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 03:57:55.409815 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406850    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 03:57:55.409815 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406852    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 03:57:55.409815 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406854    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 03:57:55.409815 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406857    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 03:57:55.409815 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406860    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 03:57:55.409815 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406862    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 03:57:55.409815 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406865    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 03:57:55.409815 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406867    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 03:57:55.409815 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406870    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 03:57:55.409815 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406872    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 03:57:55.409815 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406874    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 03:57:55.409815 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406877    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 03:57:55.409815 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406880    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 03:57:55.409815 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406883    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 03:57:55.409815 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406885    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 03:57:55.409815 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406887    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 03:57:55.409815 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406890    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 03:57:55.409815 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406892    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 03:57:55.409815 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406895    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 03:57:55.409815 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406897    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 03:57:55.410305 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406900    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 03:57:55.410305 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406902    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 03:57:55.410305 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406905    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 03:57:55.410305 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406907    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 03:57:55.410305 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406910    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 03:57:55.410305 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406912    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 03:57:55.410305 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406914    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 03:57:55.410305 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406917    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 03:57:55.410305 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406919    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 03:57:55.410305 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406936    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 03:57:55.410305 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406940    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 03:57:55.410305 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406943    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 03:57:55.410305 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406946    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 03:57:55.410305 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406948    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 03:57:55.410305 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406951    2557 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 03:57:55.410305 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406953    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 03:57:55.410305 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406960    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 03:57:55.410305 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406963    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 03:57:55.410305 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406965    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 03:57:55.410305 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406968    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 03:57:55.410810 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406971    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 03:57:55.410810 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406973    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 03:57:55.410810 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406977    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 03:57:55.410810 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406979    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 03:57:55.410810 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406982    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 03:57:55.410810 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406984    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 03:57:55.410810 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406987    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 03:57:55.410810 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406990    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 03:57:55.410810 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406993    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 03:57:55.410810 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406995    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 03:57:55.410810 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.406998    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 03:57:55.410810 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.407000    2557 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 03:57:55.410810 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.407003    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 03:57:55.410810 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.407005    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 03:57:55.410810 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.407008    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 03:57:55.410810 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.407010    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 03:57:55.410810 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.407013    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 03:57:55.410810 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.407015    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 03:57:55.410810 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.407018    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 03:57:55.411273 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.407020    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 03:57:55.411273 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.407023    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 03:57:55.411273 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.407025    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 03:57:55.411273 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.407028    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 03:57:55.411273 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.407030    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 03:57:55.411273 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.407033    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 03:57:55.411273 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.407035    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 03:57:55.411273 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.407038    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 03:57:55.411273 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.407040    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 03:57:55.411273 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.407043    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 03:57:55.411273 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.407046    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 03:57:55.411273 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.407048    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 03:57:55.411273 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.407051    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 03:57:55.411273 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.407053    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 03:57:55.411273 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.407055    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 03:57:55.411273 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.407058    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 03:57:55.411273 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.407061    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 03:57:55.411273 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.407063    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 03:57:55.411273 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.407065    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 03:57:55.411741 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:55.407068    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 03:57:55.411741 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.407072    2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 03:57:55.411741 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.407182    2557 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 09 03:57:55.411741 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.409464    2557 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 09 03:57:55.411741 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.410596    2557 server.go:1019] "Starting client certificate rotation"
Sep 09 03:57:55.411741 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.410697    2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 09 03:57:55.411741 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.410727    2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 09 03:57:55.413548 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.413534    2557 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 09 03:57:55.414692 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.414675    2557 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 09 03:57:55.426505 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.426486    2557 log.go:25] "Validated CRI v1 runtime API"
Sep 09 03:57:55.430422 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.430404    2557 log.go:25] "Validated CRI v1 image API"
Sep 09 03:57:55.432146 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.432131    2557 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 09 03:57:55.439345 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.439325    2557 fs.go:135] Filesystem UUIDs: map[254dd8b3-f5bb-43d6-830a-0972398834c7:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 f6e2b121-3d1d-492f-8a77-5cdddc50b6e7:/dev/nvme0n1p4]
Sep 09 03:57:55.439423 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.439343    2557 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 09 03:57:55.444871 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.444853    2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 09 03:57:55.445442 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.445335    2557 manager.go:216] Machine: {Timestamp:2026-09-09 03:57:55.443945505 +0000 UTC m=+0.363116404 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3197152 MemoryCapacity:33164468224 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ff4ef780f0a6b77e700b3cd8f787d SystemUUID:ec2ff4ef-780f-0a6b-77e7-00b3cd8f787d BootID:bb186297-1f76-4b97-9b9c-7237fc297405 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582234112 Type:vfs Inodes:4048397 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632894464 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582234112 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:73:f9:e7:64:87 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:73:f9:e7:64:87 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5a:b1:b9:ec:2e:a4 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164468224 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 09 03:57:55.445442 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.445441    2557 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 09 03:57:55.445560 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.445549    2557 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 09 03:57:55.449000 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.448980    2557 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 09 03:57:55.449168 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.449003    2557 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-71.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 09 03:57:55.449856 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.449847    2557 topology_manager.go:138] "Creating topology manager with none policy"
Sep 09 03:57:55.449885 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.449858    2557 container_manager_linux.go:309] "Creating device plugin manager"
Sep 09 03:57:55.449885 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.449866    2557 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 09 03:57:55.449885 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.449879    2557 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 09 03:57:55.451161 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.451151    2557 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 09 03:57:55.455892 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.455882    2557 state_mem.go:36] "Initialized new in-memory state store"
Sep 09 03:57:55.456004 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.455995    2557 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 09 03:57:55.456055 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.456022    2557 kubelet.go:486] "Attempting to sync node with API server"
Sep 09 03:57:55.456055 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.456038    2557 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 09 03:57:55.456055 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.456049    2557 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 09 03:57:55.456150 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.456058    2557 kubelet.go:398] "Adding apiserver pod source"
Sep 09 03:57:55.456150 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.456066    2557 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 09 03:57:55.456392 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.456378    2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 09 03:57:55.456454 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.456399    2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 09 03:57:55.458717 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.458700    2557 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 09 03:57:55.461523 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.461510    2557 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 09 03:57:55.461562 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.461540    2557 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 09 03:57:55.462967 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.462956    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 09 03:57:55.463002 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.462972    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 09 03:57:55.463002 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.462978    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 09 03:57:55.463002 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.462983    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 09 03:57:55.463002 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.462992    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 09 03:57:55.463002 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.462997    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 09 03:57:55.463155 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.463010    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 09 03:57:55.463155 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.463016    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 09 03:57:55.463155 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.463022    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 09 03:57:55.463155 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.463027    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 09 03:57:55.463155 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.463040    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 09 03:57:55.463155 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.463045    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 09 03:57:55.463897 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.463885    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 09 03:57:55.463928 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.463900    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 09 03:57:55.467623 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.467595    2557 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-71.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 09 03:57:55.467756 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:55.467724    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 09 03:57:55.467815 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:55.467798    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-71.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 09 03:57:55.467989 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.467977    2557 server.go:1268] "Started kubelet"
Sep 09 03:57:55.472532 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.472391    2557 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 09 03:57:55.472532 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.472476    2557 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 09 03:57:55.472844 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.472818    2557 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 09 03:57:55.472976 ip-10-0-141-71 systemd[1]: Started Kubernetes Kubelet.
Sep 09 03:57:55.474405 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.474381    2557 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 09 03:57:55.476196 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.476161    2557 server.go:310] "Adding debug handlers to kubelet server"
Sep 09 03:57:55.479063 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:55.479042    2557 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 09 03:57:55.479849 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:55.478761    2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-71.ec2.internal.18d38af82aa13e17  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-71.ec2.internal,UID:ip-10-0-141-71.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-71.ec2.internal,},FirstTimestamp:2026-09-09 03:57:55.467951639 +0000 UTC m=+0.387122538,LastTimestamp:2026-09-09 03:57:55.467951639 +0000 UTC m=+0.387122538,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-71.ec2.internal,}"
Sep 09 03:57:55.484094 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.484079    2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 09 03:57:55.484778 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.484765    2557 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 09 03:57:55.484874 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:55.484851    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-71.ec2.internal\" not found"
Sep 09 03:57:55.485667 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.485505    2557 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 09 03:57:55.485667 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.485519    2557 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 09 03:57:55.485667 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.485595    2557 reconstruct.go:97] "Volume reconstruction finished"
Sep 09 03:57:55.485667 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.485604    2557 reconciler.go:29] "Reconciler: start to sync state"
Sep 09 03:57:55.485844 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.485670    2557 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 09 03:57:55.485844 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.485838    2557 factory.go:153] Registering CRI-O factory
Sep 09 03:57:55.485936 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.485853    2557 factory.go:223] Registration of the crio container factory successfully
Sep 09 03:57:55.485936 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.485897    2557 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 09 03:57:55.485936 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.485905    2557 factory.go:55] Registering systemd factory
Sep 09 03:57:55.485936 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.485913    2557 factory.go:223] Registration of the systemd container factory successfully
Sep 09 03:57:55.485936 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.485926    2557 factory.go:103] Registering Raw factory
Sep 09 03:57:55.485936 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.485936    2557 manager.go:1196] Started watching for new ooms in manager
Sep 09 03:57:55.486273 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.486261    2557 manager.go:349] Starting recovery of all containers
Sep 09 03:57:55.487061 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:55.486736    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 09 03:57:55.487061 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:55.486840    2557 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-71.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 09 03:57:55.493131 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.493089    2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xsplg"
Sep 09 03:57:55.495375 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.495360    2557 manager.go:354] Recovery completed
Sep 09 03:57:55.500008 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.499996    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 03:57:55.501654 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.501637    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-71.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 03:57:55.501719 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.501668    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-71.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 03:57:55.501719 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.501680    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-71.ec2.internal" event="NodeHasSufficientPID"
Sep 09 03:57:55.501719 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.501682    2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xsplg"
Sep 09 03:57:55.502269 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.502251    2557 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 09 03:57:55.502269 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.502263    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 03:57:55.502269 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.502267    2557 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 09 03:57:55.502416 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.502284    2557 state_mem.go:36] "Initialized new in-memory state store"
Sep 09 03:57:55.502940 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.502915    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-71.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 03:57:55.502986 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.502946    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-71.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 03:57:55.502986 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.502960    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-71.ec2.internal" event="NodeHasSufficientPID"
Sep 09 03:57:55.512369 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:55.512301    2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-71.ec2.internal.18d38af82ca37ec6  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-71.ec2.internal,UID:ip-10-0-141-71.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-71.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-71.ec2.internal,},FirstTimestamp:2026-09-09 03:57:55.501653702 +0000 UTC m=+0.420824602,LastTimestamp:2026-09-09 03:57:55.501653702 +0000 UTC m=+0.420824602,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-71.ec2.internal,}"
Sep 09 03:57:55.515889 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:55.515834    2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-71.ec2.internal.18d38af82ca3cdbd  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-71.ec2.internal,UID:ip-10-0-141-71.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-141-71.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-141-71.ec2.internal,},FirstTimestamp:2026-09-09 03:57:55.501673917 +0000 UTC m=+0.420844816,LastTimestamp:2026-09-09 03:57:55.501673917 +0000 UTC m=+0.420844816,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-71.ec2.internal,}"
Sep 09 03:57:55.523326 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:55.523259    2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-71.ec2.internal.18d38af82ca3f328  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-71.ec2.internal,UID:ip-10-0-141-71.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-141-71.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-141-71.ec2.internal,},FirstTimestamp:2026-09-09 03:57:55.501683496 +0000 UTC m=+0.420854396,LastTimestamp:2026-09-09 03:57:55.501683496 +0000 UTC m=+0.420854396,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-71.ec2.internal,}"
Sep 09 03:57:55.526226 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:55.526167    2557 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-71.ec2.internal.18d38af82ca37ec6\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-71.ec2.internal.18d38af82ca37ec6  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-71.ec2.internal,UID:ip-10-0-141-71.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-71.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-71.ec2.internal,},FirstTimestamp:2026-09-09 03:57:55.501653702 +0000 UTC m=+0.420824602,LastTimestamp:2026-09-09 03:57:55.502930824 +0000 UTC m=+0.422101725,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-71.ec2.internal,}"
Sep 09 03:57:55.533534 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:55.533478    2557 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-71.ec2.internal.18d38af82ca3cdbd\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-71.ec2.internal.18d38af82ca3cdbd  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-71.ec2.internal,UID:ip-10-0-141-71.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-141-71.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-141-71.ec2.internal,},FirstTimestamp:2026-09-09 03:57:55.501673917 +0000 UTC m=+0.420844816,LastTimestamp:2026-09-09 03:57:55.502952049 +0000 UTC m=+0.422122954,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-71.ec2.internal,}"
Sep 09 03:57:55.541968 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:55.541913    2557 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-71.ec2.internal.18d38af82ca3f328\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-71.ec2.internal.18d38af82ca3f328  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-71.ec2.internal,UID:ip-10-0-141-71.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-141-71.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-141-71.ec2.internal,},FirstTimestamp:2026-09-09 03:57:55.501683496 +0000 UTC m=+0.420854396,LastTimestamp:2026-09-09 03:57:55.502964669 +0000 UTC m=+0.422135570,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-71.ec2.internal,}"
Sep 09 03:57:55.644969 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:55.584951    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-71.ec2.internal\" not found"
Sep 09 03:57:55.644969 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.611597    2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 09 03:57:55.644969 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.612655    2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 09 03:57:55.644969 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.612714    2557 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 09 03:57:55.644969 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.612740    2557 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 09 03:57:55.644969 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:55.612781    2557 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Sep 09 03:57:55.644969 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:55.622002    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Sep 09 03:57:55.645586 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.645572    2557 policy_none.go:49] "None policy: Start"
Sep 09 03:57:55.645645 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.645593    2557 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 09 03:57:55.645645 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.645607    2557 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 09 03:57:55.647593 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.647581    2557 policy_none.go:47] "Start"
Sep 09 03:57:55.679658 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.679644    2557 manager.go:337] "Starting Device Plugin manager"
Sep 09 03:57:55.679746 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:55.679697    2557 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 09 03:57:55.679746 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.679710    2557 server.go:87] "Starting device plugin registration server"
Sep 09 03:57:55.679994 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.679976    2557 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 09 03:57:55.680084 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.679990    2557 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 09 03:57:55.680084 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.680076    2557 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 09 03:57:55.680212 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.680173    2557 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 09 03:57:55.680212 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.680182    2557 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 09 03:57:55.680660 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:55.680641    2557 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 09 03:57:55.680720 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:55.680699    2557 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-71.ec2.internal\" not found"
Sep 09 03:57:55.695078 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:55.695056    2557 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-141-71.ec2.internal\" not found" node="ip-10-0-141-71.ec2.internal"
Sep 09 03:57:55.712851 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.712827    2557 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-71.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-71.ec2.internal"]
Sep 09 03:57:55.712917 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.712908    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 03:57:55.714233 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.714217    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-71.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 03:57:55.714310 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.714246    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-71.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 03:57:55.714310 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.714256    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-71.ec2.internal" event="NodeHasSufficientPID"
Sep 09 03:57:55.716285 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.716275    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 03:57:55.716387 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.716375    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-71.ec2.internal"
Sep 09 03:57:55.716430 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.716400    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 03:57:55.717739 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.717219    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-71.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 03:57:55.717739 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.717245    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-71.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 03:57:55.717739 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.717258    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-71.ec2.internal" event="NodeHasSufficientPID"
Sep 09 03:57:55.717739 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.717636    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-71.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 03:57:55.717739 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.717655    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-71.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 03:57:55.717739 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.717685    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-71.ec2.internal" event="NodeHasSufficientPID"
Sep 09 03:57:55.719760 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.719746    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-71.ec2.internal"
Sep 09 03:57:55.719827 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.719770    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 03:57:55.720443 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.720428    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-71.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 03:57:55.720509 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.720457    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-71.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 03:57:55.720509 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.720469    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-71.ec2.internal" event="NodeHasSufficientPID"
Sep 09 03:57:55.749735 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:55.749715    2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-71.ec2.internal\" not found" node="ip-10-0-141-71.ec2.internal"
Sep 09 03:57:55.754004 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:55.753989    2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-71.ec2.internal\" not found" node="ip-10-0-141-71.ec2.internal"
Sep 09 03:57:55.780599 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.780582    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 03:57:55.781378 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.781364    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-71.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 03:57:55.781438 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.781387    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-71.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 03:57:55.781438 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.781397    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-71.ec2.internal" event="NodeHasSufficientPID"
Sep 09 03:57:55.781438 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.781414    2557 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-71.ec2.internal"
Sep 09 03:57:55.791568 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.791550    2557 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-71.ec2.internal"
Sep 09 03:57:55.791568 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:55.791569    2557 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-71.ec2.internal\": node \"ip-10-0-141-71.ec2.internal\" not found"
Sep 09 03:57:55.854318 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:55.854293    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-71.ec2.internal\" not found"
Sep 09 03:57:55.887694 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.887671    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dcbf869fa4c13085efd0490573ad1b8f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-71.ec2.internal\" (UID: \"dcbf869fa4c13085efd0490573ad1b8f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-71.ec2.internal"
Sep 09 03:57:55.887779 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.887695    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2c06b6aa4345f7858e0b9cc81a5b603f-config\") pod \"kube-apiserver-proxy-ip-10-0-141-71.ec2.internal\" (UID: \"2c06b6aa4345f7858e0b9cc81a5b603f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-71.ec2.internal"
Sep 09 03:57:55.887779 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.887711    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dcbf869fa4c13085efd0490573ad1b8f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-71.ec2.internal\" (UID: \"dcbf869fa4c13085efd0490573ad1b8f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-71.ec2.internal"
Sep 09 03:57:55.955134 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:55.955049    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-71.ec2.internal\" not found"
Sep 09 03:57:55.988606 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.988571    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dcbf869fa4c13085efd0490573ad1b8f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-71.ec2.internal\" (UID: \"dcbf869fa4c13085efd0490573ad1b8f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-71.ec2.internal"
Sep 09 03:57:55.988606 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.988609    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dcbf869fa4c13085efd0490573ad1b8f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-71.ec2.internal\" (UID: \"dcbf869fa4c13085efd0490573ad1b8f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-71.ec2.internal"
Sep 09 03:57:55.988697 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.988626    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2c06b6aa4345f7858e0b9cc81a5b603f-config\") pod \"kube-apiserver-proxy-ip-10-0-141-71.ec2.internal\" (UID: \"2c06b6aa4345f7858e0b9cc81a5b603f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-71.ec2.internal"
Sep 09 03:57:55.988697 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.988671    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dcbf869fa4c13085efd0490573ad1b8f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-71.ec2.internal\" (UID: \"dcbf869fa4c13085efd0490573ad1b8f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-71.ec2.internal"
Sep 09 03:57:55.988767 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.988673    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2c06b6aa4345f7858e0b9cc81a5b603f-config\") pod \"kube-apiserver-proxy-ip-10-0-141-71.ec2.internal\" (UID: \"2c06b6aa4345f7858e0b9cc81a5b603f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-71.ec2.internal"
Sep 09 03:57:55.988767 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:55.988708    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dcbf869fa4c13085efd0490573ad1b8f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-71.ec2.internal\" (UID: \"dcbf869fa4c13085efd0490573ad1b8f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-71.ec2.internal"
Sep 09 03:57:56.051756 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:56.051717    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-71.ec2.internal"
Sep 09 03:57:56.055125 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:56.055089    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-71.ec2.internal\" not found"
Sep 09 03:57:56.057427 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:56.057412    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-71.ec2.internal"
Sep 09 03:57:56.155881 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:56.155841    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-71.ec2.internal\" not found"
Sep 09 03:57:56.261233 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:56.261164    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-71.ec2.internal\" not found"
Sep 09 03:57:56.362311 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:56.362279    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-71.ec2.internal\" not found"
Sep 09 03:57:56.403845 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:56.403820    2557 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 03:57:56.410637 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:56.410617    2557 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 09 03:57:56.410744 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:56.410724    2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 09 03:57:56.463057 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:56.463029    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-71.ec2.internal\" not found"
Sep 09 03:57:56.487126 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:56.487092    2557 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 03:57:56.531468 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:56.531388    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 03:57:56.532396 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:56.532358    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-71.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 03:57:56.532396 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:56.532398    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-71.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 03:57:56.532537 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:56.532415    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-71.ec2.internal" event="NodeHasSufficientPID"
Sep 09 03:57:56.563654 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:56.563493    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-71.ec2.internal\" not found"
Sep 09 03:57:56.647608 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:56.647550    2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-08 03:52:56 +0000 UTC" deadline="2028-03-26 20:07:13.712424031 +0000 UTC"
Sep 09 03:57:56.647608 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:56.647603    2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13552h9m17.064825003s"
Sep 09 03:57:56.663321 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:56.663296    2557 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 03:57:56.685698 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:56.685677    2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-71.ec2.internal"
Sep 09 03:57:56.699733 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:56.699714    2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 09 03:57:56.700669 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:56.700654    2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-71.ec2.internal"
Sep 09 03:57:56.707833 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:56.707807    2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 09 03:57:56.745903 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:56.745872    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcbf869fa4c13085efd0490573ad1b8f.slice/crio-cb74ae454d8ce77368f3735d75281d048afed7d196e4a58f3482b203751bc620 WatchSource:0}: Error finding container cb74ae454d8ce77368f3735d75281d048afed7d196e4a58f3482b203751bc620: Status 404 returned error can't find the container with id cb74ae454d8ce77368f3735d75281d048afed7d196e4a58f3482b203751bc620
Sep 09 03:57:56.747020 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:56.746998    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c06b6aa4345f7858e0b9cc81a5b603f.slice/crio-c95de41a268d6e0922f68c57fd42649fa4e1f5589441b3e0851a7150283717b9 WatchSource:0}: Error finding container c95de41a268d6e0922f68c57fd42649fa4e1f5589441b3e0851a7150283717b9: Status 404 returned error can't find the container with id c95de41a268d6e0922f68c57fd42649fa4e1f5589441b3e0851a7150283717b9
Sep 09 03:57:56.751132 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:56.751114    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 03:57:56.825958 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:56.825904    2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 03:57:57.460464 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.460436    2557 apiserver.go:52] "Watching apiserver"
Sep 09 03:57:57.469005 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.468975    2557 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 09 03:57:57.470815 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.470790    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-rfvrc","kube-system/global-pull-secret-syncer-78hdg","openshift-cluster-node-tuning-operator/tuned-fhllb","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-71.ec2.internal","openshift-multus/multus-additional-cni-plugins-g6ggd","openshift-multus/multus-gfd87","openshift-multus/network-metrics-daemon-9lrvt","kube-system/konnectivity-agent-scn7w","kube-system/kube-apiserver-proxy-ip-10-0-141-71.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l5vh","openshift-dns/node-resolver-hn6zn","openshift-image-registry/node-ca-kvnzq","openshift-network-diagnostics/network-check-target-bvnr6","openshift-network-operator/iptables-alerter-85krs"]
Sep 09 03:57:57.473467 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.473445    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l5vh"
Sep 09 03:57:57.475758 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.475735    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvnr6"
Sep 09 03:57:57.475855 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:57.475816    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvnr6" podUID="0b01bf6d-392c-4015-b166-11deff0ffbaa"
Sep 09 03:57:57.476525 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.476504    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 09 03:57:57.476732 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.476714    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 09 03:57:57.476812 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.476770    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 09 03:57:57.477052 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.477032    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-26d7m\""
Sep 09 03:57:57.478169 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.478148    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.480647 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.480378    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 09 03:57:57.480647 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.480417    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-85krs"
Sep 09 03:57:57.480647 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.480526    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 09 03:57:57.480927 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.480909    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-h7965\""
Sep 09 03:57:57.482756 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.482735    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 09 03:57:57.483042 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.483022    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 09 03:57:57.483123 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.483043    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-g6ggd"
Sep 09 03:57:57.483590 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.483570    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7zlmw\""
Sep 09 03:57:57.483669 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.483623    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 09 03:57:57.484629 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.484610    2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 09 03:57:57.485430 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.485414    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.485814 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.485795    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 09 03:57:57.486075 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.486054    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 09 03:57:57.486175 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.486089    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 09 03:57:57.486248 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.486213    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-x8zjl\""
Sep 09 03:57:57.486324 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.486298    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 09 03:57:57.487569 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.487527    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 09 03:57:57.488098 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.488056    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 09 03:57:57.490074 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.488589    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-scn7w"
Sep 09 03:57:57.490317 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.490156    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-dkvrf\""
Sep 09 03:57:57.491728 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.491637    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 09 03:57:57.492508 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.492104    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-vtkl7\""
Sep 09 03:57:57.492508 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.492175    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 09 03:57:57.492693 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.492671    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9lrvt"
Sep 09 03:57:57.492798 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:57.492767    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9lrvt" podUID="58280195-fa67-47e6-af03-e39d2bab379b"
Sep 09 03:57:57.495274 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.495256    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78hdg"
Sep 09 03:57:57.495369 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:57.495318    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78hdg" podUID="85fe16e4-d362-49b1-b379-451d29551d1b"
Sep 09 03:57:57.495939 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.495914    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4dd10eaf-1bc1-449e-bb29-3ef1cc334409-socket-dir\") pod \"aws-ebs-csi-driver-node-2l5vh\" (UID: \"4dd10eaf-1bc1-449e-bb29-3ef1cc334409\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l5vh"
Sep 09 03:57:57.496031 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.495951    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4dd10eaf-1bc1-449e-bb29-3ef1cc334409-device-dir\") pod \"aws-ebs-csi-driver-node-2l5vh\" (UID: \"4dd10eaf-1bc1-449e-bb29-3ef1cc334409\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l5vh"
Sep 09 03:57:57.496031 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.495978    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcqdm\" (UniqueName: \"kubernetes.io/projected/4dd10eaf-1bc1-449e-bb29-3ef1cc334409-kube-api-access-qcqdm\") pod \"aws-ebs-csi-driver-node-2l5vh\" (UID: \"4dd10eaf-1bc1-449e-bb29-3ef1cc334409\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l5vh"
Sep 09 03:57:57.496031 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.496003    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-host-var-lib-kubelet\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.496031 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.496028    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-multus-conf-dir\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.496258 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.496051    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-host-run-multus-certs\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.496258 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.496075    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfn7h\" (UniqueName: \"kubernetes.io/projected/f26faf1a-61a1-4ff3-a8fb-5321f27480bc-kube-api-access-qfn7h\") pod \"multus-additional-cni-plugins-g6ggd\" (UID: \"f26faf1a-61a1-4ff3-a8fb-5321f27480bc\") " pod="openshift-multus/multus-additional-cni-plugins-g6ggd"
Sep 09 03:57:57.496258 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.496102    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/30cb71f3-8f94-4b63-a43c-07f84af039e9-iptables-alerter-script\") pod \"iptables-alerter-85krs\" (UID: \"30cb71f3-8f94-4b63-a43c-07f84af039e9\") " pod="openshift-network-operator/iptables-alerter-85krs"
Sep 09 03:57:57.496258 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.496128    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96s47\" (UniqueName: \"kubernetes.io/projected/30cb71f3-8f94-4b63-a43c-07f84af039e9-kube-api-access-96s47\") pod \"iptables-alerter-85krs\" (UID: \"30cb71f3-8f94-4b63-a43c-07f84af039e9\") " pod="openshift-network-operator/iptables-alerter-85krs"
Sep 09 03:57:57.496258 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.496152    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-cnibin\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.496258 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.496176    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-host-var-lib-cni-multus\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.496517 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.496387    2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 09 03:57:57.496635 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.496616    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-etc-kubernetes\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.496699 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.496649    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjszz\" (UniqueName: \"kubernetes.io/projected/1c7a8c99-53ac-4578-aad2-25908242ff17-kube-api-access-sjszz\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.496699 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.496674    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5452b40e-6ceb-4f8d-93d5-3dc83a424314-agent-certs\") pod \"konnectivity-agent-scn7w\" (UID: \"5452b40e-6ceb-4f8d-93d5-3dc83a424314\") " pod="kube-system/konnectivity-agent-scn7w"
Sep 09 03:57:57.496808 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.496697    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4dd10eaf-1bc1-449e-bb29-3ef1cc334409-sys-fs\") pod \"aws-ebs-csi-driver-node-2l5vh\" (UID: \"4dd10eaf-1bc1-449e-bb29-3ef1cc334409\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l5vh"
Sep 09 03:57:57.496808 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.496720    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/30cb71f3-8f94-4b63-a43c-07f84af039e9-host-slash\") pod \"iptables-alerter-85krs\" (UID: \"30cb71f3-8f94-4b63-a43c-07f84af039e9\") " pod="openshift-network-operator/iptables-alerter-85krs"
Sep 09 03:57:57.496808 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.496743    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/148ac06c-2830-439f-8046-0024406a5951-etc-sysconfig\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.496808 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.496765    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/148ac06c-2830-439f-8046-0024406a5951-etc-sysctl-conf\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.496808 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.496787    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/148ac06c-2830-439f-8046-0024406a5951-run\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.497050 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.496808    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/148ac06c-2830-439f-8046-0024406a5951-sys\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.497050 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.496831    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/148ac06c-2830-439f-8046-0024406a5951-host\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.497050 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.496854    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/148ac06c-2830-439f-8046-0024406a5951-etc-tuned\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.497050 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.496877    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4dd10eaf-1bc1-449e-bb29-3ef1cc334409-registration-dir\") pod \"aws-ebs-csi-driver-node-2l5vh\" (UID: \"4dd10eaf-1bc1-449e-bb29-3ef1cc334409\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l5vh"
Sep 09 03:57:57.497050 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.496908    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1c7a8c99-53ac-4578-aad2-25908242ff17-multus-daemon-config\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.497050 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.496932    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5452b40e-6ceb-4f8d-93d5-3dc83a424314-konnectivity-ca\") pod \"konnectivity-agent-scn7w\" (UID: \"5452b40e-6ceb-4f8d-93d5-3dc83a424314\") " pod="kube-system/konnectivity-agent-scn7w"
Sep 09 03:57:57.497050 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.496955    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/148ac06c-2830-439f-8046-0024406a5951-var-lib-kubelet\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.497050 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.496979    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6nh6\" (UniqueName: \"kubernetes.io/projected/148ac06c-2830-439f-8046-0024406a5951-kube-api-access-h6nh6\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.497050 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.497003    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f26faf1a-61a1-4ff3-a8fb-5321f27480bc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-g6ggd\" (UID: \"f26faf1a-61a1-4ff3-a8fb-5321f27480bc\") " pod="openshift-multus/multus-additional-cni-plugins-g6ggd"
Sep 09 03:57:57.497050 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.497026    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4dd10eaf-1bc1-449e-bb29-3ef1cc334409-etc-selinux\") pod \"aws-ebs-csi-driver-node-2l5vh\" (UID: \"4dd10eaf-1bc1-449e-bb29-3ef1cc334409\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l5vh"
Sep 09 03:57:57.497050 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.497051    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f26faf1a-61a1-4ff3-a8fb-5321f27480bc-system-cni-dir\") pod \"multus-additional-cni-plugins-g6ggd\" (UID: \"f26faf1a-61a1-4ff3-a8fb-5321f27480bc\") " pod="openshift-multus/multus-additional-cni-plugins-g6ggd"
Sep 09 03:57:57.497596 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.497077    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f26faf1a-61a1-4ff3-a8fb-5321f27480bc-cni-binary-copy\") pod \"multus-additional-cni-plugins-g6ggd\" (UID: \"f26faf1a-61a1-4ff3-a8fb-5321f27480bc\") " pod="openshift-multus/multus-additional-cni-plugins-g6ggd"
Sep 09 03:57:57.497596 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.497104    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-multus-cni-dir\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.497596 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.497128    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1c7a8c99-53ac-4578-aad2-25908242ff17-cni-binary-copy\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.497596 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.497152    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-multus-socket-dir-parent\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.497596 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.497228    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84bvb\" (UniqueName: \"kubernetes.io/projected/0b01bf6d-392c-4015-b166-11deff0ffbaa-kube-api-access-84bvb\") pod \"network-check-target-bvnr6\" (UID: \"0b01bf6d-392c-4015-b166-11deff0ffbaa\") " pod="openshift-network-diagnostics/network-check-target-bvnr6"
Sep 09 03:57:57.497596 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.497273    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/148ac06c-2830-439f-8046-0024406a5951-etc-modprobe-d\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.497596 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.497344    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/148ac06c-2830-439f-8046-0024406a5951-etc-kubernetes\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.497596 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.497378    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f26faf1a-61a1-4ff3-a8fb-5321f27480bc-cnibin\") pod \"multus-additional-cni-plugins-g6ggd\" (UID: \"f26faf1a-61a1-4ff3-a8fb-5321f27480bc\") " pod="openshift-multus/multus-additional-cni-plugins-g6ggd"
Sep 09 03:57:57.497596 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.497402    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-system-cni-dir\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.497596 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.497443    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-host-run-k8s-cni-cncf-io\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.497596 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.497479    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-host-var-lib-cni-bin\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.497596 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.497524    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f26faf1a-61a1-4ff3-a8fb-5321f27480bc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-g6ggd\" (UID: \"f26faf1a-61a1-4ff3-a8fb-5321f27480bc\") " pod="openshift-multus/multus-additional-cni-plugins-g6ggd"
Sep 09 03:57:57.497596 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.497556    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-host-run-netns\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.497596 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.497585    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-hostroot\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.498286 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.497606    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/148ac06c-2830-439f-8046-0024406a5951-etc-sysctl-d\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.498286 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.497628    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/148ac06c-2830-439f-8046-0024406a5951-etc-systemd\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.498286 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.497656    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/148ac06c-2830-439f-8046-0024406a5951-lib-modules\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.498286 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.497679    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/148ac06c-2830-439f-8046-0024406a5951-tmp\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.498286 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.497703    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f26faf1a-61a1-4ff3-a8fb-5321f27480bc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-g6ggd\" (UID: \"f26faf1a-61a1-4ff3-a8fb-5321f27480bc\") " pod="openshift-multus/multus-additional-cni-plugins-g6ggd"
Sep 09 03:57:57.498286 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.497729    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4dd10eaf-1bc1-449e-bb29-3ef1cc334409-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2l5vh\" (UID: \"4dd10eaf-1bc1-449e-bb29-3ef1cc334409\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l5vh"
Sep 09 03:57:57.498742 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.498441    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.500865 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.500846    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 09 03:57:57.501757 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.501283    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hn6zn"
Sep 09 03:57:57.504221 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.503956    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kvnzq"
Sep 09 03:57:57.504221 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.504041    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 09 03:57:57.504221 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.504065    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 09 03:57:57.504772 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.504560    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6tc4d\""
Sep 09 03:57:57.504772 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.504592    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 09 03:57:57.504772 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.504599    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 09 03:57:57.504772 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.504698    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 09 03:57:57.504991 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.504855    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 09 03:57:57.505173 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.505154    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-jjf5g\""
Sep 09 03:57:57.505279 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.505162    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 09 03:57:57.506305 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.506284    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 09 03:57:57.507516 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.507491    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 09 03:57:57.507745 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.507725    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 09 03:57:57.507823 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.507808    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-r9xnc\""
Sep 09 03:57:57.528941 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.528908    2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-6htps"
Sep 09 03:57:57.537589 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.537571    2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6htps"
Sep 09 03:57:57.587109 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.587086    2557 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 09 03:57:57.597891 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.597867    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-host-run-netns\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.598053 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.597901    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-hostroot\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.598053 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.597927    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/148ac06c-2830-439f-8046-0024406a5951-etc-sysctl-d\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.598053 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.597949    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/148ac06c-2830-439f-8046-0024406a5951-lib-modules\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.598053 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.597971    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/148ac06c-2830-439f-8046-0024406a5951-tmp\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.598053 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.597994    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f26faf1a-61a1-4ff3-a8fb-5321f27480bc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-g6ggd\" (UID: \"f26faf1a-61a1-4ff3-a8fb-5321f27480bc\") " pod="openshift-multus/multus-additional-cni-plugins-g6ggd"
Sep 09 03:57:57.598053 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.598023    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-host-run-ovn-kubernetes\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.598053 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.598049    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4dd10eaf-1bc1-449e-bb29-3ef1cc334409-device-dir\") pod \"aws-ebs-csi-driver-node-2l5vh\" (UID: \"4dd10eaf-1bc1-449e-bb29-3ef1cc334409\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l5vh"
Sep 09 03:57:57.598434 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.598074    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjcqf\" (UniqueName: \"kubernetes.io/projected/1828d4d8-0d24-4e4d-a502-aee013a3b116-kube-api-access-vjcqf\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.598434 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.598101    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bhdk\" (UniqueName: \"kubernetes.io/projected/6f01d502-4aa7-42a0-9617-093232b1f6a6-kube-api-access-9bhdk\") pod \"node-ca-kvnzq\" (UID: \"6f01d502-4aa7-42a0-9617-093232b1f6a6\") " pod="openshift-image-registry/node-ca-kvnzq"
Sep 09 03:57:57.598434 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.598125    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-cnibin\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.598434 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.598159    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-host-var-lib-cni-multus\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.598434 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.598184    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-etc-kubernetes\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.598434 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.598224    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjszz\" (UniqueName: \"kubernetes.io/projected/1c7a8c99-53ac-4578-aad2-25908242ff17-kube-api-access-sjszz\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.598434 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.598248    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/148ac06c-2830-439f-8046-0024406a5951-etc-sysctl-conf\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.598434 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.598273    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7pck\" (UniqueName: \"kubernetes.io/projected/58280195-fa67-47e6-af03-e39d2bab379b-kube-api-access-p7pck\") pod \"network-metrics-daemon-9lrvt\" (UID: \"58280195-fa67-47e6-af03-e39d2bab379b\") " pod="openshift-multus/network-metrics-daemon-9lrvt"
Sep 09 03:57:57.598434 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.598300    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-host-cni-bin\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.598434 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.598326    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1828d4d8-0d24-4e4d-a502-aee013a3b116-env-overrides\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.598434 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.598351    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/148ac06c-2830-439f-8046-0024406a5951-etc-tuned\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.598925 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.598648    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-systemd-units\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.598925 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.598693    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-host-slash\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.598925 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.598721    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/34f28b0d-79cb-47df-9cc4-44f781c379fa-hosts-file\") pod \"node-resolver-hn6zn\" (UID: \"34f28b0d-79cb-47df-9cc4-44f781c379fa\") " pod="openshift-dns/node-resolver-hn6zn"
Sep 09 03:57:57.598925 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.598777    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5452b40e-6ceb-4f8d-93d5-3dc83a424314-konnectivity-ca\") pod \"konnectivity-agent-scn7w\" (UID: \"5452b40e-6ceb-4f8d-93d5-3dc83a424314\") " pod="kube-system/konnectivity-agent-scn7w"
Sep 09 03:57:57.598925 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.598815    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6nh6\" (UniqueName: \"kubernetes.io/projected/148ac06c-2830-439f-8046-0024406a5951-kube-api-access-h6nh6\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.598925 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.598851    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f26faf1a-61a1-4ff3-a8fb-5321f27480bc-cni-binary-copy\") pod \"multus-additional-cni-plugins-g6ggd\" (UID: \"f26faf1a-61a1-4ff3-a8fb-5321f27480bc\") " pod="openshift-multus/multus-additional-cni-plugins-g6ggd"
Sep 09 03:57:57.598925 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.598883    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-log-socket\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.600080 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.599327    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4dd10eaf-1bc1-449e-bb29-3ef1cc334409-device-dir\") pod \"aws-ebs-csi-driver-node-2l5vh\" (UID: \"4dd10eaf-1bc1-449e-bb29-3ef1cc334409\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l5vh"
Sep 09 03:57:57.600080 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.599342    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-host-run-netns\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.600080 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.599443    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-hostroot\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.600080 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.599575    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/148ac06c-2830-439f-8046-0024406a5951-etc-sysctl-d\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.600080 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.599639    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/148ac06c-2830-439f-8046-0024406a5951-lib-modules\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.600080 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.599667    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f26faf1a-61a1-4ff3-a8fb-5321f27480bc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-g6ggd\" (UID: \"f26faf1a-61a1-4ff3-a8fb-5321f27480bc\") " pod="openshift-multus/multus-additional-cni-plugins-g6ggd"
Sep 09 03:57:57.600080 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.599687    2557 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 09 03:57:57.600080 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.599768    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-cnibin\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.600080 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.599810    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-multus-socket-dir-parent\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.600080 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.599847    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f26faf1a-61a1-4ff3-a8fb-5321f27480bc-cnibin\") pod \"multus-additional-cni-plugins-g6ggd\" (UID: \"f26faf1a-61a1-4ff3-a8fb-5321f27480bc\") " pod="openshift-multus/multus-additional-cni-plugins-g6ggd"
Sep 09 03:57:57.600080 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.599881    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f26faf1a-61a1-4ff3-a8fb-5321f27480bc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-g6ggd\" (UID: \"f26faf1a-61a1-4ff3-a8fb-5321f27480bc\") " pod="openshift-multus/multus-additional-cni-plugins-g6ggd"
Sep 09 03:57:57.600080 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.599931    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/85fe16e4-d362-49b1-b379-451d29551d1b-original-pull-secret\") pod \"global-pull-secret-syncer-78hdg\" (UID: \"85fe16e4-d362-49b1-b379-451d29551d1b\") " pod="kube-system/global-pull-secret-syncer-78hdg"
Sep 09 03:57:57.600080 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.599969    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-host-run-netns\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.600080 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.600004    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-host-cni-netd\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.600080 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.600037    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-system-cni-dir\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.600080 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.600067    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-host-run-k8s-cni-cncf-io\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.600887 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.600099    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-host-var-lib-cni-bin\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.600887 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.600130    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1828d4d8-0d24-4e4d-a502-aee013a3b116-ovnkube-script-lib\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.600887 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.600166    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/148ac06c-2830-439f-8046-0024406a5951-etc-systemd\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.600887 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.600210    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4dd10eaf-1bc1-449e-bb29-3ef1cc334409-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2l5vh\" (UID: \"4dd10eaf-1bc1-449e-bb29-3ef1cc334409\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l5vh"
Sep 09 03:57:57.600887 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.600242    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4dd10eaf-1bc1-449e-bb29-3ef1cc334409-socket-dir\") pod \"aws-ebs-csi-driver-node-2l5vh\" (UID: \"4dd10eaf-1bc1-449e-bb29-3ef1cc334409\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l5vh"
Sep 09 03:57:57.600887 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.600274    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcqdm\" (UniqueName: \"kubernetes.io/projected/4dd10eaf-1bc1-449e-bb29-3ef1cc334409-kube-api-access-qcqdm\") pod \"aws-ebs-csi-driver-node-2l5vh\" (UID: \"4dd10eaf-1bc1-449e-bb29-3ef1cc334409\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l5vh"
Sep 09 03:57:57.600887 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.600306    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-host-var-lib-kubelet\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.600887 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.600337    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-multus-conf-dir\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.600887 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.600363    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-host-run-multus-certs\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.600887 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.600404    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfn7h\" (UniqueName: \"kubernetes.io/projected/f26faf1a-61a1-4ff3-a8fb-5321f27480bc-kube-api-access-qfn7h\") pod \"multus-additional-cni-plugins-g6ggd\" (UID: \"f26faf1a-61a1-4ff3-a8fb-5321f27480bc\") " pod="openshift-multus/multus-additional-cni-plugins-g6ggd"
Sep 09 03:57:57.600887 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.600445    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-var-lib-openvswitch\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.600887 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.600497    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/30cb71f3-8f94-4b63-a43c-07f84af039e9-iptables-alerter-script\") pod \"iptables-alerter-85krs\" (UID: \"30cb71f3-8f94-4b63-a43c-07f84af039e9\") " pod="openshift-network-operator/iptables-alerter-85krs"
Sep 09 03:57:57.600887 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.600532    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96s47\" (UniqueName: \"kubernetes.io/projected/30cb71f3-8f94-4b63-a43c-07f84af039e9-kube-api-access-96s47\") pod \"iptables-alerter-85krs\" (UID: \"30cb71f3-8f94-4b63-a43c-07f84af039e9\") " pod="openshift-network-operator/iptables-alerter-85krs"
Sep 09 03:57:57.600887 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.600516    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-etc-kubernetes\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.600887 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.600565    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5452b40e-6ceb-4f8d-93d5-3dc83a424314-agent-certs\") pod \"konnectivity-agent-scn7w\" (UID: \"5452b40e-6ceb-4f8d-93d5-3dc83a424314\") " pod="kube-system/konnectivity-agent-scn7w"
Sep 09 03:57:57.600887 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.600607    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/148ac06c-2830-439f-8046-0024406a5951-etc-sysconfig\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.600887 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.600670    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/148ac06c-2830-439f-8046-0024406a5951-sys\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.601689 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.600722    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/148ac06c-2830-439f-8046-0024406a5951-host\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.601689 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.600757    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f26faf1a-61a1-4ff3-a8fb-5321f27480bc-system-cni-dir\") pod \"multus-additional-cni-plugins-g6ggd\" (UID: \"f26faf1a-61a1-4ff3-a8fb-5321f27480bc\") " pod="openshift-multus/multus-additional-cni-plugins-g6ggd"
Sep 09 03:57:57.601689 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.600768    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-multus-socket-dir-parent\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.601689 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.600802    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f26faf1a-61a1-4ff3-a8fb-5321f27480bc-cnibin\") pod \"multus-additional-cni-plugins-g6ggd\" (UID: \"f26faf1a-61a1-4ff3-a8fb-5321f27480bc\") " pod="openshift-multus/multus-additional-cni-plugins-g6ggd"
Sep 09 03:57:57.601689 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.600926    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f26faf1a-61a1-4ff3-a8fb-5321f27480bc-system-cni-dir\") pod \"multus-additional-cni-plugins-g6ggd\" (UID: \"f26faf1a-61a1-4ff3-a8fb-5321f27480bc\") " pod="openshift-multus/multus-additional-cni-plugins-g6ggd"
Sep 09 03:57:57.601689 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.601110    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/148ac06c-2830-439f-8046-0024406a5951-etc-sysctl-conf\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.601689 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.601332    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f26faf1a-61a1-4ff3-a8fb-5321f27480bc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-g6ggd\" (UID: \"f26faf1a-61a1-4ff3-a8fb-5321f27480bc\") " pod="openshift-multus/multus-additional-cni-plugins-g6ggd"
Sep 09 03:57:57.601689 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.601402    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/148ac06c-2830-439f-8046-0024406a5951-host\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.601689 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.601561    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-system-cni-dir\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.602060 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.601798    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-multus-conf-dir\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.602060 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.601878    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-host-run-multus-certs\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.602147 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.602083    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-host-run-k8s-cni-cncf-io\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.603502 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.602201    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-host-var-lib-cni-bin\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.603502 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.602343    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4dd10eaf-1bc1-449e-bb29-3ef1cc334409-socket-dir\") pod \"aws-ebs-csi-driver-node-2l5vh\" (UID: \"4dd10eaf-1bc1-449e-bb29-3ef1cc334409\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l5vh"
Sep 09 03:57:57.603502 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.602402    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-host-var-lib-kubelet\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.603502 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.602481    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4dd10eaf-1bc1-449e-bb29-3ef1cc334409-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2l5vh\" (UID: \"4dd10eaf-1bc1-449e-bb29-3ef1cc334409\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l5vh"
Sep 09 03:57:57.603502 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.602513    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/148ac06c-2830-439f-8046-0024406a5951-etc-systemd\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.603502 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.602596    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/148ac06c-2830-439f-8046-0024406a5951-etc-sysconfig\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.603502 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.598462    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-host-var-lib-cni-multus\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.603502 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.603203    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/30cb71f3-8f94-4b63-a43c-07f84af039e9-iptables-alerter-script\") pod \"iptables-alerter-85krs\" (UID: \"30cb71f3-8f94-4b63-a43c-07f84af039e9\") " pod="openshift-network-operator/iptables-alerter-85krs"
Sep 09 03:57:57.603502 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.603280    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5452b40e-6ceb-4f8d-93d5-3dc83a424314-konnectivity-ca\") pod \"konnectivity-agent-scn7w\" (UID: \"5452b40e-6ceb-4f8d-93d5-3dc83a424314\") " pod="kube-system/konnectivity-agent-scn7w"
Sep 09 03:57:57.603980 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.603705    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4dd10eaf-1bc1-449e-bb29-3ef1cc334409-sys-fs\") pod \"aws-ebs-csi-driver-node-2l5vh\" (UID: \"4dd10eaf-1bc1-449e-bb29-3ef1cc334409\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l5vh"
Sep 09 03:57:57.603980 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.603776    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/30cb71f3-8f94-4b63-a43c-07f84af039e9-host-slash\") pod \"iptables-alerter-85krs\" (UID: \"30cb71f3-8f94-4b63-a43c-07f84af039e9\") " pod="openshift-network-operator/iptables-alerter-85krs"
Sep 09 03:57:57.603980 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.603781    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f26faf1a-61a1-4ff3-a8fb-5321f27480bc-cni-binary-copy\") pod \"multus-additional-cni-plugins-g6ggd\" (UID: \"f26faf1a-61a1-4ff3-a8fb-5321f27480bc\") " pod="openshift-multus/multus-additional-cni-plugins-g6ggd"
Sep 09 03:57:57.603980 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.603812    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/148ac06c-2830-439f-8046-0024406a5951-run\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.603980 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.603845    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f26faf1a-61a1-4ff3-a8fb-5321f27480bc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-g6ggd\" (UID: \"f26faf1a-61a1-4ff3-a8fb-5321f27480bc\") " pod="openshift-multus/multus-additional-cni-plugins-g6ggd"
Sep 09 03:57:57.603980 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.603865    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4dd10eaf-1bc1-449e-bb29-3ef1cc334409-sys-fs\") pod \"aws-ebs-csi-driver-node-2l5vh\" (UID: \"4dd10eaf-1bc1-449e-bb29-3ef1cc334409\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l5vh"
Sep 09 03:57:57.603980 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.603896    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58280195-fa67-47e6-af03-e39d2bab379b-metrics-certs\") pod \"network-metrics-daemon-9lrvt\" (UID: \"58280195-fa67-47e6-af03-e39d2bab379b\") " pod="openshift-multus/network-metrics-daemon-9lrvt"
Sep 09 03:57:57.603980 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.603929    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-run-systemd\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.603980 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.603953    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/30cb71f3-8f94-4b63-a43c-07f84af039e9-host-slash\") pod \"iptables-alerter-85krs\" (UID: \"30cb71f3-8f94-4b63-a43c-07f84af039e9\") " pod="openshift-network-operator/iptables-alerter-85krs"
Sep 09 03:57:57.604428 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.604063    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-node-log\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.604428 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.604111    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1828d4d8-0d24-4e4d-a502-aee013a3b116-ovn-node-metrics-cert\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.604428 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.604182    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4dd10eaf-1bc1-449e-bb29-3ef1cc334409-registration-dir\") pod \"aws-ebs-csi-driver-node-2l5vh\" (UID: \"4dd10eaf-1bc1-449e-bb29-3ef1cc334409\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l5vh"
Sep 09 03:57:57.604428 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.604314    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1c7a8c99-53ac-4578-aad2-25908242ff17-multus-daemon-config\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.604428 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.604321    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/148ac06c-2830-439f-8046-0024406a5951-run\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.604428 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.604342    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/148ac06c-2830-439f-8046-0024406a5951-var-lib-kubelet\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.604679 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.604488    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f26faf1a-61a1-4ff3-a8fb-5321f27480bc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-g6ggd\" (UID: \"f26faf1a-61a1-4ff3-a8fb-5321f27480bc\") " pod="openshift-multus/multus-additional-cni-plugins-g6ggd"
Sep 09 03:57:57.604679 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.604497    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/85fe16e4-d362-49b1-b379-451d29551d1b-dbus\") pod \"global-pull-secret-syncer-78hdg\" (UID: \"85fe16e4-d362-49b1-b379-451d29551d1b\") " pod="kube-system/global-pull-secret-syncer-78hdg"
Sep 09 03:57:57.604679 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.604531    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-host-kubelet\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.604679 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.604559    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4dd10eaf-1bc1-449e-bb29-3ef1cc334409-registration-dir\") pod \"aws-ebs-csi-driver-node-2l5vh\" (UID: \"4dd10eaf-1bc1-449e-bb29-3ef1cc334409\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l5vh"
Sep 09 03:57:57.604679 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.604562    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-etc-openvswitch\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.604679 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.604613    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-run-openvswitch\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.604679 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.604639    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-run-ovn\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.604679 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.604653    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/148ac06c-2830-439f-8046-0024406a5951-var-lib-kubelet\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.605006 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.604671    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4dd10eaf-1bc1-449e-bb29-3ef1cc334409-etc-selinux\") pod \"aws-ebs-csi-driver-node-2l5vh\" (UID: \"4dd10eaf-1bc1-449e-bb29-3ef1cc334409\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l5vh"
Sep 09 03:57:57.605006 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.604716    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1828d4d8-0d24-4e4d-a502-aee013a3b116-ovnkube-config\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.605006 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.604752    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5m47\" (UniqueName: \"kubernetes.io/projected/34f28b0d-79cb-47df-9cc4-44f781c379fa-kube-api-access-b5m47\") pod \"node-resolver-hn6zn\" (UID: \"34f28b0d-79cb-47df-9cc4-44f781c379fa\") " pod="openshift-dns/node-resolver-hn6zn"
Sep 09 03:57:57.605006 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.604782    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6f01d502-4aa7-42a0-9617-093232b1f6a6-host\") pod \"node-ca-kvnzq\" (UID: \"6f01d502-4aa7-42a0-9617-093232b1f6a6\") " pod="openshift-image-registry/node-ca-kvnzq"
Sep 09 03:57:57.605006 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.604831    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-multus-cni-dir\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.605006 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.604892    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1c7a8c99-53ac-4578-aad2-25908242ff17-cni-binary-copy\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.605006 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.604936    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84bvb\" (UniqueName: \"kubernetes.io/projected/0b01bf6d-392c-4015-b166-11deff0ffbaa-kube-api-access-84bvb\") pod \"network-check-target-bvnr6\" (UID: \"0b01bf6d-392c-4015-b166-11deff0ffbaa\") " pod="openshift-network-diagnostics/network-check-target-bvnr6"
Sep 09 03:57:57.605308 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.605103    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/148ac06c-2830-439f-8046-0024406a5951-etc-modprobe-d\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.605308 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.605135    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/148ac06c-2830-439f-8046-0024406a5951-etc-kubernetes\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.605308 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.605175    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/85fe16e4-d362-49b1-b379-451d29551d1b-kubelet-config\") pod \"global-pull-secret-syncer-78hdg\" (UID: \"85fe16e4-d362-49b1-b379-451d29551d1b\") " pod="kube-system/global-pull-secret-syncer-78hdg"
Sep 09 03:57:57.605308 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.605211    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1c7a8c99-53ac-4578-aad2-25908242ff17-multus-daemon-config\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.605308 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.605251    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.605514 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.605296    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6f01d502-4aa7-42a0-9617-093232b1f6a6-serviceca\") pod \"node-ca-kvnzq\" (UID: \"6f01d502-4aa7-42a0-9617-093232b1f6a6\") " pod="openshift-image-registry/node-ca-kvnzq"
Sep 09 03:57:57.605514 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.605447    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34f28b0d-79cb-47df-9cc4-44f781c379fa-tmp-dir\") pod \"node-resolver-hn6zn\" (UID: \"34f28b0d-79cb-47df-9cc4-44f781c379fa\") " pod="openshift-dns/node-resolver-hn6zn"
Sep 09 03:57:57.605600 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.605514    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c7a8c99-53ac-4578-aad2-25908242ff17-multus-cni-dir\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.606361 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.605674    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4dd10eaf-1bc1-449e-bb29-3ef1cc334409-etc-selinux\") pod \"aws-ebs-csi-driver-node-2l5vh\" (UID: \"4dd10eaf-1bc1-449e-bb29-3ef1cc334409\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l5vh"
Sep 09 03:57:57.606361 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.605802    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/148ac06c-2830-439f-8046-0024406a5951-etc-kubernetes\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.606361 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.605946    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/148ac06c-2830-439f-8046-0024406a5951-etc-modprobe-d\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.606361 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.606301    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/148ac06c-2830-439f-8046-0024406a5951-sys\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.606585 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.606426    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1c7a8c99-53ac-4578-aad2-25908242ff17-cni-binary-copy\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.613295 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:57.613263    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 03:57:57.613391 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:57.613321    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 03:57:57.613391 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:57.613348    2557 projected.go:196] Error preparing data for projected volume kube-api-access-84bvb for pod openshift-network-diagnostics/network-check-target-bvnr6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:57:57.613550 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:57.613534    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b01bf6d-392c-4015-b166-11deff0ffbaa-kube-api-access-84bvb podName:0b01bf6d-392c-4015-b166-11deff0ffbaa nodeName:}" failed. No retries permitted until 2026-09-09 03:57:58.113486415 +0000 UTC m=+3.032657313 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-84bvb" (UniqueName: "kubernetes.io/projected/0b01bf6d-392c-4015-b166-11deff0ffbaa-kube-api-access-84bvb") pod "network-check-target-bvnr6" (UID: "0b01bf6d-392c-4015-b166-11deff0ffbaa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:57:57.618678 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.618652    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfn7h\" (UniqueName: \"kubernetes.io/projected/f26faf1a-61a1-4ff3-a8fb-5321f27480bc-kube-api-access-qfn7h\") pod \"multus-additional-cni-plugins-g6ggd\" (UID: \"f26faf1a-61a1-4ff3-a8fb-5321f27480bc\") " pod="openshift-multus/multus-additional-cni-plugins-g6ggd"
Sep 09 03:57:57.619087 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.619037    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/148ac06c-2830-439f-8046-0024406a5951-etc-tuned\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.619500 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.619463    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6nh6\" (UniqueName: \"kubernetes.io/projected/148ac06c-2830-439f-8046-0024406a5951-kube-api-access-h6nh6\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.620935 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.620914    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5452b40e-6ceb-4f8d-93d5-3dc83a424314-agent-certs\") pod \"konnectivity-agent-scn7w\" (UID: \"5452b40e-6ceb-4f8d-93d5-3dc83a424314\") " pod="kube-system/konnectivity-agent-scn7w"
Sep 09 03:57:57.622478 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.621220    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/148ac06c-2830-439f-8046-0024406a5951-tmp\") pod \"tuned-fhllb\" (UID: \"148ac06c-2830-439f-8046-0024406a5951\") " pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.622478 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.621887    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcqdm\" (UniqueName: \"kubernetes.io/projected/4dd10eaf-1bc1-449e-bb29-3ef1cc334409-kube-api-access-qcqdm\") pod \"aws-ebs-csi-driver-node-2l5vh\" (UID: \"4dd10eaf-1bc1-449e-bb29-3ef1cc334409\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l5vh"
Sep 09 03:57:57.622478 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.622048    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjszz\" (UniqueName: \"kubernetes.io/projected/1c7a8c99-53ac-4578-aad2-25908242ff17-kube-api-access-sjszz\") pod \"multus-gfd87\" (UID: \"1c7a8c99-53ac-4578-aad2-25908242ff17\") " pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.623543 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.623520    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96s47\" (UniqueName: \"kubernetes.io/projected/30cb71f3-8f94-4b63-a43c-07f84af039e9-kube-api-access-96s47\") pod \"iptables-alerter-85krs\" (UID: \"30cb71f3-8f94-4b63-a43c-07f84af039e9\") " pod="openshift-network-operator/iptables-alerter-85krs"
Sep 09 03:57:57.623751 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.623704    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-71.ec2.internal" event={"ID":"dcbf869fa4c13085efd0490573ad1b8f","Type":"ContainerStarted","Data":"cb74ae454d8ce77368f3735d75281d048afed7d196e4a58f3482b203751bc620"}
Sep 09 03:57:57.625710 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.625682    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-71.ec2.internal" event={"ID":"2c06b6aa4345f7858e0b9cc81a5b603f","Type":"ContainerStarted","Data":"c95de41a268d6e0922f68c57fd42649fa4e1f5589441b3e0851a7150283717b9"}
Sep 09 03:57:57.706222 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.706136    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.706391 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.706286    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.706391 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.706293    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6f01d502-4aa7-42a0-9617-093232b1f6a6-serviceca\") pod \"node-ca-kvnzq\" (UID: \"6f01d502-4aa7-42a0-9617-093232b1f6a6\") " pod="openshift-image-registry/node-ca-kvnzq"
Sep 09 03:57:57.706391 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.706363    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34f28b0d-79cb-47df-9cc4-44f781c379fa-tmp-dir\") pod \"node-resolver-hn6zn\" (UID: \"34f28b0d-79cb-47df-9cc4-44f781c379fa\") " pod="openshift-dns/node-resolver-hn6zn"
Sep 09 03:57:57.706556 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.706397    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-host-run-ovn-kubernetes\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.706556 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.706424    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjcqf\" (UniqueName: \"kubernetes.io/projected/1828d4d8-0d24-4e4d-a502-aee013a3b116-kube-api-access-vjcqf\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.706556 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.706448    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bhdk\" (UniqueName: \"kubernetes.io/projected/6f01d502-4aa7-42a0-9617-093232b1f6a6-kube-api-access-9bhdk\") pod \"node-ca-kvnzq\" (UID: \"6f01d502-4aa7-42a0-9617-093232b1f6a6\") " pod="openshift-image-registry/node-ca-kvnzq"
Sep 09 03:57:57.706556 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.706479    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7pck\" (UniqueName: \"kubernetes.io/projected/58280195-fa67-47e6-af03-e39d2bab379b-kube-api-access-p7pck\") pod \"network-metrics-daemon-9lrvt\" (UID: \"58280195-fa67-47e6-af03-e39d2bab379b\") " pod="openshift-multus/network-metrics-daemon-9lrvt"
Sep 09 03:57:57.706556 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.706505    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-host-cni-bin\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.706556 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.706528    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1828d4d8-0d24-4e4d-a502-aee013a3b116-env-overrides\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.706556 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.706554    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-systemd-units\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.706945 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.706577    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-host-slash\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.706945 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.706622    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/34f28b0d-79cb-47df-9cc4-44f781c379fa-hosts-file\") pod \"node-resolver-hn6zn\" (UID: \"34f28b0d-79cb-47df-9cc4-44f781c379fa\") " pod="openshift-dns/node-resolver-hn6zn"
Sep 09 03:57:57.706945 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.706650    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-log-socket\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.706945 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.706679    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/85fe16e4-d362-49b1-b379-451d29551d1b-original-pull-secret\") pod \"global-pull-secret-syncer-78hdg\" (UID: \"85fe16e4-d362-49b1-b379-451d29551d1b\") " pod="kube-system/global-pull-secret-syncer-78hdg"
Sep 09 03:57:57.706945 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.706704    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-host-run-netns\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.706945 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.706745    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-host-cni-netd\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.706945 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.706770    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6f01d502-4aa7-42a0-9617-093232b1f6a6-serviceca\") pod \"node-ca-kvnzq\" (UID: \"6f01d502-4aa7-42a0-9617-093232b1f6a6\") " pod="openshift-image-registry/node-ca-kvnzq"
Sep 09 03:57:57.706945 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.706773    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1828d4d8-0d24-4e4d-a502-aee013a3b116-ovnkube-script-lib\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.706945 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.706846    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-var-lib-openvswitch\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.706945 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.706890    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58280195-fa67-47e6-af03-e39d2bab379b-metrics-certs\") pod \"network-metrics-daemon-9lrvt\" (UID: \"58280195-fa67-47e6-af03-e39d2bab379b\") " pod="openshift-multus/network-metrics-daemon-9lrvt"
Sep 09 03:57:57.706945 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.706926    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-run-systemd\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.707468 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.706955    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-node-log\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.707468 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.706980    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1828d4d8-0d24-4e4d-a502-aee013a3b116-ovn-node-metrics-cert\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.707468 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.707007    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/85fe16e4-d362-49b1-b379-451d29551d1b-dbus\") pod \"global-pull-secret-syncer-78hdg\" (UID: \"85fe16e4-d362-49b1-b379-451d29551d1b\") " pod="kube-system/global-pull-secret-syncer-78hdg"
Sep 09 03:57:57.707468 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.707030    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-host-kubelet\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.707468 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.707054    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-etc-openvswitch\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.707468 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.707076    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-run-openvswitch\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.707468 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.707135    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-run-openvswitch\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.707468 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.707116    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-run-ovn\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.707468 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.707209    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1828d4d8-0d24-4e4d-a502-aee013a3b116-ovnkube-config\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.707468 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.707238    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5m47\" (UniqueName: \"kubernetes.io/projected/34f28b0d-79cb-47df-9cc4-44f781c379fa-kube-api-access-b5m47\") pod \"node-resolver-hn6zn\" (UID: \"34f28b0d-79cb-47df-9cc4-44f781c379fa\") " pod="openshift-dns/node-resolver-hn6zn"
Sep 09 03:57:57.707468 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.707264    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6f01d502-4aa7-42a0-9617-093232b1f6a6-host\") pod \"node-ca-kvnzq\" (UID: \"6f01d502-4aa7-42a0-9617-093232b1f6a6\") " pod="openshift-image-registry/node-ca-kvnzq"
Sep 09 03:57:57.707468 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.707309    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/85fe16e4-d362-49b1-b379-451d29551d1b-kubelet-config\") pod \"global-pull-secret-syncer-78hdg\" (UID: \"85fe16e4-d362-49b1-b379-451d29551d1b\") " pod="kube-system/global-pull-secret-syncer-78hdg"
Sep 09 03:57:57.707468 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.707392    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/85fe16e4-d362-49b1-b379-451d29551d1b-kubelet-config\") pod \"global-pull-secret-syncer-78hdg\" (UID: \"85fe16e4-d362-49b1-b379-451d29551d1b\") " pod="kube-system/global-pull-secret-syncer-78hdg"
Sep 09 03:57:57.707468 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.707459    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1828d4d8-0d24-4e4d-a502-aee013a3b116-ovnkube-script-lib\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.708140 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.707531    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-systemd-units\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.708140 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.707578    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1828d4d8-0d24-4e4d-a502-aee013a3b116-env-overrides\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.708140 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.707593    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-host-slash\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.708140 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.707647    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34f28b0d-79cb-47df-9cc4-44f781c379fa-tmp-dir\") pod \"node-resolver-hn6zn\" (UID: \"34f28b0d-79cb-47df-9cc4-44f781c379fa\") " pod="openshift-dns/node-resolver-hn6zn"
Sep 09 03:57:57.708140 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.707655    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/34f28b0d-79cb-47df-9cc4-44f781c379fa-hosts-file\") pod \"node-resolver-hn6zn\" (UID: \"34f28b0d-79cb-47df-9cc4-44f781c379fa\") " pod="openshift-dns/node-resolver-hn6zn"
Sep 09 03:57:57.708140 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:57.707675    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:57:57.708140 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.707703    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-log-socket\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.708140 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.707715    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-host-run-ovn-kubernetes\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.708140 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:57.707762    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58280195-fa67-47e6-af03-e39d2bab379b-metrics-certs podName:58280195-fa67-47e6-af03-e39d2bab379b nodeName:}" failed. No retries permitted until 2026-09-09 03:57:58.207744367 +0000 UTC m=+3.126915276 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58280195-fa67-47e6-af03-e39d2bab379b-metrics-certs") pod "network-metrics-daemon-9lrvt" (UID: "58280195-fa67-47e6-af03-e39d2bab379b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:57:57.708140 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.707760    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-host-run-netns\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.708140 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.707818    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-run-systemd\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.708140 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:57.707831    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 03:57:57.708140 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.707859    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-node-log\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.708140 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:57.707879    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85fe16e4-d362-49b1-b379-451d29551d1b-original-pull-secret podName:85fe16e4-d362-49b1-b379-451d29551d1b nodeName:}" failed. No retries permitted until 2026-09-09 03:57:58.20786238 +0000 UTC m=+3.127033289 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/85fe16e4-d362-49b1-b379-451d29551d1b-original-pull-secret") pod "global-pull-secret-syncer-78hdg" (UID: "85fe16e4-d362-49b1-b379-451d29551d1b") : object "kube-system"/"original-pull-secret" not registered
Sep 09 03:57:57.708140 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.707916    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-host-cni-netd\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.708898 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.708311    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-host-cni-bin\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.708898 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.708374    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-run-ovn\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.708898 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.708387    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/85fe16e4-d362-49b1-b379-451d29551d1b-dbus\") pod \"global-pull-secret-syncer-78hdg\" (UID: \"85fe16e4-d362-49b1-b379-451d29551d1b\") " pod="kube-system/global-pull-secret-syncer-78hdg"
Sep 09 03:57:57.708898 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.708417    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6f01d502-4aa7-42a0-9617-093232b1f6a6-host\") pod \"node-ca-kvnzq\" (UID: \"6f01d502-4aa7-42a0-9617-093232b1f6a6\") " pod="openshift-image-registry/node-ca-kvnzq"
Sep 09 03:57:57.708898 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.708461    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-host-kubelet\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.708898 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.708467    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-var-lib-openvswitch\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.708898 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.708480    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1828d4d8-0d24-4e4d-a502-aee013a3b116-etc-openvswitch\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.708898 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.708702    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1828d4d8-0d24-4e4d-a502-aee013a3b116-ovnkube-config\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.712626 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.712560    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1828d4d8-0d24-4e4d-a502-aee013a3b116-ovn-node-metrics-cert\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.718822 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.718797    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjcqf\" (UniqueName: \"kubernetes.io/projected/1828d4d8-0d24-4e4d-a502-aee013a3b116-kube-api-access-vjcqf\") pod \"ovnkube-node-rfvrc\" (UID: \"1828d4d8-0d24-4e4d-a502-aee013a3b116\") " pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.719765 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.719721    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5m47\" (UniqueName: \"kubernetes.io/projected/34f28b0d-79cb-47df-9cc4-44f781c379fa-kube-api-access-b5m47\") pod \"node-resolver-hn6zn\" (UID: \"34f28b0d-79cb-47df-9cc4-44f781c379fa\") " pod="openshift-dns/node-resolver-hn6zn"
Sep 09 03:57:57.721123 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.721097    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bhdk\" (UniqueName: \"kubernetes.io/projected/6f01d502-4aa7-42a0-9617-093232b1f6a6-kube-api-access-9bhdk\") pod \"node-ca-kvnzq\" (UID: \"6f01d502-4aa7-42a0-9617-093232b1f6a6\") " pod="openshift-image-registry/node-ca-kvnzq"
Sep 09 03:57:57.721286 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.721250    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7pck\" (UniqueName: \"kubernetes.io/projected/58280195-fa67-47e6-af03-e39d2bab379b-kube-api-access-p7pck\") pod \"network-metrics-daemon-9lrvt\" (UID: \"58280195-fa67-47e6-af03-e39d2bab379b\") " pod="openshift-multus/network-metrics-daemon-9lrvt"
Sep 09 03:57:57.785981 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.785943    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l5vh"
Sep 09 03:57:57.794915 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.794724    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fhllb"
Sep 09 03:57:57.799044 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:57.799016    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dd10eaf_1bc1_449e_bb29_3ef1cc334409.slice/crio-a7a8e2c02300ab800c7616cd6cd3397e7af4053536fdd853f5ef9ec844ef8c94 WatchSource:0}: Error finding container a7a8e2c02300ab800c7616cd6cd3397e7af4053536fdd853f5ef9ec844ef8c94: Status 404 returned error can't find the container with id a7a8e2c02300ab800c7616cd6cd3397e7af4053536fdd853f5ef9ec844ef8c94
Sep 09 03:57:57.806099 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.806074    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-85krs"
Sep 09 03:57:57.806490 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:57.806457    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod148ac06c_2830_439f_8046_0024406a5951.slice/crio-cd95381d3a40643b4bad647436ccb314d6525854c6bbbcfdfb219655fb958d15 WatchSource:0}: Error finding container cd95381d3a40643b4bad647436ccb314d6525854c6bbbcfdfb219655fb958d15: Status 404 returned error can't find the container with id cd95381d3a40643b4bad647436ccb314d6525854c6bbbcfdfb219655fb958d15
Sep 09 03:57:57.813458 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.813433    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-g6ggd"
Sep 09 03:57:57.817863 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:57.817838    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30cb71f3_8f94_4b63_a43c_07f84af039e9.slice/crio-1e8da346176d40e5c23518a6cb44b97d1c40636ba1572f0960e5497f91f626b7 WatchSource:0}: Error finding container 1e8da346176d40e5c23518a6cb44b97d1c40636ba1572f0960e5497f91f626b7: Status 404 returned error can't find the container with id 1e8da346176d40e5c23518a6cb44b97d1c40636ba1572f0960e5497f91f626b7
Sep 09 03:57:57.820833 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.820813    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gfd87"
Sep 09 03:57:57.824548 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:57.824526    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf26faf1a_61a1_4ff3_a8fb_5321f27480bc.slice/crio-b9315618cf8607633beebd8c45011f1d77cf09b5bb0d5b731661aafa4b023af8 WatchSource:0}: Error finding container b9315618cf8607633beebd8c45011f1d77cf09b5bb0d5b731661aafa4b023af8: Status 404 returned error can't find the container with id b9315618cf8607633beebd8c45011f1d77cf09b5bb0d5b731661aafa4b023af8
Sep 09 03:57:57.828686 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.828625    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-scn7w"
Sep 09 03:57:57.832396 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:57.832362    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c7a8c99_53ac_4578_aad2_25908242ff17.slice/crio-f6be34bcdedc87a88ebc02359fb25ab125a83028d2686b46d8c7cbb7acfcfd85 WatchSource:0}: Error finding container f6be34bcdedc87a88ebc02359fb25ab125a83028d2686b46d8c7cbb7acfcfd85: Status 404 returned error can't find the container with id f6be34bcdedc87a88ebc02359fb25ab125a83028d2686b46d8c7cbb7acfcfd85
Sep 09 03:57:57.838512 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.838349    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:57:57.842294 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:57.842270    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5452b40e_6ceb_4f8d_93d5_3dc83a424314.slice/crio-011b89b48a7c15213865fa382a42de83783194cd85ae33ad75e1c6ed538f2385 WatchSource:0}: Error finding container 011b89b48a7c15213865fa382a42de83783194cd85ae33ad75e1c6ed538f2385: Status 404 returned error can't find the container with id 011b89b48a7c15213865fa382a42de83783194cd85ae33ad75e1c6ed538f2385
Sep 09 03:57:57.846556 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.846503    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hn6zn"
Sep 09 03:57:57.848787 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:57.848758    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1828d4d8_0d24_4e4d_a502_aee013a3b116.slice/crio-169a537df86c824c824ced17ff3ebda8c4012d5496ed8904d0441ff81a93b05d WatchSource:0}: Error finding container 169a537df86c824c824ced17ff3ebda8c4012d5496ed8904d0441ff81a93b05d: Status 404 returned error can't find the container with id 169a537df86c824c824ced17ff3ebda8c4012d5496ed8904d0441ff81a93b05d
Sep 09 03:57:57.857945 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:57.857920    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34f28b0d_79cb_47df_9cc4_44f781c379fa.slice/crio-cc60d993de7c587f3527756e7e7c741b086643923994c0c16bf9475d55e12025 WatchSource:0}: Error finding container cc60d993de7c587f3527756e7e7c741b086643923994c0c16bf9475d55e12025: Status 404 returned error can't find the container with id cc60d993de7c587f3527756e7e7c741b086643923994c0c16bf9475d55e12025
Sep 09 03:57:57.858046 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:57.857979    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kvnzq"
Sep 09 03:57:57.869766 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:57:57.869744    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f01d502_4aa7_42a0_9617_093232b1f6a6.slice/crio-9b830c66e41d93431c11ec8ff5f4d9fde5f2c37b2a07efe6d575eb5956d66399 WatchSource:0}: Error finding container 9b830c66e41d93431c11ec8ff5f4d9fde5f2c37b2a07efe6d575eb5956d66399: Status 404 returned error can't find the container with id 9b830c66e41d93431c11ec8ff5f4d9fde5f2c37b2a07efe6d575eb5956d66399
Sep 09 03:57:58.211630 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:58.211544    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58280195-fa67-47e6-af03-e39d2bab379b-metrics-certs\") pod \"network-metrics-daemon-9lrvt\" (UID: \"58280195-fa67-47e6-af03-e39d2bab379b\") " pod="openshift-multus/network-metrics-daemon-9lrvt"
Sep 09 03:57:58.211630 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:58.211603    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84bvb\" (UniqueName: \"kubernetes.io/projected/0b01bf6d-392c-4015-b166-11deff0ffbaa-kube-api-access-84bvb\") pod \"network-check-target-bvnr6\" (UID: \"0b01bf6d-392c-4015-b166-11deff0ffbaa\") " pod="openshift-network-diagnostics/network-check-target-bvnr6"
Sep 09 03:57:58.211885 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:58.211658    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/85fe16e4-d362-49b1-b379-451d29551d1b-original-pull-secret\") pod \"global-pull-secret-syncer-78hdg\" (UID: \"85fe16e4-d362-49b1-b379-451d29551d1b\") " pod="kube-system/global-pull-secret-syncer-78hdg"
Sep 09 03:57:58.211885 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:58.211753    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:57:58.211885 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:58.211768    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 03:57:58.211885 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:58.211803    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 03:57:58.211885 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:58.211825    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 03:57:58.211885 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:58.211835    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58280195-fa67-47e6-af03-e39d2bab379b-metrics-certs podName:58280195-fa67-47e6-af03-e39d2bab379b nodeName:}" failed. No retries permitted until 2026-09-09 03:57:59.211813446 +0000 UTC m=+4.130984342 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58280195-fa67-47e6-af03-e39d2bab379b-metrics-certs") pod "network-metrics-daemon-9lrvt" (UID: "58280195-fa67-47e6-af03-e39d2bab379b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:57:58.211885 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:58.211839    2557 projected.go:196] Error preparing data for projected volume kube-api-access-84bvb for pod openshift-network-diagnostics/network-check-target-bvnr6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:57:58.211885 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:58.211860    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85fe16e4-d362-49b1-b379-451d29551d1b-original-pull-secret podName:85fe16e4-d362-49b1-b379-451d29551d1b nodeName:}" failed. No retries permitted until 2026-09-09 03:57:59.211846908 +0000 UTC m=+4.131017804 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/85fe16e4-d362-49b1-b379-451d29551d1b-original-pull-secret") pod "global-pull-secret-syncer-78hdg" (UID: "85fe16e4-d362-49b1-b379-451d29551d1b") : object "kube-system"/"original-pull-secret" not registered
Sep 09 03:57:58.212299 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:58.211903    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b01bf6d-392c-4015-b166-11deff0ffbaa-kube-api-access-84bvb podName:0b01bf6d-392c-4015-b166-11deff0ffbaa nodeName:}" failed. No retries permitted until 2026-09-09 03:57:59.211885996 +0000 UTC m=+4.131056899 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-84bvb" (UniqueName: "kubernetes.io/projected/0b01bf6d-392c-4015-b166-11deff0ffbaa-kube-api-access-84bvb") pod "network-check-target-bvnr6" (UID: "0b01bf6d-392c-4015-b166-11deff0ffbaa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:57:58.541668 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:58.541569    2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 03:52:58 +0000 UTC" deadline="2028-03-11 19:39:07.827622606 +0000 UTC"
Sep 09 03:57:58.541668 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:58.541620    2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13191h41m9.286014043s"
Sep 09 03:57:58.628745 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:58.628710    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g6ggd" event={"ID":"f26faf1a-61a1-4ff3-a8fb-5321f27480bc","Type":"ContainerStarted","Data":"b9315618cf8607633beebd8c45011f1d77cf09b5bb0d5b731661aafa4b023af8"}
Sep 09 03:57:58.629852 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:58.629822    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fhllb" event={"ID":"148ac06c-2830-439f-8046-0024406a5951","Type":"ContainerStarted","Data":"cd95381d3a40643b4bad647436ccb314d6525854c6bbbcfdfb219655fb958d15"}
Sep 09 03:57:58.630872 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:58.630848    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-scn7w" event={"ID":"5452b40e-6ceb-4f8d-93d5-3dc83a424314","Type":"ContainerStarted","Data":"011b89b48a7c15213865fa382a42de83783194cd85ae33ad75e1c6ed538f2385"}
Sep 09 03:57:58.631974 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:58.631954    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-85krs" event={"ID":"30cb71f3-8f94-4b63-a43c-07f84af039e9","Type":"ContainerStarted","Data":"1e8da346176d40e5c23518a6cb44b97d1c40636ba1572f0960e5497f91f626b7"}
Sep 09 03:57:58.633155 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:58.633135    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l5vh" event={"ID":"4dd10eaf-1bc1-449e-bb29-3ef1cc334409","Type":"ContainerStarted","Data":"a7a8e2c02300ab800c7616cd6cd3397e7af4053536fdd853f5ef9ec844ef8c94"}
Sep 09 03:57:58.634245 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:58.634220    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kvnzq" event={"ID":"6f01d502-4aa7-42a0-9617-093232b1f6a6","Type":"ContainerStarted","Data":"9b830c66e41d93431c11ec8ff5f4d9fde5f2c37b2a07efe6d575eb5956d66399"}
Sep 09 03:57:58.635636 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:58.635610    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hn6zn" event={"ID":"34f28b0d-79cb-47df-9cc4-44f781c379fa","Type":"ContainerStarted","Data":"cc60d993de7c587f3527756e7e7c741b086643923994c0c16bf9475d55e12025"}
Sep 09 03:57:58.637970 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:58.637946    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc" event={"ID":"1828d4d8-0d24-4e4d-a502-aee013a3b116","Type":"ContainerStarted","Data":"169a537df86c824c824ced17ff3ebda8c4012d5496ed8904d0441ff81a93b05d"}
Sep 09 03:57:58.639328 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:58.639303    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gfd87" event={"ID":"1c7a8c99-53ac-4578-aad2-25908242ff17","Type":"ContainerStarted","Data":"f6be34bcdedc87a88ebc02359fb25ab125a83028d2686b46d8c7cbb7acfcfd85"}
Sep 09 03:57:58.736304 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:58.736273    2557 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 03:57:59.219952 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:59.219907    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/85fe16e4-d362-49b1-b379-451d29551d1b-original-pull-secret\") pod \"global-pull-secret-syncer-78hdg\" (UID: \"85fe16e4-d362-49b1-b379-451d29551d1b\") " pod="kube-system/global-pull-secret-syncer-78hdg"
Sep 09 03:57:59.220121 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:59.219982    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58280195-fa67-47e6-af03-e39d2bab379b-metrics-certs\") pod \"network-metrics-daemon-9lrvt\" (UID: \"58280195-fa67-47e6-af03-e39d2bab379b\") " pod="openshift-multus/network-metrics-daemon-9lrvt"
Sep 09 03:57:59.220121 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:59.220023    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84bvb\" (UniqueName: \"kubernetes.io/projected/0b01bf6d-392c-4015-b166-11deff0ffbaa-kube-api-access-84bvb\") pod \"network-check-target-bvnr6\" (UID: \"0b01bf6d-392c-4015-b166-11deff0ffbaa\") " pod="openshift-network-diagnostics/network-check-target-bvnr6"
Sep 09 03:57:59.220334 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:59.220214    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 03:57:59.220334 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:59.220235    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 03:57:59.220334 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:59.220249    2557 projected.go:196] Error preparing data for projected volume kube-api-access-84bvb for pod openshift-network-diagnostics/network-check-target-bvnr6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:57:59.220334 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:59.220311    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b01bf6d-392c-4015-b166-11deff0ffbaa-kube-api-access-84bvb podName:0b01bf6d-392c-4015-b166-11deff0ffbaa nodeName:}" failed. No retries permitted until 2026-09-09 03:58:01.220291166 +0000 UTC m=+6.139462072 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-84bvb" (UniqueName: "kubernetes.io/projected/0b01bf6d-392c-4015-b166-11deff0ffbaa-kube-api-access-84bvb") pod "network-check-target-bvnr6" (UID: "0b01bf6d-392c-4015-b166-11deff0ffbaa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:57:59.220766 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:59.220740    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 03:57:59.220846 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:59.220806    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85fe16e4-d362-49b1-b379-451d29551d1b-original-pull-secret podName:85fe16e4-d362-49b1-b379-451d29551d1b nodeName:}" failed. No retries permitted until 2026-09-09 03:58:01.220789212 +0000 UTC m=+6.139960125 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/85fe16e4-d362-49b1-b379-451d29551d1b-original-pull-secret") pod "global-pull-secret-syncer-78hdg" (UID: "85fe16e4-d362-49b1-b379-451d29551d1b") : object "kube-system"/"original-pull-secret" not registered
Sep 09 03:57:59.220921 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:59.220869    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:57:59.220921 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:59.220904    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58280195-fa67-47e6-af03-e39d2bab379b-metrics-certs podName:58280195-fa67-47e6-af03-e39d2bab379b nodeName:}" failed. No retries permitted until 2026-09-09 03:58:01.220891431 +0000 UTC m=+6.140062325 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58280195-fa67-47e6-af03-e39d2bab379b-metrics-certs") pod "network-metrics-daemon-9lrvt" (UID: "58280195-fa67-47e6-af03-e39d2bab379b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:57:59.542810 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:59.542686    2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 03:52:58 +0000 UTC" deadline="2028-02-29 20:58:57.128372803 +0000 UTC"
Sep 09 03:57:59.542810 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:59.542748    2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12929h0m57.585629186s"
Sep 09 03:57:59.616428 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:59.615555    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9lrvt"
Sep 09 03:57:59.616428 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:59.615697    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9lrvt" podUID="58280195-fa67-47e6-af03-e39d2bab379b"
Sep 09 03:57:59.616428 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:59.616103    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78hdg"
Sep 09 03:57:59.616428 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:59.616213    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78hdg" podUID="85fe16e4-d362-49b1-b379-451d29551d1b"
Sep 09 03:57:59.616428 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:59.616276    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvnr6"
Sep 09 03:57:59.616428 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:57:59.616346    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvnr6" podUID="0b01bf6d-392c-4015-b166-11deff0ffbaa"
Sep 09 03:57:59.654931 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:57:59.654118    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-71.ec2.internal" event={"ID":"2c06b6aa4345f7858e0b9cc81a5b603f","Type":"ContainerStarted","Data":"f1047e4d1920b4277b97414d21338c5c083ad252d166142a8424d3fc693239bf"}
Sep 09 03:58:00.684354 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:00.684312    2557 generic.go:356] "Generic (PLEG): container finished" podID="dcbf869fa4c13085efd0490573ad1b8f" containerID="e8e84a8a67662b571d43658f49b4bbc5d01ec7adfe9e31321de89996f751eda9" exitCode=0
Sep 09 03:58:00.685345 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:00.685316    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-71.ec2.internal" event={"ID":"dcbf869fa4c13085efd0490573ad1b8f","Type":"ContainerDied","Data":"e8e84a8a67662b571d43658f49b4bbc5d01ec7adfe9e31321de89996f751eda9"}
Sep 09 03:58:00.707914 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:00.706642    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-71.ec2.internal" podStartSLOduration=3.706620316 podStartE2EDuration="3.706620316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 03:57:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:57:59.673072323 +0000 UTC m=+4.592243237" watchObservedRunningTime="2026-09-09 03:58:00.706620316 +0000 UTC m=+5.625791218"
Sep 09 03:58:01.240504 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:01.240463    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84bvb\" (UniqueName: \"kubernetes.io/projected/0b01bf6d-392c-4015-b166-11deff0ffbaa-kube-api-access-84bvb\") pod \"network-check-target-bvnr6\" (UID: \"0b01bf6d-392c-4015-b166-11deff0ffbaa\") " pod="openshift-network-diagnostics/network-check-target-bvnr6"
Sep 09 03:58:01.240737 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:01.240537    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/85fe16e4-d362-49b1-b379-451d29551d1b-original-pull-secret\") pod \"global-pull-secret-syncer-78hdg\" (UID: \"85fe16e4-d362-49b1-b379-451d29551d1b\") " pod="kube-system/global-pull-secret-syncer-78hdg"
Sep 09 03:58:01.240737 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:01.240580    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58280195-fa67-47e6-af03-e39d2bab379b-metrics-certs\") pod \"network-metrics-daemon-9lrvt\" (UID: \"58280195-fa67-47e6-af03-e39d2bab379b\") " pod="openshift-multus/network-metrics-daemon-9lrvt"
Sep 09 03:58:01.240737 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:01.240702    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:58:01.240910 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:01.240767    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58280195-fa67-47e6-af03-e39d2bab379b-metrics-certs podName:58280195-fa67-47e6-af03-e39d2bab379b nodeName:}" failed. No retries permitted until 2026-09-09 03:58:05.240748701 +0000 UTC m=+10.159919597 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58280195-fa67-47e6-af03-e39d2bab379b-metrics-certs") pod "network-metrics-daemon-9lrvt" (UID: "58280195-fa67-47e6-af03-e39d2bab379b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:58:01.241227 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:01.241205    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 03:58:01.241314 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:01.241238    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 03:58:01.241314 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:01.241252    2557 projected.go:196] Error preparing data for projected volume kube-api-access-84bvb for pod openshift-network-diagnostics/network-check-target-bvnr6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:58:01.241431 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:01.241335    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b01bf6d-392c-4015-b166-11deff0ffbaa-kube-api-access-84bvb podName:0b01bf6d-392c-4015-b166-11deff0ffbaa nodeName:}" failed. No retries permitted until 2026-09-09 03:58:05.241317101 +0000 UTC m=+10.160487996 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-84bvb" (UniqueName: "kubernetes.io/projected/0b01bf6d-392c-4015-b166-11deff0ffbaa-kube-api-access-84bvb") pod "network-check-target-bvnr6" (UID: "0b01bf6d-392c-4015-b166-11deff0ffbaa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:58:01.241431 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:01.241401    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 03:58:01.241543 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:01.241440    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85fe16e4-d362-49b1-b379-451d29551d1b-original-pull-secret podName:85fe16e4-d362-49b1-b379-451d29551d1b nodeName:}" failed. No retries permitted until 2026-09-09 03:58:05.241423788 +0000 UTC m=+10.160594722 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/85fe16e4-d362-49b1-b379-451d29551d1b-original-pull-secret") pod "global-pull-secret-syncer-78hdg" (UID: "85fe16e4-d362-49b1-b379-451d29551d1b") : object "kube-system"/"original-pull-secret" not registered
Sep 09 03:58:01.614333 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:01.614302    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9lrvt"
Sep 09 03:58:01.614504 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:01.614453    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9lrvt" podUID="58280195-fa67-47e6-af03-e39d2bab379b"
Sep 09 03:58:01.614843 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:01.614820    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78hdg"
Sep 09 03:58:01.614948 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:01.614929    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78hdg" podUID="85fe16e4-d362-49b1-b379-451d29551d1b"
Sep 09 03:58:01.615095 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:01.615078    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvnr6"
Sep 09 03:58:01.615204 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:01.615183    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvnr6" podUID="0b01bf6d-392c-4015-b166-11deff0ffbaa"
Sep 09 03:58:01.690558 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:01.690507    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-71.ec2.internal" event={"ID":"dcbf869fa4c13085efd0490573ad1b8f","Type":"ContainerStarted","Data":"372d452ce66a44c9d4e437530d92d9dc2f143d684401f97472287833f45f2842"}
Sep 09 03:58:01.727374 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:01.727307    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-71.ec2.internal" podStartSLOduration=4.727282322 podStartE2EDuration="4.727282322s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 03:57:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:58:01.726123835 +0000 UTC m=+6.645294749" watchObservedRunningTime="2026-09-09 03:58:01.727282322 +0000 UTC m=+6.646453235"
Sep 09 03:58:03.614104 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:03.614060    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvnr6"
Sep 09 03:58:03.614584 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:03.614221    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvnr6" podUID="0b01bf6d-392c-4015-b166-11deff0ffbaa"
Sep 09 03:58:03.614584 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:03.614266    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9lrvt"
Sep 09 03:58:03.614584 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:03.614391    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9lrvt" podUID="58280195-fa67-47e6-af03-e39d2bab379b"
Sep 09 03:58:03.615929 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:03.615906    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78hdg"
Sep 09 03:58:03.616031 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:03.616005    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78hdg" podUID="85fe16e4-d362-49b1-b379-451d29551d1b"
Sep 09 03:58:05.278896 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:05.278523    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/85fe16e4-d362-49b1-b379-451d29551d1b-original-pull-secret\") pod \"global-pull-secret-syncer-78hdg\" (UID: \"85fe16e4-d362-49b1-b379-451d29551d1b\") " pod="kube-system/global-pull-secret-syncer-78hdg"
Sep 09 03:58:05.278896 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:05.278595    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58280195-fa67-47e6-af03-e39d2bab379b-metrics-certs\") pod \"network-metrics-daemon-9lrvt\" (UID: \"58280195-fa67-47e6-af03-e39d2bab379b\") " pod="openshift-multus/network-metrics-daemon-9lrvt"
Sep 09 03:58:05.278896 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:05.278635    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84bvb\" (UniqueName: \"kubernetes.io/projected/0b01bf6d-392c-4015-b166-11deff0ffbaa-kube-api-access-84bvb\") pod \"network-check-target-bvnr6\" (UID: \"0b01bf6d-392c-4015-b166-11deff0ffbaa\") " pod="openshift-network-diagnostics/network-check-target-bvnr6"
Sep 09 03:58:05.278896 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:05.278705    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 03:58:05.278896 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:05.278755    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:58:05.278896 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:05.278777    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 03:58:05.278896 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:05.278793    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85fe16e4-d362-49b1-b379-451d29551d1b-original-pull-secret podName:85fe16e4-d362-49b1-b379-451d29551d1b nodeName:}" failed. No retries permitted until 2026-09-09 03:58:13.278770166 +0000 UTC m=+18.197941058 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/85fe16e4-d362-49b1-b379-451d29551d1b-original-pull-secret") pod "global-pull-secret-syncer-78hdg" (UID: "85fe16e4-d362-49b1-b379-451d29551d1b") : object "kube-system"/"original-pull-secret" not registered
Sep 09 03:58:05.278896 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:05.278794    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 03:58:05.278896 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:05.278816    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58280195-fa67-47e6-af03-e39d2bab379b-metrics-certs podName:58280195-fa67-47e6-af03-e39d2bab379b nodeName:}" failed. No retries permitted until 2026-09-09 03:58:13.278804306 +0000 UTC m=+18.197975197 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58280195-fa67-47e6-af03-e39d2bab379b-metrics-certs") pod "network-metrics-daemon-9lrvt" (UID: "58280195-fa67-47e6-af03-e39d2bab379b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:58:05.278896 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:05.278818    2557 projected.go:196] Error preparing data for projected volume kube-api-access-84bvb for pod openshift-network-diagnostics/network-check-target-bvnr6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:58:05.278896 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:05.278871    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b01bf6d-392c-4015-b166-11deff0ffbaa-kube-api-access-84bvb podName:0b01bf6d-392c-4015-b166-11deff0ffbaa nodeName:}" failed. No retries permitted until 2026-09-09 03:58:13.278855419 +0000 UTC m=+18.198026311 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-84bvb" (UniqueName: "kubernetes.io/projected/0b01bf6d-392c-4015-b166-11deff0ffbaa-kube-api-access-84bvb") pod "network-check-target-bvnr6" (UID: "0b01bf6d-392c-4015-b166-11deff0ffbaa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:58:05.614796 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:05.614756    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvnr6"
Sep 09 03:58:05.614976 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:05.614868    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvnr6" podUID="0b01bf6d-392c-4015-b166-11deff0ffbaa"
Sep 09 03:58:05.614976 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:05.614875    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9lrvt"
Sep 09 03:58:05.615091 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:05.614972    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9lrvt" podUID="58280195-fa67-47e6-af03-e39d2bab379b"
Sep 09 03:58:05.615091 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:05.614975    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78hdg"
Sep 09 03:58:05.615091 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:05.615052    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78hdg" podUID="85fe16e4-d362-49b1-b379-451d29551d1b"
Sep 09 03:58:07.616737 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:07.616699    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvnr6"
Sep 09 03:58:07.616737 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:07.616741    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9lrvt"
Sep 09 03:58:07.617217 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:07.616871    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvnr6" podUID="0b01bf6d-392c-4015-b166-11deff0ffbaa"
Sep 09 03:58:07.617217 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:07.616895    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78hdg"
Sep 09 03:58:07.617217 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:07.616998    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9lrvt" podUID="58280195-fa67-47e6-af03-e39d2bab379b"
Sep 09 03:58:07.617217 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:07.617073    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78hdg" podUID="85fe16e4-d362-49b1-b379-451d29551d1b"
Sep 09 03:58:09.613309 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:09.613276    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9lrvt"
Sep 09 03:58:09.613309 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:09.613296    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78hdg"
Sep 09 03:58:09.613791 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:09.613276    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvnr6"
Sep 09 03:58:09.613791 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:09.613417    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9lrvt" podUID="58280195-fa67-47e6-af03-e39d2bab379b"
Sep 09 03:58:09.613791 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:09.613495    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvnr6" podUID="0b01bf6d-392c-4015-b166-11deff0ffbaa"
Sep 09 03:58:09.614269 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:09.614221    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78hdg" podUID="85fe16e4-d362-49b1-b379-451d29551d1b"
Sep 09 03:58:11.614016 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:11.613982    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvnr6"
Sep 09 03:58:11.614455 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:11.613993    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9lrvt"
Sep 09 03:58:11.614455 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:11.614102    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78hdg"
Sep 09 03:58:11.614455 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:11.614107    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvnr6" podUID="0b01bf6d-392c-4015-b166-11deff0ffbaa"
Sep 09 03:58:11.614455 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:11.614199    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9lrvt" podUID="58280195-fa67-47e6-af03-e39d2bab379b"
Sep 09 03:58:11.614455 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:11.614267    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78hdg" podUID="85fe16e4-d362-49b1-b379-451d29551d1b"
Sep 09 03:58:13.332781 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:13.332744    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/85fe16e4-d362-49b1-b379-451d29551d1b-original-pull-secret\") pod \"global-pull-secret-syncer-78hdg\" (UID: \"85fe16e4-d362-49b1-b379-451d29551d1b\") " pod="kube-system/global-pull-secret-syncer-78hdg"
Sep 09 03:58:13.332781 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:13.332794    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58280195-fa67-47e6-af03-e39d2bab379b-metrics-certs\") pod \"network-metrics-daemon-9lrvt\" (UID: \"58280195-fa67-47e6-af03-e39d2bab379b\") " pod="openshift-multus/network-metrics-daemon-9lrvt"
Sep 09 03:58:13.333415 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:13.332821    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84bvb\" (UniqueName: \"kubernetes.io/projected/0b01bf6d-392c-4015-b166-11deff0ffbaa-kube-api-access-84bvb\") pod \"network-check-target-bvnr6\" (UID: \"0b01bf6d-392c-4015-b166-11deff0ffbaa\") " pod="openshift-network-diagnostics/network-check-target-bvnr6"
Sep 09 03:58:13.333415 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:13.332901    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 03:58:13.333415 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:13.332938    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 03:58:13.333415 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:13.332951    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:58:13.333415 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:13.332972    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85fe16e4-d362-49b1-b379-451d29551d1b-original-pull-secret podName:85fe16e4-d362-49b1-b379-451d29551d1b nodeName:}" failed. No retries permitted until 2026-09-09 03:58:29.3329584 +0000 UTC m=+34.252129292 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/85fe16e4-d362-49b1-b379-451d29551d1b-original-pull-secret") pod "global-pull-secret-syncer-78hdg" (UID: "85fe16e4-d362-49b1-b379-451d29551d1b") : object "kube-system"/"original-pull-secret" not registered
Sep 09 03:58:13.333415 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:13.333007    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58280195-fa67-47e6-af03-e39d2bab379b-metrics-certs podName:58280195-fa67-47e6-af03-e39d2bab379b nodeName:}" failed. No retries permitted until 2026-09-09 03:58:29.332990326 +0000 UTC m=+34.252161220 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58280195-fa67-47e6-af03-e39d2bab379b-metrics-certs") pod "network-metrics-daemon-9lrvt" (UID: "58280195-fa67-47e6-af03-e39d2bab379b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:58:13.333415 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:13.332957    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 03:58:13.333415 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:13.333032    2557 projected.go:196] Error preparing data for projected volume kube-api-access-84bvb for pod openshift-network-diagnostics/network-check-target-bvnr6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:58:13.333415 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:13.333067    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b01bf6d-392c-4015-b166-11deff0ffbaa-kube-api-access-84bvb podName:0b01bf6d-392c-4015-b166-11deff0ffbaa nodeName:}" failed. No retries permitted until 2026-09-09 03:58:29.333059875 +0000 UTC m=+34.252230762 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-84bvb" (UniqueName: "kubernetes.io/projected/0b01bf6d-392c-4015-b166-11deff0ffbaa-kube-api-access-84bvb") pod "network-check-target-bvnr6" (UID: "0b01bf6d-392c-4015-b166-11deff0ffbaa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 03:58:13.618908 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:13.618877    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78hdg"
Sep 09 03:58:13.619084 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:13.618974    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvnr6"
Sep 09 03:58:13.619084 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:13.618985    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78hdg" podUID="85fe16e4-d362-49b1-b379-451d29551d1b"
Sep 09 03:58:13.619084 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:13.619007    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9lrvt"
Sep 09 03:58:13.619243 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:13.619093    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvnr6" podUID="0b01bf6d-392c-4015-b166-11deff0ffbaa"
Sep 09 03:58:13.619301 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:13.619230    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9lrvt" podUID="58280195-fa67-47e6-af03-e39d2bab379b"
Sep 09 03:58:15.614196 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:15.614160    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvnr6"
Sep 09 03:58:15.614670 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:15.614264    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvnr6" podUID="0b01bf6d-392c-4015-b166-11deff0ffbaa"
Sep 09 03:58:15.614670 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:15.614360    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9lrvt"
Sep 09 03:58:15.614670 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:15.614494    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9lrvt" podUID="58280195-fa67-47e6-af03-e39d2bab379b"
Sep 09 03:58:15.614670 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:15.614519    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78hdg"
Sep 09 03:58:15.614670 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:15.614575    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78hdg" podUID="85fe16e4-d362-49b1-b379-451d29551d1b"
Sep 09 03:58:16.722463 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:16.722049    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kvnzq" event={"ID":"6f01d502-4aa7-42a0-9617-093232b1f6a6","Type":"ContainerStarted","Data":"30003faba1d6c534e9f95aa6882fb74cf1d6057717345bebae524dd31dbf0adf"}
Sep 09 03:58:16.723588 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:16.723563    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hn6zn" event={"ID":"34f28b0d-79cb-47df-9cc4-44f781c379fa","Type":"ContainerStarted","Data":"567a22e3bd85cde7a6a7e9956158ed5e559351dbaa63644ef7e5fe1a93f1b9e0"}
Sep 09 03:58:16.726334 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:16.726317    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rfvrc_1828d4d8-0d24-4e4d-a502-aee013a3b116/ovn-acl-logging/0.log"
Sep 09 03:58:16.726653 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:16.726636    2557 generic.go:356] "Generic (PLEG): container finished" podID="1828d4d8-0d24-4e4d-a502-aee013a3b116" containerID="0dc4e2782d3732f2d40126826c38fc206894726f6ccf47591a05db2a4b675caf" exitCode=1
Sep 09 03:58:16.726716 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:16.726695    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc" event={"ID":"1828d4d8-0d24-4e4d-a502-aee013a3b116","Type":"ContainerStarted","Data":"c3522d2e199f24bf433a1b7d6779c645a6360a4a103906e201493be98dd27d05"}
Sep 09 03:58:16.726764 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:16.726745    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc" event={"ID":"1828d4d8-0d24-4e4d-a502-aee013a3b116","Type":"ContainerStarted","Data":"978fc7ed401e0d6ae6b60554d85dc87fb725a83e592b9ed874c49701d03cbb81"}
Sep 09 03:58:16.726807 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:16.726789    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc" event={"ID":"1828d4d8-0d24-4e4d-a502-aee013a3b116","Type":"ContainerStarted","Data":"348e7c496513f82638b6bf268d1cea066619e4ab4bf1024994fd9a3c6dc6025f"}
Sep 09 03:58:16.726843 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:16.726824    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc" event={"ID":"1828d4d8-0d24-4e4d-a502-aee013a3b116","Type":"ContainerStarted","Data":"3383476e59c541017d8c648b51af488e59d63375dd7d25fd552be9461d433339"}
Sep 09 03:58:16.726886 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:16.726866    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc" event={"ID":"1828d4d8-0d24-4e4d-a502-aee013a3b116","Type":"ContainerDied","Data":"0dc4e2782d3732f2d40126826c38fc206894726f6ccf47591a05db2a4b675caf"}
Sep 09 03:58:16.726946 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:16.726906    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc" event={"ID":"1828d4d8-0d24-4e4d-a502-aee013a3b116","Type":"ContainerStarted","Data":"80c9296e950335fe37f65466c272d35f0b863c67cd19d8ab4d170f4b291d6294"}
Sep 09 03:58:16.727930 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:16.727913    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gfd87" event={"ID":"1c7a8c99-53ac-4578-aad2-25908242ff17","Type":"ContainerStarted","Data":"23f8009fcf66703ee31290c12e461b65e6933b2a292ec76a13b695c517a912c1"}
Sep 09 03:58:16.729339 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:16.729318    2557 generic.go:356] "Generic (PLEG): container finished" podID="f26faf1a-61a1-4ff3-a8fb-5321f27480bc" containerID="9115898f88fdc7015af44911665e3e88a4ac7191e60937de4dac8bb78e01519c" exitCode=0
Sep 09 03:58:16.729416 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:16.729385    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g6ggd" event={"ID":"f26faf1a-61a1-4ff3-a8fb-5321f27480bc","Type":"ContainerDied","Data":"9115898f88fdc7015af44911665e3e88a4ac7191e60937de4dac8bb78e01519c"}
Sep 09 03:58:16.730857 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:16.730842    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fhllb" event={"ID":"148ac06c-2830-439f-8046-0024406a5951","Type":"ContainerStarted","Data":"6987d27b4b715294cfb4695808c4d60360a11448199f496c5a48e9d50aeb93ee"}
Sep 09 03:58:16.732275 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:16.732255    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-scn7w" event={"ID":"5452b40e-6ceb-4f8d-93d5-3dc83a424314","Type":"ContainerStarted","Data":"182bb8aef056917bed2819c7d6bc2c28639e5edf9b5a711723d0600f6eebce41"}
Sep 09 03:58:16.733714 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:16.733696    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l5vh" event={"ID":"4dd10eaf-1bc1-449e-bb29-3ef1cc334409","Type":"ContainerStarted","Data":"8441d1cfc7b8a346dbd4fd1b81463b161d8f2db5376e1bb534fd0ab00f634088"}
Sep 09 03:58:16.739225 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:16.739190    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-kvnzq" podStartSLOduration=2.690949339 podStartE2EDuration="20.739178715s" totalImagesPullingTime="18.048229376s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 03:57:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:58:16.73839031 +0000 UTC m=+21.657561197" watchObservedRunningTime="2026-09-09 03:58:16.739178715 +0000 UTC m=+21.658349622"
Sep 09 03:58:16.755354 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:16.755318    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fhllb" podStartSLOduration=2.52538315 podStartE2EDuration="20.755307079s" totalImagesPullingTime="18.229923929s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 03:57:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:58:16.754915423 +0000 UTC m=+21.674086325" watchObservedRunningTime="2026-09-09 03:58:16.755307079 +0000 UTC m=+21.674477984"
Sep 09 03:58:16.811171 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:16.811076    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-gfd87" podStartSLOduration=2.403495652 podStartE2EDuration="20.811062063s" totalImagesPullingTime="18.407566411s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 03:57:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:58:16.810994618 +0000 UTC m=+21.730165526" watchObservedRunningTime="2026-09-09 03:58:16.811062063 +0000 UTC m=+21.730232962"
Sep 09 03:58:16.829388 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:16.829339    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-scn7w" podStartSLOduration=2.654705818 podStartE2EDuration="20.829328039s" totalImagesPullingTime="18.174622221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:57:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:58:16.828834974 +0000 UTC m=+21.748005883" watchObservedRunningTime="2026-09-09 03:58:16.829328039 +0000 UTC m=+21.748498947"
Sep 09 03:58:16.847524 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:16.847486    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-hn6zn" podStartSLOduration=2.688671012 podStartE2EDuration="20.847473302s" totalImagesPullingTime="18.15880229s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 03:57:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:58:16.847158137 +0000 UTC m=+21.766329045" watchObservedRunningTime="2026-09-09 03:58:16.847473302 +0000 UTC m=+21.766644212"
Sep 09 03:58:17.613487 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:17.613462    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78hdg"
Sep 09 03:58:17.613615 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:17.613462    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9lrvt"
Sep 09 03:58:17.613615 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:17.613585    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78hdg" podUID="85fe16e4-d362-49b1-b379-451d29551d1b"
Sep 09 03:58:17.613718 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:17.613468    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvnr6"
Sep 09 03:58:17.613718 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:17.613672    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9lrvt" podUID="58280195-fa67-47e6-af03-e39d2bab379b"
Sep 09 03:58:17.613799 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:17.613726    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvnr6" podUID="0b01bf6d-392c-4015-b166-11deff0ffbaa"
Sep 09 03:58:17.626004 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:17.625874    2557 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 09 03:58:17.695253 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:17.695160    2557 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-09T03:58:17.625891297Z","UUID":"767e45af-5d94-4553-b9cf-6d62793c3223","Handler":null,"Name":"","Endpoint":""}
Sep 09 03:58:17.696583 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:17.696566    2557 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 09 03:58:17.696691 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:17.696596    2557 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 09 03:58:17.738069 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:17.738041    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-85krs" event={"ID":"30cb71f3-8f94-4b63-a43c-07f84af039e9","Type":"ContainerStarted","Data":"d7aa6ae3c9f9be47594fe67deee1d0da7f100e2bbc40a819d308e0c685dd1150"}
Sep 09 03:58:17.740103 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:17.740073    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l5vh" event={"ID":"4dd10eaf-1bc1-449e-bb29-3ef1cc334409","Type":"ContainerStarted","Data":"14a140a2bbbe57405baaff49477b42396d62c64df368da7e0cf125672566a180"}
Sep 09 03:58:17.753402 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:17.753325    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-85krs" podStartSLOduration=7.15447776 podStartE2EDuration="21.753309289s" totalImagesPullingTime="14.598831529s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 03:57:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:58:17.753245372 +0000 UTC m=+22.672416285" watchObservedRunningTime="2026-09-09 03:58:17.753309289 +0000 UTC m=+22.672480198"
Sep 09 03:58:19.613904 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:19.613875    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvnr6"
Sep 09 03:58:19.613904 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:19.613912    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9lrvt"
Sep 09 03:58:19.614416 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:19.613990    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvnr6" podUID="0b01bf6d-392c-4015-b166-11deff0ffbaa"
Sep 09 03:58:19.614416 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:19.614015    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78hdg"
Sep 09 03:58:19.614416 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:19.614131    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78hdg" podUID="85fe16e4-d362-49b1-b379-451d29551d1b"
Sep 09 03:58:19.614416 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:19.614187    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9lrvt" podUID="58280195-fa67-47e6-af03-e39d2bab379b"
Sep 09 03:58:19.749631 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:19.749595    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l5vh" event={"ID":"4dd10eaf-1bc1-449e-bb29-3ef1cc334409","Type":"ContainerStarted","Data":"4c95bbfd4a5300e1b6311069198cab218bc6dbc00b12951231a335bdfa2e9654"}
Sep 09 03:58:19.753168 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:19.753149    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rfvrc_1828d4d8-0d24-4e4d-a502-aee013a3b116/ovn-acl-logging/0.log"
Sep 09 03:58:19.753574 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:19.753551    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc" event={"ID":"1828d4d8-0d24-4e4d-a502-aee013a3b116","Type":"ContainerStarted","Data":"14ffd115f724f9f8365571db0fc7c7e4937e89675765bdcd94a71a99c8654480"}
Sep 09 03:58:19.768574 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:19.768524    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l5vh" podStartSLOduration=3.002118765 podStartE2EDuration="23.768508737s" totalImagesPullingTime="20.766389972s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 03:57:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:58:19.768371535 +0000 UTC m=+24.687542445" watchObservedRunningTime="2026-09-09 03:58:19.768508737 +0000 UTC m=+24.687679647"
Sep 09 03:58:20.312130 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:20.311877    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-scn7w"
Sep 09 03:58:20.312506 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:20.312485    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-scn7w"
Sep 09 03:58:20.756159 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:20.756124    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-scn7w"
Sep 09 03:58:20.756641 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:20.756411    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-scn7w"
Sep 09 03:58:21.613228 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:21.613203    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvnr6"
Sep 09 03:58:21.613327 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:21.613203    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9lrvt"
Sep 09 03:58:21.613394 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:21.613332    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvnr6" podUID="0b01bf6d-392c-4015-b166-11deff0ffbaa"
Sep 09 03:58:21.613394 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:21.613203    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78hdg"
Sep 09 03:58:21.613489 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:21.613388    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9lrvt" podUID="58280195-fa67-47e6-af03-e39d2bab379b"
Sep 09 03:58:21.613489 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:21.613457    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78hdg" podUID="85fe16e4-d362-49b1-b379-451d29551d1b"
Sep 09 03:58:21.760743 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:21.760722    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rfvrc_1828d4d8-0d24-4e4d-a502-aee013a3b116/ovn-acl-logging/0.log"
Sep 09 03:58:21.761524 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:21.761077    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc" event={"ID":"1828d4d8-0d24-4e4d-a502-aee013a3b116","Type":"ContainerStarted","Data":"3ad4fc38d813a1ee15d13d68242df3b990e233f873eff92db426f1f92e4b6526"}
Sep 09 03:58:21.761524 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:21.761320    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:58:21.761524 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:21.761482    2557 scope.go:117] "RemoveContainer" containerID="0dc4e2782d3732f2d40126826c38fc206894726f6ccf47591a05db2a4b675caf"
Sep 09 03:58:21.763040 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:21.763016    2557 generic.go:356] "Generic (PLEG): container finished" podID="f26faf1a-61a1-4ff3-a8fb-5321f27480bc" containerID="e1ce1cfc03230b865977440075345857ac0c475eaf1e1707c58070430a3df4a2" exitCode=0
Sep 09 03:58:21.763146 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:21.763082    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g6ggd" event={"ID":"f26faf1a-61a1-4ff3-a8fb-5321f27480bc","Type":"ContainerDied","Data":"e1ce1cfc03230b865977440075345857ac0c475eaf1e1707c58070430a3df4a2"}
Sep 09 03:58:21.776889 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:21.776872    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:58:22.770541 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:22.770320    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rfvrc_1828d4d8-0d24-4e4d-a502-aee013a3b116/ovn-acl-logging/0.log"
Sep 09 03:58:22.770991 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:22.770959    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc" event={"ID":"1828d4d8-0d24-4e4d-a502-aee013a3b116","Type":"ContainerStarted","Data":"a59f152d692390eae6b7615855b416076f4f7ef3825a805d76c9f523cd968a78"}
Sep 09 03:58:22.771205 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:22.771184    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:58:22.771337 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:22.771251    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:58:22.786590 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:22.786564    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:58:22.803712 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:22.803615    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc" podStartSLOduration=8.581592738 podStartE2EDuration="26.803598364s" totalImagesPullingTime="18.222005626s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 03:57:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:58:22.803548673 +0000 UTC m=+27.722719597" watchObservedRunningTime="2026-09-09 03:58:22.803598364 +0000 UTC m=+27.722769266"
Sep 09 03:58:23.236626 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:23.236589    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-78hdg"]
Sep 09 03:58:23.236793 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:23.236712    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78hdg"
Sep 09 03:58:23.236843 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:23.236811    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78hdg" podUID="85fe16e4-d362-49b1-b379-451d29551d1b"
Sep 09 03:58:23.239869 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:23.239844    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bvnr6"]
Sep 09 03:58:23.239998 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:23.239943    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvnr6"
Sep 09 03:58:23.240049 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:23.240031    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvnr6" podUID="0b01bf6d-392c-4015-b166-11deff0ffbaa"
Sep 09 03:58:23.240485 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:23.240467    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9lrvt"]
Sep 09 03:58:23.240576 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:23.240553    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9lrvt"
Sep 09 03:58:23.240646 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:23.240627    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9lrvt" podUID="58280195-fa67-47e6-af03-e39d2bab379b"
Sep 09 03:58:23.774885 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:23.774849    2557 generic.go:356] "Generic (PLEG): container finished" podID="f26faf1a-61a1-4ff3-a8fb-5321f27480bc" containerID="c488527c4ac41840db14685135b38d902f5203460a9cfcfbc4e8176dd0699fd0" exitCode=0
Sep 09 03:58:23.775276 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:23.774934    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g6ggd" event={"ID":"f26faf1a-61a1-4ff3-a8fb-5321f27480bc","Type":"ContainerDied","Data":"c488527c4ac41840db14685135b38d902f5203460a9cfcfbc4e8176dd0699fd0"}
Sep 09 03:58:24.613209 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:24.613185    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78hdg"
Sep 09 03:58:24.613297 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:24.613185    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9lrvt"
Sep 09 03:58:24.613347 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:24.613319    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78hdg" podUID="85fe16e4-d362-49b1-b379-451d29551d1b"
Sep 09 03:58:24.613392 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:24.613376    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9lrvt" podUID="58280195-fa67-47e6-af03-e39d2bab379b"
Sep 09 03:58:24.779036 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:24.778966    2557 generic.go:356] "Generic (PLEG): container finished" podID="f26faf1a-61a1-4ff3-a8fb-5321f27480bc" containerID="3e1d05ebdbe0f58cbbbcadd154f0f8680dd94e5028e7734a5d509cb3ba3c3cc8" exitCode=0
Sep 09 03:58:24.779421 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:24.779046    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g6ggd" event={"ID":"f26faf1a-61a1-4ff3-a8fb-5321f27480bc","Type":"ContainerDied","Data":"3e1d05ebdbe0f58cbbbcadd154f0f8680dd94e5028e7734a5d509cb3ba3c3cc8"}
Sep 09 03:58:25.614228 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:25.614198    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvnr6"
Sep 09 03:58:25.614361 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:25.614273    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvnr6" podUID="0b01bf6d-392c-4015-b166-11deff0ffbaa"
Sep 09 03:58:26.613062 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:26.613031    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78hdg"
Sep 09 03:58:26.613442 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:26.613033    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9lrvt"
Sep 09 03:58:26.613442 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:26.613154    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78hdg" podUID="85fe16e4-d362-49b1-b379-451d29551d1b"
Sep 09 03:58:26.613442 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:26.613213    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9lrvt" podUID="58280195-fa67-47e6-af03-e39d2bab379b"
Sep 09 03:58:27.613586 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:27.613555    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvnr6"
Sep 09 03:58:27.613930 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:27.613650    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bvnr6" podUID="0b01bf6d-392c-4015-b166-11deff0ffbaa"
Sep 09 03:58:28.613924 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:28.613890    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9lrvt"
Sep 09 03:58:28.614454 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:28.614041    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9lrvt" podUID="58280195-fa67-47e6-af03-e39d2bab379b"
Sep 09 03:58:28.614454 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:28.614100    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78hdg"
Sep 09 03:58:28.614454 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:28.614223    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78hdg" podUID="85fe16e4-d362-49b1-b379-451d29551d1b"
Sep 09 03:58:28.855039 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:28.855011    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-71.ec2.internal" event="NodeReady"
Sep 09 03:58:28.855302 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:28.855208    2557 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 09 03:58:28.910885 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:28.910803    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6f66c76d89-h8rdt"]
Sep 09 03:58:28.932285 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:28.932255    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6f7c46fdb6-89xrm"]
Sep 09 03:58:28.932460 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:28.932395    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt"
Sep 09 03:58:28.947770 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:28.947741    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 09 03:58:28.947915 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:28.947815    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 09 03:58:28.947915 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:28.947867    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 09 03:58:28.951136 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:28.950410    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-rtwn6\""
Sep 09 03:58:28.954894 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:28.954859    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-ln2j9"]
Sep 09 03:58:28.955018 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:28.954995    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6f7c46fdb6-89xrm"
Sep 09 03:58:28.958653 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:28.958296    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 09 03:58:28.958653 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:28.958307    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 09 03:58:28.958653 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:28.958418    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 09 03:58:28.958653 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:28.958466    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 09 03:58:28.958653 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:28.958563    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-csp64\""
Sep 09 03:58:28.958653 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:28.958564    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 09 03:58:28.960902 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:28.960885    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 09 03:58:28.971767 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:28.971340    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 09 03:58:28.974629 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:28.974398    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-zqtzf"]
Sep 09 03:58:28.974629 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:28.974540    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-ln2j9"
Sep 09 03:58:28.985646 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:28.985621    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 09 03:58:28.987083 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:28.987060    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 09 03:58:28.987218 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:28.987151    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-2f9js\""
Sep 09 03:58:28.987218 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:28.987205    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 09 03:58:28.987457 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:28.987439    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 09 03:58:29.000187 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.000165    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-57s54"]
Sep 09 03:58:29.000331 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.000284    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zqtzf"
Sep 09 03:58:29.010878 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.010839    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 09 03:58:29.011056 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.011033    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 09 03:58:29.011724 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.011703    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 09 03:58:29.011846 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.011828    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 09 03:58:29.011975 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.011956    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-pcwfx\""
Sep 09 03:58:29.014289 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.014269    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-4xg2m"]
Sep 09 03:58:29.014451 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.014430    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-57s54"
Sep 09 03:58:29.017454 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.017432    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 09 03:58:29.021309 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.021245    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 09 03:58:29.021309 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.021251    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-ght7p\""
Sep 09 03:58:29.021660 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.021367    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 09 03:58:29.021660 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.021256    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 09 03:58:29.025073 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.025052    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 09 03:58:29.032618 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.032597    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-tg4dz"]
Sep 09 03:58:29.032728 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.032716    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-4xg2m"
Sep 09 03:58:29.038627 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.038608    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 09 03:58:29.038766 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.038711    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-rhnfg\""
Sep 09 03:58:29.039081 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.039061    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 09 03:58:29.039202 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.039073    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 09 03:58:29.039280 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.039261    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 09 03:58:29.045139 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.045099    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/54f58d60-214f-4fb6-9ae8-02334c258100-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-ln2j9\" (UID: \"54f58d60-214f-4fb6-9ae8-02334c258100\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-ln2j9"
Sep 09 03:58:29.045226 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.045155    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcm8f\" (UniqueName: \"kubernetes.io/projected/54f58d60-214f-4fb6-9ae8-02334c258100-kube-api-access-hcm8f\") pod \"cluster-monitoring-operator-6cdcc96965-ln2j9\" (UID: \"54f58d60-214f-4fb6-9ae8-02334c258100\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-ln2j9"
Sep 09 03:58:29.045226 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.045182    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/48e91752-c7e6-4208-85f8-1c8386f0aacc-registry-certificates\") pod \"image-registry-6f66c76d89-h8rdt\" (UID: \"48e91752-c7e6-4208-85f8-1c8386f0aacc\") " pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt"
Sep 09 03:58:29.045226 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.045209    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nc44z\" (UniqueName: \"kubernetes.io/projected/f831999b-7d59-4577-864d-c39b88896d64-kube-api-access-nc44z\") pod \"router-default-6f7c46fdb6-89xrm\" (UID: \"f831999b-7d59-4577-864d-c39b88896d64\") " pod="openshift-ingress/router-default-6f7c46fdb6-89xrm"
Sep 09 03:58:29.045371 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.045263    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/48e91752-c7e6-4208-85f8-1c8386f0aacc-image-registry-private-configuration\") pod \"image-registry-6f66c76d89-h8rdt\" (UID: \"48e91752-c7e6-4208-85f8-1c8386f0aacc\") " pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt"
Sep 09 03:58:29.045371 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.045303    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpkxg\" (UniqueName: \"kubernetes.io/projected/48e91752-c7e6-4208-85f8-1c8386f0aacc-kube-api-access-dpkxg\") pod \"image-registry-6f66c76d89-h8rdt\" (UID: \"48e91752-c7e6-4208-85f8-1c8386f0aacc\") " pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt"
Sep 09 03:58:29.045371 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.045331    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48e91752-c7e6-4208-85f8-1c8386f0aacc-registry-tls\") pod \"image-registry-6f66c76d89-h8rdt\" (UID: \"48e91752-c7e6-4208-85f8-1c8386f0aacc\") " pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt"
Sep 09 03:58:29.045371 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.045351    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f831999b-7d59-4577-864d-c39b88896d64-stats-auth\") pod \"router-default-6f7c46fdb6-89xrm\" (UID: \"f831999b-7d59-4577-864d-c39b88896d64\") " pod="openshift-ingress/router-default-6f7c46fdb6-89xrm"
Sep 09 03:58:29.045531 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.045411    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48e91752-c7e6-4208-85f8-1c8386f0aacc-bound-sa-token\") pod \"image-registry-6f66c76d89-h8rdt\" (UID: \"48e91752-c7e6-4208-85f8-1c8386f0aacc\") " pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt"
Sep 09 03:58:29.045531 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.045464    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/48e91752-c7e6-4208-85f8-1c8386f0aacc-trusted-ca\") pod \"image-registry-6f66c76d89-h8rdt\" (UID: \"48e91752-c7e6-4208-85f8-1c8386f0aacc\") " pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt"
Sep 09 03:58:29.045531 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.045493    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/48e91752-c7e6-4208-85f8-1c8386f0aacc-installation-pull-secrets\") pod \"image-registry-6f66c76d89-h8rdt\" (UID: \"48e91752-c7e6-4208-85f8-1c8386f0aacc\") " pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt"
Sep 09 03:58:29.045670 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.045536    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f831999b-7d59-4577-864d-c39b88896d64-default-certificate\") pod \"router-default-6f7c46fdb6-89xrm\" (UID: \"f831999b-7d59-4577-864d-c39b88896d64\") " pod="openshift-ingress/router-default-6f7c46fdb6-89xrm"
Sep 09 03:58:29.045670 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.045566    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f831999b-7d59-4577-864d-c39b88896d64-service-ca-bundle\") pod \"router-default-6f7c46fdb6-89xrm\" (UID: \"f831999b-7d59-4577-864d-c39b88896d64\") " pod="openshift-ingress/router-default-6f7c46fdb6-89xrm"
Sep 09 03:58:29.045670 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.045588    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f831999b-7d59-4577-864d-c39b88896d64-metrics-certs\") pod \"router-default-6f7c46fdb6-89xrm\" (UID: \"f831999b-7d59-4577-864d-c39b88896d64\") " pod="openshift-ingress/router-default-6f7c46fdb6-89xrm"
Sep 09 03:58:29.045670 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.045634    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/54f58d60-214f-4fb6-9ae8-02334c258100-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-ln2j9\" (UID: \"54f58d60-214f-4fb6-9ae8-02334c258100\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-ln2j9"
Sep 09 03:58:29.045670 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.045659    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/48e91752-c7e6-4208-85f8-1c8386f0aacc-ca-trust-extracted\") pod \"image-registry-6f66c76d89-h8rdt\" (UID: \"48e91752-c7e6-4208-85f8-1c8386f0aacc\") " pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt"
Sep 09 03:58:29.051288 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.051270    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-pnxkj"]
Sep 09 03:58:29.051403 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.051389    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-tg4dz"
Sep 09 03:58:29.056077 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.056060    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 09 03:58:29.056283 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.056267    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-7zcl9\""
Sep 09 03:58:29.058413 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.058398    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 09 03:58:29.058532 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.058521    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 09 03:58:29.061181 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.061163    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-gj7h7"]
Sep 09 03:58:29.061294 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.061277    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-pnxkj"
Sep 09 03:58:29.065934 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.065913    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 09 03:58:29.066072 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.066053    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-kl289\""
Sep 09 03:58:29.066385 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.066368    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 09 03:58:29.076812 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.076797    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-gj7h7"
Sep 09 03:58:29.076902 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.076795    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-kkw2d"]
Sep 09 03:58:29.079567 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.079550    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 09 03:58:29.079661 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.079600    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-qn5xc\""
Sep 09 03:58:29.079720 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.079669    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 09 03:58:29.079906 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.079892    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 09 03:58:29.079981 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.079930    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 09 03:58:29.085282 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.085264    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 09 03:58:29.091010 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.090990    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65684fbc5d-f8nqq"]
Sep 09 03:58:29.091157 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.091142    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-kkw2d"
Sep 09 03:58:29.093846 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.093830    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 09 03:58:29.094095 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.094077    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-b6m49\""
Sep 09 03:58:29.094250 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.094227    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 09 03:58:29.101022 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.101004    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-865d5755c7-2v95v"]
Sep 09 03:58:29.101173 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.101156    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65684fbc5d-f8nqq"
Sep 09 03:58:29.104138 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.104090    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-kmzjq\""
Sep 09 03:58:29.104368 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.104352    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 09 03:58:29.104731 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.104712    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 09 03:58:29.105355 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.105329    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 09 03:58:29.105954 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.105934    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 09 03:58:29.117131 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.117089    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f674d7db9-jhbnd"]
Sep 09 03:58:29.117247 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.117226    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-865d5755c7-2v95v"
Sep 09 03:58:29.119743 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.119724    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 09 03:58:29.127290 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.127257    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6f7c46fdb6-89xrm"]
Sep 09 03:58:29.127382 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.127295    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-57s54"]
Sep 09 03:58:29.127382 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.127317    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-ln2j9"]
Sep 09 03:58:29.127382 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.127337    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6f66c76d89-h8rdt"]
Sep 09 03:58:29.127382 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.127357    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-tg4dz"]
Sep 09 03:58:29.127382 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.127381    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-4xg2m"]
Sep 09 03:58:29.127615 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.127394    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f674d7db9-jhbnd"
Sep 09 03:58:29.127615 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.127402    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-pwxcn"]
Sep 09 03:58:29.132918 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.132832    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 09 03:58:29.133646 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.133627    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 09 03:58:29.133836 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.133815    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 09 03:58:29.134778 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.134764    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 09 03:58:29.141721 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.141705    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9d9xs"]
Sep 09 03:58:29.141824 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.141811    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pwxcn"
Sep 09 03:58:29.145883 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.145865    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 09 03:58:29.145976 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.145910    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 09 03:58:29.146033 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.145989    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/48e91752-c7e6-4208-85f8-1c8386f0aacc-trusted-ca\") pod \"image-registry-6f66c76d89-h8rdt\" (UID: \"48e91752-c7e6-4208-85f8-1c8386f0aacc\") " pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt"
Sep 09 03:58:29.146033 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.146026    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f831999b-7d59-4577-864d-c39b88896d64-metrics-certs\") pod \"router-default-6f7c46fdb6-89xrm\" (UID: \"f831999b-7d59-4577-864d-c39b88896d64\") " pod="openshift-ingress/router-default-6f7c46fdb6-89xrm"
Sep 09 03:58:29.146128 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.146061    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6964ed13-fd43-48b3-aff1-8e37bcd05287-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-4xg2m\" (UID: \"6964ed13-fd43-48b3-aff1-8e37bcd05287\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-4xg2m"
Sep 09 03:58:29.146128 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.146094    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7816cd49-e9c2-4f7b-b8ec-4e5711b675a5-serving-cert\") pod \"console-operator-85b454874b-gj7h7\" (UID: \"7816cd49-e9c2-4f7b-b8ec-4e5711b675a5\") " pod="openshift-console-operator/console-operator-85b454874b-gj7h7"
Sep 09 03:58:29.146218 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.146133    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7816cd49-e9c2-4f7b-b8ec-4e5711b675a5-config\") pod \"console-operator-85b454874b-gj7h7\" (UID: \"7816cd49-e9c2-4f7b-b8ec-4e5711b675a5\") " pod="openshift-console-operator/console-operator-85b454874b-gj7h7"
Sep 09 03:58:29.146218 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.146138    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2ccw8\""
Sep 09 03:58:29.146218 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.146156    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gd6hf\" (UniqueName: \"kubernetes.io/projected/7816cd49-e9c2-4f7b-b8ec-4e5711b675a5-kube-api-access-gd6hf\") pod \"console-operator-85b454874b-gj7h7\" (UID: \"7816cd49-e9c2-4f7b-b8ec-4e5711b675a5\") " pod="openshift-console-operator/console-operator-85b454874b-gj7h7"
Sep 09 03:58:29.146218 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:29.146183    2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 09 03:58:29.146350 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.146226    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 09 03:58:29.146350 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:29.146246    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f831999b-7d59-4577-864d-c39b88896d64-metrics-certs podName:f831999b-7d59-4577-864d-c39b88896d64 nodeName:}" failed. No retries permitted until 2026-09-09 03:58:29.646228016 +0000 UTC m=+34.565398922 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f831999b-7d59-4577-864d-c39b88896d64-metrics-certs") pod "router-default-6f7c46fdb6-89xrm" (UID: "f831999b-7d59-4577-864d-c39b88896d64") : secret "router-metrics-certs-default" not found
Sep 09 03:58:29.146350 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.146303    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/54f58d60-214f-4fb6-9ae8-02334c258100-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-ln2j9\" (UID: \"54f58d60-214f-4fb6-9ae8-02334c258100\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-ln2j9"
Sep 09 03:58:29.146350 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.146332    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/54f58d60-214f-4fb6-9ae8-02334c258100-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-ln2j9\" (UID: \"54f58d60-214f-4fb6-9ae8-02334c258100\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-ln2j9"
Sep 09 03:58:29.146527 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.146370    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fftph\" (UniqueName: \"kubernetes.io/projected/6964ed13-fd43-48b3-aff1-8e37bcd05287-kube-api-access-fftph\") pod \"kube-storage-version-migrator-operator-8d7dd4675-4xg2m\" (UID: \"6964ed13-fd43-48b3-aff1-8e37bcd05287\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-4xg2m"
Sep 09 03:58:29.146527 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.146398    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6964ed13-fd43-48b3-aff1-8e37bcd05287-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-4xg2m\" (UID: \"6964ed13-fd43-48b3-aff1-8e37bcd05287\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-4xg2m"
Sep 09 03:58:29.146527 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.146420    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7816cd49-e9c2-4f7b-b8ec-4e5711b675a5-trusted-ca\") pod \"console-operator-85b454874b-gj7h7\" (UID: \"7816cd49-e9c2-4f7b-b8ec-4e5711b675a5\") " pod="openshift-console-operator/console-operator-85b454874b-gj7h7"
Sep 09 03:58:29.146527 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:29.146424    2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 09 03:58:29.146527 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.146449    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/48e91752-c7e6-4208-85f8-1c8386f0aacc-image-registry-private-configuration\") pod \"image-registry-6f66c76d89-h8rdt\" (UID: \"48e91752-c7e6-4208-85f8-1c8386f0aacc\") " pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt"
Sep 09 03:58:29.146527 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.146488    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpkxg\" (UniqueName: \"kubernetes.io/projected/48e91752-c7e6-4208-85f8-1c8386f0aacc-kube-api-access-dpkxg\") pod \"image-registry-6f66c76d89-h8rdt\" (UID: \"48e91752-c7e6-4208-85f8-1c8386f0aacc\") " pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt"
Sep 09 03:58:29.146527 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:29.146500    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54f58d60-214f-4fb6-9ae8-02334c258100-cluster-monitoring-operator-tls podName:54f58d60-214f-4fb6-9ae8-02334c258100 nodeName:}" failed. No retries permitted until 2026-09-09 03:58:29.64648213 +0000 UTC m=+34.565653035 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/54f58d60-214f-4fb6-9ae8-02334c258100-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-ln2j9" (UID: "54f58d60-214f-4fb6-9ae8-02334c258100") : secret "cluster-monitoring-operator-tls" not found
Sep 09 03:58:29.146853 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.146531    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-57s54\" (UID: \"55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7\") " pod="openshift-insights/insights-operator-76bcd6ccf5-57s54"
Sep 09 03:58:29.146853 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.146561    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8s4pm\" (UniqueName: \"kubernetes.io/projected/1318985b-585a-4a94-a73e-c5856d2d2075-kube-api-access-8s4pm\") pod \"service-ca-operator-859f94c4b-zqtzf\" (UID: \"1318985b-585a-4a94-a73e-c5856d2d2075\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zqtzf"
Sep 09 03:58:29.146853 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.146589    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48e91752-c7e6-4208-85f8-1c8386f0aacc-registry-tls\") pod \"image-registry-6f66c76d89-h8rdt\" (UID: \"48e91752-c7e6-4208-85f8-1c8386f0aacc\") " pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt"
Sep 09 03:58:29.146853 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.146613    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f831999b-7d59-4577-864d-c39b88896d64-stats-auth\") pod \"router-default-6f7c46fdb6-89xrm\" (UID: \"f831999b-7d59-4577-864d-c39b88896d64\") " pod="openshift-ingress/router-default-6f7c46fdb6-89xrm"
Sep 09 03:58:29.146853 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.146638    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/68bc0dde-1b15-48ca-a42c-f169a59f3ae4-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-tg4dz\" (UID: \"68bc0dde-1b15-48ca-a42c-f169a59f3ae4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-tg4dz"
Sep 09 03:58:29.146853 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.146662    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7-snapshots\") pod \"insights-operator-76bcd6ccf5-57s54\" (UID: \"55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7\") " pod="openshift-insights/insights-operator-76bcd6ccf5-57s54"
Sep 09 03:58:29.146853 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:29.146685    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 03:58:29.146853 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:29.146698    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f66c76d89-h8rdt: secret "image-registry-tls" not found
Sep 09 03:58:29.146853 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.146697    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48e91752-c7e6-4208-85f8-1c8386f0aacc-bound-sa-token\") pod \"image-registry-6f66c76d89-h8rdt\" (UID: \"48e91752-c7e6-4208-85f8-1c8386f0aacc\") " pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt"
Sep 09 03:58:29.146853 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.146721    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7-serving-cert\") pod \"insights-operator-76bcd6ccf5-57s54\" (UID: \"55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7\") " pod="openshift-insights/insights-operator-76bcd6ccf5-57s54"
Sep 09 03:58:29.146853 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:29.146732    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48e91752-c7e6-4208-85f8-1c8386f0aacc-registry-tls podName:48e91752-c7e6-4208-85f8-1c8386f0aacc nodeName:}" failed. No retries permitted until 2026-09-09 03:58:29.646720335 +0000 UTC m=+34.565891226 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/48e91752-c7e6-4208-85f8-1c8386f0aacc-registry-tls") pod "image-registry-6f66c76d89-h8rdt" (UID: "48e91752-c7e6-4208-85f8-1c8386f0aacc") : secret "image-registry-tls" not found
Sep 09 03:58:29.146853 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.146764    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vs992\" (UniqueName: \"kubernetes.io/projected/55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7-kube-api-access-vs992\") pod \"insights-operator-76bcd6ccf5-57s54\" (UID: \"55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7\") " pod="openshift-insights/insights-operator-76bcd6ccf5-57s54"
Sep 09 03:58:29.146853 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.146850    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/48e91752-c7e6-4208-85f8-1c8386f0aacc-installation-pull-secrets\") pod \"image-registry-6f66c76d89-h8rdt\" (UID: \"48e91752-c7e6-4208-85f8-1c8386f0aacc\") " pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt"
Sep 09 03:58:29.147458 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.146994    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/48e91752-c7e6-4208-85f8-1c8386f0aacc-trusted-ca\") pod \"image-registry-6f66c76d89-h8rdt\" (UID: \"48e91752-c7e6-4208-85f8-1c8386f0aacc\") " pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt"
Sep 09 03:58:29.147458 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.147021    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/54f58d60-214f-4fb6-9ae8-02334c258100-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-ln2j9\" (UID: \"54f58d60-214f-4fb6-9ae8-02334c258100\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-ln2j9"
Sep 09 03:58:29.147458 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.147022    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f831999b-7d59-4577-864d-c39b88896d64-default-certificate\") pod \"router-default-6f7c46fdb6-89xrm\" (UID: \"f831999b-7d59-4577-864d-c39b88896d64\") " pod="openshift-ingress/router-default-6f7c46fdb6-89xrm"
Sep 09 03:58:29.147458 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.147060    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f831999b-7d59-4577-864d-c39b88896d64-service-ca-bundle\") pod \"router-default-6f7c46fdb6-89xrm\" (UID: \"f831999b-7d59-4577-864d-c39b88896d64\") " pod="openshift-ingress/router-default-6f7c46fdb6-89xrm"
Sep 09 03:58:29.147458 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.147089    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/48e91752-c7e6-4208-85f8-1c8386f0aacc-registry-certificates\") pod \"image-registry-6f66c76d89-h8rdt\" (UID: \"48e91752-c7e6-4208-85f8-1c8386f0aacc\") " pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt"
Sep 09 03:58:29.147458 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.147134    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nc44z\" (UniqueName: \"kubernetes.io/projected/f831999b-7d59-4577-864d-c39b88896d64-kube-api-access-nc44z\") pod \"router-default-6f7c46fdb6-89xrm\" (UID: \"f831999b-7d59-4577-864d-c39b88896d64\") " pod="openshift-ingress/router-default-6f7c46fdb6-89xrm"
Sep 09 03:58:29.147458 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.147164    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7-tmp\") pod \"insights-operator-76bcd6ccf5-57s54\" (UID: \"55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7\") " pod="openshift-insights/insights-operator-76bcd6ccf5-57s54"
Sep 09 03:58:29.147458 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:29.147194    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f831999b-7d59-4577-864d-c39b88896d64-service-ca-bundle podName:f831999b-7d59-4577-864d-c39b88896d64 nodeName:}" failed. No retries permitted until 2026-09-09 03:58:29.647180064 +0000 UTC m=+34.566350963 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f831999b-7d59-4577-864d-c39b88896d64-service-ca-bundle") pod "router-default-6f7c46fdb6-89xrm" (UID: "f831999b-7d59-4577-864d-c39b88896d64") : configmap references non-existent config key: service-ca.crt
Sep 09 03:58:29.147458 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.147223    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-57s54\" (UID: \"55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7\") " pod="openshift-insights/insights-operator-76bcd6ccf5-57s54"
Sep 09 03:58:29.147458 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.147264    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/48e91752-c7e6-4208-85f8-1c8386f0aacc-ca-trust-extracted\") pod \"image-registry-6f66c76d89-h8rdt\" (UID: \"48e91752-c7e6-4208-85f8-1c8386f0aacc\") " pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt"
Sep 09 03:58:29.147458 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.147325    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fc4t4\" (UniqueName: \"kubernetes.io/projected/d55cca64-4230-4d1b-9e52-0d7379772d4b-kube-api-access-fc4t4\") pod \"volume-data-source-validator-695c895bd6-pnxkj\" (UID: \"d55cca64-4230-4d1b-9e52-0d7379772d4b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-pnxkj"
Sep 09 03:58:29.147458 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.147371    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcm8f\" (UniqueName: \"kubernetes.io/projected/54f58d60-214f-4fb6-9ae8-02334c258100-kube-api-access-hcm8f\") pod \"cluster-monitoring-operator-6cdcc96965-ln2j9\" (UID: \"54f58d60-214f-4fb6-9ae8-02334c258100\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-ln2j9"
Sep 09 03:58:29.147458 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.147396    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpfbz\" (UniqueName: \"kubernetes.io/projected/68bc0dde-1b15-48ca-a42c-f169a59f3ae4-kube-api-access-fpfbz\") pod \"cluster-samples-operator-7cdcfcf76c-tg4dz\" (UID: \"68bc0dde-1b15-48ca-a42c-f169a59f3ae4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-tg4dz"
Sep 09 03:58:29.147458 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.147421    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1318985b-585a-4a94-a73e-c5856d2d2075-serving-cert\") pod \"service-ca-operator-859f94c4b-zqtzf\" (UID: \"1318985b-585a-4a94-a73e-c5856d2d2075\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zqtzf"
Sep 09 03:58:29.147458 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.147455    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1318985b-585a-4a94-a73e-c5856d2d2075-config\") pod \"service-ca-operator-859f94c4b-zqtzf\" (UID: \"1318985b-585a-4a94-a73e-c5856d2d2075\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zqtzf"
Sep 09 03:58:29.148100 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.147594    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/48e91752-c7e6-4208-85f8-1c8386f0aacc-registry-certificates\") pod \"image-registry-6f66c76d89-h8rdt\" (UID: \"48e91752-c7e6-4208-85f8-1c8386f0aacc\") " pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt"
Sep 09 03:58:29.148100 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.147703    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/48e91752-c7e6-4208-85f8-1c8386f0aacc-ca-trust-extracted\") pod \"image-registry-6f66c76d89-h8rdt\" (UID: \"48e91752-c7e6-4208-85f8-1c8386f0aacc\") " pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt"
Sep 09 03:58:29.150865 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.150842    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/48e91752-c7e6-4208-85f8-1c8386f0aacc-image-registry-private-configuration\") pod \"image-registry-6f66c76d89-h8rdt\" (UID: \"48e91752-c7e6-4208-85f8-1c8386f0aacc\") " pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt"
Sep 09 03:58:29.150965 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.150947    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f831999b-7d59-4577-864d-c39b88896d64-stats-auth\") pod \"router-default-6f7c46fdb6-89xrm\" (UID: \"f831999b-7d59-4577-864d-c39b88896d64\") " pod="openshift-ingress/router-default-6f7c46fdb6-89xrm"
Sep 09 03:58:29.151314 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.151282    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/48e91752-c7e6-4208-85f8-1c8386f0aacc-installation-pull-secrets\") pod \"image-registry-6f66c76d89-h8rdt\" (UID: \"48e91752-c7e6-4208-85f8-1c8386f0aacc\") " pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt"
Sep 09 03:58:29.151397 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.151346    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f831999b-7d59-4577-864d-c39b88896d64-default-certificate\") pod \"router-default-6f7c46fdb6-89xrm\" (UID: \"f831999b-7d59-4577-864d-c39b88896d64\") " pod="openshift-ingress/router-default-6f7c46fdb6-89xrm"
Sep 09 03:58:29.159820 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.159782    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpkxg\" (UniqueName: \"kubernetes.io/projected/48e91752-c7e6-4208-85f8-1c8386f0aacc-kube-api-access-dpkxg\") pod \"image-registry-6f66c76d89-h8rdt\" (UID: \"48e91752-c7e6-4208-85f8-1c8386f0aacc\") " pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt"
Sep 09 03:58:29.161190 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.161142    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcm8f\" (UniqueName: \"kubernetes.io/projected/54f58d60-214f-4fb6-9ae8-02334c258100-kube-api-access-hcm8f\") pod \"cluster-monitoring-operator-6cdcc96965-ln2j9\" (UID: \"54f58d60-214f-4fb6-9ae8-02334c258100\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-ln2j9"
Sep 09 03:58:29.161585 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.161568    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nc44z\" (UniqueName: \"kubernetes.io/projected/f831999b-7d59-4577-864d-c39b88896d64-kube-api-access-nc44z\") pod \"router-default-6f7c46fdb6-89xrm\" (UID: \"f831999b-7d59-4577-864d-c39b88896d64\") " pod="openshift-ingress/router-default-6f7c46fdb6-89xrm"
Sep 09 03:58:29.162247 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.162215    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48e91752-c7e6-4208-85f8-1c8386f0aacc-bound-sa-token\") pod \"image-registry-6f66c76d89-h8rdt\" (UID: \"48e91752-c7e6-4208-85f8-1c8386f0aacc\") " pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt"
Sep 09 03:58:29.175003 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.174983    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65684fbc5d-f8nqq"]
Sep 09 03:58:29.175094 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.175015    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-gj7h7"]
Sep 09 03:58:29.175183 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.175095    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9d9xs"
Sep 09 03:58:29.175183 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.175125    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-pnxkj"]
Sep 09 03:58:29.175183 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.175149    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9d9xs"]
Sep 09 03:58:29.175183 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.175174    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-kkw2d"]
Sep 09 03:58:29.175353 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.175191    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-865d5755c7-2v95v"]
Sep 09 03:58:29.175353 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.175208    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-zqtzf"]
Sep 09 03:58:29.175353 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.175220    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pwxcn"]
Sep 09 03:58:29.175353 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.175231    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f674d7db9-jhbnd"]
Sep 09 03:58:29.177687 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.177670    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 09 03:58:29.178014 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.177987    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 09 03:58:29.178226 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.178205    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-nrdh9\""
Sep 09 03:58:29.248330 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.248300    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/80439eab-81e8-4a3b-a2de-44c45f992052-hub\") pod \"cluster-proxy-proxy-agent-7f674d7db9-jhbnd\" (UID: \"80439eab-81e8-4a3b-a2de-44c45f992052\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f674d7db9-jhbnd"
Sep 09 03:58:29.248330 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.248330    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1a0b3b4-b53f-4870-8849-6ee3e393374f-tmp\") pod \"klusterlet-addon-workmgr-865d5755c7-2v95v\" (UID: \"e1a0b3b4-b53f-4870-8849-6ee3e393374f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-865d5755c7-2v95v"
Sep 09 03:58:29.248533 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.248366    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e1a0b3b4-b53f-4870-8849-6ee3e393374f-klusterlet-config\") pod \"klusterlet-addon-workmgr-865d5755c7-2v95v\" (UID: \"e1a0b3b4-b53f-4870-8849-6ee3e393374f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-865d5755c7-2v95v"
Sep 09 03:58:29.248533 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.248413    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7-tmp\") pod \"insights-operator-76bcd6ccf5-57s54\" (UID: \"55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7\") " pod="openshift-insights/insights-operator-76bcd6ccf5-57s54"
Sep 09 03:58:29.248533 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.248433    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-57s54\" (UID: \"55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7\") " pod="openshift-insights/insights-operator-76bcd6ccf5-57s54"
Sep 09 03:58:29.248533 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.248455    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/80439eab-81e8-4a3b-a2de-44c45f992052-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7f674d7db9-jhbnd\" (UID: \"80439eab-81e8-4a3b-a2de-44c45f992052\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f674d7db9-jhbnd"
Sep 09 03:58:29.248533 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.248477    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/80439eab-81e8-4a3b-a2de-44c45f992052-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7f674d7db9-jhbnd\" (UID: \"80439eab-81e8-4a3b-a2de-44c45f992052\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f674d7db9-jhbnd"
Sep 09 03:58:29.248533 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.248503    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fc4t4\" (UniqueName: \"kubernetes.io/projected/d55cca64-4230-4d1b-9e52-0d7379772d4b-kube-api-access-fc4t4\") pod \"volume-data-source-validator-695c895bd6-pnxkj\" (UID: \"d55cca64-4230-4d1b-9e52-0d7379772d4b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-pnxkj"
Sep 09 03:58:29.248533 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.248526    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/80439eab-81e8-4a3b-a2de-44c45f992052-ca\") pod \"cluster-proxy-proxy-agent-7f674d7db9-jhbnd\" (UID: \"80439eab-81e8-4a3b-a2de-44c45f992052\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f674d7db9-jhbnd"
Sep 09 03:58:29.248834 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.248552    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa51a428-9bbd-492a-98ae-e4bd03fab5ce-cert\") pod \"ingress-canary-pwxcn\" (UID: \"aa51a428-9bbd-492a-98ae-e4bd03fab5ce\") " pod="openshift-ingress-canary/ingress-canary-pwxcn"
Sep 09 03:58:29.248834 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.248570    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpfbz\" (UniqueName: \"kubernetes.io/projected/68bc0dde-1b15-48ca-a42c-f169a59f3ae4-kube-api-access-fpfbz\") pod \"cluster-samples-operator-7cdcfcf76c-tg4dz\" (UID: \"68bc0dde-1b15-48ca-a42c-f169a59f3ae4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-tg4dz"
Sep 09 03:58:29.248834 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.248592    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1318985b-585a-4a94-a73e-c5856d2d2075-serving-cert\") pod \"service-ca-operator-859f94c4b-zqtzf\" (UID: \"1318985b-585a-4a94-a73e-c5856d2d2075\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zqtzf"
Sep 09 03:58:29.248834 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.248654    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1318985b-585a-4a94-a73e-c5856d2d2075-config\") pod \"service-ca-operator-859f94c4b-zqtzf\" (UID: \"1318985b-585a-4a94-a73e-c5856d2d2075\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zqtzf"
Sep 09 03:58:29.248834 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.248676    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5lb6\" (UniqueName: \"kubernetes.io/projected/b73c2563-ee66-481a-ba6e-122139fb3abc-kube-api-access-m5lb6\") pod \"dns-default-9d9xs\" (UID: \"b73c2563-ee66-481a-ba6e-122139fb3abc\") " pod="openshift-dns/dns-default-9d9xs"
Sep 09 03:58:29.248834 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.248703    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkgzw\" (UniqueName: \"kubernetes.io/projected/39402424-bac6-470f-ba07-996f9acd8313-kube-api-access-bkgzw\") pod \"network-check-source-685456655f-kkw2d\" (UID: \"39402424-bac6-470f-ba07-996f9acd8313\") " pod="openshift-network-diagnostics/network-check-source-685456655f-kkw2d"
Sep 09 03:58:29.248834 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.248724    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b73c2563-ee66-481a-ba6e-122139fb3abc-metrics-tls\") pod \"dns-default-9d9xs\" (UID: \"b73c2563-ee66-481a-ba6e-122139fb3abc\") " pod="openshift-dns/dns-default-9d9xs"
Sep 09 03:58:29.248834 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.248777    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpbzb\" (UniqueName: \"kubernetes.io/projected/e1a0b3b4-b53f-4870-8849-6ee3e393374f-kube-api-access-lpbzb\") pod \"klusterlet-addon-workmgr-865d5755c7-2v95v\" (UID: \"e1a0b3b4-b53f-4870-8849-6ee3e393374f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-865d5755c7-2v95v"
Sep 09 03:58:29.248834 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.248802    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vzbg\" (UniqueName: \"kubernetes.io/projected/9cf8ade1-fc61-4570-9a58-ab1b4c4bb291-kube-api-access-6vzbg\") pod \"managed-serviceaccount-addon-agent-65684fbc5d-f8nqq\" (UID: \"9cf8ade1-fc61-4570-9a58-ab1b4c4bb291\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65684fbc5d-f8nqq"
Sep 09 03:58:29.248834 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.248826    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b73c2563-ee66-481a-ba6e-122139fb3abc-config-volume\") pod \"dns-default-9d9xs\" (UID: \"b73c2563-ee66-481a-ba6e-122139fb3abc\") " pod="openshift-dns/dns-default-9d9xs"
Sep 09 03:58:29.249166 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.248859    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6964ed13-fd43-48b3-aff1-8e37bcd05287-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-4xg2m\" (UID: \"6964ed13-fd43-48b3-aff1-8e37bcd05287\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-4xg2m"
Sep 09 03:58:29.249166 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.248803    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7-tmp\") pod \"insights-operator-76bcd6ccf5-57s54\" (UID: \"55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7\") " pod="openshift-insights/insights-operator-76bcd6ccf5-57s54"
Sep 09 03:58:29.249166 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.248882    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99dss\" (UniqueName: \"kubernetes.io/projected/aa51a428-9bbd-492a-98ae-e4bd03fab5ce-kube-api-access-99dss\") pod \"ingress-canary-pwxcn\" (UID: \"aa51a428-9bbd-492a-98ae-e4bd03fab5ce\") " pod="openshift-ingress-canary/ingress-canary-pwxcn"
Sep 09 03:58:29.249166 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.248921    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7816cd49-e9c2-4f7b-b8ec-4e5711b675a5-serving-cert\") pod \"console-operator-85b454874b-gj7h7\" (UID: \"7816cd49-e9c2-4f7b-b8ec-4e5711b675a5\") " pod="openshift-console-operator/console-operator-85b454874b-gj7h7"
Sep 09 03:58:29.249166 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.248947    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7816cd49-e9c2-4f7b-b8ec-4e5711b675a5-config\") pod \"console-operator-85b454874b-gj7h7\" (UID: \"7816cd49-e9c2-4f7b-b8ec-4e5711b675a5\") " pod="openshift-console-operator/console-operator-85b454874b-gj7h7"
Sep 09 03:58:29.249166 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.248970    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gd6hf\" (UniqueName: \"kubernetes.io/projected/7816cd49-e9c2-4f7b-b8ec-4e5711b675a5-kube-api-access-gd6hf\") pod \"console-operator-85b454874b-gj7h7\" (UID: \"7816cd49-e9c2-4f7b-b8ec-4e5711b675a5\") " pod="openshift-console-operator/console-operator-85b454874b-gj7h7"
Sep 09 03:58:29.249166 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.249008    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9cf8ade1-fc61-4570-9a58-ab1b4c4bb291-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-65684fbc5d-f8nqq\" (UID: \"9cf8ade1-fc61-4570-9a58-ab1b4c4bb291\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65684fbc5d-f8nqq"
Sep 09 03:58:29.249166 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.249051    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fftph\" (UniqueName: \"kubernetes.io/projected/6964ed13-fd43-48b3-aff1-8e37bcd05287-kube-api-access-fftph\") pod \"kube-storage-version-migrator-operator-8d7dd4675-4xg2m\" (UID: \"6964ed13-fd43-48b3-aff1-8e37bcd05287\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-4xg2m"
Sep 09 03:58:29.249166 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.249085    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6964ed13-fd43-48b3-aff1-8e37bcd05287-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-4xg2m\" (UID: \"6964ed13-fd43-48b3-aff1-8e37bcd05287\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-4xg2m"
Sep 09 03:58:29.249166 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.249133    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7816cd49-e9c2-4f7b-b8ec-4e5711b675a5-trusted-ca\") pod \"console-operator-85b454874b-gj7h7\" (UID: \"7816cd49-e9c2-4f7b-b8ec-4e5711b675a5\") " pod="openshift-console-operator/console-operator-85b454874b-gj7h7"
Sep 09 03:58:29.249166 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.249162    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b73c2563-ee66-481a-ba6e-122139fb3abc-tmp-dir\") pod \"dns-default-9d9xs\" (UID: \"b73c2563-ee66-481a-ba6e-122139fb3abc\") " pod="openshift-dns/dns-default-9d9xs"
Sep 09 03:58:29.249658 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.249189    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-57s54\" (UID: \"55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7\") " pod="openshift-insights/insights-operator-76bcd6ccf5-57s54"
Sep 09 03:58:29.249658 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.249218    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8s4pm\" (UniqueName: \"kubernetes.io/projected/1318985b-585a-4a94-a73e-c5856d2d2075-kube-api-access-8s4pm\") pod \"service-ca-operator-859f94c4b-zqtzf\" (UID: \"1318985b-585a-4a94-a73e-c5856d2d2075\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zqtzf"
Sep 09 03:58:29.249658 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.249248    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrgz9\" (UniqueName: \"kubernetes.io/projected/80439eab-81e8-4a3b-a2de-44c45f992052-kube-api-access-rrgz9\") pod \"cluster-proxy-proxy-agent-7f674d7db9-jhbnd\" (UID: \"80439eab-81e8-4a3b-a2de-44c45f992052\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f674d7db9-jhbnd"
Sep 09 03:58:29.249658 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.249260    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1318985b-585a-4a94-a73e-c5856d2d2075-config\") pod \"service-ca-operator-859f94c4b-zqtzf\" (UID: \"1318985b-585a-4a94-a73e-c5856d2d2075\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zqtzf"
Sep 09 03:58:29.249658 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.249322    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/68bc0dde-1b15-48ca-a42c-f169a59f3ae4-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-tg4dz\" (UID: \"68bc0dde-1b15-48ca-a42c-f169a59f3ae4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-tg4dz"
Sep 09 03:58:29.249658 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.249353    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7-snapshots\") pod \"insights-operator-76bcd6ccf5-57s54\" (UID: \"55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7\") " pod="openshift-insights/insights-operator-76bcd6ccf5-57s54"
Sep 09 03:58:29.249658 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.249381    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7-serving-cert\") pod \"insights-operator-76bcd6ccf5-57s54\" (UID: \"55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7\") " pod="openshift-insights/insights-operator-76bcd6ccf5-57s54"
Sep 09 03:58:29.249658 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.249401    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-57s54\" (UID: \"55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7\") " pod="openshift-insights/insights-operator-76bcd6ccf5-57s54"
Sep 09 03:58:29.249658 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.249407    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vs992\" (UniqueName: \"kubernetes.io/projected/55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7-kube-api-access-vs992\") pod \"insights-operator-76bcd6ccf5-57s54\" (UID: \"55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7\") " pod="openshift-insights/insights-operator-76bcd6ccf5-57s54"
Sep 09 03:58:29.249658 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.249435    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/80439eab-81e8-4a3b-a2de-44c45f992052-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7f674d7db9-jhbnd\" (UID: \"80439eab-81e8-4a3b-a2de-44c45f992052\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f674d7db9-jhbnd"
Sep 09 03:58:29.249658 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:29.249554    2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 03:58:29.249658 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:29.249618    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68bc0dde-1b15-48ca-a42c-f169a59f3ae4-samples-operator-tls podName:68bc0dde-1b15-48ca-a42c-f169a59f3ae4 nodeName:}" failed. No retries permitted until 2026-09-09 03:58:29.749601228 +0000 UTC m=+34.668772116 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/68bc0dde-1b15-48ca-a42c-f169a59f3ae4-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-tg4dz" (UID: "68bc0dde-1b15-48ca-a42c-f169a59f3ae4") : secret "samples-operator-tls" not found
Sep 09 03:58:29.250230 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.249917    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6964ed13-fd43-48b3-aff1-8e37bcd05287-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-4xg2m\" (UID: \"6964ed13-fd43-48b3-aff1-8e37bcd05287\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-4xg2m"
Sep 09 03:58:29.250230 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.249924    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7816cd49-e9c2-4f7b-b8ec-4e5711b675a5-config\") pod \"console-operator-85b454874b-gj7h7\" (UID: \"7816cd49-e9c2-4f7b-b8ec-4e5711b675a5\") " pod="openshift-console-operator/console-operator-85b454874b-gj7h7"
Sep 09 03:58:29.250348 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.250324    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7-snapshots\") pod \"insights-operator-76bcd6ccf5-57s54\" (UID: \"55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7\") " pod="openshift-insights/insights-operator-76bcd6ccf5-57s54"
Sep 09 03:58:29.251097 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.250758    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7816cd49-e9c2-4f7b-b8ec-4e5711b675a5-trusted-ca\") pod \"console-operator-85b454874b-gj7h7\" (UID: \"7816cd49-e9c2-4f7b-b8ec-4e5711b675a5\") " pod="openshift-console-operator/console-operator-85b454874b-gj7h7"
Sep 09 03:58:29.251097 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.250949    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-57s54\" (UID: \"55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7\") " pod="openshift-insights/insights-operator-76bcd6ccf5-57s54"
Sep 09 03:58:29.251812 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.251204    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7816cd49-e9c2-4f7b-b8ec-4e5711b675a5-serving-cert\") pod \"console-operator-85b454874b-gj7h7\" (UID: \"7816cd49-e9c2-4f7b-b8ec-4e5711b675a5\") " pod="openshift-console-operator/console-operator-85b454874b-gj7h7"
Sep 09 03:58:29.251812 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.251243    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1318985b-585a-4a94-a73e-c5856d2d2075-serving-cert\") pod \"service-ca-operator-859f94c4b-zqtzf\" (UID: \"1318985b-585a-4a94-a73e-c5856d2d2075\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zqtzf"
Sep 09 03:58:29.251812 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.251564    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6964ed13-fd43-48b3-aff1-8e37bcd05287-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-4xg2m\" (UID: \"6964ed13-fd43-48b3-aff1-8e37bcd05287\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-4xg2m"
Sep 09 03:58:29.252492 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.252472    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7-serving-cert\") pod \"insights-operator-76bcd6ccf5-57s54\" (UID: \"55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7\") " pod="openshift-insights/insights-operator-76bcd6ccf5-57s54"
Sep 09 03:58:29.256893 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.256864    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpfbz\" (UniqueName: \"kubernetes.io/projected/68bc0dde-1b15-48ca-a42c-f169a59f3ae4-kube-api-access-fpfbz\") pod \"cluster-samples-operator-7cdcfcf76c-tg4dz\" (UID: \"68bc0dde-1b15-48ca-a42c-f169a59f3ae4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-tg4dz"
Sep 09 03:58:29.257220 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.257193    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fc4t4\" (UniqueName: \"kubernetes.io/projected/d55cca64-4230-4d1b-9e52-0d7379772d4b-kube-api-access-fc4t4\") pod \"volume-data-source-validator-695c895bd6-pnxkj\" (UID: \"d55cca64-4230-4d1b-9e52-0d7379772d4b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-pnxkj"
Sep 09 03:58:29.262042 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.262002    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fftph\" (UniqueName: \"kubernetes.io/projected/6964ed13-fd43-48b3-aff1-8e37bcd05287-kube-api-access-fftph\") pod \"kube-storage-version-migrator-operator-8d7dd4675-4xg2m\" (UID: \"6964ed13-fd43-48b3-aff1-8e37bcd05287\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-4xg2m"
Sep 09 03:58:29.262410 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.262388    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gd6hf\" (UniqueName: \"kubernetes.io/projected/7816cd49-e9c2-4f7b-b8ec-4e5711b675a5-kube-api-access-gd6hf\") pod \"console-operator-85b454874b-gj7h7\" (UID: \"7816cd49-e9c2-4f7b-b8ec-4e5711b675a5\") " pod="openshift-console-operator/console-operator-85b454874b-gj7h7"
Sep 09 03:58:29.263088 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.263068    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8s4pm\" (UniqueName: \"kubernetes.io/projected/1318985b-585a-4a94-a73e-c5856d2d2075-kube-api-access-8s4pm\") pod \"service-ca-operator-859f94c4b-zqtzf\" (UID: \"1318985b-585a-4a94-a73e-c5856d2d2075\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zqtzf"
Sep 09 03:58:29.263513 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.263496    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vs992\" (UniqueName: \"kubernetes.io/projected/55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7-kube-api-access-vs992\") pod \"insights-operator-76bcd6ccf5-57s54\" (UID: \"55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7\") " pod="openshift-insights/insights-operator-76bcd6ccf5-57s54"
Sep 09 03:58:29.310259 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.310227    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zqtzf"
Sep 09 03:58:29.327363 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.327334    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-57s54"
Sep 09 03:58:29.342463 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.342441    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-4xg2m"
Sep 09 03:58:29.349905 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.349878    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/80439eab-81e8-4a3b-a2de-44c45f992052-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7f674d7db9-jhbnd\" (UID: \"80439eab-81e8-4a3b-a2de-44c45f992052\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f674d7db9-jhbnd"
Sep 09 03:58:29.350025 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.349922    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/80439eab-81e8-4a3b-a2de-44c45f992052-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7f674d7db9-jhbnd\" (UID: \"80439eab-81e8-4a3b-a2de-44c45f992052\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f674d7db9-jhbnd"
Sep 09 03:58:29.350025 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.349948    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/80439eab-81e8-4a3b-a2de-44c45f992052-ca\") pod \"cluster-proxy-proxy-agent-7f674d7db9-jhbnd\" (UID: \"80439eab-81e8-4a3b-a2de-44c45f992052\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f674d7db9-jhbnd"
Sep 09 03:58:29.350025 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.349975    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84bvb\" (UniqueName: \"kubernetes.io/projected/0b01bf6d-392c-4015-b166-11deff0ffbaa-kube-api-access-84bvb\") pod \"network-check-target-bvnr6\" (UID: \"0b01bf6d-392c-4015-b166-11deff0ffbaa\") " pod="openshift-network-diagnostics/network-check-target-bvnr6"
Sep 09 03:58:29.350025 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.349999    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa51a428-9bbd-492a-98ae-e4bd03fab5ce-cert\") pod \"ingress-canary-pwxcn\" (UID: \"aa51a428-9bbd-492a-98ae-e4bd03fab5ce\") " pod="openshift-ingress-canary/ingress-canary-pwxcn"
Sep 09 03:58:29.350240 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.350035    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5lb6\" (UniqueName: \"kubernetes.io/projected/b73c2563-ee66-481a-ba6e-122139fb3abc-kube-api-access-m5lb6\") pod \"dns-default-9d9xs\" (UID: \"b73c2563-ee66-481a-ba6e-122139fb3abc\") " pod="openshift-dns/dns-default-9d9xs"
Sep 09 03:58:29.350240 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.350058    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkgzw\" (UniqueName: \"kubernetes.io/projected/39402424-bac6-470f-ba07-996f9acd8313-kube-api-access-bkgzw\") pod \"network-check-source-685456655f-kkw2d\" (UID: \"39402424-bac6-470f-ba07-996f9acd8313\") " pod="openshift-network-diagnostics/network-check-source-685456655f-kkw2d"
Sep 09 03:58:29.350240 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.350079    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b73c2563-ee66-481a-ba6e-122139fb3abc-metrics-tls\") pod \"dns-default-9d9xs\" (UID: \"b73c2563-ee66-481a-ba6e-122139fb3abc\") " pod="openshift-dns/dns-default-9d9xs"
Sep 09 03:58:29.350240 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.350096    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/85fe16e4-d362-49b1-b379-451d29551d1b-original-pull-secret\") pod \"global-pull-secret-syncer-78hdg\" (UID: \"85fe16e4-d362-49b1-b379-451d29551d1b\") " pod="kube-system/global-pull-secret-syncer-78hdg"
Sep 09 03:58:29.350240 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.350140    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpbzb\" (UniqueName: \"kubernetes.io/projected/e1a0b3b4-b53f-4870-8849-6ee3e393374f-kube-api-access-lpbzb\") pod \"klusterlet-addon-workmgr-865d5755c7-2v95v\" (UID: \"e1a0b3b4-b53f-4870-8849-6ee3e393374f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-865d5755c7-2v95v"
Sep 09 03:58:29.350240 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.350158    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vzbg\" (UniqueName: \"kubernetes.io/projected/9cf8ade1-fc61-4570-9a58-ab1b4c4bb291-kube-api-access-6vzbg\") pod \"managed-serviceaccount-addon-agent-65684fbc5d-f8nqq\" (UID: \"9cf8ade1-fc61-4570-9a58-ab1b4c4bb291\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65684fbc5d-f8nqq"
Sep 09 03:58:29.350240 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.350173    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b73c2563-ee66-481a-ba6e-122139fb3abc-config-volume\") pod \"dns-default-9d9xs\" (UID: \"b73c2563-ee66-481a-ba6e-122139fb3abc\") " pod="openshift-dns/dns-default-9d9xs"
Sep 09 03:58:29.350240 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.350203    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99dss\" (UniqueName: \"kubernetes.io/projected/aa51a428-9bbd-492a-98ae-e4bd03fab5ce-kube-api-access-99dss\") pod \"ingress-canary-pwxcn\" (UID: \"aa51a428-9bbd-492a-98ae-e4bd03fab5ce\") " pod="openshift-ingress-canary/ingress-canary-pwxcn"
Sep 09 03:58:29.350240 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.350236    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58280195-fa67-47e6-af03-e39d2bab379b-metrics-certs\") pod \"network-metrics-daemon-9lrvt\" (UID: \"58280195-fa67-47e6-af03-e39d2bab379b\") " pod="openshift-multus/network-metrics-daemon-9lrvt"
Sep 09 03:58:29.350644 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.350266    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9cf8ade1-fc61-4570-9a58-ab1b4c4bb291-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-65684fbc5d-f8nqq\" (UID: \"9cf8ade1-fc61-4570-9a58-ab1b4c4bb291\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65684fbc5d-f8nqq"
Sep 09 03:58:29.350644 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.350310    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b73c2563-ee66-481a-ba6e-122139fb3abc-tmp-dir\") pod \"dns-default-9d9xs\" (UID: \"b73c2563-ee66-481a-ba6e-122139fb3abc\") " pod="openshift-dns/dns-default-9d9xs"
Sep 09 03:58:29.350644 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.350330    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrgz9\" (UniqueName: \"kubernetes.io/projected/80439eab-81e8-4a3b-a2de-44c45f992052-kube-api-access-rrgz9\") pod \"cluster-proxy-proxy-agent-7f674d7db9-jhbnd\" (UID: \"80439eab-81e8-4a3b-a2de-44c45f992052\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f674d7db9-jhbnd"
Sep 09 03:58:29.350644 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.350391    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/80439eab-81e8-4a3b-a2de-44c45f992052-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7f674d7db9-jhbnd\" (UID: \"80439eab-81e8-4a3b-a2de-44c45f992052\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f674d7db9-jhbnd"
Sep 09 03:58:29.350644 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.350420    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/80439eab-81e8-4a3b-a2de-44c45f992052-hub\") pod \"cluster-proxy-proxy-agent-7f674d7db9-jhbnd\" (UID: \"80439eab-81e8-4a3b-a2de-44c45f992052\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f674d7db9-jhbnd"
Sep 09 03:58:29.350644 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.350444    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1a0b3b4-b53f-4870-8849-6ee3e393374f-tmp\") pod \"klusterlet-addon-workmgr-865d5755c7-2v95v\" (UID: \"e1a0b3b4-b53f-4870-8849-6ee3e393374f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-865d5755c7-2v95v"
Sep 09 03:58:29.350644 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.350475    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e1a0b3b4-b53f-4870-8849-6ee3e393374f-klusterlet-config\") pod \"klusterlet-addon-workmgr-865d5755c7-2v95v\" (UID: \"e1a0b3b4-b53f-4870-8849-6ee3e393374f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-865d5755c7-2v95v"
Sep 09 03:58:29.350644 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.350603    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/80439eab-81e8-4a3b-a2de-44c45f992052-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7f674d7db9-jhbnd\" (UID: \"80439eab-81e8-4a3b-a2de-44c45f992052\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f674d7db9-jhbnd"
Sep 09 03:58:29.351037 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:29.350710    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:58:29.351037 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:29.350778    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58280195-fa67-47e6-af03-e39d2bab379b-metrics-certs podName:58280195-fa67-47e6-af03-e39d2bab379b nodeName:}" failed. No retries permitted until 2026-09-09 03:59:01.350760224 +0000 UTC m=+66.269931116 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58280195-fa67-47e6-af03-e39d2bab379b-metrics-certs") pod "network-metrics-daemon-9lrvt" (UID: "58280195-fa67-47e6-af03-e39d2bab379b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 03:58:29.351037 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.350786    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b73c2563-ee66-481a-ba6e-122139fb3abc-tmp-dir\") pod \"dns-default-9d9xs\" (UID: \"b73c2563-ee66-481a-ba6e-122139fb3abc\") " pod="openshift-dns/dns-default-9d9xs"
Sep 09 03:58:29.351246 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.351089    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b73c2563-ee66-481a-ba6e-122139fb3abc-config-volume\") pod \"dns-default-9d9xs\" (UID: \"b73c2563-ee66-481a-ba6e-122139fb3abc\") " pod="openshift-dns/dns-default-9d9xs"
Sep 09 03:58:29.351246 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:29.351208    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 03:58:29.351345 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:29.351267    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b73c2563-ee66-481a-ba6e-122139fb3abc-metrics-tls podName:b73c2563-ee66-481a-ba6e-122139fb3abc nodeName:}" failed. No retries permitted until 2026-09-09 03:58:29.85125131 +0000 UTC m=+34.770422211 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b73c2563-ee66-481a-ba6e-122139fb3abc-metrics-tls") pod "dns-default-9d9xs" (UID: "b73c2563-ee66-481a-ba6e-122139fb3abc") : secret "dns-default-metrics-tls" not found
Sep 09 03:58:29.352093 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:29.351990    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 03:58:29.352093 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:29.352055    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85fe16e4-d362-49b1-b379-451d29551d1b-original-pull-secret podName:85fe16e4-d362-49b1-b379-451d29551d1b nodeName:}" failed. No retries permitted until 2026-09-09 03:59:01.352039152 +0000 UTC m=+66.271210053 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/85fe16e4-d362-49b1-b379-451d29551d1b-original-pull-secret") pod "global-pull-secret-syncer-78hdg" (UID: "85fe16e4-d362-49b1-b379-451d29551d1b") : object "kube-system"/"original-pull-secret" not registered
Sep 09 03:58:29.352261 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.352176    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1a0b3b4-b53f-4870-8849-6ee3e393374f-tmp\") pod \"klusterlet-addon-workmgr-865d5755c7-2v95v\" (UID: \"e1a0b3b4-b53f-4870-8849-6ee3e393374f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-865d5755c7-2v95v"
Sep 09 03:58:29.353242 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.353217    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/80439eab-81e8-4a3b-a2de-44c45f992052-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7f674d7db9-jhbnd\" (UID: \"80439eab-81e8-4a3b-a2de-44c45f992052\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f674d7db9-jhbnd"
Sep 09 03:58:29.353323 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.353223    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e1a0b3b4-b53f-4870-8849-6ee3e393374f-klusterlet-config\") pod \"klusterlet-addon-workmgr-865d5755c7-2v95v\" (UID: \"e1a0b3b4-b53f-4870-8849-6ee3e393374f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-865d5755c7-2v95v"
Sep 09 03:58:29.353520 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.353496    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84bvb\" (UniqueName: \"kubernetes.io/projected/0b01bf6d-392c-4015-b166-11deff0ffbaa-kube-api-access-84bvb\") pod \"network-check-target-bvnr6\" (UID: \"0b01bf6d-392c-4015-b166-11deff0ffbaa\") " pod="openshift-network-diagnostics/network-check-target-bvnr6"
Sep 09 03:58:29.353790 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.353764    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa51a428-9bbd-492a-98ae-e4bd03fab5ce-cert\") pod \"ingress-canary-pwxcn\" (UID: \"aa51a428-9bbd-492a-98ae-e4bd03fab5ce\") " pod="openshift-ingress-canary/ingress-canary-pwxcn"
Sep 09 03:58:29.354346 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.354326    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9cf8ade1-fc61-4570-9a58-ab1b4c4bb291-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-65684fbc5d-f8nqq\" (UID: \"9cf8ade1-fc61-4570-9a58-ab1b4c4bb291\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65684fbc5d-f8nqq"
Sep 09 03:58:29.354529 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.354512    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/80439eab-81e8-4a3b-a2de-44c45f992052-ca\") pod \"cluster-proxy-proxy-agent-7f674d7db9-jhbnd\" (UID: \"80439eab-81e8-4a3b-a2de-44c45f992052\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f674d7db9-jhbnd"
Sep 09 03:58:29.354689 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.354667    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/80439eab-81e8-4a3b-a2de-44c45f992052-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7f674d7db9-jhbnd\" (UID: \"80439eab-81e8-4a3b-a2de-44c45f992052\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f674d7db9-jhbnd"
Sep 09 03:58:29.355370 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.355350    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/80439eab-81e8-4a3b-a2de-44c45f992052-hub\") pod \"cluster-proxy-proxy-agent-7f674d7db9-jhbnd\" (UID: \"80439eab-81e8-4a3b-a2de-44c45f992052\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f674d7db9-jhbnd"
Sep 09 03:58:29.361952 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.361803    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vzbg\" (UniqueName: \"kubernetes.io/projected/9cf8ade1-fc61-4570-9a58-ab1b4c4bb291-kube-api-access-6vzbg\") pod \"managed-serviceaccount-addon-agent-65684fbc5d-f8nqq\" (UID: \"9cf8ade1-fc61-4570-9a58-ab1b4c4bb291\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65684fbc5d-f8nqq"
Sep 09 03:58:29.362214 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.362186    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrgz9\" (UniqueName: \"kubernetes.io/projected/80439eab-81e8-4a3b-a2de-44c45f992052-kube-api-access-rrgz9\") pod \"cluster-proxy-proxy-agent-7f674d7db9-jhbnd\" (UID: \"80439eab-81e8-4a3b-a2de-44c45f992052\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f674d7db9-jhbnd"
Sep 09 03:58:29.362819 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.362791    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99dss\" (UniqueName: \"kubernetes.io/projected/aa51a428-9bbd-492a-98ae-e4bd03fab5ce-kube-api-access-99dss\") pod \"ingress-canary-pwxcn\" (UID: \"aa51a428-9bbd-492a-98ae-e4bd03fab5ce\") " pod="openshift-ingress-canary/ingress-canary-pwxcn"
Sep 09 03:58:29.363097 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.363074    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5lb6\" (UniqueName: \"kubernetes.io/projected/b73c2563-ee66-481a-ba6e-122139fb3abc-kube-api-access-m5lb6\") pod \"dns-default-9d9xs\" (UID: \"b73c2563-ee66-481a-ba6e-122139fb3abc\") " pod="openshift-dns/dns-default-9d9xs"
Sep 09 03:58:29.363216 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.363185    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpbzb\" (UniqueName: \"kubernetes.io/projected/e1a0b3b4-b53f-4870-8849-6ee3e393374f-kube-api-access-lpbzb\") pod \"klusterlet-addon-workmgr-865d5755c7-2v95v\" (UID: \"e1a0b3b4-b53f-4870-8849-6ee3e393374f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-865d5755c7-2v95v"
Sep 09 03:58:29.363354 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.363336    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkgzw\" (UniqueName: \"kubernetes.io/projected/39402424-bac6-470f-ba07-996f9acd8313-kube-api-access-bkgzw\") pod \"network-check-source-685456655f-kkw2d\" (UID: \"39402424-bac6-470f-ba07-996f9acd8313\") " pod="openshift-network-diagnostics/network-check-source-685456655f-kkw2d"
Sep 09 03:58:29.369878 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.369860    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-pnxkj"
Sep 09 03:58:29.386668 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.386649    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-gj7h7"
Sep 09 03:58:29.400754 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.400732    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-kkw2d"
Sep 09 03:58:29.427490 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.427471    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65684fbc5d-f8nqq"
Sep 09 03:58:29.436591 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.436572    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-865d5755c7-2v95v"
Sep 09 03:58:29.441995 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.441975    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f674d7db9-jhbnd"
Sep 09 03:58:29.474273 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.474257    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pwxcn"
Sep 09 03:58:29.613407 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.613377    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvnr6"
Sep 09 03:58:29.616220 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.616196    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-slrsq\""
Sep 09 03:58:29.622567 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.622550    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bvnr6"
Sep 09 03:58:29.653575 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.653546    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/54f58d60-214f-4fb6-9ae8-02334c258100-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-ln2j9\" (UID: \"54f58d60-214f-4fb6-9ae8-02334c258100\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-ln2j9"
Sep 09 03:58:29.653703 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.653593    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48e91752-c7e6-4208-85f8-1c8386f0aacc-registry-tls\") pod \"image-registry-6f66c76d89-h8rdt\" (UID: \"48e91752-c7e6-4208-85f8-1c8386f0aacc\") " pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt"
Sep 09 03:58:29.653703 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.653633    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f831999b-7d59-4577-864d-c39b88896d64-service-ca-bundle\") pod \"router-default-6f7c46fdb6-89xrm\" (UID: \"f831999b-7d59-4577-864d-c39b88896d64\") " pod="openshift-ingress/router-default-6f7c46fdb6-89xrm"
Sep 09 03:58:29.653703 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.653692    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f831999b-7d59-4577-864d-c39b88896d64-metrics-certs\") pod \"router-default-6f7c46fdb6-89xrm\" (UID: \"f831999b-7d59-4577-864d-c39b88896d64\") " pod="openshift-ingress/router-default-6f7c46fdb6-89xrm"
Sep 09 03:58:29.653845 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:29.653700    2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 09 03:58:29.653845 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:29.653764    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 03:58:29.653845 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:29.653783    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f66c76d89-h8rdt: secret "image-registry-tls" not found
Sep 09 03:58:29.653845 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:29.653770    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54f58d60-214f-4fb6-9ae8-02334c258100-cluster-monitoring-operator-tls podName:54f58d60-214f-4fb6-9ae8-02334c258100 nodeName:}" failed. No retries permitted until 2026-09-09 03:58:30.653747946 +0000 UTC m=+35.572918836 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/54f58d60-214f-4fb6-9ae8-02334c258100-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-ln2j9" (UID: "54f58d60-214f-4fb6-9ae8-02334c258100") : secret "cluster-monitoring-operator-tls" not found
Sep 09 03:58:29.653845 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:29.653810    2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 09 03:58:29.654057 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:29.653852    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f831999b-7d59-4577-864d-c39b88896d64-service-ca-bundle podName:f831999b-7d59-4577-864d-c39b88896d64 nodeName:}" failed. No retries permitted until 2026-09-09 03:58:30.653821625 +0000 UTC m=+35.572992529 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f831999b-7d59-4577-864d-c39b88896d64-service-ca-bundle") pod "router-default-6f7c46fdb6-89xrm" (UID: "f831999b-7d59-4577-864d-c39b88896d64") : configmap references non-existent config key: service-ca.crt
Sep 09 03:58:29.654057 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:29.653872    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48e91752-c7e6-4208-85f8-1c8386f0aacc-registry-tls podName:48e91752-c7e6-4208-85f8-1c8386f0aacc nodeName:}" failed. No retries permitted until 2026-09-09 03:58:30.653860889 +0000 UTC m=+35.573031778 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/48e91752-c7e6-4208-85f8-1c8386f0aacc-registry-tls") pod "image-registry-6f66c76d89-h8rdt" (UID: "48e91752-c7e6-4208-85f8-1c8386f0aacc") : secret "image-registry-tls" not found
Sep 09 03:58:29.654057 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:29.653889    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f831999b-7d59-4577-864d-c39b88896d64-metrics-certs podName:f831999b-7d59-4577-864d-c39b88896d64 nodeName:}" failed. No retries permitted until 2026-09-09 03:58:30.653880463 +0000 UTC m=+35.573051358 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f831999b-7d59-4577-864d-c39b88896d64-metrics-certs") pod "router-default-6f7c46fdb6-89xrm" (UID: "f831999b-7d59-4577-864d-c39b88896d64") : secret "router-metrics-certs-default" not found
Sep 09 03:58:29.754348 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.754273    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/68bc0dde-1b15-48ca-a42c-f169a59f3ae4-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-tg4dz\" (UID: \"68bc0dde-1b15-48ca-a42c-f169a59f3ae4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-tg4dz"
Sep 09 03:58:29.754510 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:29.754452    2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 03:58:29.754559 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:29.754533    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68bc0dde-1b15-48ca-a42c-f169a59f3ae4-samples-operator-tls podName:68bc0dde-1b15-48ca-a42c-f169a59f3ae4 nodeName:}" failed. No retries permitted until 2026-09-09 03:58:30.754513186 +0000 UTC m=+35.673684086 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/68bc0dde-1b15-48ca-a42c-f169a59f3ae4-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-tg4dz" (UID: "68bc0dde-1b15-48ca-a42c-f169a59f3ae4") : secret "samples-operator-tls" not found
Sep 09 03:58:29.855793 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:29.855757    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b73c2563-ee66-481a-ba6e-122139fb3abc-metrics-tls\") pod \"dns-default-9d9xs\" (UID: \"b73c2563-ee66-481a-ba6e-122139fb3abc\") " pod="openshift-dns/dns-default-9d9xs"
Sep 09 03:58:29.855955 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:29.855921    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 03:58:29.856015 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:29.855992    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b73c2563-ee66-481a-ba6e-122139fb3abc-metrics-tls podName:b73c2563-ee66-481a-ba6e-122139fb3abc nodeName:}" failed. No retries permitted until 2026-09-09 03:58:30.855975344 +0000 UTC m=+35.775146231 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b73c2563-ee66-481a-ba6e-122139fb3abc-metrics-tls") pod "dns-default-9d9xs" (UID: "b73c2563-ee66-481a-ba6e-122139fb3abc") : secret "dns-default-metrics-tls" not found
Sep 09 03:58:30.337554 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:30.337284    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-4xg2m"]
Sep 09 03:58:30.343301 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:58:30.342714    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6964ed13_fd43_48b3_aff1_8e37bcd05287.slice/crio-a80db076523f4b3293c3eb418d28cd93b82861d63e2a90db07e14b8a60fc8a7f WatchSource:0}: Error finding container a80db076523f4b3293c3eb418d28cd93b82861d63e2a90db07e14b8a60fc8a7f: Status 404 returned error can't find the container with id a80db076523f4b3293c3eb418d28cd93b82861d63e2a90db07e14b8a60fc8a7f
Sep 09 03:58:30.345087 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:30.345049    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pwxcn"]
Sep 09 03:58:30.347609 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:30.347570    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-57s54"]
Sep 09 03:58:30.352508 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:30.352490    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-gj7h7"]
Sep 09 03:58:30.354278 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:30.354255    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bvnr6"]
Sep 09 03:58:30.357475 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:30.356684    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-kkw2d"]
Sep 09 03:58:30.365438 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:58:30.365341    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39402424_bac6_470f_ba07_996f9acd8313.slice/crio-d9e0645d56f0009e3ab3d9f4be8dfa190b6e63d78e87ce4d5f00b7aade3840a0 WatchSource:0}: Error finding container d9e0645d56f0009e3ab3d9f4be8dfa190b6e63d78e87ce4d5f00b7aade3840a0: Status 404 returned error can't find the container with id d9e0645d56f0009e3ab3d9f4be8dfa190b6e63d78e87ce4d5f00b7aade3840a0
Sep 09 03:58:30.380081 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:30.379978    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-zqtzf"]
Sep 09 03:58:30.381390 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:30.381323    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-865d5755c7-2v95v"]
Sep 09 03:58:30.385423 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:30.385397    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f674d7db9-jhbnd"]
Sep 09 03:58:30.386738 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:30.386702    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65684fbc5d-f8nqq"]
Sep 09 03:58:30.391429 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:30.391409    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-pnxkj"]
Sep 09 03:58:30.394071 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:58:30.394051    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd55cca64_4230_4d1b_9e52_0d7379772d4b.slice/crio-eca806c9ef002e36795e9677ac588aeb066eb3efc04e191b613df3caedcc6682 WatchSource:0}: Error finding container eca806c9ef002e36795e9677ac588aeb066eb3efc04e191b613df3caedcc6682: Status 404 returned error can't find the container with id eca806c9ef002e36795e9677ac588aeb066eb3efc04e191b613df3caedcc6682
Sep 09 03:58:30.395243 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:58:30.395224    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod80439eab_81e8_4a3b_a2de_44c45f992052.slice/crio-f796c2b09fc496778c82ea83b8d447de3db833919a93ff6f9b66761499edd8bd WatchSource:0}: Error finding container f796c2b09fc496778c82ea83b8d447de3db833919a93ff6f9b66761499edd8bd: Status 404 returned error can't find the container with id f796c2b09fc496778c82ea83b8d447de3db833919a93ff6f9b66761499edd8bd
Sep 09 03:58:30.396752 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:58:30.396641    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9cf8ade1_fc61_4570_9a58_ab1b4c4bb291.slice/crio-f7a708df5810d201a5bc07453e9745a3d1c25856d15360957f0d0b2f8e8f55bb WatchSource:0}: Error finding container f7a708df5810d201a5bc07453e9745a3d1c25856d15360957f0d0b2f8e8f55bb: Status 404 returned error can't find the container with id f7a708df5810d201a5bc07453e9745a3d1c25856d15360957f0d0b2f8e8f55bb
Sep 09 03:58:30.397697 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:58:30.397664    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1318985b_585a_4a94_a73e_c5856d2d2075.slice/crio-65f1947f91baa32839a84ff01947505bb43faae4130d8af5d94b271ce5eb6de5 WatchSource:0}: Error finding container 65f1947f91baa32839a84ff01947505bb43faae4130d8af5d94b271ce5eb6de5: Status 404 returned error can't find the container with id 65f1947f91baa32839a84ff01947505bb43faae4130d8af5d94b271ce5eb6de5
Sep 09 03:58:30.398683 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:58:30.398602    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1a0b3b4_b53f_4870_8849_6ee3e393374f.slice/crio-da9f7fb656dd5e107d02ed7b4d0026f9428f0e39fae796fe3633d2406a1168bc WatchSource:0}: Error finding container da9f7fb656dd5e107d02ed7b4d0026f9428f0e39fae796fe3633d2406a1168bc: Status 404 returned error can't find the container with id da9f7fb656dd5e107d02ed7b4d0026f9428f0e39fae796fe3633d2406a1168bc
Sep 09 03:58:30.400803 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:30.400778    2557 kuberuntime_manager.go:1459] "Unhandled Error" err="container acm-agent start failed in pod klusterlet-addon-workmgr-865d5755c7-2v95v_open-cluster-management-agent-addon(e1a0b3b4-b53f-4870-8849-6ee3e393374f): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Sep 09 03:58:30.400868 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:30.400834    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with ErrImagePull: \"pull QPS exceeded\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-865d5755c7-2v95v" podUID="e1a0b3b4-b53f-4870-8849-6ee3e393374f"
Sep 09 03:58:30.613926 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:30.613898    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78hdg"
Sep 09 03:58:30.614077 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:30.614059    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9lrvt"
Sep 09 03:58:30.616537 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:30.616514    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 09 03:58:30.617180 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:30.616666    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 09 03:58:30.617180 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:30.616871    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-krf5h\""
Sep 09 03:58:30.666263 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:30.666243    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f831999b-7d59-4577-864d-c39b88896d64-metrics-certs\") pod \"router-default-6f7c46fdb6-89xrm\" (UID: \"f831999b-7d59-4577-864d-c39b88896d64\") " pod="openshift-ingress/router-default-6f7c46fdb6-89xrm"
Sep 09 03:58:30.666363 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:30.666287    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/54f58d60-214f-4fb6-9ae8-02334c258100-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-ln2j9\" (UID: \"54f58d60-214f-4fb6-9ae8-02334c258100\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-ln2j9"
Sep 09 03:58:30.666363 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:30.666314    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48e91752-c7e6-4208-85f8-1c8386f0aacc-registry-tls\") pod \"image-registry-6f66c76d89-h8rdt\" (UID: \"48e91752-c7e6-4208-85f8-1c8386f0aacc\") " pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt"
Sep 09 03:58:30.666363 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:30.666342    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f831999b-7d59-4577-864d-c39b88896d64-service-ca-bundle\") pod \"router-default-6f7c46fdb6-89xrm\" (UID: \"f831999b-7d59-4577-864d-c39b88896d64\") " pod="openshift-ingress/router-default-6f7c46fdb6-89xrm"
Sep 09 03:58:30.666496 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:30.666391    2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 09 03:58:30.666496 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:30.666451    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 03:58:30.666496 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:30.666394    2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 09 03:58:30.666496 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:30.666463    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f66c76d89-h8rdt: secret "image-registry-tls" not found
Sep 09 03:58:30.666496 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:30.666456    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54f58d60-214f-4fb6-9ae8-02334c258100-cluster-monitoring-operator-tls podName:54f58d60-214f-4fb6-9ae8-02334c258100 nodeName:}" failed. No retries permitted until 2026-09-09 03:58:32.666437888 +0000 UTC m=+37.585608798 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/54f58d60-214f-4fb6-9ae8-02334c258100-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-ln2j9" (UID: "54f58d60-214f-4fb6-9ae8-02334c258100") : secret "cluster-monitoring-operator-tls" not found
Sep 09 03:58:30.666496 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:30.666493    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f831999b-7d59-4577-864d-c39b88896d64-service-ca-bundle podName:f831999b-7d59-4577-864d-c39b88896d64 nodeName:}" failed. No retries permitted until 2026-09-09 03:58:32.666481081 +0000 UTC m=+37.585651969 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f831999b-7d59-4577-864d-c39b88896d64-service-ca-bundle") pod "router-default-6f7c46fdb6-89xrm" (UID: "f831999b-7d59-4577-864d-c39b88896d64") : configmap references non-existent config key: service-ca.crt
Sep 09 03:58:30.666684 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:30.666505    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f831999b-7d59-4577-864d-c39b88896d64-metrics-certs podName:f831999b-7d59-4577-864d-c39b88896d64 nodeName:}" failed. No retries permitted until 2026-09-09 03:58:32.666497934 +0000 UTC m=+37.585668821 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f831999b-7d59-4577-864d-c39b88896d64-metrics-certs") pod "router-default-6f7c46fdb6-89xrm" (UID: "f831999b-7d59-4577-864d-c39b88896d64") : secret "router-metrics-certs-default" not found
Sep 09 03:58:30.666684 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:30.666515    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48e91752-c7e6-4208-85f8-1c8386f0aacc-registry-tls podName:48e91752-c7e6-4208-85f8-1c8386f0aacc nodeName:}" failed. No retries permitted until 2026-09-09 03:58:32.666509475 +0000 UTC m=+37.585680362 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/48e91752-c7e6-4208-85f8-1c8386f0aacc-registry-tls") pod "image-registry-6f66c76d89-h8rdt" (UID: "48e91752-c7e6-4208-85f8-1c8386f0aacc") : secret "image-registry-tls" not found
Sep 09 03:58:30.767031 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:30.766996    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/68bc0dde-1b15-48ca-a42c-f169a59f3ae4-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-tg4dz\" (UID: \"68bc0dde-1b15-48ca-a42c-f169a59f3ae4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-tg4dz"
Sep 09 03:58:30.767169 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:30.767138    2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 03:58:30.767213 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:30.767196    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68bc0dde-1b15-48ca-a42c-f169a59f3ae4-samples-operator-tls podName:68bc0dde-1b15-48ca-a42c-f169a59f3ae4 nodeName:}" failed. No retries permitted until 2026-09-09 03:58:32.767180647 +0000 UTC m=+37.686351534 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/68bc0dde-1b15-48ca-a42c-f169a59f3ae4-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-tg4dz" (UID: "68bc0dde-1b15-48ca-a42c-f169a59f3ae4") : secret "samples-operator-tls" not found
Sep 09 03:58:30.793639 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:30.793611    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-865d5755c7-2v95v" event={"ID":"e1a0b3b4-b53f-4870-8849-6ee3e393374f","Type":"ContainerStarted","Data":"da9f7fb656dd5e107d02ed7b4d0026f9428f0e39fae796fe3633d2406a1168bc"}
Sep 09 03:58:30.794662 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:30.794628    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.redhat.io/multicluster-engine/multicloud-manager-rhel9@sha256:1c1d101eca90478fef96f30032ecf3baf4b7980dd83856f8c6bc2763329ac70b\\\": ErrImagePull: pull QPS exceeded\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-865d5755c7-2v95v" podUID="e1a0b3b4-b53f-4870-8849-6ee3e393374f"
Sep 09 03:58:30.794762 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:30.794744    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pwxcn" event={"ID":"aa51a428-9bbd-492a-98ae-e4bd03fab5ce","Type":"ContainerStarted","Data":"63259e35d9ae9862e694643d9f35cd8b8b76738ee37573e1cde6a1e54c1e74b9"}
Sep 09 03:58:30.795857 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:30.795817    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-gj7h7" event={"ID":"7816cd49-e9c2-4f7b-b8ec-4e5711b675a5","Type":"ContainerStarted","Data":"efe6865b3eb8c751bf7eab3a9e905eeac835a0d8d4e333450288d0e9ce99f9b8"}
Sep 09 03:58:30.796940 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:30.796921    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bvnr6" event={"ID":"0b01bf6d-392c-4015-b166-11deff0ffbaa","Type":"ContainerStarted","Data":"0c2836bedbbb84e0cecf06282c5ef4424f17a497174aad141d6979ed5aab7906"}
Sep 09 03:58:30.797993 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:30.797975    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-4xg2m" event={"ID":"6964ed13-fd43-48b3-aff1-8e37bcd05287","Type":"ContainerStarted","Data":"a80db076523f4b3293c3eb418d28cd93b82861d63e2a90db07e14b8a60fc8a7f"}
Sep 09 03:58:30.799033 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:30.799015    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65684fbc5d-f8nqq" event={"ID":"9cf8ade1-fc61-4570-9a58-ab1b4c4bb291","Type":"ContainerStarted","Data":"f7a708df5810d201a5bc07453e9745a3d1c25856d15360957f0d0b2f8e8f55bb"}
Sep 09 03:58:30.800059 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:30.800040    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-kkw2d" event={"ID":"39402424-bac6-470f-ba07-996f9acd8313","Type":"ContainerStarted","Data":"d9e0645d56f0009e3ab3d9f4be8dfa190b6e63d78e87ce4d5f00b7aade3840a0"}
Sep 09 03:58:30.801156 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:30.801135    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-57s54" event={"ID":"55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7","Type":"ContainerStarted","Data":"864681fccba54b7d74dca9d1bbc6dafe5173fe79306614749faf788edfa1ebe0"}
Sep 09 03:58:30.806061 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:30.806035    2557 generic.go:356] "Generic (PLEG): container finished" podID="f26faf1a-61a1-4ff3-a8fb-5321f27480bc" containerID="202c6242493ca791d48ca48bb03e0c1b7c09ddffbb85811c2a43e81ab00a7f6f" exitCode=0
Sep 09 03:58:30.806154 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:30.806103    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g6ggd" event={"ID":"f26faf1a-61a1-4ff3-a8fb-5321f27480bc","Type":"ContainerDied","Data":"202c6242493ca791d48ca48bb03e0c1b7c09ddffbb85811c2a43e81ab00a7f6f"}
Sep 09 03:58:30.807623 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:30.807599    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-pnxkj" event={"ID":"d55cca64-4230-4d1b-9e52-0d7379772d4b","Type":"ContainerStarted","Data":"eca806c9ef002e36795e9677ac588aeb066eb3efc04e191b613df3caedcc6682"}
Sep 09 03:58:30.808603 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:30.808582    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f674d7db9-jhbnd" event={"ID":"80439eab-81e8-4a3b-a2de-44c45f992052","Type":"ContainerStarted","Data":"f796c2b09fc496778c82ea83b8d447de3db833919a93ff6f9b66761499edd8bd"}
Sep 09 03:58:30.809632 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:30.809612    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zqtzf" event={"ID":"1318985b-585a-4a94-a73e-c5856d2d2075","Type":"ContainerStarted","Data":"65f1947f91baa32839a84ff01947505bb43faae4130d8af5d94b271ce5eb6de5"}
Sep 09 03:58:30.868168 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:30.867617    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b73c2563-ee66-481a-ba6e-122139fb3abc-metrics-tls\") pod \"dns-default-9d9xs\" (UID: \"b73c2563-ee66-481a-ba6e-122139fb3abc\") " pod="openshift-dns/dns-default-9d9xs"
Sep 09 03:58:30.868168 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:30.867763    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 03:58:30.868168 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:30.867831    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b73c2563-ee66-481a-ba6e-122139fb3abc-metrics-tls podName:b73c2563-ee66-481a-ba6e-122139fb3abc nodeName:}" failed. No retries permitted until 2026-09-09 03:58:32.86781208 +0000 UTC m=+37.786982974 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b73c2563-ee66-481a-ba6e-122139fb3abc-metrics-tls") pod "dns-default-9d9xs" (UID: "b73c2563-ee66-481a-ba6e-122139fb3abc") : secret "dns-default-metrics-tls" not found
Sep 09 03:58:31.849832 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:31.849794    2557 generic.go:356] "Generic (PLEG): container finished" podID="f26faf1a-61a1-4ff3-a8fb-5321f27480bc" containerID="c86bc11e48f73ac4ddf2a7d4608d6ae7d3c0f1d18808dc28cc67c9d99c50248c" exitCode=0
Sep 09 03:58:31.851096 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:31.851066    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g6ggd" event={"ID":"f26faf1a-61a1-4ff3-a8fb-5321f27480bc","Type":"ContainerDied","Data":"c86bc11e48f73ac4ddf2a7d4608d6ae7d3c0f1d18808dc28cc67c9d99c50248c"}
Sep 09 03:58:31.853647 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:31.853591    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.redhat.io/multicluster-engine/multicloud-manager-rhel9@sha256:1c1d101eca90478fef96f30032ecf3baf4b7980dd83856f8c6bc2763329ac70b\\\": ErrImagePull: pull QPS exceeded\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-865d5755c7-2v95v" podUID="e1a0b3b4-b53f-4870-8849-6ee3e393374f"
Sep 09 03:58:32.689200 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:32.689161    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f831999b-7d59-4577-864d-c39b88896d64-metrics-certs\") pod \"router-default-6f7c46fdb6-89xrm\" (UID: \"f831999b-7d59-4577-864d-c39b88896d64\") " pod="openshift-ingress/router-default-6f7c46fdb6-89xrm"
Sep 09 03:58:32.689378 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:32.689246    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/54f58d60-214f-4fb6-9ae8-02334c258100-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-ln2j9\" (UID: \"54f58d60-214f-4fb6-9ae8-02334c258100\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-ln2j9"
Sep 09 03:58:32.689378 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:32.689296    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48e91752-c7e6-4208-85f8-1c8386f0aacc-registry-tls\") pod \"image-registry-6f66c76d89-h8rdt\" (UID: \"48e91752-c7e6-4208-85f8-1c8386f0aacc\") " pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt"
Sep 09 03:58:32.689378 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:32.689343    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f831999b-7d59-4577-864d-c39b88896d64-service-ca-bundle\") pod \"router-default-6f7c46fdb6-89xrm\" (UID: \"f831999b-7d59-4577-864d-c39b88896d64\") " pod="openshift-ingress/router-default-6f7c46fdb6-89xrm"
Sep 09 03:58:32.689592 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:32.689509    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f831999b-7d59-4577-864d-c39b88896d64-service-ca-bundle podName:f831999b-7d59-4577-864d-c39b88896d64 nodeName:}" failed. No retries permitted until 2026-09-09 03:58:36.689490573 +0000 UTC m=+41.608661474 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f831999b-7d59-4577-864d-c39b88896d64-service-ca-bundle") pod "router-default-6f7c46fdb6-89xrm" (UID: "f831999b-7d59-4577-864d-c39b88896d64") : configmap references non-existent config key: service-ca.crt
Sep 09 03:58:32.689925 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:32.689906    2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 09 03:58:32.690000 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:32.689962    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f831999b-7d59-4577-864d-c39b88896d64-metrics-certs podName:f831999b-7d59-4577-864d-c39b88896d64 nodeName:}" failed. No retries permitted until 2026-09-09 03:58:36.689946662 +0000 UTC m=+41.609117553 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f831999b-7d59-4577-864d-c39b88896d64-metrics-certs") pod "router-default-6f7c46fdb6-89xrm" (UID: "f831999b-7d59-4577-864d-c39b88896d64") : secret "router-metrics-certs-default" not found
Sep 09 03:58:32.690059 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:32.690013    2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 09 03:58:32.690059 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:32.690044    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54f58d60-214f-4fb6-9ae8-02334c258100-cluster-monitoring-operator-tls podName:54f58d60-214f-4fb6-9ae8-02334c258100 nodeName:}" failed. No retries permitted until 2026-09-09 03:58:36.690034378 +0000 UTC m=+41.609205267 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/54f58d60-214f-4fb6-9ae8-02334c258100-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-ln2j9" (UID: "54f58d60-214f-4fb6-9ae8-02334c258100") : secret "cluster-monitoring-operator-tls" not found
Sep 09 03:58:32.690196 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:32.690098    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 03:58:32.690196 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:32.690124    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f66c76d89-h8rdt: secret "image-registry-tls" not found
Sep 09 03:58:32.690196 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:32.690154    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48e91752-c7e6-4208-85f8-1c8386f0aacc-registry-tls podName:48e91752-c7e6-4208-85f8-1c8386f0aacc nodeName:}" failed. No retries permitted until 2026-09-09 03:58:36.690145122 +0000 UTC m=+41.609316012 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/48e91752-c7e6-4208-85f8-1c8386f0aacc-registry-tls") pod "image-registry-6f66c76d89-h8rdt" (UID: "48e91752-c7e6-4208-85f8-1c8386f0aacc") : secret "image-registry-tls" not found
Sep 09 03:58:32.790567 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:32.790474    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/68bc0dde-1b15-48ca-a42c-f169a59f3ae4-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-tg4dz\" (UID: \"68bc0dde-1b15-48ca-a42c-f169a59f3ae4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-tg4dz"
Sep 09 03:58:32.790766 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:32.790659    2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 03:58:32.790766 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:32.790726    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68bc0dde-1b15-48ca-a42c-f169a59f3ae4-samples-operator-tls podName:68bc0dde-1b15-48ca-a42c-f169a59f3ae4 nodeName:}" failed. No retries permitted until 2026-09-09 03:58:36.790706907 +0000 UTC m=+41.709877817 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/68bc0dde-1b15-48ca-a42c-f169a59f3ae4-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-tg4dz" (UID: "68bc0dde-1b15-48ca-a42c-f169a59f3ae4") : secret "samples-operator-tls" not found
Sep 09 03:58:32.887731 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:32.887021    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-g6ggd" event={"ID":"f26faf1a-61a1-4ff3-a8fb-5321f27480bc","Type":"ContainerStarted","Data":"be2eeeb64cc430539f4e6cb6dd04ae7529c355a9f9681ec70a54d0fbe8e14eac"}
Sep 09 03:58:32.892014 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:32.891981    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b73c2563-ee66-481a-ba6e-122139fb3abc-metrics-tls\") pod \"dns-default-9d9xs\" (UID: \"b73c2563-ee66-481a-ba6e-122139fb3abc\") " pod="openshift-dns/dns-default-9d9xs"
Sep 09 03:58:32.892176 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:32.892160    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 03:58:32.892244 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:32.892224    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b73c2563-ee66-481a-ba6e-122139fb3abc-metrics-tls podName:b73c2563-ee66-481a-ba6e-122139fb3abc nodeName:}" failed. No retries permitted until 2026-09-09 03:58:36.892205224 +0000 UTC m=+41.811376115 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b73c2563-ee66-481a-ba6e-122139fb3abc-metrics-tls") pod "dns-default-9d9xs" (UID: "b73c2563-ee66-481a-ba6e-122139fb3abc") : secret "dns-default-metrics-tls" not found
Sep 09 03:58:32.912940 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:32.912888    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-g6ggd" podStartSLOduration=6.727593698 podStartE2EDuration="36.912867985s" totalImagesPullingTime="30.185274287s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 03:57:56 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:58:32.911768137 +0000 UTC m=+37.830939044" watchObservedRunningTime="2026-09-09 03:58:32.912867985 +0000 UTC m=+37.832038896"
Sep 09 03:58:36.730207 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:36.730170    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f831999b-7d59-4577-864d-c39b88896d64-metrics-certs\") pod \"router-default-6f7c46fdb6-89xrm\" (UID: \"f831999b-7d59-4577-864d-c39b88896d64\") " pod="openshift-ingress/router-default-6f7c46fdb6-89xrm"
Sep 09 03:58:36.730662 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:36.730252    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/54f58d60-214f-4fb6-9ae8-02334c258100-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-ln2j9\" (UID: \"54f58d60-214f-4fb6-9ae8-02334c258100\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-ln2j9"
Sep 09 03:58:36.730662 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:36.730293    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48e91752-c7e6-4208-85f8-1c8386f0aacc-registry-tls\") pod \"image-registry-6f66c76d89-h8rdt\" (UID: \"48e91752-c7e6-4208-85f8-1c8386f0aacc\") " pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt"
Sep 09 03:58:36.730662 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:36.730317    2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 09 03:58:36.730662 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:36.730352    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f831999b-7d59-4577-864d-c39b88896d64-service-ca-bundle\") pod \"router-default-6f7c46fdb6-89xrm\" (UID: \"f831999b-7d59-4577-864d-c39b88896d64\") " pod="openshift-ingress/router-default-6f7c46fdb6-89xrm"
Sep 09 03:58:36.730662 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:36.730404    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f831999b-7d59-4577-864d-c39b88896d64-metrics-certs podName:f831999b-7d59-4577-864d-c39b88896d64 nodeName:}" failed. No retries permitted until 2026-09-09 03:58:44.730382253 +0000 UTC m=+49.649553143 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f831999b-7d59-4577-864d-c39b88896d64-metrics-certs") pod "router-default-6f7c46fdb6-89xrm" (UID: "f831999b-7d59-4577-864d-c39b88896d64") : secret "router-metrics-certs-default" not found
Sep 09 03:58:36.730662 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:36.730420    2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 09 03:58:36.730662 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:36.730449    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 03:58:36.730662 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:36.730470    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f66c76d89-h8rdt: secret "image-registry-tls" not found
Sep 09 03:58:36.730662 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:36.730474    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f831999b-7d59-4577-864d-c39b88896d64-service-ca-bundle podName:f831999b-7d59-4577-864d-c39b88896d64 nodeName:}" failed. No retries permitted until 2026-09-09 03:58:44.73045872 +0000 UTC m=+49.649629627 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f831999b-7d59-4577-864d-c39b88896d64-service-ca-bundle") pod "router-default-6f7c46fdb6-89xrm" (UID: "f831999b-7d59-4577-864d-c39b88896d64") : configmap references non-existent config key: service-ca.crt
Sep 09 03:58:36.730662 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:36.730501    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54f58d60-214f-4fb6-9ae8-02334c258100-cluster-monitoring-operator-tls podName:54f58d60-214f-4fb6-9ae8-02334c258100 nodeName:}" failed. No retries permitted until 2026-09-09 03:58:44.730490899 +0000 UTC m=+49.649661791 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/54f58d60-214f-4fb6-9ae8-02334c258100-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-ln2j9" (UID: "54f58d60-214f-4fb6-9ae8-02334c258100") : secret "cluster-monitoring-operator-tls" not found
Sep 09 03:58:36.730662 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:36.730517    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48e91752-c7e6-4208-85f8-1c8386f0aacc-registry-tls podName:48e91752-c7e6-4208-85f8-1c8386f0aacc nodeName:}" failed. No retries permitted until 2026-09-09 03:58:44.730508744 +0000 UTC m=+49.649679635 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/48e91752-c7e6-4208-85f8-1c8386f0aacc-registry-tls") pod "image-registry-6f66c76d89-h8rdt" (UID: "48e91752-c7e6-4208-85f8-1c8386f0aacc") : secret "image-registry-tls" not found
Sep 09 03:58:36.831629 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:36.831586    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/68bc0dde-1b15-48ca-a42c-f169a59f3ae4-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-tg4dz\" (UID: \"68bc0dde-1b15-48ca-a42c-f169a59f3ae4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-tg4dz"
Sep 09 03:58:36.831796 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:36.831710    2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 03:58:36.831796 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:36.831789    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68bc0dde-1b15-48ca-a42c-f169a59f3ae4-samples-operator-tls podName:68bc0dde-1b15-48ca-a42c-f169a59f3ae4 nodeName:}" failed. No retries permitted until 2026-09-09 03:58:44.831766802 +0000 UTC m=+49.750937717 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/68bc0dde-1b15-48ca-a42c-f169a59f3ae4-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-tg4dz" (UID: "68bc0dde-1b15-48ca-a42c-f169a59f3ae4") : secret "samples-operator-tls" not found
Sep 09 03:58:36.932939 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:36.932911    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b73c2563-ee66-481a-ba6e-122139fb3abc-metrics-tls\") pod \"dns-default-9d9xs\" (UID: \"b73c2563-ee66-481a-ba6e-122139fb3abc\") " pod="openshift-dns/dns-default-9d9xs"
Sep 09 03:58:36.933161 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:36.933063    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 03:58:36.933161 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:36.933134    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b73c2563-ee66-481a-ba6e-122139fb3abc-metrics-tls podName:b73c2563-ee66-481a-ba6e-122139fb3abc nodeName:}" failed. No retries permitted until 2026-09-09 03:58:44.933103016 +0000 UTC m=+49.852273913 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b73c2563-ee66-481a-ba6e-122139fb3abc-metrics-tls") pod "dns-default-9d9xs" (UID: "b73c2563-ee66-481a-ba6e-122139fb3abc") : secret "dns-default-metrics-tls" not found
Sep 09 03:58:42.929184 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:42.924409    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pwxcn" event={"ID":"aa51a428-9bbd-492a-98ae-e4bd03fab5ce","Type":"ContainerStarted","Data":"6a9a1079f3d6ba78ed092e2f6632afa39c2d4a39a9db3c6696c901929b6a0775"}
Sep 09 03:58:42.929184 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:42.928348    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gj7h7_7816cd49-e9c2-4f7b-b8ec-4e5711b675a5/console-operator/0.log"
Sep 09 03:58:42.929184 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:42.928384    2557 generic.go:356] "Generic (PLEG): container finished" podID="7816cd49-e9c2-4f7b-b8ec-4e5711b675a5" containerID="30b15ca02dce3b7ba44cca43b474164fe05c877e225935d383b8fa6223a59f1e" exitCode=255
Sep 09 03:58:42.929184 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:42.928455    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-gj7h7" event={"ID":"7816cd49-e9c2-4f7b-b8ec-4e5711b675a5","Type":"ContainerDied","Data":"30b15ca02dce3b7ba44cca43b474164fe05c877e225935d383b8fa6223a59f1e"}
Sep 09 03:58:42.929184 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:42.928783    2557 scope.go:117] "RemoveContainer" containerID="30b15ca02dce3b7ba44cca43b474164fe05c877e225935d383b8fa6223a59f1e"
Sep 09 03:58:42.934234 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:42.933923    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bvnr6" event={"ID":"0b01bf6d-392c-4015-b166-11deff0ffbaa","Type":"ContainerStarted","Data":"493cacc1e361a90b9f0bd0687c0abb3656bfcb66f79fa8fe2d62d2ef04efc62f"}
Sep 09 03:58:42.936814 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:42.935702    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-4xg2m" event={"ID":"6964ed13-fd43-48b3-aff1-8e37bcd05287","Type":"ContainerStarted","Data":"89c877aba0e71455048846d5a639aa1fcad2bed66108d7ec1026ce8eb97143a8"}
Sep 09 03:58:42.940294 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:42.940227    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65684fbc5d-f8nqq" event={"ID":"9cf8ade1-fc61-4570-9a58-ab1b4c4bb291","Type":"ContainerStarted","Data":"1e4ec448a30093a9df3e63e1deb836f45ca56a74f4697b40cbc2c3f637229236"}
Sep 09 03:58:42.941945 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:42.941885    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-kkw2d" event={"ID":"39402424-bac6-470f-ba07-996f9acd8313","Type":"ContainerStarted","Data":"e4393844ef3dcdd44670af356e28d8c61236a3cbe7693300e3a5ae52ad67e30e"}
Sep 09 03:58:42.943638 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:42.943583    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-57s54" event={"ID":"55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7","Type":"ContainerStarted","Data":"546c750fe56629278f8a0d63d437bcf748ab1e4a28e414532836e20609fe29d1"}
Sep 09 03:58:42.945304 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:42.945247    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-pnxkj" event={"ID":"d55cca64-4230-4d1b-9e52-0d7379772d4b","Type":"ContainerStarted","Data":"5fa23f93a2a7d0de7c415e164746c4f08856059557a86ef201ac5f2f1a7d0c0a"}
Sep 09 03:58:42.948065 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:42.948020    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-pwxcn" podStartSLOduration=2.829583104 podStartE2EDuration="14.948003968s" totalImagesPullingTime="12.118420864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:58:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:58:42.946432418 +0000 UTC m=+47.865603328" watchObservedRunningTime="2026-09-09 03:58:42.948003968 +0000 UTC m=+47.867174877"
Sep 09 03:58:42.950967 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:42.949149    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-bvnr6"
Sep 09 03:58:42.956129 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:42.956046    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f674d7db9-jhbnd" event={"ID":"80439eab-81e8-4a3b-a2de-44c45f992052","Type":"ContainerStarted","Data":"4fe35f0a16614f4a5f064a607f66ca76c14fb521af86baeb2285fbb9d35e1de8"}
Sep 09 03:58:42.956129 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:42.956078    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f674d7db9-jhbnd" event={"ID":"80439eab-81e8-4a3b-a2de-44c45f992052","Type":"ContainerStarted","Data":"460133ce1b8589dca3f01d32f0807759734f39033605a9458afbe4e643bb0c4c"}
Sep 09 03:58:42.968497 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:42.967994    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zqtzf" event={"ID":"1318985b-585a-4a94-a73e-c5856d2d2075","Type":"ContainerStarted","Data":"5f47b222d08d3dfbed959cf8c5fcb6aea0cce6d3e9dcb2dc3ac5bee554bb9bc8"}
Sep 09 03:58:43.009146 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:43.008640    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-57s54" podStartSLOduration=4.885949171 podStartE2EDuration="17.008618296s" totalImagesPullingTime="12.122669125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:58:42.989060567 +0000 UTC m=+47.908231477" watchObservedRunningTime="2026-09-09 03:58:43.008618296 +0000 UTC m=+47.927789205"
Sep 09 03:58:43.033133 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:43.029908    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-4xg2m" podStartSLOduration=4.903632604 podStartE2EDuration="17.029886259s" totalImagesPullingTime="12.126253655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:58:43.010059941 +0000 UTC m=+47.929230843" watchObservedRunningTime="2026-09-09 03:58:43.029886259 +0000 UTC m=+47.949057168"
Sep 09 03:58:43.050003 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:43.049954    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65684fbc5d-f8nqq" podStartSLOduration=22.963833196 podStartE2EDuration="35.049933545s" totalImagesPullingTime="12.086100349s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:58:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:58:43.047843354 +0000 UTC m=+47.967014261" watchObservedRunningTime="2026-09-09 03:58:43.049933545 +0000 UTC m=+47.969104447"
Sep 09 03:58:43.051025 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:43.050976    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-pnxkj" podStartSLOduration=4.9714984829999995 podStartE2EDuration="17.050960956s" totalImagesPullingTime="12.079462473s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:58:43.031915502 +0000 UTC m=+47.951086427" watchObservedRunningTime="2026-09-09 03:58:43.050960956 +0000 UTC m=+47.970131865"
Sep 09 03:58:43.068645 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:43.068597    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-bvnr6" podStartSLOduration=34.972299292 podStartE2EDuration="47.06858011s" totalImagesPullingTime="12.096280818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:57:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:58:43.06744348 +0000 UTC m=+47.986614390" watchObservedRunningTime="2026-09-09 03:58:43.06858011 +0000 UTC m=+47.987751020"
Sep 09 03:58:43.089512 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:43.087996    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-kkw2d" podStartSLOduration=4.977764448 podStartE2EDuration="17.087979216s" totalImagesPullingTime="12.110214768s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:58:43.087412165 +0000 UTC m=+48.006583076" watchObservedRunningTime="2026-09-09 03:58:43.087979216 +0000 UTC m=+48.007150131"
Sep 09 03:58:43.976316 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:43.976275    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f674d7db9-jhbnd" event={"ID":"80439eab-81e8-4a3b-a2de-44c45f992052","Type":"ContainerStarted","Data":"97b8fe42f1c788325ec77d4ff1fb833e47aea413653c48e1610e725e2e08acda"}
Sep 09 03:58:43.978180 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:43.978157    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gj7h7_7816cd49-e9c2-4f7b-b8ec-4e5711b675a5/console-operator/1.log"
Sep 09 03:58:43.978620 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:43.978603    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gj7h7_7816cd49-e9c2-4f7b-b8ec-4e5711b675a5/console-operator/0.log"
Sep 09 03:58:43.978698 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:43.978642    2557 generic.go:356] "Generic (PLEG): container finished" podID="7816cd49-e9c2-4f7b-b8ec-4e5711b675a5" containerID="d9631056996bb24a1237624ae0c24fc157389cb200a56802a21d28e4f4c9cab4" exitCode=255
Sep 09 03:58:43.978773 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:43.978748    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-gj7h7" event={"ID":"7816cd49-e9c2-4f7b-b8ec-4e5711b675a5","Type":"ContainerDied","Data":"d9631056996bb24a1237624ae0c24fc157389cb200a56802a21d28e4f4c9cab4"}
Sep 09 03:58:43.978830 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:43.978817    2557 scope.go:117] "RemoveContainer" containerID="30b15ca02dce3b7ba44cca43b474164fe05c877e225935d383b8fa6223a59f1e"
Sep 09 03:58:43.978939 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:43.978925    2557 scope.go:117] "RemoveContainer" containerID="d9631056996bb24a1237624ae0c24fc157389cb200a56802a21d28e4f4c9cab4"
Sep 09 03:58:43.979124 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:43.979089    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-gj7h7_openshift-console-operator(7816cd49-e9c2-4f7b-b8ec-4e5711b675a5)\"" pod="openshift-console-operator/console-operator-85b454874b-gj7h7" podUID="7816cd49-e9c2-4f7b-b8ec-4e5711b675a5"
Sep 09 03:58:43.999026 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:43.998968    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zqtzf" podStartSLOduration=5.971512918 podStartE2EDuration="17.998948666s" totalImagesPullingTime="12.027435748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:58:43.111842616 +0000 UTC m=+48.031013526" watchObservedRunningTime="2026-09-09 03:58:43.998948666 +0000 UTC m=+48.918119575"
Sep 09 03:58:43.999337 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:43.999311    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f674d7db9-jhbnd" podStartSLOduration=23.919998488 podStartE2EDuration="35.99930148s" totalImagesPullingTime="12.079302992s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:58:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:58:43.996751321 +0000 UTC m=+48.915922230" watchObservedRunningTime="2026-09-09 03:58:43.99930148 +0000 UTC m=+48.918472391"
Sep 09 03:58:44.198288 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:44.198255    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5dtmp"]
Sep 09 03:58:44.230366 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:44.230296    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5dtmp"]
Sep 09 03:58:44.230484 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:44.230433    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5dtmp"
Sep 09 03:58:44.232881 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:44.232855    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 09 03:58:44.233107 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:44.233091    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4kqm2\""
Sep 09 03:58:44.233345 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:44.233322    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 09 03:58:44.309484 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:44.309458    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c7e4f11-324d-4e2f-9d4b-97a2186beabc-data-volume\") pod \"insights-runtime-extractor-5dtmp\" (UID: \"0c7e4f11-324d-4e2f-9d4b-97a2186beabc\") " pod="openshift-insights/insights-runtime-extractor-5dtmp"
Sep 09 03:58:44.309643 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:44.309490    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0c7e4f11-324d-4e2f-9d4b-97a2186beabc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5dtmp\" (UID: \"0c7e4f11-324d-4e2f-9d4b-97a2186beabc\") " pod="openshift-insights/insights-runtime-extractor-5dtmp"
Sep 09 03:58:44.309643 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:44.309572    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p22ql\" (UniqueName: \"kubernetes.io/projected/0c7e4f11-324d-4e2f-9d4b-97a2186beabc-kube-api-access-p22ql\") pod \"insights-runtime-extractor-5dtmp\" (UID: \"0c7e4f11-324d-4e2f-9d4b-97a2186beabc\") " pod="openshift-insights/insights-runtime-extractor-5dtmp"
Sep 09 03:58:44.309754 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:44.309650    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0c7e4f11-324d-4e2f-9d4b-97a2186beabc-crio-socket\") pod \"insights-runtime-extractor-5dtmp\" (UID: \"0c7e4f11-324d-4e2f-9d4b-97a2186beabc\") " pod="openshift-insights/insights-runtime-extractor-5dtmp"
Sep 09 03:58:44.309754 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:44.309686    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0c7e4f11-324d-4e2f-9d4b-97a2186beabc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5dtmp\" (UID: \"0c7e4f11-324d-4e2f-9d4b-97a2186beabc\") " pod="openshift-insights/insights-runtime-extractor-5dtmp"
Sep 09 03:58:44.410942 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:44.410908    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0c7e4f11-324d-4e2f-9d4b-97a2186beabc-crio-socket\") pod \"insights-runtime-extractor-5dtmp\" (UID: \"0c7e4f11-324d-4e2f-9d4b-97a2186beabc\") " pod="openshift-insights/insights-runtime-extractor-5dtmp"
Sep 09 03:58:44.411108 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:44.410961    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0c7e4f11-324d-4e2f-9d4b-97a2186beabc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5dtmp\" (UID: \"0c7e4f11-324d-4e2f-9d4b-97a2186beabc\") " pod="openshift-insights/insights-runtime-extractor-5dtmp"
Sep 09 03:58:44.411108 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:44.410995    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c7e4f11-324d-4e2f-9d4b-97a2186beabc-data-volume\") pod \"insights-runtime-extractor-5dtmp\" (UID: \"0c7e4f11-324d-4e2f-9d4b-97a2186beabc\") " pod="openshift-insights/insights-runtime-extractor-5dtmp"
Sep 09 03:58:44.411108 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:44.411011    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0c7e4f11-324d-4e2f-9d4b-97a2186beabc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5dtmp\" (UID: \"0c7e4f11-324d-4e2f-9d4b-97a2186beabc\") " pod="openshift-insights/insights-runtime-extractor-5dtmp"
Sep 09 03:58:44.411108 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:44.411047    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0c7e4f11-324d-4e2f-9d4b-97a2186beabc-crio-socket\") pod \"insights-runtime-extractor-5dtmp\" (UID: \"0c7e4f11-324d-4e2f-9d4b-97a2186beabc\") " pod="openshift-insights/insights-runtime-extractor-5dtmp"
Sep 09 03:58:44.411108 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:44.411055    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p22ql\" (UniqueName: \"kubernetes.io/projected/0c7e4f11-324d-4e2f-9d4b-97a2186beabc-kube-api-access-p22ql\") pod \"insights-runtime-extractor-5dtmp\" (UID: \"0c7e4f11-324d-4e2f-9d4b-97a2186beabc\") " pod="openshift-insights/insights-runtime-extractor-5dtmp"
Sep 09 03:58:44.411389 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:44.411189    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 03:58:44.411389 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:44.411257    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c7e4f11-324d-4e2f-9d4b-97a2186beabc-insights-runtime-extractor-tls podName:0c7e4f11-324d-4e2f-9d4b-97a2186beabc nodeName:}" failed. No retries permitted until 2026-09-09 03:58:44.911238183 +0000 UTC m=+49.830409073 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0c7e4f11-324d-4e2f-9d4b-97a2186beabc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5dtmp" (UID: "0c7e4f11-324d-4e2f-9d4b-97a2186beabc") : secret "insights-runtime-extractor-tls" not found
Sep 09 03:58:44.411389 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:44.411189    2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 03:58:44.411389 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:44.411341    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0c7e4f11-324d-4e2f-9d4b-97a2186beabc-kube-rbac-proxy-cm podName:0c7e4f11-324d-4e2f-9d4b-97a2186beabc nodeName:}" failed. No retries permitted until 2026-09-09 03:58:44.911323113 +0000 UTC m=+49.830494019 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0c7e4f11-324d-4e2f-9d4b-97a2186beabc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5dtmp" (UID: "0c7e4f11-324d-4e2f-9d4b-97a2186beabc") : configmap "kube-rbac-proxy" not found
Sep 09 03:58:44.411575 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:44.411496    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c7e4f11-324d-4e2f-9d4b-97a2186beabc-data-volume\") pod \"insights-runtime-extractor-5dtmp\" (UID: \"0c7e4f11-324d-4e2f-9d4b-97a2186beabc\") " pod="openshift-insights/insights-runtime-extractor-5dtmp"
Sep 09 03:58:44.423039 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:44.423017    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p22ql\" (UniqueName: \"kubernetes.io/projected/0c7e4f11-324d-4e2f-9d4b-97a2186beabc-kube-api-access-p22ql\") pod \"insights-runtime-extractor-5dtmp\" (UID: \"0c7e4f11-324d-4e2f-9d4b-97a2186beabc\") " pod="openshift-insights/insights-runtime-extractor-5dtmp"
Sep 09 03:58:44.813433 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:44.813398    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48e91752-c7e6-4208-85f8-1c8386f0aacc-registry-tls\") pod \"image-registry-6f66c76d89-h8rdt\" (UID: \"48e91752-c7e6-4208-85f8-1c8386f0aacc\") " pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt"
Sep 09 03:58:44.813618 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:44.813459    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f831999b-7d59-4577-864d-c39b88896d64-service-ca-bundle\") pod \"router-default-6f7c46fdb6-89xrm\" (UID: \"f831999b-7d59-4577-864d-c39b88896d64\") " pod="openshift-ingress/router-default-6f7c46fdb6-89xrm"
Sep 09 03:58:44.813618 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:44.813542    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f831999b-7d59-4577-864d-c39b88896d64-metrics-certs\") pod \"router-default-6f7c46fdb6-89xrm\" (UID: \"f831999b-7d59-4577-864d-c39b88896d64\") " pod="openshift-ingress/router-default-6f7c46fdb6-89xrm"
Sep 09 03:58:44.813618 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:44.813545    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 03:58:44.813618 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:44.813562    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f66c76d89-h8rdt: secret "image-registry-tls" not found
Sep 09 03:58:44.813618 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:44.813569    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f831999b-7d59-4577-864d-c39b88896d64-service-ca-bundle podName:f831999b-7d59-4577-864d-c39b88896d64 nodeName:}" failed. No retries permitted until 2026-09-09 03:59:00.813554562 +0000 UTC m=+65.732725463 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f831999b-7d59-4577-864d-c39b88896d64-service-ca-bundle") pod "router-default-6f7c46fdb6-89xrm" (UID: "f831999b-7d59-4577-864d-c39b88896d64") : configmap references non-existent config key: service-ca.crt
Sep 09 03:58:44.813618 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:44.813613    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48e91752-c7e6-4208-85f8-1c8386f0aacc-registry-tls podName:48e91752-c7e6-4208-85f8-1c8386f0aacc nodeName:}" failed. No retries permitted until 2026-09-09 03:59:00.813598167 +0000 UTC m=+65.732769055 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/48e91752-c7e6-4208-85f8-1c8386f0aacc-registry-tls") pod "image-registry-6f66c76d89-h8rdt" (UID: "48e91752-c7e6-4208-85f8-1c8386f0aacc") : secret "image-registry-tls" not found
Sep 09 03:58:44.813618 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:44.813618    2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 09 03:58:44.813890 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:44.813656    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/54f58d60-214f-4fb6-9ae8-02334c258100-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-ln2j9\" (UID: \"54f58d60-214f-4fb6-9ae8-02334c258100\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-ln2j9"
Sep 09 03:58:44.813890 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:44.813664    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f831999b-7d59-4577-864d-c39b88896d64-metrics-certs podName:f831999b-7d59-4577-864d-c39b88896d64 nodeName:}" failed. No retries permitted until 2026-09-09 03:59:00.81365102 +0000 UTC m=+65.732821920 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f831999b-7d59-4577-864d-c39b88896d64-metrics-certs") pod "router-default-6f7c46fdb6-89xrm" (UID: "f831999b-7d59-4577-864d-c39b88896d64") : secret "router-metrics-certs-default" not found
Sep 09 03:58:44.813890 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:44.813700    2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 09 03:58:44.813890 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:44.813724    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54f58d60-214f-4fb6-9ae8-02334c258100-cluster-monitoring-operator-tls podName:54f58d60-214f-4fb6-9ae8-02334c258100 nodeName:}" failed. No retries permitted until 2026-09-09 03:59:00.813715452 +0000 UTC m=+65.732886339 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/54f58d60-214f-4fb6-9ae8-02334c258100-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-ln2j9" (UID: "54f58d60-214f-4fb6-9ae8-02334c258100") : secret "cluster-monitoring-operator-tls" not found
Sep 09 03:58:44.914662 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:44.914632    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0c7e4f11-324d-4e2f-9d4b-97a2186beabc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5dtmp\" (UID: \"0c7e4f11-324d-4e2f-9d4b-97a2186beabc\") " pod="openshift-insights/insights-runtime-extractor-5dtmp"
Sep 09 03:58:44.914828 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:44.914687    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0c7e4f11-324d-4e2f-9d4b-97a2186beabc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5dtmp\" (UID: \"0c7e4f11-324d-4e2f-9d4b-97a2186beabc\") " pod="openshift-insights/insights-runtime-extractor-5dtmp"
Sep 09 03:58:44.914828 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:44.914769    2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 03:58:44.914828 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:44.914800    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/68bc0dde-1b15-48ca-a42c-f169a59f3ae4-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-tg4dz\" (UID: \"68bc0dde-1b15-48ca-a42c-f169a59f3ae4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-tg4dz"
Sep 09 03:58:44.914972 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:44.914835    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0c7e4f11-324d-4e2f-9d4b-97a2186beabc-kube-rbac-proxy-cm podName:0c7e4f11-324d-4e2f-9d4b-97a2186beabc nodeName:}" failed. No retries permitted until 2026-09-09 03:58:45.914817266 +0000 UTC m=+50.833988166 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0c7e4f11-324d-4e2f-9d4b-97a2186beabc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5dtmp" (UID: "0c7e4f11-324d-4e2f-9d4b-97a2186beabc") : configmap "kube-rbac-proxy" not found
Sep 09 03:58:44.914972 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:44.914888    2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 03:58:44.914972 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:44.914911    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 03:58:44.914972 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:44.914940    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68bc0dde-1b15-48ca-a42c-f169a59f3ae4-samples-operator-tls podName:68bc0dde-1b15-48ca-a42c-f169a59f3ae4 nodeName:}" failed. No retries permitted until 2026-09-09 03:59:00.914926601 +0000 UTC m=+65.834097502 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/68bc0dde-1b15-48ca-a42c-f169a59f3ae4-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-tg4dz" (UID: "68bc0dde-1b15-48ca-a42c-f169a59f3ae4") : secret "samples-operator-tls" not found
Sep 09 03:58:44.915129 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:44.914981    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c7e4f11-324d-4e2f-9d4b-97a2186beabc-insights-runtime-extractor-tls podName:0c7e4f11-324d-4e2f-9d4b-97a2186beabc nodeName:}" failed. No retries permitted until 2026-09-09 03:58:45.914974792 +0000 UTC m=+50.834145679 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0c7e4f11-324d-4e2f-9d4b-97a2186beabc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5dtmp" (UID: "0c7e4f11-324d-4e2f-9d4b-97a2186beabc") : secret "insights-runtime-extractor-tls" not found
Sep 09 03:58:44.985012 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:44.984980    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gj7h7_7816cd49-e9c2-4f7b-b8ec-4e5711b675a5/console-operator/1.log"
Sep 09 03:58:44.985447 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:44.985331    2557 scope.go:117] "RemoveContainer" containerID="d9631056996bb24a1237624ae0c24fc157389cb200a56802a21d28e4f4c9cab4"
Sep 09 03:58:44.985529 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:44.985504    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-gj7h7_openshift-console-operator(7816cd49-e9c2-4f7b-b8ec-4e5711b675a5)\"" pod="openshift-console-operator/console-operator-85b454874b-gj7h7" podUID="7816cd49-e9c2-4f7b-b8ec-4e5711b675a5"
Sep 09 03:58:45.015457 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:45.015431    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b73c2563-ee66-481a-ba6e-122139fb3abc-metrics-tls\") pod \"dns-default-9d9xs\" (UID: \"b73c2563-ee66-481a-ba6e-122139fb3abc\") " pod="openshift-dns/dns-default-9d9xs"
Sep 09 03:58:45.015603 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:45.015509    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 03:58:45.015603 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:45.015543    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b73c2563-ee66-481a-ba6e-122139fb3abc-metrics-tls podName:b73c2563-ee66-481a-ba6e-122139fb3abc nodeName:}" failed. No retries permitted until 2026-09-09 03:59:01.015532624 +0000 UTC m=+65.934703511 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b73c2563-ee66-481a-ba6e-122139fb3abc-metrics-tls") pod "dns-default-9d9xs" (UID: "b73c2563-ee66-481a-ba6e-122139fb3abc") : secret "dns-default-metrics-tls" not found
Sep 09 03:58:45.923913 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:45.923873    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0c7e4f11-324d-4e2f-9d4b-97a2186beabc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5dtmp\" (UID: \"0c7e4f11-324d-4e2f-9d4b-97a2186beabc\") " pod="openshift-insights/insights-runtime-extractor-5dtmp"
Sep 09 03:58:45.924061 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:45.923929    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0c7e4f11-324d-4e2f-9d4b-97a2186beabc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5dtmp\" (UID: \"0c7e4f11-324d-4e2f-9d4b-97a2186beabc\") " pod="openshift-insights/insights-runtime-extractor-5dtmp"
Sep 09 03:58:45.924061 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:45.924040    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 03:58:45.924188 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:45.924091    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c7e4f11-324d-4e2f-9d4b-97a2186beabc-insights-runtime-extractor-tls podName:0c7e4f11-324d-4e2f-9d4b-97a2186beabc nodeName:}" failed. No retries permitted until 2026-09-09 03:58:47.924078964 +0000 UTC m=+52.843249851 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0c7e4f11-324d-4e2f-9d4b-97a2186beabc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5dtmp" (UID: "0c7e4f11-324d-4e2f-9d4b-97a2186beabc") : secret "insights-runtime-extractor-tls" not found
Sep 09 03:58:45.924456 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:45.924436    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0c7e4f11-324d-4e2f-9d4b-97a2186beabc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5dtmp\" (UID: \"0c7e4f11-324d-4e2f-9d4b-97a2186beabc\") " pod="openshift-insights/insights-runtime-extractor-5dtmp"
Sep 09 03:58:45.961243 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:45.961220    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hn6zn_34f28b0d-79cb-47df-9cc4-44f781c379fa/dns-node-resolver/0.log"
Sep 09 03:58:46.960687 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:46.960620    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kvnzq_6f01d502-4aa7-42a0-9617-093232b1f6a6/node-ca/0.log"
Sep 09 03:58:47.361382 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:47.361355    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-pwxcn_aa51a428-9bbd-492a-98ae-e4bd03fab5ce/serve-healthcheck-canary/0.log"
Sep 09 03:58:47.940184 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:47.940144    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0c7e4f11-324d-4e2f-9d4b-97a2186beabc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5dtmp\" (UID: \"0c7e4f11-324d-4e2f-9d4b-97a2186beabc\") " pod="openshift-insights/insights-runtime-extractor-5dtmp"
Sep 09 03:58:47.940416 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:47.940277    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 03:58:47.940416 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:47.940355    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c7e4f11-324d-4e2f-9d4b-97a2186beabc-insights-runtime-extractor-tls podName:0c7e4f11-324d-4e2f-9d4b-97a2186beabc nodeName:}" failed. No retries permitted until 2026-09-09 03:58:51.940335754 +0000 UTC m=+56.859506658 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0c7e4f11-324d-4e2f-9d4b-97a2186beabc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5dtmp" (UID: "0c7e4f11-324d-4e2f-9d4b-97a2186beabc") : secret "insights-runtime-extractor-tls" not found
Sep 09 03:58:49.387591 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:49.387551    2557 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-gj7h7"
Sep 09 03:58:49.388142 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:49.387623    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-gj7h7"
Sep 09 03:58:49.388142 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:49.388062    2557 scope.go:117] "RemoveContainer" containerID="d9631056996bb24a1237624ae0c24fc157389cb200a56802a21d28e4f4c9cab4"
Sep 09 03:58:49.388301 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:49.388279    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-gj7h7_openshift-console-operator(7816cd49-e9c2-4f7b-b8ec-4e5711b675a5)\"" pod="openshift-console-operator/console-operator-85b454874b-gj7h7" podUID="7816cd49-e9c2-4f7b-b8ec-4e5711b675a5"
Sep 09 03:58:51.014972 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:51.014892    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-865d5755c7-2v95v" event={"ID":"e1a0b3b4-b53f-4870-8849-6ee3e393374f","Type":"ContainerStarted","Data":"79d1893a02f39e74ca3b384abd50b3cdb1075138d512b259e684454e3caf406c"}
Sep 09 03:58:51.015384 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:51.015241    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-865d5755c7-2v95v"
Sep 09 03:58:51.016658 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:51.016637    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-865d5755c7-2v95v"
Sep 09 03:58:51.034106 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:51.034064    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-865d5755c7-2v95v" podStartSLOduration=22.80445799 podStartE2EDuration="43.034051485s" totalImagesPullingTime="20.229593495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:58:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-09-09 03:58:51.032228925 +0000 UTC m=+55.951399834" watchObservedRunningTime="2026-09-09 03:58:51.034051485 +0000 UTC m=+55.953222454"
Sep 09 03:58:51.978246 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:51.978211    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0c7e4f11-324d-4e2f-9d4b-97a2186beabc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5dtmp\" (UID: \"0c7e4f11-324d-4e2f-9d4b-97a2186beabc\") " pod="openshift-insights/insights-runtime-extractor-5dtmp"
Sep 09 03:58:51.978403 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:51.978344    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 03:58:51.978446 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:58:51.978403    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c7e4f11-324d-4e2f-9d4b-97a2186beabc-insights-runtime-extractor-tls podName:0c7e4f11-324d-4e2f-9d4b-97a2186beabc nodeName:}" failed. No retries permitted until 2026-09-09 03:58:59.978390642 +0000 UTC m=+64.897561534 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0c7e4f11-324d-4e2f-9d4b-97a2186beabc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5dtmp" (UID: "0c7e4f11-324d-4e2f-9d4b-97a2186beabc") : secret "insights-runtime-extractor-tls" not found
Sep 09 03:58:54.790195 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:58:54.790170    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rfvrc"
Sep 09 03:59:00.047408 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:00.047376    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0c7e4f11-324d-4e2f-9d4b-97a2186beabc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5dtmp\" (UID: \"0c7e4f11-324d-4e2f-9d4b-97a2186beabc\") " pod="openshift-insights/insights-runtime-extractor-5dtmp"
Sep 09 03:59:00.049621 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:00.049601    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0c7e4f11-324d-4e2f-9d4b-97a2186beabc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5dtmp\" (UID: \"0c7e4f11-324d-4e2f-9d4b-97a2186beabc\") " pod="openshift-insights/insights-runtime-extractor-5dtmp"
Sep 09 03:59:00.142161 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:00.142138    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4kqm2\""
Sep 09 03:59:00.150293 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:00.150275    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5dtmp"
Sep 09 03:59:00.429814 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:00.429787    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5dtmp"]
Sep 09 03:59:00.433443 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:59:00.433417    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c7e4f11_324d_4e2f_9d4b_97a2186beabc.slice/crio-c6d607ed286ee952160d4c3c7f1ec6d97400ed011356d50fcdacf27117ec0224 WatchSource:0}: Error finding container c6d607ed286ee952160d4c3c7f1ec6d97400ed011356d50fcdacf27117ec0224: Status 404 returned error can't find the container with id c6d607ed286ee952160d4c3c7f1ec6d97400ed011356d50fcdacf27117ec0224
Sep 09 03:59:00.613765 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:00.613739    2557 scope.go:117] "RemoveContainer" containerID="d9631056996bb24a1237624ae0c24fc157389cb200a56802a21d28e4f4c9cab4"
Sep 09 03:59:00.853643 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:00.853614    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f831999b-7d59-4577-864d-c39b88896d64-metrics-certs\") pod \"router-default-6f7c46fdb6-89xrm\" (UID: \"f831999b-7d59-4577-864d-c39b88896d64\") " pod="openshift-ingress/router-default-6f7c46fdb6-89xrm"
Sep 09 03:59:00.853797 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:00.853666    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/54f58d60-214f-4fb6-9ae8-02334c258100-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-ln2j9\" (UID: \"54f58d60-214f-4fb6-9ae8-02334c258100\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-ln2j9"
Sep 09 03:59:00.853797 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:00.853700    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48e91752-c7e6-4208-85f8-1c8386f0aacc-registry-tls\") pod \"image-registry-6f66c76d89-h8rdt\" (UID: \"48e91752-c7e6-4208-85f8-1c8386f0aacc\") " pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt"
Sep 09 03:59:00.853797 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:00.853731    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f831999b-7d59-4577-864d-c39b88896d64-service-ca-bundle\") pod \"router-default-6f7c46fdb6-89xrm\" (UID: \"f831999b-7d59-4577-864d-c39b88896d64\") " pod="openshift-ingress/router-default-6f7c46fdb6-89xrm"
Sep 09 03:59:00.854359 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:00.854325    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f831999b-7d59-4577-864d-c39b88896d64-service-ca-bundle\") pod \"router-default-6f7c46fdb6-89xrm\" (UID: \"f831999b-7d59-4577-864d-c39b88896d64\") " pod="openshift-ingress/router-default-6f7c46fdb6-89xrm"
Sep 09 03:59:00.855937 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:00.855913    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/54f58d60-214f-4fb6-9ae8-02334c258100-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-ln2j9\" (UID: \"54f58d60-214f-4fb6-9ae8-02334c258100\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-ln2j9"
Sep 09 03:59:00.856026 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:00.855938    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48e91752-c7e6-4208-85f8-1c8386f0aacc-registry-tls\") pod \"image-registry-6f66c76d89-h8rdt\" (UID: \"48e91752-c7e6-4208-85f8-1c8386f0aacc\") " pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt"
Sep 09 03:59:00.856026 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:00.855948    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f831999b-7d59-4577-864d-c39b88896d64-metrics-certs\") pod \"router-default-6f7c46fdb6-89xrm\" (UID: \"f831999b-7d59-4577-864d-c39b88896d64\") " pod="openshift-ingress/router-default-6f7c46fdb6-89xrm"
Sep 09 03:59:00.954920 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:00.954886    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/68bc0dde-1b15-48ca-a42c-f169a59f3ae4-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-tg4dz\" (UID: \"68bc0dde-1b15-48ca-a42c-f169a59f3ae4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-tg4dz"
Sep 09 03:59:00.957240 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:00.957217    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/68bc0dde-1b15-48ca-a42c-f169a59f3ae4-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-tg4dz\" (UID: \"68bc0dde-1b15-48ca-a42c-f169a59f3ae4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-tg4dz"
Sep 09 03:59:01.049271 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:01.049195    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-rtwn6\""
Sep 09 03:59:01.049685 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:01.049626    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gj7h7_7816cd49-e9c2-4f7b-b8ec-4e5711b675a5/console-operator/2.log"
Sep 09 03:59:01.050036 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:01.050021    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gj7h7_7816cd49-e9c2-4f7b-b8ec-4e5711b675a5/console-operator/1.log"
Sep 09 03:59:01.050078 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:01.050052    2557 generic.go:356] "Generic (PLEG): container finished" podID="7816cd49-e9c2-4f7b-b8ec-4e5711b675a5" containerID="5d55ad7c5ccc150e47b82b536ccec9952fec4ea00c2aa2116a8d2b2da276b5d4" exitCode=255
Sep 09 03:59:01.050168 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:01.050141    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-gj7h7" event={"ID":"7816cd49-e9c2-4f7b-b8ec-4e5711b675a5","Type":"ContainerDied","Data":"5d55ad7c5ccc150e47b82b536ccec9952fec4ea00c2aa2116a8d2b2da276b5d4"}
Sep 09 03:59:01.050284 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:01.050196    2557 scope.go:117] "RemoveContainer" containerID="d9631056996bb24a1237624ae0c24fc157389cb200a56802a21d28e4f4c9cab4"
Sep 09 03:59:01.050566 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:01.050545    2557 scope.go:117] "RemoveContainer" containerID="5d55ad7c5ccc150e47b82b536ccec9952fec4ea00c2aa2116a8d2b2da276b5d4"
Sep 09 03:59:01.050796 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:59:01.050774    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-gj7h7_openshift-console-operator(7816cd49-e9c2-4f7b-b8ec-4e5711b675a5)\"" pod="openshift-console-operator/console-operator-85b454874b-gj7h7" podUID="7816cd49-e9c2-4f7b-b8ec-4e5711b675a5"
Sep 09 03:59:01.051871 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:01.051846    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5dtmp" event={"ID":"0c7e4f11-324d-4e2f-9d4b-97a2186beabc","Type":"ContainerStarted","Data":"9b685c9311dfd84030030b0bfc4142d197c5cc8bc34191b3bdf152ddcb28ee68"}
Sep 09 03:59:01.051942 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:01.051906    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5dtmp" event={"ID":"0c7e4f11-324d-4e2f-9d4b-97a2186beabc","Type":"ContainerStarted","Data":"c6d607ed286ee952160d4c3c7f1ec6d97400ed011356d50fcdacf27117ec0224"}
Sep 09 03:59:01.055890 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:01.055847    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b73c2563-ee66-481a-ba6e-122139fb3abc-metrics-tls\") pod \"dns-default-9d9xs\" (UID: \"b73c2563-ee66-481a-ba6e-122139fb3abc\") " pod="openshift-dns/dns-default-9d9xs"
Sep 09 03:59:01.056181 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:01.056139    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt"
Sep 09 03:59:01.058211 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:01.058193    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b73c2563-ee66-481a-ba6e-122139fb3abc-metrics-tls\") pod \"dns-default-9d9xs\" (UID: \"b73c2563-ee66-481a-ba6e-122139fb3abc\") " pod="openshift-dns/dns-default-9d9xs"
Sep 09 03:59:01.068823 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:01.068803    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-csp64\""
Sep 09 03:59:01.076401 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:01.076376    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6f7c46fdb6-89xrm"
Sep 09 03:59:01.088851 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:01.088831    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-2f9js\""
Sep 09 03:59:01.095906 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:01.095702    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-ln2j9"
Sep 09 03:59:01.163959 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:01.163899    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-7zcl9\""
Sep 09 03:59:01.171594 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:01.171573    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-tg4dz"
Sep 09 03:59:01.287020 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:01.286989    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-nrdh9\""
Sep 09 03:59:01.294551 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:01.294537    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9d9xs"
Sep 09 03:59:01.335868 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:01.335830    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6f66c76d89-h8rdt"]
Sep 09 03:59:01.339971 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:59:01.339943    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48e91752_c7e6_4208_85f8_1c8386f0aacc.slice/crio-116f9b7a248834cc9f4e557aa7dc0dbc58453bc968711467a07afcd294fb86f1 WatchSource:0}: Error finding container 116f9b7a248834cc9f4e557aa7dc0dbc58453bc968711467a07afcd294fb86f1: Status 404 returned error can't find the container with id 116f9b7a248834cc9f4e557aa7dc0dbc58453bc968711467a07afcd294fb86f1
Sep 09 03:59:01.360222 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:01.360044    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58280195-fa67-47e6-af03-e39d2bab379b-metrics-certs\") pod \"network-metrics-daemon-9lrvt\" (UID: \"58280195-fa67-47e6-af03-e39d2bab379b\") " pod="openshift-multus/network-metrics-daemon-9lrvt"
Sep 09 03:59:01.360222 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:01.360187    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/85fe16e4-d362-49b1-b379-451d29551d1b-original-pull-secret\") pod \"global-pull-secret-syncer-78hdg\" (UID: \"85fe16e4-d362-49b1-b379-451d29551d1b\") " pod="kube-system/global-pull-secret-syncer-78hdg"
Sep 09 03:59:01.360494 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:01.360470    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6f7c46fdb6-89xrm"]
Sep 09 03:59:01.362861 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:01.362669    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 09 03:59:01.363494 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:01.363473    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 09 03:59:01.365640 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:59:01.363719    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf831999b_7d59_4577_864d_c39b88896d64.slice/crio-f3f2114df4847fa6525c8ecc32deaecad4eeae5cf573f68445279e167acc10d7 WatchSource:0}: Error finding container f3f2114df4847fa6525c8ecc32deaecad4eeae5cf573f68445279e167acc10d7: Status 404 returned error can't find the container with id f3f2114df4847fa6525c8ecc32deaecad4eeae5cf573f68445279e167acc10d7
Sep 09 03:59:01.372657 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:01.372634    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/85fe16e4-d362-49b1-b379-451d29551d1b-original-pull-secret\") pod \"global-pull-secret-syncer-78hdg\" (UID: \"85fe16e4-d362-49b1-b379-451d29551d1b\") " pod="kube-system/global-pull-secret-syncer-78hdg"
Sep 09 03:59:01.372657 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:01.372650    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58280195-fa67-47e6-af03-e39d2bab379b-metrics-certs\") pod \"network-metrics-daemon-9lrvt\" (UID: \"58280195-fa67-47e6-af03-e39d2bab379b\") " pod="openshift-multus/network-metrics-daemon-9lrvt"
Sep 09 03:59:01.382010 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:01.381989    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-ln2j9"]
Sep 09 03:59:01.391058 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:59:01.391035    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54f58d60_214f_4fb6_9ae8_02334c258100.slice/crio-d9245a3c327c1a6725f5aea204efb464b4a5ec7b10b367c9b634d6a698e42ac4 WatchSource:0}: Error finding container d9245a3c327c1a6725f5aea204efb464b4a5ec7b10b367c9b634d6a698e42ac4: Status 404 returned error can't find the container with id d9245a3c327c1a6725f5aea204efb464b4a5ec7b10b367c9b634d6a698e42ac4
Sep 09 03:59:01.442351 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:01.442324    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-tg4dz"]
Sep 09 03:59:01.524026 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:01.523992    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78hdg"
Sep 09 03:59:01.531808 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:01.531788    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-krf5h\""
Sep 09 03:59:01.539125 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:01.539094    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9lrvt"
Sep 09 03:59:01.570087 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:01.569808    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9d9xs"]
Sep 09 03:59:01.572785 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:59:01.572753    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb73c2563_ee66_481a_ba6e_122139fb3abc.slice/crio-2d4a3290992a11e28d33e7cf3726c1d4055157f9e00cf746f7bb9e698f45c445 WatchSource:0}: Error finding container 2d4a3290992a11e28d33e7cf3726c1d4055157f9e00cf746f7bb9e698f45c445: Status 404 returned error can't find the container with id 2d4a3290992a11e28d33e7cf3726c1d4055157f9e00cf746f7bb9e698f45c445
Sep 09 03:59:01.804549 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:01.804516    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-78hdg"]
Sep 09 03:59:01.809263 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:59:01.809238    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85fe16e4_d362_49b1_b379_451d29551d1b.slice/crio-6f10fd4a36bde46b65c339f1127aa1867ac678fc550dc112ce2c8bc47683c05e WatchSource:0}: Error finding container 6f10fd4a36bde46b65c339f1127aa1867ac678fc550dc112ce2c8bc47683c05e: Status 404 returned error can't find the container with id 6f10fd4a36bde46b65c339f1127aa1867ac678fc550dc112ce2c8bc47683c05e
Sep 09 03:59:01.831197 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:01.831138    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9lrvt"]
Sep 09 03:59:01.834654 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:59:01.834633    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58280195_fa67_47e6_af03_e39d2bab379b.slice/crio-08285c20a7bd89e20d06add52d2a8b0c5d4c0acbe3c0f6b795e8afbb0c30fb13 WatchSource:0}: Error finding container 08285c20a7bd89e20d06add52d2a8b0c5d4c0acbe3c0f6b795e8afbb0c30fb13: Status 404 returned error can't find the container with id 08285c20a7bd89e20d06add52d2a8b0c5d4c0acbe3c0f6b795e8afbb0c30fb13
Sep 09 03:59:02.058269 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:02.058243    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gj7h7_7816cd49-e9c2-4f7b-b8ec-4e5711b675a5/console-operator/2.log"
Sep 09 03:59:02.059853 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:02.059823    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-78hdg" event={"ID":"85fe16e4-d362-49b1-b379-451d29551d1b","Type":"ContainerStarted","Data":"6f10fd4a36bde46b65c339f1127aa1867ac678fc550dc112ce2c8bc47683c05e"}
Sep 09 03:59:02.062137 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:02.062093    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5dtmp" event={"ID":"0c7e4f11-324d-4e2f-9d4b-97a2186beabc","Type":"ContainerStarted","Data":"5389c4da5906a9fdd3342234a7f433580b1a441354a2af65ecdf5a720124cf90"}
Sep 09 03:59:02.063371 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:02.063336    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9d9xs" event={"ID":"b73c2563-ee66-481a-ba6e-122139fb3abc","Type":"ContainerStarted","Data":"2d4a3290992a11e28d33e7cf3726c1d4055157f9e00cf746f7bb9e698f45c445"}
Sep 09 03:59:02.064747 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:02.064723    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9lrvt" event={"ID":"58280195-fa67-47e6-af03-e39d2bab379b","Type":"ContainerStarted","Data":"08285c20a7bd89e20d06add52d2a8b0c5d4c0acbe3c0f6b795e8afbb0c30fb13"}
Sep 09 03:59:02.066304 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:02.066270    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-tg4dz" event={"ID":"68bc0dde-1b15-48ca-a42c-f169a59f3ae4","Type":"ContainerStarted","Data":"3918bb00155e68ee7f4dd0c7b8f86936cd61341df2c5b48da525bfbe5db832b7"}
Sep 09 03:59:02.067771 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:02.067750    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-ln2j9" event={"ID":"54f58d60-214f-4fb6-9ae8-02334c258100","Type":"ContainerStarted","Data":"d9245a3c327c1a6725f5aea204efb464b4a5ec7b10b367c9b634d6a698e42ac4"}
Sep 09 03:59:02.069560 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:02.069537    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6f7c46fdb6-89xrm" event={"ID":"f831999b-7d59-4577-864d-c39b88896d64","Type":"ContainerStarted","Data":"758022b68c07a20a57418b5c893a40dd75fa0571dc361fc86a746d164489a32a"}
Sep 09 03:59:02.069645 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:02.069606    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6f7c46fdb6-89xrm" event={"ID":"f831999b-7d59-4577-864d-c39b88896d64","Type":"ContainerStarted","Data":"f3f2114df4847fa6525c8ecc32deaecad4eeae5cf573f68445279e167acc10d7"}
Sep 09 03:59:02.071873 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:02.071853    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt" event={"ID":"48e91752-c7e6-4208-85f8-1c8386f0aacc","Type":"ContainerStarted","Data":"2eb613630673a20cc35b2965b23b1913a8d84444de688c9b0d05159054714d71"}
Sep 09 03:59:02.071954 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:02.071888    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt" event={"ID":"48e91752-c7e6-4208-85f8-1c8386f0aacc","Type":"ContainerStarted","Data":"116f9b7a248834cc9f4e557aa7dc0dbc58453bc968711467a07afcd294fb86f1"}
Sep 09 03:59:02.072027 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:02.071982    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt"
Sep 09 03:59:02.077545 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:02.077500    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6f7c46fdb6-89xrm"
Sep 09 03:59:02.080211 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:02.080192    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6f7c46fdb6-89xrm"
Sep 09 03:59:02.100830 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:02.100726    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6f7c46fdb6-89xrm" podStartSLOduration=37.100709091 podStartE2EDuration="37.100709091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:58:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:59:02.09877989 +0000 UTC m=+67.017950798" watchObservedRunningTime="2026-09-09 03:59:02.100709091 +0000 UTC m=+67.019880000"
Sep 09 03:59:02.146313 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:02.145643    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt" podStartSLOduration=65.145627472 podStartE2EDuration="1m5.145627472s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:57:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:59:02.145521519 +0000 UTC m=+67.064692427" watchObservedRunningTime="2026-09-09 03:59:02.145627472 +0000 UTC m=+67.064798382"
Sep 09 03:59:03.080422 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:03.080316    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6f7c46fdb6-89xrm"
Sep 09 03:59:03.082424 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:03.082198    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6f7c46fdb6-89xrm"
Sep 09 03:59:08.100197 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:08.100160    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9d9xs" event={"ID":"b73c2563-ee66-481a-ba6e-122139fb3abc","Type":"ContainerStarted","Data":"9d8a5c9beb628a8adb5c76813ee20ab402bb7ef5743dc974f7faa48610f4bca8"}
Sep 09 03:59:08.101886 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:08.101821    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9lrvt" event={"ID":"58280195-fa67-47e6-af03-e39d2bab379b","Type":"ContainerStarted","Data":"19673a1e14b96c5db15adc3e37cba31f74f658b9d2950c1002f339b80e28e807"}
Sep 09 03:59:08.103387 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:08.103353    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-ln2j9" event={"ID":"54f58d60-214f-4fb6-9ae8-02334c258100","Type":"ContainerStarted","Data":"4e2a6bc280ccced01fac7d7185af3656c958fe4d71373172033c5dfbdd443605"}
Sep 09 03:59:08.106368 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:08.106343    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5dtmp" event={"ID":"0c7e4f11-324d-4e2f-9d4b-97a2186beabc","Type":"ContainerStarted","Data":"ae285a236aafaefcef529112cc931ef46f8e8bbdd7d0937c313f27d4064cc9da"}
Sep 09 03:59:08.138780 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:08.137000    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-ln2j9" podStartSLOduration=35.723563852 podStartE2EDuration="42.136981291s" totalImagesPullingTime="6.413417439s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:59:08.135753068 +0000 UTC m=+73.054923976" watchObservedRunningTime="2026-09-09 03:59:08.136981291 +0000 UTC m=+73.056152204"
Sep 09 03:59:08.159220 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:08.159128    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-5dtmp" podStartSLOduration=16.924752466 podStartE2EDuration="24.159089063s" totalImagesPullingTime="7.234336597s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 03:58:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:59:08.157524398 +0000 UTC m=+73.076695307" watchObservedRunningTime="2026-09-09 03:59:08.159089063 +0000 UTC m=+73.078259973"
Sep 09 03:59:09.111686 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:09.111647    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-78hdg" event={"ID":"85fe16e4-d362-49b1-b379-451d29551d1b","Type":"ContainerStarted","Data":"2728d04a140f15c573fad2b37b954130491f8cfeac45e6df93ab8f9e6610e6b8"}
Sep 09 03:59:09.113392 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:09.113367    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9d9xs" event={"ID":"b73c2563-ee66-481a-ba6e-122139fb3abc","Type":"ContainerStarted","Data":"f45c803614b6ae71c8cc0e3a2c5851da40b5347d279ad84176e8b7ea0f26f13f"}
Sep 09 03:59:09.113508 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:09.113411    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9d9xs"
Sep 09 03:59:09.115124 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:09.115088    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9lrvt" event={"ID":"58280195-fa67-47e6-af03-e39d2bab379b","Type":"ContainerStarted","Data":"5ee95995ab38e58e981bd059edb7786cf7a2b14340ebc1bdfde5afcd3f303b9f"}
Sep 09 03:59:09.116761 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:09.116736    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-tg4dz" event={"ID":"68bc0dde-1b15-48ca-a42c-f169a59f3ae4","Type":"ContainerStarted","Data":"e53ab7cc5de8ff5998bea1c0101d4a689e8087be8cf59e58323b3149d430fcee"}
Sep 09 03:59:09.116872 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:09.116773    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-tg4dz" event={"ID":"68bc0dde-1b15-48ca-a42c-f169a59f3ae4","Type":"ContainerStarted","Data":"ffdd6dcb5b43c60baede3b0fa43c1ffab284024b5d1509b31b21c1f089d89dea"}
Sep 09 03:59:09.125888 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:09.125848    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-78hdg" podStartSLOduration=66.831679482 podStartE2EDuration="1m13.125835367s" totalImagesPullingTime="6.294155885s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 03:57:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:59:09.125396283 +0000 UTC m=+74.044567203" watchObservedRunningTime="2026-09-09 03:59:09.125835367 +0000 UTC m=+74.045006315"
Sep 09 03:59:09.140902 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:09.140860    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9lrvt" podStartSLOduration=67.205164129 podStartE2EDuration="1m13.140848729s" totalImagesPullingTime="5.9356846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:57:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:59:09.140357682 +0000 UTC m=+74.059528590" watchObservedRunningTime="2026-09-09 03:59:09.140848729 +0000 UTC m=+74.060019637"
Sep 09 03:59:09.180752 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:09.180709    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-tg4dz" podStartSLOduration=36.850588986 podStartE2EDuration="43.180693014s" totalImagesPullingTime="6.330104028s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:59:09.163299814 +0000 UTC m=+74.082470723" watchObservedRunningTime="2026-09-09 03:59:09.180693014 +0000 UTC m=+74.099863924"
Sep 09 03:59:09.180905 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:09.180862    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9d9xs" podStartSLOduration=34.983845813 podStartE2EDuration="41.180854892s" totalImagesPullingTime="6.197009079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:58:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:59:09.180282406 +0000 UTC m=+74.099453314" watchObservedRunningTime="2026-09-09 03:59:09.180854892 +0000 UTC m=+74.100025801"
Sep 09 03:59:09.386920 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:09.386842    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-gj7h7"
Sep 09 03:59:09.386920 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:09.386898    2557 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-gj7h7"
Sep 09 03:59:09.387279 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:09.387265    2557 scope.go:117] "RemoveContainer" containerID="5d55ad7c5ccc150e47b82b536ccec9952fec4ea00c2aa2116a8d2b2da276b5d4"
Sep 09 03:59:09.387449 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:59:09.387431    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-gj7h7_openshift-console-operator(7816cd49-e9c2-4f7b-b8ec-4e5711b675a5)\"" pod="openshift-console-operator/console-operator-85b454874b-gj7h7" podUID="7816cd49-e9c2-4f7b-b8ec-4e5711b675a5"
Sep 09 03:59:14.987608 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:14.987579    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-bvnr6"
Sep 09 03:59:17.841886 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:17.841852    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-wz2tp"]
Sep 09 03:59:17.845107 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:17.845088    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wz2tp"
Sep 09 03:59:17.847455 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:17.847427    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 09 03:59:17.847567 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:17.847493    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 09 03:59:17.847567 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:17.847517    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-rxzhr\""
Sep 09 03:59:17.847567 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:17.847554    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 09 03:59:17.848222 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:17.848206    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 09 03:59:17.988806 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:17.988702    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d48875b3-ed83-4a71-818e-b56a6d54e394-node-exporter-tls\") pod \"node-exporter-wz2tp\" (UID: \"d48875b3-ed83-4a71-818e-b56a6d54e394\") " pod="openshift-monitoring/node-exporter-wz2tp"
Sep 09 03:59:17.988806 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:17.988744    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bw7zt\" (UniqueName: \"kubernetes.io/projected/d48875b3-ed83-4a71-818e-b56a6d54e394-kube-api-access-bw7zt\") pod \"node-exporter-wz2tp\" (UID: \"d48875b3-ed83-4a71-818e-b56a6d54e394\") " pod="openshift-monitoring/node-exporter-wz2tp"
Sep 09 03:59:17.988806 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:17.988766    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d48875b3-ed83-4a71-818e-b56a6d54e394-root\") pod \"node-exporter-wz2tp\" (UID: \"d48875b3-ed83-4a71-818e-b56a6d54e394\") " pod="openshift-monitoring/node-exporter-wz2tp"
Sep 09 03:59:17.988973 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:17.988835    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d48875b3-ed83-4a71-818e-b56a6d54e394-node-exporter-accelerators-collector-config\") pod \"node-exporter-wz2tp\" (UID: \"d48875b3-ed83-4a71-818e-b56a6d54e394\") " pod="openshift-monitoring/node-exporter-wz2tp"
Sep 09 03:59:17.988973 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:17.988864    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d48875b3-ed83-4a71-818e-b56a6d54e394-node-exporter-textfile\") pod \"node-exporter-wz2tp\" (UID: \"d48875b3-ed83-4a71-818e-b56a6d54e394\") " pod="openshift-monitoring/node-exporter-wz2tp"
Sep 09 03:59:17.988973 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:17.988881    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d48875b3-ed83-4a71-818e-b56a6d54e394-node-exporter-wtmp\") pod \"node-exporter-wz2tp\" (UID: \"d48875b3-ed83-4a71-818e-b56a6d54e394\") " pod="openshift-monitoring/node-exporter-wz2tp"
Sep 09 03:59:17.988973 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:17.988897    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d48875b3-ed83-4a71-818e-b56a6d54e394-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wz2tp\" (UID: \"d48875b3-ed83-4a71-818e-b56a6d54e394\") " pod="openshift-monitoring/node-exporter-wz2tp"
Sep 09 03:59:17.988973 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:17.988917    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d48875b3-ed83-4a71-818e-b56a6d54e394-sys\") pod \"node-exporter-wz2tp\" (UID: \"d48875b3-ed83-4a71-818e-b56a6d54e394\") " pod="openshift-monitoring/node-exporter-wz2tp"
Sep 09 03:59:17.988973 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:17.988956    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d48875b3-ed83-4a71-818e-b56a6d54e394-metrics-client-ca\") pod \"node-exporter-wz2tp\" (UID: \"d48875b3-ed83-4a71-818e-b56a6d54e394\") " pod="openshift-monitoring/node-exporter-wz2tp"
Sep 09 03:59:18.090184 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:18.090154    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d48875b3-ed83-4a71-818e-b56a6d54e394-node-exporter-tls\") pod \"node-exporter-wz2tp\" (UID: \"d48875b3-ed83-4a71-818e-b56a6d54e394\") " pod="openshift-monitoring/node-exporter-wz2tp"
Sep 09 03:59:18.090327 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:18.090199    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bw7zt\" (UniqueName: \"kubernetes.io/projected/d48875b3-ed83-4a71-818e-b56a6d54e394-kube-api-access-bw7zt\") pod \"node-exporter-wz2tp\" (UID: \"d48875b3-ed83-4a71-818e-b56a6d54e394\") " pod="openshift-monitoring/node-exporter-wz2tp"
Sep 09 03:59:18.090327 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:18.090217    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d48875b3-ed83-4a71-818e-b56a6d54e394-root\") pod \"node-exporter-wz2tp\" (UID: \"d48875b3-ed83-4a71-818e-b56a6d54e394\") " pod="openshift-monitoring/node-exporter-wz2tp"
Sep 09 03:59:18.090327 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:18.090246    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d48875b3-ed83-4a71-818e-b56a6d54e394-node-exporter-accelerators-collector-config\") pod \"node-exporter-wz2tp\" (UID: \"d48875b3-ed83-4a71-818e-b56a6d54e394\") " pod="openshift-monitoring/node-exporter-wz2tp"
Sep 09 03:59:18.090327 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:18.090291    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d48875b3-ed83-4a71-818e-b56a6d54e394-root\") pod \"node-exporter-wz2tp\" (UID: \"d48875b3-ed83-4a71-818e-b56a6d54e394\") " pod="openshift-monitoring/node-exporter-wz2tp"
Sep 09 03:59:18.090327 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:59:18.090319    2557 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 09 03:59:18.090598 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:18.090349    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d48875b3-ed83-4a71-818e-b56a6d54e394-node-exporter-textfile\") pod \"node-exporter-wz2tp\" (UID: \"d48875b3-ed83-4a71-818e-b56a6d54e394\") " pod="openshift-monitoring/node-exporter-wz2tp"
Sep 09 03:59:18.090598 ip-10-0-141-71 kubenswrapper[2557]: E0909 03:59:18.090387    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d48875b3-ed83-4a71-818e-b56a6d54e394-node-exporter-tls podName:d48875b3-ed83-4a71-818e-b56a6d54e394 nodeName:}" failed. No retries permitted until 2026-09-09 03:59:18.590366104 +0000 UTC m=+83.509536994 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/d48875b3-ed83-4a71-818e-b56a6d54e394-node-exporter-tls") pod "node-exporter-wz2tp" (UID: "d48875b3-ed83-4a71-818e-b56a6d54e394") : secret "node-exporter-tls" not found
Sep 09 03:59:18.090598 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:18.090425    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d48875b3-ed83-4a71-818e-b56a6d54e394-node-exporter-wtmp\") pod \"node-exporter-wz2tp\" (UID: \"d48875b3-ed83-4a71-818e-b56a6d54e394\") " pod="openshift-monitoring/node-exporter-wz2tp"
Sep 09 03:59:18.090598 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:18.090456    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d48875b3-ed83-4a71-818e-b56a6d54e394-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wz2tp\" (UID: \"d48875b3-ed83-4a71-818e-b56a6d54e394\") " pod="openshift-monitoring/node-exporter-wz2tp"
Sep 09 03:59:18.090598 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:18.090487    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d48875b3-ed83-4a71-818e-b56a6d54e394-sys\") pod \"node-exporter-wz2tp\" (UID: \"d48875b3-ed83-4a71-818e-b56a6d54e394\") " pod="openshift-monitoring/node-exporter-wz2tp"
Sep 09 03:59:18.090598 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:18.090510    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d48875b3-ed83-4a71-818e-b56a6d54e394-metrics-client-ca\") pod \"node-exporter-wz2tp\" (UID: \"d48875b3-ed83-4a71-818e-b56a6d54e394\") " pod="openshift-monitoring/node-exporter-wz2tp"
Sep 09 03:59:18.090598 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:18.090554    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d48875b3-ed83-4a71-818e-b56a6d54e394-sys\") pod \"node-exporter-wz2tp\" (UID: \"d48875b3-ed83-4a71-818e-b56a6d54e394\") " pod="openshift-monitoring/node-exporter-wz2tp"
Sep 09 03:59:18.090939 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:18.090606    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d48875b3-ed83-4a71-818e-b56a6d54e394-node-exporter-wtmp\") pod \"node-exporter-wz2tp\" (UID: \"d48875b3-ed83-4a71-818e-b56a6d54e394\") " pod="openshift-monitoring/node-exporter-wz2tp"
Sep 09 03:59:18.090939 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:18.090632    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d48875b3-ed83-4a71-818e-b56a6d54e394-node-exporter-textfile\") pod \"node-exporter-wz2tp\" (UID: \"d48875b3-ed83-4a71-818e-b56a6d54e394\") " pod="openshift-monitoring/node-exporter-wz2tp"
Sep 09 03:59:18.090939 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:18.090776    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d48875b3-ed83-4a71-818e-b56a6d54e394-node-exporter-accelerators-collector-config\") pod \"node-exporter-wz2tp\" (UID: \"d48875b3-ed83-4a71-818e-b56a6d54e394\") " pod="openshift-monitoring/node-exporter-wz2tp"
Sep 09 03:59:18.091084 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:18.091022    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d48875b3-ed83-4a71-818e-b56a6d54e394-metrics-client-ca\") pod \"node-exporter-wz2tp\" (UID: \"d48875b3-ed83-4a71-818e-b56a6d54e394\") " pod="openshift-monitoring/node-exporter-wz2tp"
Sep 09 03:59:18.092652 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:18.092611    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d48875b3-ed83-4a71-818e-b56a6d54e394-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wz2tp\" (UID: \"d48875b3-ed83-4a71-818e-b56a6d54e394\") " pod="openshift-monitoring/node-exporter-wz2tp"
Sep 09 03:59:18.099347 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:18.099319    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bw7zt\" (UniqueName: \"kubernetes.io/projected/d48875b3-ed83-4a71-818e-b56a6d54e394-kube-api-access-bw7zt\") pod \"node-exporter-wz2tp\" (UID: \"d48875b3-ed83-4a71-818e-b56a6d54e394\") " pod="openshift-monitoring/node-exporter-wz2tp"
Sep 09 03:59:18.594718 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:18.594684    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d48875b3-ed83-4a71-818e-b56a6d54e394-node-exporter-tls\") pod \"node-exporter-wz2tp\" (UID: \"d48875b3-ed83-4a71-818e-b56a6d54e394\") " pod="openshift-monitoring/node-exporter-wz2tp"
Sep 09 03:59:18.596849 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:18.596822    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d48875b3-ed83-4a71-818e-b56a6d54e394-node-exporter-tls\") pod \"node-exporter-wz2tp\" (UID: \"d48875b3-ed83-4a71-818e-b56a6d54e394\") " pod="openshift-monitoring/node-exporter-wz2tp"
Sep 09 03:59:18.754098 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:18.754069    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wz2tp"
Sep 09 03:59:18.765930 ip-10-0-141-71 kubenswrapper[2557]: W0909 03:59:18.765899    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd48875b3_ed83_4a71_818e_b56a6d54e394.slice/crio-ff2a607b0f49e8a1ff79059c2fe8dc7f540f20c5825acbb1f2ef0ad3bc75c2d2 WatchSource:0}: Error finding container ff2a607b0f49e8a1ff79059c2fe8dc7f540f20c5825acbb1f2ef0ad3bc75c2d2: Status 404 returned error can't find the container with id ff2a607b0f49e8a1ff79059c2fe8dc7f540f20c5825acbb1f2ef0ad3bc75c2d2
Sep 09 03:59:19.123855 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:19.123817    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9d9xs"
Sep 09 03:59:19.155548 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:19.155517    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wz2tp" event={"ID":"d48875b3-ed83-4a71-818e-b56a6d54e394","Type":"ContainerStarted","Data":"ff2a607b0f49e8a1ff79059c2fe8dc7f540f20c5825acbb1f2ef0ad3bc75c2d2"}
Sep 09 03:59:20.160888 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:20.160850    2557 generic.go:356] "Generic (PLEG): container finished" podID="d48875b3-ed83-4a71-818e-b56a6d54e394" containerID="f125ff8f5e03f2028b8a3926bb9e44eff511296529bdf48044a5b919ce3d75da" exitCode=0
Sep 09 03:59:20.161279 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:20.160930    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wz2tp" event={"ID":"d48875b3-ed83-4a71-818e-b56a6d54e394","Type":"ContainerDied","Data":"f125ff8f5e03f2028b8a3926bb9e44eff511296529bdf48044a5b919ce3d75da"}
Sep 09 03:59:21.060598 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:21.060561    2557 patch_prober.go:28] interesting pod/image-registry-6f66c76d89-h8rdt container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 09 03:59:21.060757 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:21.060612    2557 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt" podUID="48e91752-c7e6-4208-85f8-1c8386f0aacc" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 03:59:21.167627 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:21.167594    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wz2tp" event={"ID":"d48875b3-ed83-4a71-818e-b56a6d54e394","Type":"ContainerStarted","Data":"82d2f171e3b0ec774550260dd25abcb0708201421cdea0aa49138ae6d3262fb9"}
Sep 09 03:59:21.167962 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:21.167653    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wz2tp" event={"ID":"d48875b3-ed83-4a71-818e-b56a6d54e394","Type":"ContainerStarted","Data":"24caa7153566489b56c505e5e7a8754924585551cf29241aa9245860aa1ba9f3"}
Sep 09 03:59:21.187822 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:21.187778    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-wz2tp" podStartSLOduration=3.128764487 podStartE2EDuration="4.18776189s" totalImagesPullingTime="1.000143928s" totalInitContainerRuntime="58.853475ms" isStatefulPod=true podCreationTimestamp="2026-09-09 03:59:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:59:21.186487055 +0000 UTC m=+86.105657963" watchObservedRunningTime="2026-09-09 03:59:21.18776189 +0000 UTC m=+86.106932798"
Sep 09 03:59:23.084253 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:23.084186    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt"
Sep 09 03:59:23.613468 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:23.613431    2557 scope.go:117] "RemoveContainer" containerID="5d55ad7c5ccc150e47b82b536ccec9952fec4ea00c2aa2116a8d2b2da276b5d4"
Sep 09 03:59:24.180425 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:24.180397    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gj7h7_7816cd49-e9c2-4f7b-b8ec-4e5711b675a5/console-operator/2.log"
Sep 09 03:59:24.180780 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:24.180491    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-gj7h7" event={"ID":"7816cd49-e9c2-4f7b-b8ec-4e5711b675a5","Type":"ContainerStarted","Data":"3633c802678677ebbffc727c7bac115965dc2a74078e35c090b49f277dd85457"}
Sep 09 03:59:24.180780 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:24.180757    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-gj7h7"
Sep 09 03:59:24.202265 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:24.202197    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-gj7h7" podStartSLOduration=46.103553469 podStartE2EDuration="58.202182596s" totalImagesPullingTime="12.098629127s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 03:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 03:59:24.201610146 +0000 UTC m=+89.120781055" watchObservedRunningTime="2026-09-09 03:59:24.202182596 +0000 UTC m=+89.121353504"
Sep 09 03:59:24.333427 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:24.333395    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-gj7h7"
Sep 09 03:59:33.120433 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:33.120403    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6f66c76d89-h8rdt"]
Sep 09 03:59:49.274722 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:49.274690    2557 generic.go:356] "Generic (PLEG): container finished" podID="1318985b-585a-4a94-a73e-c5856d2d2075" containerID="5f47b222d08d3dfbed959cf8c5fcb6aea0cce6d3e9dcb2dc3ac5bee554bb9bc8" exitCode=0
Sep 09 03:59:49.275200 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:49.274731    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zqtzf" event={"ID":"1318985b-585a-4a94-a73e-c5856d2d2075","Type":"ContainerDied","Data":"5f47b222d08d3dfbed959cf8c5fcb6aea0cce6d3e9dcb2dc3ac5bee554bb9bc8"}
Sep 09 03:59:49.275200 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:49.275037    2557 scope.go:117] "RemoveContainer" containerID="5f47b222d08d3dfbed959cf8c5fcb6aea0cce6d3e9dcb2dc3ac5bee554bb9bc8"
Sep 09 03:59:50.280731 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:50.280695    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zqtzf" event={"ID":"1318985b-585a-4a94-a73e-c5856d2d2075","Type":"ContainerStarted","Data":"91f1342a323c473baba65f40cb41fc70a0e5ff65775baad9b54a31bec5eb395e"}
Sep 09 03:59:54.302195 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:54.302155    2557 generic.go:356] "Generic (PLEG): container finished" podID="6964ed13-fd43-48b3-aff1-8e37bcd05287" containerID="89c877aba0e71455048846d5a639aa1fcad2bed66108d7ec1026ce8eb97143a8" exitCode=0
Sep 09 03:59:54.302647 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:54.302230    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-4xg2m" event={"ID":"6964ed13-fd43-48b3-aff1-8e37bcd05287","Type":"ContainerDied","Data":"89c877aba0e71455048846d5a639aa1fcad2bed66108d7ec1026ce8eb97143a8"}
Sep 09 03:59:54.302647 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:54.302572    2557 scope.go:117] "RemoveContainer" containerID="89c877aba0e71455048846d5a639aa1fcad2bed66108d7ec1026ce8eb97143a8"
Sep 09 03:59:55.308271 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:55.308235    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-4xg2m" event={"ID":"6964ed13-fd43-48b3-aff1-8e37bcd05287","Type":"ContainerStarted","Data":"241aa207cf615e5258957a52ed291989aa86e75d6d2e9586f7c257be292e873e"}
Sep 09 03:59:58.139448 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:58.139390    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt" podUID="48e91752-c7e6-4208-85f8-1c8386f0aacc" containerName="registry" containerID="cri-o://2eb613630673a20cc35b2965b23b1913a8d84444de688c9b0d05159054714d71" gracePeriod=30
Sep 09 03:59:58.320306 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:58.320277    2557 generic.go:356] "Generic (PLEG): container finished" podID="48e91752-c7e6-4208-85f8-1c8386f0aacc" containerID="2eb613630673a20cc35b2965b23b1913a8d84444de688c9b0d05159054714d71" exitCode=0
Sep 09 03:59:58.320430 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:58.320316    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt" event={"ID":"48e91752-c7e6-4208-85f8-1c8386f0aacc","Type":"ContainerDied","Data":"2eb613630673a20cc35b2965b23b1913a8d84444de688c9b0d05159054714d71"}
Sep 09 03:59:58.383906 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:58.383884    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt"
Sep 09 03:59:58.495049 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:58.494951    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/48e91752-c7e6-4208-85f8-1c8386f0aacc-ca-trust-extracted\") pod \"48e91752-c7e6-4208-85f8-1c8386f0aacc\" (UID: \"48e91752-c7e6-4208-85f8-1c8386f0aacc\") "
Sep 09 03:59:58.495049 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:58.495000    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/48e91752-c7e6-4208-85f8-1c8386f0aacc-trusted-ca\") pod \"48e91752-c7e6-4208-85f8-1c8386f0aacc\" (UID: \"48e91752-c7e6-4208-85f8-1c8386f0aacc\") "
Sep 09 03:59:58.495049 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:58.495019    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48e91752-c7e6-4208-85f8-1c8386f0aacc-registry-tls\") pod \"48e91752-c7e6-4208-85f8-1c8386f0aacc\" (UID: \"48e91752-c7e6-4208-85f8-1c8386f0aacc\") "
Sep 09 03:59:58.495049 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:58.495062    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/48e91752-c7e6-4208-85f8-1c8386f0aacc-image-registry-private-configuration\") pod \"48e91752-c7e6-4208-85f8-1c8386f0aacc\" (UID: \"48e91752-c7e6-4208-85f8-1c8386f0aacc\") "
Sep 09 03:59:58.495425 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:58.495079    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/48e91752-c7e6-4208-85f8-1c8386f0aacc-installation-pull-secrets\") pod \"48e91752-c7e6-4208-85f8-1c8386f0aacc\" (UID: \"48e91752-c7e6-4208-85f8-1c8386f0aacc\") "
Sep 09 03:59:58.495425 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:58.495096    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/48e91752-c7e6-4208-85f8-1c8386f0aacc-registry-certificates\") pod \"48e91752-c7e6-4208-85f8-1c8386f0aacc\" (UID: \"48e91752-c7e6-4208-85f8-1c8386f0aacc\") "
Sep 09 03:59:58.495425 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:58.495136    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48e91752-c7e6-4208-85f8-1c8386f0aacc-bound-sa-token\") pod \"48e91752-c7e6-4208-85f8-1c8386f0aacc\" (UID: \"48e91752-c7e6-4208-85f8-1c8386f0aacc\") "
Sep 09 03:59:58.495425 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:58.495168    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dpkxg\" (UniqueName: \"kubernetes.io/projected/48e91752-c7e6-4208-85f8-1c8386f0aacc-kube-api-access-dpkxg\") pod \"48e91752-c7e6-4208-85f8-1c8386f0aacc\" (UID: \"48e91752-c7e6-4208-85f8-1c8386f0aacc\") "
Sep 09 03:59:58.495613 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:58.495455    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48e91752-c7e6-4208-85f8-1c8386f0aacc-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "48e91752-c7e6-4208-85f8-1c8386f0aacc" (UID: "48e91752-c7e6-4208-85f8-1c8386f0aacc"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:59:58.495734 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:58.495702    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48e91752-c7e6-4208-85f8-1c8386f0aacc-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "48e91752-c7e6-4208-85f8-1c8386f0aacc" (UID: "48e91752-c7e6-4208-85f8-1c8386f0aacc"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 03:59:58.495823 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:58.495806    2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/48e91752-c7e6-4208-85f8-1c8386f0aacc-trusted-ca\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 03:59:58.495881 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:58.495831    2557 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/48e91752-c7e6-4208-85f8-1c8386f0aacc-registry-certificates\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 03:59:58.497856 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:58.497794    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48e91752-c7e6-4208-85f8-1c8386f0aacc-kube-api-access-dpkxg" (OuterVolumeSpecName: "kube-api-access-dpkxg") pod "48e91752-c7e6-4208-85f8-1c8386f0aacc" (UID: "48e91752-c7e6-4208-85f8-1c8386f0aacc"). InnerVolumeSpecName "kube-api-access-dpkxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:59:58.498023 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:58.497998    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48e91752-c7e6-4208-85f8-1c8386f0aacc-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "48e91752-c7e6-4208-85f8-1c8386f0aacc" (UID: "48e91752-c7e6-4208-85f8-1c8386f0aacc"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:59:58.498098 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:58.498080    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48e91752-c7e6-4208-85f8-1c8386f0aacc-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "48e91752-c7e6-4208-85f8-1c8386f0aacc" (UID: "48e91752-c7e6-4208-85f8-1c8386f0aacc"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 03:59:58.498187 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:58.498101    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48e91752-c7e6-4208-85f8-1c8386f0aacc-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "48e91752-c7e6-4208-85f8-1c8386f0aacc" (UID: "48e91752-c7e6-4208-85f8-1c8386f0aacc"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 03:59:58.498187 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:58.498167    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48e91752-c7e6-4208-85f8-1c8386f0aacc-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "48e91752-c7e6-4208-85f8-1c8386f0aacc" (UID: "48e91752-c7e6-4208-85f8-1c8386f0aacc"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 03:59:58.503091 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:58.503065    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48e91752-c7e6-4208-85f8-1c8386f0aacc-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "48e91752-c7e6-4208-85f8-1c8386f0aacc" (UID: "48e91752-c7e6-4208-85f8-1c8386f0aacc"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 03:59:58.597201 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:58.597171    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpkxg\" (UniqueName: \"kubernetes.io/projected/48e91752-c7e6-4208-85f8-1c8386f0aacc-kube-api-access-dpkxg\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 03:59:58.597201 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:58.597195    2557 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/48e91752-c7e6-4208-85f8-1c8386f0aacc-ca-trust-extracted\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 03:59:58.597201 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:58.597206    2557 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/48e91752-c7e6-4208-85f8-1c8386f0aacc-registry-tls\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 03:59:58.597389 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:58.597217    2557 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/48e91752-c7e6-4208-85f8-1c8386f0aacc-image-registry-private-configuration\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 03:59:58.597389 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:58.597228    2557 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/48e91752-c7e6-4208-85f8-1c8386f0aacc-installation-pull-secrets\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 03:59:58.597389 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:58.597236    2557 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48e91752-c7e6-4208-85f8-1c8386f0aacc-bound-sa-token\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 03:59:59.326230 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:59.326204    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt"
Sep 09 03:59:59.326230 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:59.326223    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f66c76d89-h8rdt" event={"ID":"48e91752-c7e6-4208-85f8-1c8386f0aacc","Type":"ContainerDied","Data":"116f9b7a248834cc9f4e557aa7dc0dbc58453bc968711467a07afcd294fb86f1"}
Sep 09 03:59:59.326673 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:59.326279    2557 scope.go:117] "RemoveContainer" containerID="2eb613630673a20cc35b2965b23b1913a8d84444de688c9b0d05159054714d71"
Sep 09 03:59:59.348429 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:59.348405    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6f66c76d89-h8rdt"]
Sep 09 03:59:59.353714 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:59.353693    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6f66c76d89-h8rdt"]
Sep 09 03:59:59.443529 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:59.443495    2557 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f674d7db9-jhbnd" podUID="80439eab-81e8-4a3b-a2de-44c45f992052" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 09 03:59:59.618429 ip-10-0-141-71 kubenswrapper[2557]: I0909 03:59:59.618400    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48e91752-c7e6-4208-85f8-1c8386f0aacc" path="/var/lib/kubelet/pods/48e91752-c7e6-4208-85f8-1c8386f0aacc/volumes"
Sep 09 04:00:08.365275 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:00:08.365246    2557 generic.go:356] "Generic (PLEG): container finished" podID="55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7" containerID="546c750fe56629278f8a0d63d437bcf748ab1e4a28e414532836e20609fe29d1" exitCode=0
Sep 09 04:00:08.365557 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:00:08.365318    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-57s54" event={"ID":"55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7","Type":"ContainerDied","Data":"546c750fe56629278f8a0d63d437bcf748ab1e4a28e414532836e20609fe29d1"}
Sep 09 04:00:08.365651 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:00:08.365636    2557 scope.go:117] "RemoveContainer" containerID="546c750fe56629278f8a0d63d437bcf748ab1e4a28e414532836e20609fe29d1"
Sep 09 04:00:09.372977 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:00:09.372936    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-57s54" event={"ID":"55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7","Type":"ContainerStarted","Data":"fceffa99605a53555656a1196bbb3ceec19b6cef1f9b7311e8a7d29c12fb5100"}
Sep 09 04:00:09.443256 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:00:09.443222    2557 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f674d7db9-jhbnd" podUID="80439eab-81e8-4a3b-a2de-44c45f992052" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 09 04:00:19.444430 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:00:19.444388    2557 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f674d7db9-jhbnd" podUID="80439eab-81e8-4a3b-a2de-44c45f992052" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 09 04:00:19.444788 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:00:19.444476    2557 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f674d7db9-jhbnd"
Sep 09 04:00:19.445071 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:00:19.445037    2557 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"97b8fe42f1c788325ec77d4ff1fb833e47aea413653c48e1610e725e2e08acda"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f674d7db9-jhbnd" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Sep 09 04:00:19.445139 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:00:19.445094    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f674d7db9-jhbnd" podUID="80439eab-81e8-4a3b-a2de-44c45f992052" containerName="service-proxy" containerID="cri-o://97b8fe42f1c788325ec77d4ff1fb833e47aea413653c48e1610e725e2e08acda" gracePeriod=30
Sep 09 04:00:20.417466 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:00:20.417434    2557 generic.go:356] "Generic (PLEG): container finished" podID="80439eab-81e8-4a3b-a2de-44c45f992052" containerID="97b8fe42f1c788325ec77d4ff1fb833e47aea413653c48e1610e725e2e08acda" exitCode=2
Sep 09 04:00:20.417647 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:00:20.417487    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f674d7db9-jhbnd" event={"ID":"80439eab-81e8-4a3b-a2de-44c45f992052","Type":"ContainerDied","Data":"97b8fe42f1c788325ec77d4ff1fb833e47aea413653c48e1610e725e2e08acda"}
Sep 09 04:00:20.417647 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:00:20.417526    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f674d7db9-jhbnd" event={"ID":"80439eab-81e8-4a3b-a2de-44c45f992052","Type":"ContainerStarted","Data":"a5d7dda4b51a7c242909689cc0b81891a6ff4c48851c82541deaae837ccf5b82"}
Sep 09 04:02:43.773944 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:43.773910    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5996d6cf95-w4tx6"]
Sep 09 04:02:43.774530 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:43.774311    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48e91752-c7e6-4208-85f8-1c8386f0aacc" containerName="registry"
Sep 09 04:02:43.774530 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:43.774330    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="48e91752-c7e6-4208-85f8-1c8386f0aacc" containerName="registry"
Sep 09 04:02:43.774530 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:43.774443    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="48e91752-c7e6-4208-85f8-1c8386f0aacc" containerName="registry"
Sep 09 04:02:43.777378 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:43.777356    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5996d6cf95-w4tx6"
Sep 09 04:02:43.779584 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:43.779560    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 09 04:02:43.779713 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:43.779601    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 09 04:02:43.779713 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:43.779611    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-2jwxl\""
Sep 09 04:02:43.779828 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:43.779741    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 09 04:02:43.780285 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:43.780269    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 09 04:02:43.789912 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:43.789891    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5996d6cf95-w4tx6"]
Sep 09 04:02:43.842183 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:43.842151    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9e4b9910-12de-4341-89be-3863b2274210-webhook-cert\") pod \"opendatahub-operator-controller-manager-5996d6cf95-w4tx6\" (UID: \"9e4b9910-12de-4341-89be-3863b2274210\") " pod="opendatahub/opendatahub-operator-controller-manager-5996d6cf95-w4tx6"
Sep 09 04:02:43.842377 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:43.842203    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9e4b9910-12de-4341-89be-3863b2274210-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5996d6cf95-w4tx6\" (UID: \"9e4b9910-12de-4341-89be-3863b2274210\") " pod="opendatahub/opendatahub-operator-controller-manager-5996d6cf95-w4tx6"
Sep 09 04:02:43.842377 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:43.842224    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjtt5\" (UniqueName: \"kubernetes.io/projected/9e4b9910-12de-4341-89be-3863b2274210-kube-api-access-tjtt5\") pod \"opendatahub-operator-controller-manager-5996d6cf95-w4tx6\" (UID: \"9e4b9910-12de-4341-89be-3863b2274210\") " pod="opendatahub/opendatahub-operator-controller-manager-5996d6cf95-w4tx6"
Sep 09 04:02:43.842377 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:43.842283    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9e4b9910-12de-4341-89be-3863b2274210-tmp\") pod \"opendatahub-operator-controller-manager-5996d6cf95-w4tx6\" (UID: \"9e4b9910-12de-4341-89be-3863b2274210\") " pod="opendatahub/opendatahub-operator-controller-manager-5996d6cf95-w4tx6"
Sep 09 04:02:43.842377 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:43.842337    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9e4b9910-12de-4341-89be-3863b2274210-manifests\") pod \"opendatahub-operator-controller-manager-5996d6cf95-w4tx6\" (UID: \"9e4b9910-12de-4341-89be-3863b2274210\") " pod="opendatahub/opendatahub-operator-controller-manager-5996d6cf95-w4tx6"
Sep 09 04:02:43.942708 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:43.942678    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9e4b9910-12de-4341-89be-3863b2274210-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5996d6cf95-w4tx6\" (UID: \"9e4b9910-12de-4341-89be-3863b2274210\") " pod="opendatahub/opendatahub-operator-controller-manager-5996d6cf95-w4tx6"
Sep 09 04:02:43.942708 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:43.942711    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjtt5\" (UniqueName: \"kubernetes.io/projected/9e4b9910-12de-4341-89be-3863b2274210-kube-api-access-tjtt5\") pod \"opendatahub-operator-controller-manager-5996d6cf95-w4tx6\" (UID: \"9e4b9910-12de-4341-89be-3863b2274210\") " pod="opendatahub/opendatahub-operator-controller-manager-5996d6cf95-w4tx6"
Sep 09 04:02:43.942966 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:43.942736    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9e4b9910-12de-4341-89be-3863b2274210-tmp\") pod \"opendatahub-operator-controller-manager-5996d6cf95-w4tx6\" (UID: \"9e4b9910-12de-4341-89be-3863b2274210\") " pod="opendatahub/opendatahub-operator-controller-manager-5996d6cf95-w4tx6"
Sep 09 04:02:43.942966 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:43.942755    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9e4b9910-12de-4341-89be-3863b2274210-manifests\") pod \"opendatahub-operator-controller-manager-5996d6cf95-w4tx6\" (UID: \"9e4b9910-12de-4341-89be-3863b2274210\") " pod="opendatahub/opendatahub-operator-controller-manager-5996d6cf95-w4tx6"
Sep 09 04:02:43.942966 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:43.942796    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9e4b9910-12de-4341-89be-3863b2274210-webhook-cert\") pod \"opendatahub-operator-controller-manager-5996d6cf95-w4tx6\" (UID: \"9e4b9910-12de-4341-89be-3863b2274210\") " pod="opendatahub/opendatahub-operator-controller-manager-5996d6cf95-w4tx6"
Sep 09 04:02:43.943301 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:43.943272    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9e4b9910-12de-4341-89be-3863b2274210-manifests\") pod \"opendatahub-operator-controller-manager-5996d6cf95-w4tx6\" (UID: \"9e4b9910-12de-4341-89be-3863b2274210\") " pod="opendatahub/opendatahub-operator-controller-manager-5996d6cf95-w4tx6"
Sep 09 04:02:43.945263 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:43.945232    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9e4b9910-12de-4341-89be-3863b2274210-tmp\") pod \"opendatahub-operator-controller-manager-5996d6cf95-w4tx6\" (UID: \"9e4b9910-12de-4341-89be-3863b2274210\") " pod="opendatahub/opendatahub-operator-controller-manager-5996d6cf95-w4tx6"
Sep 09 04:02:43.945388 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:43.945354    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9e4b9910-12de-4341-89be-3863b2274210-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5996d6cf95-w4tx6\" (UID: \"9e4b9910-12de-4341-89be-3863b2274210\") " pod="opendatahub/opendatahub-operator-controller-manager-5996d6cf95-w4tx6"
Sep 09 04:02:43.945514 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:43.945496    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9e4b9910-12de-4341-89be-3863b2274210-webhook-cert\") pod \"opendatahub-operator-controller-manager-5996d6cf95-w4tx6\" (UID: \"9e4b9910-12de-4341-89be-3863b2274210\") " pod="opendatahub/opendatahub-operator-controller-manager-5996d6cf95-w4tx6"
Sep 09 04:02:43.952963 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:43.952939    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjtt5\" (UniqueName: \"kubernetes.io/projected/9e4b9910-12de-4341-89be-3863b2274210-kube-api-access-tjtt5\") pod \"opendatahub-operator-controller-manager-5996d6cf95-w4tx6\" (UID: \"9e4b9910-12de-4341-89be-3863b2274210\") " pod="opendatahub/opendatahub-operator-controller-manager-5996d6cf95-w4tx6"
Sep 09 04:02:44.087467 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:44.087381    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5996d6cf95-w4tx6"
Sep 09 04:02:44.365816 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:44.365788    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5996d6cf95-w4tx6"]
Sep 09 04:02:44.369236 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:02:44.369209    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e4b9910_12de_4341_89be_3863b2274210.slice/crio-7f9ab7c5dac55bcbe528c9739439d7cb004d03cfcfbfba1471921e2260a854ed WatchSource:0}: Error finding container 7f9ab7c5dac55bcbe528c9739439d7cb004d03cfcfbfba1471921e2260a854ed: Status 404 returned error can't find the container with id 7f9ab7c5dac55bcbe528c9739439d7cb004d03cfcfbfba1471921e2260a854ed
Sep 09 04:02:44.952392 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:44.952356    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5996d6cf95-w4tx6" event={"ID":"9e4b9910-12de-4341-89be-3863b2274210","Type":"ContainerStarted","Data":"7f9ab7c5dac55bcbe528c9739439d7cb004d03cfcfbfba1471921e2260a854ed"}
Sep 09 04:02:47.967344 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:47.967308    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5996d6cf95-w4tx6" event={"ID":"9e4b9910-12de-4341-89be-3863b2274210","Type":"ContainerStarted","Data":"f543b35879a444f315578ca9509b9bafcf1fac69ac8bc7091cb91171835a8cb2"}
Sep 09 04:02:48.972550 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:48.972463    2557 generic.go:356] "Generic (PLEG): container finished" podID="9e4b9910-12de-4341-89be-3863b2274210" containerID="f543b35879a444f315578ca9509b9bafcf1fac69ac8bc7091cb91171835a8cb2" exitCode=0
Sep 09 04:02:48.972915 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:48.972549    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5996d6cf95-w4tx6" event={"ID":"9e4b9910-12de-4341-89be-3863b2274210","Type":"ContainerDied","Data":"f543b35879a444f315578ca9509b9bafcf1fac69ac8bc7091cb91171835a8cb2"}
Sep 09 04:02:49.980261 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:49.980226    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5996d6cf95-w4tx6" event={"ID":"9e4b9910-12de-4341-89be-3863b2274210","Type":"ContainerStarted","Data":"a0c3f62197274ae15eac750fa31a25924230bae070e3bc6d17b4dc0ffcafce0a"}
Sep 09 04:02:49.980633 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:49.980354    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5996d6cf95-w4tx6"
Sep 09 04:02:50.001280 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:50.001225    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5996d6cf95-w4tx6" podStartSLOduration=2.462121869 podStartE2EDuration="7.001208615s" totalImagesPullingTime="3.480014812s" totalInitContainerRuntime="1.059071934s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:02:43 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:02:50.000216019 +0000 UTC m=+294.919386959" watchObservedRunningTime="2026-09-09 04:02:50.001208615 +0000 UTC m=+294.920379524"
Sep 09 04:02:55.493374 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:55.493343    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gj7h7_7816cd49-e9c2-4f7b-b8ec-4e5711b675a5/console-operator/2.log"
Sep 09 04:02:55.499499 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:55.499475    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rfvrc_1828d4d8-0d24-4e4d-a502-aee013a3b116/ovn-acl-logging/0.log"
Sep 09 04:02:55.499651 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:55.499628    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gj7h7_7816cd49-e9c2-4f7b-b8ec-4e5711b675a5/console-operator/2.log"
Sep 09 04:02:55.506223 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:55.506200    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rfvrc_1828d4d8-0d24-4e4d-a502-aee013a3b116/ovn-acl-logging/0.log"
Sep 09 04:02:55.507207 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:02:55.507191    2557 kubelet.go:1648] "Image garbage collection succeeded"
Sep 09 04:03:00.986361 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:00.986327    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5996d6cf95-w4tx6"
Sep 09 04:03:14.053318 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:14.053276    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-fzkbx"]
Sep 09 04:03:14.056005 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:14.055976    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-fzkbx"
Sep 09 04:03:14.058233 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:14.058211    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 09 04:03:14.058500 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:14.058485    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-mb7rz\""
Sep 09 04:03:14.066697 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:14.066670    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-fzkbx"]
Sep 09 04:03:14.093418 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:14.093379    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77n2j\" (UniqueName: \"kubernetes.io/projected/cb6e639b-01bd-4884-ab79-76b09560d3c6-kube-api-access-77n2j\") pod \"odh-model-controller-7998c67dc4-fzkbx\" (UID: \"cb6e639b-01bd-4884-ab79-76b09560d3c6\") " pod="opendatahub/odh-model-controller-7998c67dc4-fzkbx"
Sep 09 04:03:14.093418 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:14.093422    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cb6e639b-01bd-4884-ab79-76b09560d3c6-cert\") pod \"odh-model-controller-7998c67dc4-fzkbx\" (UID: \"cb6e639b-01bd-4884-ab79-76b09560d3c6\") " pod="opendatahub/odh-model-controller-7998c67dc4-fzkbx"
Sep 09 04:03:14.194451 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:14.194412    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77n2j\" (UniqueName: \"kubernetes.io/projected/cb6e639b-01bd-4884-ab79-76b09560d3c6-kube-api-access-77n2j\") pod \"odh-model-controller-7998c67dc4-fzkbx\" (UID: \"cb6e639b-01bd-4884-ab79-76b09560d3c6\") " pod="opendatahub/odh-model-controller-7998c67dc4-fzkbx"
Sep 09 04:03:14.194646 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:14.194475    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cb6e639b-01bd-4884-ab79-76b09560d3c6-cert\") pod \"odh-model-controller-7998c67dc4-fzkbx\" (UID: \"cb6e639b-01bd-4884-ab79-76b09560d3c6\") " pod="opendatahub/odh-model-controller-7998c67dc4-fzkbx"
Sep 09 04:03:14.194725 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:03:14.194644    2557 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 09 04:03:14.194725 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:03:14.194720    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb6e639b-01bd-4884-ab79-76b09560d3c6-cert podName:cb6e639b-01bd-4884-ab79-76b09560d3c6 nodeName:}" failed. No retries permitted until 2026-09-09 04:03:14.694700114 +0000 UTC m=+319.613871020 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/cb6e639b-01bd-4884-ab79-76b09560d3c6-cert") pod "odh-model-controller-7998c67dc4-fzkbx" (UID: "cb6e639b-01bd-4884-ab79-76b09560d3c6") : secret "odh-model-controller-webhook-cert" not found
Sep 09 04:03:14.202937 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:14.202908    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77n2j\" (UniqueName: \"kubernetes.io/projected/cb6e639b-01bd-4884-ab79-76b09560d3c6-kube-api-access-77n2j\") pod \"odh-model-controller-7998c67dc4-fzkbx\" (UID: \"cb6e639b-01bd-4884-ab79-76b09560d3c6\") " pod="opendatahub/odh-model-controller-7998c67dc4-fzkbx"
Sep 09 04:03:14.697504 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:14.697468    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cb6e639b-01bd-4884-ab79-76b09560d3c6-cert\") pod \"odh-model-controller-7998c67dc4-fzkbx\" (UID: \"cb6e639b-01bd-4884-ab79-76b09560d3c6\") " pod="opendatahub/odh-model-controller-7998c67dc4-fzkbx"
Sep 09 04:03:14.699858 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:14.699828    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cb6e639b-01bd-4884-ab79-76b09560d3c6-cert\") pod \"odh-model-controller-7998c67dc4-fzkbx\" (UID: \"cb6e639b-01bd-4884-ab79-76b09560d3c6\") " pod="opendatahub/odh-model-controller-7998c67dc4-fzkbx"
Sep 09 04:03:14.967632 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:14.967531    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-fzkbx"
Sep 09 04:03:15.239611 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:15.239587    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-fzkbx"]
Sep 09 04:03:15.241937 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:03:15.241911    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb6e639b_01bd_4884_ab79_76b09560d3c6.slice/crio-a986f5f9bbac03d4fe34cef6189933ce93f1ff477165ce64a36b6709567bd7eb WatchSource:0}: Error finding container a986f5f9bbac03d4fe34cef6189933ce93f1ff477165ce64a36b6709567bd7eb: Status 404 returned error can't find the container with id a986f5f9bbac03d4fe34cef6189933ce93f1ff477165ce64a36b6709567bd7eb
Sep 09 04:03:15.243310 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:15.243290    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 04:03:16.088333 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:16.088300    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-fzkbx" event={"ID":"cb6e639b-01bd-4884-ab79-76b09560d3c6","Type":"ContainerStarted","Data":"a986f5f9bbac03d4fe34cef6189933ce93f1ff477165ce64a36b6709567bd7eb"}
Sep 09 04:03:20.110043 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:20.110003    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-fzkbx" event={"ID":"cb6e639b-01bd-4884-ab79-76b09560d3c6","Type":"ContainerStarted","Data":"7e385cf067305f73c35565cf713fe2d3f10672b7f23b4b0c224a8fa0e2da2d8a"}
Sep 09 04:03:20.110477 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:20.110079    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-fzkbx"
Sep 09 04:03:20.128431 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:20.128382    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-fzkbx" podStartSLOduration=1.83129162 podStartE2EDuration="6.128366019s" totalImagesPullingTime="4.297074399s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:03:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:03:20.127147 +0000 UTC m=+325.046317908" watchObservedRunningTime="2026-09-09 04:03:20.128366019 +0000 UTC m=+325.047536929"
Sep 09 04:03:21.052079 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:21.052035    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9m5x"]
Sep 09 04:03:21.054203 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:21.054185    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9m5x"
Sep 09 04:03:21.056425 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:21.056401    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 09 04:03:21.056644 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:21.056630    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 09 04:03:21.056837 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:21.056818    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 09 04:03:21.056984 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:21.056965    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 09 04:03:21.057183 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:21.057162    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-89swh\""
Sep 09 04:03:21.067407 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:21.067382    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9m5x"]
Sep 09 04:03:21.139975 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:21.139937    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxx99\" (UniqueName: \"kubernetes.io/projected/55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5-kube-api-access-wxx99\") pod \"istiod-openshift-gateway-5677ccb49b-k9m5x\" (UID: \"55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9m5x"
Sep 09 04:03:21.139975 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:21.139981    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-k9m5x\" (UID: \"55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9m5x"
Sep 09 04:03:21.140405 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:21.140005    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-k9m5x\" (UID: \"55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9m5x"
Sep 09 04:03:21.140405 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:21.140028    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-k9m5x\" (UID: \"55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9m5x"
Sep 09 04:03:21.140405 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:21.140062    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-k9m5x\" (UID: \"55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9m5x"
Sep 09 04:03:21.140405 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:21.140095    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-k9m5x\" (UID: \"55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9m5x"
Sep 09 04:03:21.140405 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:21.140141    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-k9m5x\" (UID: \"55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9m5x"
Sep 09 04:03:21.240444 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:21.240409    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-k9m5x\" (UID: \"55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9m5x"
Sep 09 04:03:21.240627 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:21.240451    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-k9m5x\" (UID: \"55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9m5x"
Sep 09 04:03:21.240627 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:21.240480    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-k9m5x\" (UID: \"55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9m5x"
Sep 09 04:03:21.240627 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:21.240522    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-k9m5x\" (UID: \"55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9m5x"
Sep 09 04:03:21.240627 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:21.240558    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-k9m5x\" (UID: \"55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9m5x"
Sep 09 04:03:21.240627 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:21.240596    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-k9m5x\" (UID: \"55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9m5x"
Sep 09 04:03:21.240931 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:21.240628    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxx99\" (UniqueName: \"kubernetes.io/projected/55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5-kube-api-access-wxx99\") pod \"istiod-openshift-gateway-5677ccb49b-k9m5x\" (UID: \"55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9m5x"
Sep 09 04:03:21.241415 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:21.241389    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-k9m5x\" (UID: \"55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9m5x"
Sep 09 04:03:21.242852 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:21.242827    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-k9m5x\" (UID: \"55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9m5x"
Sep 09 04:03:21.243080 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:21.243058    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-k9m5x\" (UID: \"55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9m5x"
Sep 09 04:03:21.243351 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:21.243331    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-k9m5x\" (UID: \"55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9m5x"
Sep 09 04:03:21.243441 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:21.243372    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-k9m5x\" (UID: \"55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9m5x"
Sep 09 04:03:21.249059 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:21.249039    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxx99\" (UniqueName: \"kubernetes.io/projected/55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5-kube-api-access-wxx99\") pod \"istiod-openshift-gateway-5677ccb49b-k9m5x\" (UID: \"55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9m5x"
Sep 09 04:03:21.249344 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:21.249326    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-k9m5x\" (UID: \"55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9m5x"
Sep 09 04:03:21.363949 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:21.363920    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9m5x"
Sep 09 04:03:21.647643 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:21.647568    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9m5x"]
Sep 09 04:03:21.650392 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:03:21.650348    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55e1dbd3_aeeb_4e90_bfa2_ad33f8d4c2c5.slice/crio-8d1688a2c1cbd480e7de80b0b2b0abd954600328338f74f0353f6dc9814fb4fd WatchSource:0}: Error finding container 8d1688a2c1cbd480e7de80b0b2b0abd954600328338f74f0353f6dc9814fb4fd: Status 404 returned error can't find the container with id 8d1688a2c1cbd480e7de80b0b2b0abd954600328338f74f0353f6dc9814fb4fd
Sep 09 04:03:22.119184 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:22.119151    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9m5x" event={"ID":"55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5","Type":"ContainerStarted","Data":"8d1688a2c1cbd480e7de80b0b2b0abd954600328338f74f0353f6dc9814fb4fd"}
Sep 09 04:03:23.080221 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:23.080185    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7486b797d6-x52dl"]
Sep 09 04:03:23.084619 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:23.084590    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7486b797d6-x52dl"
Sep 09 04:03:23.087021 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:23.086997    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 09 04:03:23.088107 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:23.088070    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 09 04:03:23.088107 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:23.088094    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 09 04:03:23.088107 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:23.088105    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-kd6lp\""
Sep 09 04:03:23.088354 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:23.088163    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 09 04:03:23.088354 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:23.088076    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 09 04:03:23.093129 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:23.092674    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7486b797d6-x52dl"]
Sep 09 04:03:23.152840 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:23.152798    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5f62770a-e2cc-40bf-922d-009ef53d32d8-metrics-cert\") pod \"lws-controller-manager-7486b797d6-x52dl\" (UID: \"5f62770a-e2cc-40bf-922d-009ef53d32d8\") " pod="openshift-lws-operator/lws-controller-manager-7486b797d6-x52dl"
Sep 09 04:03:23.153015 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:23.152858    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5f62770a-e2cc-40bf-922d-009ef53d32d8-manager-config\") pod \"lws-controller-manager-7486b797d6-x52dl\" (UID: \"5f62770a-e2cc-40bf-922d-009ef53d32d8\") " pod="openshift-lws-operator/lws-controller-manager-7486b797d6-x52dl"
Sep 09 04:03:23.153015 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:23.152907    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k45bs\" (UniqueName: \"kubernetes.io/projected/5f62770a-e2cc-40bf-922d-009ef53d32d8-kube-api-access-k45bs\") pod \"lws-controller-manager-7486b797d6-x52dl\" (UID: \"5f62770a-e2cc-40bf-922d-009ef53d32d8\") " pod="openshift-lws-operator/lws-controller-manager-7486b797d6-x52dl"
Sep 09 04:03:23.153015 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:23.152932    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5f62770a-e2cc-40bf-922d-009ef53d32d8-cert\") pod \"lws-controller-manager-7486b797d6-x52dl\" (UID: \"5f62770a-e2cc-40bf-922d-009ef53d32d8\") " pod="openshift-lws-operator/lws-controller-manager-7486b797d6-x52dl"
Sep 09 04:03:23.253420 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:23.253380    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5f62770a-e2cc-40bf-922d-009ef53d32d8-metrics-cert\") pod \"lws-controller-manager-7486b797d6-x52dl\" (UID: \"5f62770a-e2cc-40bf-922d-009ef53d32d8\") " pod="openshift-lws-operator/lws-controller-manager-7486b797d6-x52dl"
Sep 09 04:03:23.253618 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:23.253438    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5f62770a-e2cc-40bf-922d-009ef53d32d8-manager-config\") pod \"lws-controller-manager-7486b797d6-x52dl\" (UID: \"5f62770a-e2cc-40bf-922d-009ef53d32d8\") " pod="openshift-lws-operator/lws-controller-manager-7486b797d6-x52dl"
Sep 09 04:03:23.253618 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:23.253484    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k45bs\" (UniqueName: \"kubernetes.io/projected/5f62770a-e2cc-40bf-922d-009ef53d32d8-kube-api-access-k45bs\") pod \"lws-controller-manager-7486b797d6-x52dl\" (UID: \"5f62770a-e2cc-40bf-922d-009ef53d32d8\") " pod="openshift-lws-operator/lws-controller-manager-7486b797d6-x52dl"
Sep 09 04:03:23.253618 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:23.253515    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5f62770a-e2cc-40bf-922d-009ef53d32d8-cert\") pod \"lws-controller-manager-7486b797d6-x52dl\" (UID: \"5f62770a-e2cc-40bf-922d-009ef53d32d8\") " pod="openshift-lws-operator/lws-controller-manager-7486b797d6-x52dl"
Sep 09 04:03:23.254206 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:23.254172    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5f62770a-e2cc-40bf-922d-009ef53d32d8-manager-config\") pod \"lws-controller-manager-7486b797d6-x52dl\" (UID: \"5f62770a-e2cc-40bf-922d-009ef53d32d8\") " pod="openshift-lws-operator/lws-controller-manager-7486b797d6-x52dl"
Sep 09 04:03:23.256401 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:23.256372    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5f62770a-e2cc-40bf-922d-009ef53d32d8-cert\") pod \"lws-controller-manager-7486b797d6-x52dl\" (UID: \"5f62770a-e2cc-40bf-922d-009ef53d32d8\") " pod="openshift-lws-operator/lws-controller-manager-7486b797d6-x52dl"
Sep 09 04:03:23.256520 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:23.256428    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5f62770a-e2cc-40bf-922d-009ef53d32d8-metrics-cert\") pod \"lws-controller-manager-7486b797d6-x52dl\" (UID: \"5f62770a-e2cc-40bf-922d-009ef53d32d8\") " pod="openshift-lws-operator/lws-controller-manager-7486b797d6-x52dl"
Sep 09 04:03:23.271125 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:23.271087    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k45bs\" (UniqueName: \"kubernetes.io/projected/5f62770a-e2cc-40bf-922d-009ef53d32d8-kube-api-access-k45bs\") pod \"lws-controller-manager-7486b797d6-x52dl\" (UID: \"5f62770a-e2cc-40bf-922d-009ef53d32d8\") " pod="openshift-lws-operator/lws-controller-manager-7486b797d6-x52dl"
Sep 09 04:03:23.396314 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:23.396273    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7486b797d6-x52dl"
Sep 09 04:03:23.879048 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:23.878780    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 09 04:03:23.879048 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:23.878986    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 09 04:03:24.096986 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:24.096901    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7486b797d6-x52dl"]
Sep 09 04:03:24.100032 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:03:24.100010    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f62770a_e2cc_40bf_922d_009ef53d32d8.slice/crio-409ee370d5595f0528e2b06388203e74ad47698fd372a5efec1e5483a83855af WatchSource:0}: Error finding container 409ee370d5595f0528e2b06388203e74ad47698fd372a5efec1e5483a83855af: Status 404 returned error can't find the container with id 409ee370d5595f0528e2b06388203e74ad47698fd372a5efec1e5483a83855af
Sep 09 04:03:24.129296 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:24.129267    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9m5x" event={"ID":"55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5","Type":"ContainerStarted","Data":"b545b8fca4e72b22e3aa932d583c26feba50c12b1aaf76b589de0e2cf4f4b851"}
Sep 09 04:03:24.129438 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:24.129424    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9m5x"
Sep 09 04:03:24.130547 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:24.130520    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7486b797d6-x52dl" event={"ID":"5f62770a-e2cc-40bf-922d-009ef53d32d8","Type":"ContainerStarted","Data":"409ee370d5595f0528e2b06388203e74ad47698fd372a5efec1e5483a83855af"}
Sep 09 04:03:24.131171 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:24.131148    2557 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-k9m5x container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 04:03:24.131260 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:24.131191    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9m5x" podUID="55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 04:03:24.151822 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:24.151773    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9m5x" podStartSLOduration=0.925738893 podStartE2EDuration="3.151755097s" totalImagesPullingTime="2.226016204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:03:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:03:24.150291279 +0000 UTC m=+329.069462206" watchObservedRunningTime="2026-09-09 04:03:24.151755097 +0000 UTC m=+329.070926008"
Sep 09 04:03:25.147748 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:25.147716    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9m5x"
Sep 09 04:03:27.153302 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:27.153258    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7486b797d6-x52dl" event={"ID":"5f62770a-e2cc-40bf-922d-009ef53d32d8","Type":"ContainerStarted","Data":"4579dccaa6e56f5042208c0130032253c7fdb331c0020b724f626416340c440c"}
Sep 09 04:03:27.153739 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:27.153374    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7486b797d6-x52dl"
Sep 09 04:03:27.172795 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:27.172745    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7486b797d6-x52dl" podStartSLOduration=1.789575894 podStartE2EDuration="4.172729105s" totalImagesPullingTime="2.383153211s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:03:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:03:27.171431173 +0000 UTC m=+332.090602083" watchObservedRunningTime="2026-09-09 04:03:27.172729105 +0000 UTC m=+332.091900015"
Sep 09 04:03:31.116217 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:31.116185    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-fzkbx"
Sep 09 04:03:38.160259 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:03:38.160222    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7486b797d6-x52dl"
Sep 09 04:06:59.229100 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:06:59.229023    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-q4xpl"]
Sep 09 04:06:59.232159 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:06:59.232141    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-q4xpl"
Sep 09 04:06:59.234344 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:06:59.234317    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 09 04:06:59.234440 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:06:59.234318    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-2cfl6\""
Sep 09 04:06:59.242428 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:06:59.242405    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-q4xpl"]
Sep 09 04:06:59.355472 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:06:59.355435    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ld69\" (UniqueName: \"kubernetes.io/projected/2f2c11af-63a3-41d0-8bab-12654e7b98be-kube-api-access-5ld69\") pod \"payload-processing-5b54b7fbb7-q4xpl\" (UID: \"2f2c11af-63a3-41d0-8bab-12654e7b98be\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-q4xpl"
Sep 09 04:06:59.355628 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:06:59.355492    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f2c11af-63a3-41d0-8bab-12654e7b98be-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-q4xpl\" (UID: \"2f2c11af-63a3-41d0-8bab-12654e7b98be\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-q4xpl"
Sep 09 04:06:59.456205 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:06:59.456168    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ld69\" (UniqueName: \"kubernetes.io/projected/2f2c11af-63a3-41d0-8bab-12654e7b98be-kube-api-access-5ld69\") pod \"payload-processing-5b54b7fbb7-q4xpl\" (UID: \"2f2c11af-63a3-41d0-8bab-12654e7b98be\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-q4xpl"
Sep 09 04:06:59.456393 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:06:59.456221    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f2c11af-63a3-41d0-8bab-12654e7b98be-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-q4xpl\" (UID: \"2f2c11af-63a3-41d0-8bab-12654e7b98be\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-q4xpl"
Sep 09 04:06:59.456846 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:06:59.456825    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f2c11af-63a3-41d0-8bab-12654e7b98be-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-q4xpl\" (UID: \"2f2c11af-63a3-41d0-8bab-12654e7b98be\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-q4xpl"
Sep 09 04:06:59.467008 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:06:59.466981    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ld69\" (UniqueName: \"kubernetes.io/projected/2f2c11af-63a3-41d0-8bab-12654e7b98be-kube-api-access-5ld69\") pod \"payload-processing-5b54b7fbb7-q4xpl\" (UID: \"2f2c11af-63a3-41d0-8bab-12654e7b98be\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-q4xpl"
Sep 09 04:06:59.543055 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:06:59.542969    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-q4xpl"
Sep 09 04:06:59.814193 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:06:59.814086    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-q4xpl"]
Sep 09 04:06:59.817202 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:06:59.817177    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f2c11af_63a3_41d0_8bab_12654e7b98be.slice/crio-eb92a823eaf2dcd554ee22fbf988d16cbf12ccd9cd8dcdfb72e2d54646aa1620 WatchSource:0}: Error finding container eb92a823eaf2dcd554ee22fbf988d16cbf12ccd9cd8dcdfb72e2d54646aa1620: Status 404 returned error can't find the container with id eb92a823eaf2dcd554ee22fbf988d16cbf12ccd9cd8dcdfb72e2d54646aa1620
Sep 09 04:07:00.054793 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:00.054749    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-q4xpl" event={"ID":"2f2c11af-63a3-41d0-8bab-12654e7b98be","Type":"ContainerStarted","Data":"eb92a823eaf2dcd554ee22fbf988d16cbf12ccd9cd8dcdfb72e2d54646aa1620"}
Sep 09 04:07:03.075450 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:03.075414    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-q4xpl" event={"ID":"2f2c11af-63a3-41d0-8bab-12654e7b98be","Type":"ContainerStarted","Data":"d1f62426a8139e79936798907ac9d1f89707e4eedcde942a4ab87447e057b965"}
Sep 09 04:07:03.075862 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:03.075522    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-q4xpl"
Sep 09 04:07:03.093467 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:03.093418    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-q4xpl" podStartSLOduration=1.706909631 podStartE2EDuration="4.093400959s" totalImagesPullingTime="2.386491328s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:06:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:07:03.092736017 +0000 UTC m=+548.011906928" watchObservedRunningTime="2026-09-09 04:07:03.093400959 +0000 UTC m=+548.012571868"
Sep 09 04:07:14.081915 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:14.081883    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-q4xpl"
Sep 09 04:07:42.756677 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:42.756640    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-rjnh4"]
Sep 09 04:07:42.764277 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:42.764253    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rjnh4"
Sep 09 04:07:42.766440 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:42.766418    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6qgzz\""
Sep 09 04:07:42.767367 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:42.767347    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 09 04:07:42.767658 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:42.767352    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 09 04:07:42.767754 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:42.767418    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 09 04:07:42.769554 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:42.769531    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-rjnh4"]
Sep 09 04:07:42.812974 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:42.812937    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6902ae83-be2b-47fb-b30e-f3ba0788b5f5-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rjnh4\" (UID: \"6902ae83-be2b-47fb-b30e-f3ba0788b5f5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rjnh4"
Sep 09 04:07:42.813144 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:42.812997    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6902ae83-be2b-47fb-b30e-f3ba0788b5f5-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rjnh4\" (UID: \"6902ae83-be2b-47fb-b30e-f3ba0788b5f5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rjnh4"
Sep 09 04:07:42.813144 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:42.813037    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6902ae83-be2b-47fb-b30e-f3ba0788b5f5-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rjnh4\" (UID: \"6902ae83-be2b-47fb-b30e-f3ba0788b5f5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rjnh4"
Sep 09 04:07:42.813246 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:42.813154    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6902ae83-be2b-47fb-b30e-f3ba0788b5f5-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rjnh4\" (UID: \"6902ae83-be2b-47fb-b30e-f3ba0788b5f5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rjnh4"
Sep 09 04:07:42.813246 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:42.813180    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6902ae83-be2b-47fb-b30e-f3ba0788b5f5-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rjnh4\" (UID: \"6902ae83-be2b-47fb-b30e-f3ba0788b5f5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rjnh4"
Sep 09 04:07:42.813246 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:42.813206    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dtq7\" (UniqueName: \"kubernetes.io/projected/6902ae83-be2b-47fb-b30e-f3ba0788b5f5-kube-api-access-8dtq7\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rjnh4\" (UID: \"6902ae83-be2b-47fb-b30e-f3ba0788b5f5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rjnh4"
Sep 09 04:07:42.914087 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:42.914056    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6902ae83-be2b-47fb-b30e-f3ba0788b5f5-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rjnh4\" (UID: \"6902ae83-be2b-47fb-b30e-f3ba0788b5f5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rjnh4"
Sep 09 04:07:42.914260 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:42.914095    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6902ae83-be2b-47fb-b30e-f3ba0788b5f5-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rjnh4\" (UID: \"6902ae83-be2b-47fb-b30e-f3ba0788b5f5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rjnh4"
Sep 09 04:07:42.914260 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:42.914132    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6902ae83-be2b-47fb-b30e-f3ba0788b5f5-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rjnh4\" (UID: \"6902ae83-be2b-47fb-b30e-f3ba0788b5f5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rjnh4"
Sep 09 04:07:42.914260 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:42.914172    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6902ae83-be2b-47fb-b30e-f3ba0788b5f5-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rjnh4\" (UID: \"6902ae83-be2b-47fb-b30e-f3ba0788b5f5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rjnh4"
Sep 09 04:07:42.914260 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:42.914187    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6902ae83-be2b-47fb-b30e-f3ba0788b5f5-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rjnh4\" (UID: \"6902ae83-be2b-47fb-b30e-f3ba0788b5f5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rjnh4"
Sep 09 04:07:42.914260 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:42.914209    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dtq7\" (UniqueName: \"kubernetes.io/projected/6902ae83-be2b-47fb-b30e-f3ba0788b5f5-kube-api-access-8dtq7\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rjnh4\" (UID: \"6902ae83-be2b-47fb-b30e-f3ba0788b5f5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rjnh4"
Sep 09 04:07:42.914491 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:42.914465    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6902ae83-be2b-47fb-b30e-f3ba0788b5f5-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rjnh4\" (UID: \"6902ae83-be2b-47fb-b30e-f3ba0788b5f5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rjnh4"
Sep 09 04:07:42.914619 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:42.914602    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6902ae83-be2b-47fb-b30e-f3ba0788b5f5-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rjnh4\" (UID: \"6902ae83-be2b-47fb-b30e-f3ba0788b5f5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rjnh4"
Sep 09 04:07:42.914655 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:42.914638    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6902ae83-be2b-47fb-b30e-f3ba0788b5f5-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rjnh4\" (UID: \"6902ae83-be2b-47fb-b30e-f3ba0788b5f5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rjnh4"
Sep 09 04:07:42.916399 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:42.916373    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6902ae83-be2b-47fb-b30e-f3ba0788b5f5-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rjnh4\" (UID: \"6902ae83-be2b-47fb-b30e-f3ba0788b5f5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rjnh4"
Sep 09 04:07:42.916599 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:42.916584    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6902ae83-be2b-47fb-b30e-f3ba0788b5f5-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rjnh4\" (UID: \"6902ae83-be2b-47fb-b30e-f3ba0788b5f5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rjnh4"
Sep 09 04:07:42.921926 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:42.921902    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dtq7\" (UniqueName: \"kubernetes.io/projected/6902ae83-be2b-47fb-b30e-f3ba0788b5f5-kube-api-access-8dtq7\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rjnh4\" (UID: \"6902ae83-be2b-47fb-b30e-f3ba0788b5f5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rjnh4"
Sep 09 04:07:43.076151 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:43.076053    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rjnh4"
Sep 09 04:07:43.348670 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:43.348642    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-rjnh4"]
Sep 09 04:07:43.350760 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:07:43.350731    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6902ae83_be2b_47fb_b30e_f3ba0788b5f5.slice/crio-2ab93daf88c93f66cb1be36ef103efbc8188bfd802f8dc135c9245a78f03214e WatchSource:0}: Error finding container 2ab93daf88c93f66cb1be36ef103efbc8188bfd802f8dc135c9245a78f03214e: Status 404 returned error can't find the container with id 2ab93daf88c93f66cb1be36ef103efbc8188bfd802f8dc135c9245a78f03214e
Sep 09 04:07:44.248954 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:44.248906    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rjnh4" event={"ID":"6902ae83-be2b-47fb-b30e-f3ba0788b5f5","Type":"ContainerStarted","Data":"2ab93daf88c93f66cb1be36ef103efbc8188bfd802f8dc135c9245a78f03214e"}
Sep 09 04:07:45.255554 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:45.255513    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rjnh4" event={"ID":"6902ae83-be2b-47fb-b30e-f3ba0788b5f5","Type":"ContainerStarted","Data":"e4c606c08552800cb876e0053b84c251cb1d160867ed68c45c91d5a62c94add2"}
Sep 09 04:07:45.272624 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:45.272575    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rjnh4" podStartSLOduration=2.310089553 podStartE2EDuration="3.272558434s" totalImagesPullingTime="962.468881ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:07:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:07:45.271621361 +0000 UTC m=+590.190792272" watchObservedRunningTime="2026-09-09 04:07:45.272558434 +0000 UTC m=+590.191729345"
Sep 09 04:07:53.077019 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:53.076975    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rjnh4"
Sep 09 04:07:53.077448 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:53.077277    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rjnh4"
Sep 09 04:07:53.089912 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:53.089885    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rjnh4"
Sep 09 04:07:53.303254 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:53.303228    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rjnh4"
Sep 09 04:07:55.523657 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:55.523624    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gj7h7_7816cd49-e9c2-4f7b-b8ec-4e5711b675a5/console-operator/2.log"
Sep 09 04:07:55.527275 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:55.527253    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gj7h7_7816cd49-e9c2-4f7b-b8ec-4e5711b675a5/console-operator/2.log"
Sep 09 04:07:55.529536 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:55.529515    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rfvrc_1828d4d8-0d24-4e4d-a502-aee013a3b116/ovn-acl-logging/0.log"
Sep 09 04:07:55.532610 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:07:55.532587    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rfvrc_1828d4d8-0d24-4e4d-a502-aee013a3b116/ovn-acl-logging/0.log"
Sep 09 04:10:35.789711 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:35.789666    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"]
Sep 09 04:10:35.793290 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:35.793268    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:10:35.795630 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:35.795611    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-359adf-openshift-default-dockercfg-qw9w2\""
Sep 09 04:10:35.804329 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:35.804309    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"]
Sep 09 04:10:35.910026 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:35.909989    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0cd2205-e41e-4495-83a3-29d3da0c034f-istio-token\") pod \"e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6\" (UID: \"b0cd2205-e41e-4495-83a3-29d3da0c034f\") " pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:10:35.910225 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:35.910051    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0cd2205-e41e-4495-83a3-29d3da0c034f-workload-socket\") pod \"e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6\" (UID: \"b0cd2205-e41e-4495-83a3-29d3da0c034f\") " pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:10:35.910225 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:35.910078    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0cd2205-e41e-4495-83a3-29d3da0c034f-credential-socket\") pod \"e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6\" (UID: \"b0cd2205-e41e-4495-83a3-29d3da0c034f\") " pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:10:35.910225 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:35.910138    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wstl2\" (UniqueName: \"kubernetes.io/projected/b0cd2205-e41e-4495-83a3-29d3da0c034f-kube-api-access-wstl2\") pod \"e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6\" (UID: \"b0cd2205-e41e-4495-83a3-29d3da0c034f\") " pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:10:35.910225 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:35.910176    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0cd2205-e41e-4495-83a3-29d3da0c034f-workload-certs\") pod \"e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6\" (UID: \"b0cd2205-e41e-4495-83a3-29d3da0c034f\") " pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:10:35.910388 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:35.910244    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0cd2205-e41e-4495-83a3-29d3da0c034f-istio-envoy\") pod \"e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6\" (UID: \"b0cd2205-e41e-4495-83a3-29d3da0c034f\") " pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:10:35.910388 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:35.910284    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0cd2205-e41e-4495-83a3-29d3da0c034f-istio-data\") pod \"e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6\" (UID: \"b0cd2205-e41e-4495-83a3-29d3da0c034f\") " pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:10:35.910388 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:35.910310    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0cd2205-e41e-4495-83a3-29d3da0c034f-istio-podinfo\") pod \"e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6\" (UID: \"b0cd2205-e41e-4495-83a3-29d3da0c034f\") " pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:10:35.910388 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:35.910352    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0cd2205-e41e-4495-83a3-29d3da0c034f-istiod-ca-cert\") pod \"e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6\" (UID: \"b0cd2205-e41e-4495-83a3-29d3da0c034f\") " pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:10:36.011258 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:36.011225    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0cd2205-e41e-4495-83a3-29d3da0c034f-workload-socket\") pod \"e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6\" (UID: \"b0cd2205-e41e-4495-83a3-29d3da0c034f\") " pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:10:36.011258 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:36.011258    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0cd2205-e41e-4495-83a3-29d3da0c034f-credential-socket\") pod \"e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6\" (UID: \"b0cd2205-e41e-4495-83a3-29d3da0c034f\") " pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:10:36.011493 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:36.011287    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wstl2\" (UniqueName: \"kubernetes.io/projected/b0cd2205-e41e-4495-83a3-29d3da0c034f-kube-api-access-wstl2\") pod \"e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6\" (UID: \"b0cd2205-e41e-4495-83a3-29d3da0c034f\") " pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:10:36.011493 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:36.011308    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0cd2205-e41e-4495-83a3-29d3da0c034f-workload-certs\") pod \"e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6\" (UID: \"b0cd2205-e41e-4495-83a3-29d3da0c034f\") " pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:10:36.011493 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:36.011336    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0cd2205-e41e-4495-83a3-29d3da0c034f-istio-envoy\") pod \"e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6\" (UID: \"b0cd2205-e41e-4495-83a3-29d3da0c034f\") " pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:10:36.011493 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:36.011359    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0cd2205-e41e-4495-83a3-29d3da0c034f-istio-data\") pod \"e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6\" (UID: \"b0cd2205-e41e-4495-83a3-29d3da0c034f\") " pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:10:36.011493 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:36.011376    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0cd2205-e41e-4495-83a3-29d3da0c034f-istio-podinfo\") pod \"e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6\" (UID: \"b0cd2205-e41e-4495-83a3-29d3da0c034f\") " pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:10:36.011493 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:36.011401    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0cd2205-e41e-4495-83a3-29d3da0c034f-istiod-ca-cert\") pod \"e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6\" (UID: \"b0cd2205-e41e-4495-83a3-29d3da0c034f\") " pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:10:36.011493 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:36.011458    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0cd2205-e41e-4495-83a3-29d3da0c034f-istio-token\") pod \"e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6\" (UID: \"b0cd2205-e41e-4495-83a3-29d3da0c034f\") " pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:10:36.011828 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:36.011745    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0cd2205-e41e-4495-83a3-29d3da0c034f-credential-socket\") pod \"e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6\" (UID: \"b0cd2205-e41e-4495-83a3-29d3da0c034f\") " pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:10:36.011828 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:36.011776    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0cd2205-e41e-4495-83a3-29d3da0c034f-workload-certs\") pod \"e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6\" (UID: \"b0cd2205-e41e-4495-83a3-29d3da0c034f\") " pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:10:36.011939 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:36.011872    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0cd2205-e41e-4495-83a3-29d3da0c034f-workload-socket\") pod \"e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6\" (UID: \"b0cd2205-e41e-4495-83a3-29d3da0c034f\") " pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:10:36.011939 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:36.011906    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0cd2205-e41e-4495-83a3-29d3da0c034f-istio-data\") pod \"e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6\" (UID: \"b0cd2205-e41e-4495-83a3-29d3da0c034f\") " pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:10:36.012271 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:36.012250    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0cd2205-e41e-4495-83a3-29d3da0c034f-istiod-ca-cert\") pod \"e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6\" (UID: \"b0cd2205-e41e-4495-83a3-29d3da0c034f\") " pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:10:36.013768 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:36.013744    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0cd2205-e41e-4495-83a3-29d3da0c034f-istio-envoy\") pod \"e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6\" (UID: \"b0cd2205-e41e-4495-83a3-29d3da0c034f\") " pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:10:36.013850 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:36.013829    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0cd2205-e41e-4495-83a3-29d3da0c034f-istio-podinfo\") pod \"e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6\" (UID: \"b0cd2205-e41e-4495-83a3-29d3da0c034f\") " pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:10:36.019216 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:36.019190    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wstl2\" (UniqueName: \"kubernetes.io/projected/b0cd2205-e41e-4495-83a3-29d3da0c034f-kube-api-access-wstl2\") pod \"e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6\" (UID: \"b0cd2205-e41e-4495-83a3-29d3da0c034f\") " pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:10:36.019621 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:36.019601    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0cd2205-e41e-4495-83a3-29d3da0c034f-istio-token\") pod \"e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6\" (UID: \"b0cd2205-e41e-4495-83a3-29d3da0c034f\") " pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:10:36.104861 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:36.104773    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:10:36.381965 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:36.381938    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"]
Sep 09 04:10:36.384676 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:10:36.384641    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0cd2205_e41e_4495_83a3_29d3da0c034f.slice/crio-cabc0bcc751b06e392fdc90fe4081e742db6f2196a7cc3bf9516132c4b8e9232 WatchSource:0}: Error finding container cabc0bcc751b06e392fdc90fe4081e742db6f2196a7cc3bf9516132c4b8e9232: Status 404 returned error can't find the container with id cabc0bcc751b06e392fdc90fe4081e742db6f2196a7cc3bf9516132c4b8e9232
Sep 09 04:10:36.386633 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:36.386617    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 04:10:36.997410 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:36.997376    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6" event={"ID":"b0cd2205-e41e-4495-83a3-29d3da0c034f","Type":"ContainerStarted","Data":"cabc0bcc751b06e392fdc90fe4081e742db6f2196a7cc3bf9516132c4b8e9232"}
Sep 09 04:10:39.187350 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:39.187307    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 09 04:10:39.187704 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:39.187400    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 09 04:10:39.187704 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:39.187452    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 09 04:10:40.016095 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:40.016055    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6" event={"ID":"b0cd2205-e41e-4495-83a3-29d3da0c034f","Type":"ContainerStarted","Data":"25edb2ff4fd72705f3109dbf6c4c986ee9ad91b716e654c84dc416b0490e7bc9"}
Sep 09 04:10:40.037670 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:40.037618    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6" podStartSLOduration=2.237344824 podStartE2EDuration="5.037602906s" totalImagesPullingTime="2.800258082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:10:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:10:40.035760515 +0000 UTC m=+764.954931437" watchObservedRunningTime="2026-09-09 04:10:40.037602906 +0000 UTC m=+764.956773816"
Sep 09 04:10:40.105064 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:40.105036    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:10:40.110140 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:40.110103    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:10:41.020657 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:41.020621    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:10:41.021478 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:41.021461    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:10:42.779620 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:42.779586    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-85732c-54589c45dd-ls9gc"]
Sep 09 04:10:42.784042 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:42.784014    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-85732c-54589c45dd-ls9gc"
Sep 09 04:10:42.786474 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:42.786446    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-85732c-serving-cert\""
Sep 09 04:10:42.787420 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:42.787401    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 09 04:10:42.787729 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:42.787711    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-thdks\""
Sep 09 04:10:42.787880 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:42.787723    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 04:10:42.787957 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:42.787762    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 09 04:10:42.794635 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:42.794609    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-85732c-54589c45dd-ls9gc"]
Sep 09 04:10:42.867765 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:42.867739    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljv96\" (UniqueName: \"kubernetes.io/projected/a06fb867-32f0-4993-9bc1-bca2e03a953b-kube-api-access-ljv96\") pod \"maas-api-e2e-worker-w4-85732c-54589c45dd-ls9gc\" (UID: \"a06fb867-32f0-4993-9bc1-bca2e03a953b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-85732c-54589c45dd-ls9gc"
Sep 09 04:10:42.867916 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:42.867789    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a06fb867-32f0-4993-9bc1-bca2e03a953b-metrics-certs\") pod \"maas-api-e2e-worker-w4-85732c-54589c45dd-ls9gc\" (UID: \"a06fb867-32f0-4993-9bc1-bca2e03a953b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-85732c-54589c45dd-ls9gc"
Sep 09 04:10:42.867916 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:42.867829    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a06fb867-32f0-4993-9bc1-bca2e03a953b-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-85732c-54589c45dd-ls9gc\" (UID: \"a06fb867-32f0-4993-9bc1-bca2e03a953b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-85732c-54589c45dd-ls9gc"
Sep 09 04:10:42.867916 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:42.867861    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a06fb867-32f0-4993-9bc1-bca2e03a953b-maas-api-tls\") pod \"maas-api-e2e-worker-w4-85732c-54589c45dd-ls9gc\" (UID: \"a06fb867-32f0-4993-9bc1-bca2e03a953b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-85732c-54589c45dd-ls9gc"
Sep 09 04:10:42.968673 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:42.968641    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a06fb867-32f0-4993-9bc1-bca2e03a953b-metrics-certs\") pod \"maas-api-e2e-worker-w4-85732c-54589c45dd-ls9gc\" (UID: \"a06fb867-32f0-4993-9bc1-bca2e03a953b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-85732c-54589c45dd-ls9gc"
Sep 09 04:10:42.968837 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:42.968693    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a06fb867-32f0-4993-9bc1-bca2e03a953b-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-85732c-54589c45dd-ls9gc\" (UID: \"a06fb867-32f0-4993-9bc1-bca2e03a953b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-85732c-54589c45dd-ls9gc"
Sep 09 04:10:42.968837 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:42.968723    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a06fb867-32f0-4993-9bc1-bca2e03a953b-maas-api-tls\") pod \"maas-api-e2e-worker-w4-85732c-54589c45dd-ls9gc\" (UID: \"a06fb867-32f0-4993-9bc1-bca2e03a953b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-85732c-54589c45dd-ls9gc"
Sep 09 04:10:42.968837 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:42.968754    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljv96\" (UniqueName: \"kubernetes.io/projected/a06fb867-32f0-4993-9bc1-bca2e03a953b-kube-api-access-ljv96\") pod \"maas-api-e2e-worker-w4-85732c-54589c45dd-ls9gc\" (UID: \"a06fb867-32f0-4993-9bc1-bca2e03a953b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-85732c-54589c45dd-ls9gc"
Sep 09 04:10:42.968988 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:10:42.968868    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-85732c-serving-cert: secret "maas-api-e2e-worker-w4-85732c-serving-cert" not found
Sep 09 04:10:42.968988 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:10:42.968942    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a06fb867-32f0-4993-9bc1-bca2e03a953b-maas-api-tls podName:a06fb867-32f0-4993-9bc1-bca2e03a953b nodeName:}" failed. No retries permitted until 2026-09-09 04:10:43.468915703 +0000 UTC m=+768.388086593 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a06fb867-32f0-4993-9bc1-bca2e03a953b-maas-api-tls") pod "maas-api-e2e-worker-w4-85732c-54589c45dd-ls9gc" (UID: "a06fb867-32f0-4993-9bc1-bca2e03a953b") : secret "maas-api-e2e-worker-w4-85732c-serving-cert" not found
Sep 09 04:10:42.969492 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:42.969468    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a06fb867-32f0-4993-9bc1-bca2e03a953b-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-85732c-54589c45dd-ls9gc\" (UID: \"a06fb867-32f0-4993-9bc1-bca2e03a953b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-85732c-54589c45dd-ls9gc"
Sep 09 04:10:42.971029 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:42.971007    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a06fb867-32f0-4993-9bc1-bca2e03a953b-metrics-certs\") pod \"maas-api-e2e-worker-w4-85732c-54589c45dd-ls9gc\" (UID: \"a06fb867-32f0-4993-9bc1-bca2e03a953b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-85732c-54589c45dd-ls9gc"
Sep 09 04:10:42.977417 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:42.977388    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljv96\" (UniqueName: \"kubernetes.io/projected/a06fb867-32f0-4993-9bc1-bca2e03a953b-kube-api-access-ljv96\") pod \"maas-api-e2e-worker-w4-85732c-54589c45dd-ls9gc\" (UID: \"a06fb867-32f0-4993-9bc1-bca2e03a953b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-85732c-54589c45dd-ls9gc"
Sep 09 04:10:43.431764 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:43.431732    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-85732c-6d5cd7c4b6-ffzdb"]
Sep 09 04:10:43.435233 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:43.435214    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-85732c-6d5cd7c4b6-ffzdb"
Sep 09 04:10:43.437878 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:43.437856    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-85732c-dockercfg-fdlmk\""
Sep 09 04:10:43.438002 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:43.437940    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-85732c\""
Sep 09 04:10:43.442154 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:43.442130    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-85732c-6d5cd7c4b6-ffzdb"]
Sep 09 04:10:43.445775 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:43.445751    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-85732c-65749c6df9-c844k"]
Sep 09 04:10:43.449564 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:43.449541    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-85732c-65749c6df9-c844k"
Sep 09 04:10:43.461593 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:43.461570    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-85732c-65749c6df9-c844k"]
Sep 09 04:10:43.473003 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:43.472974    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/01e315ac-1215-48b4-87fe-7a1108db4cc1-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-85732c-6d5cd7c4b6-ffzdb\" (UID: \"01e315ac-1215-48b4-87fe-7a1108db4cc1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-85732c-6d5cd7c4b6-ffzdb"
Sep 09 04:10:43.473101 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:43.473009    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-659wf\" (UniqueName: \"kubernetes.io/projected/20d1cfab-32f0-4f34-a789-b27e0c009583-kube-api-access-659wf\") pod \"payload-processing-e2e-worker-w4-85732c-65749c6df9-c844k\" (UID: \"20d1cfab-32f0-4f34-a789-b27e0c009583\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-85732c-65749c6df9-c844k"
Sep 09 04:10:43.473101 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:43.473041    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltf9l\" (UniqueName: \"kubernetes.io/projected/01e315ac-1215-48b4-87fe-7a1108db4cc1-kube-api-access-ltf9l\") pod \"payload-pre-processing-e2e-worker-w4-85732c-6d5cd7c4b6-ffzdb\" (UID: \"01e315ac-1215-48b4-87fe-7a1108db4cc1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-85732c-6d5cd7c4b6-ffzdb"
Sep 09 04:10:43.473204 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:43.473134    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20d1cfab-32f0-4f34-a789-b27e0c009583-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-85732c-65749c6df9-c844k\" (UID: \"20d1cfab-32f0-4f34-a789-b27e0c009583\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-85732c-65749c6df9-c844k"
Sep 09 04:10:43.473204 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:43.473187    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a06fb867-32f0-4993-9bc1-bca2e03a953b-maas-api-tls\") pod \"maas-api-e2e-worker-w4-85732c-54589c45dd-ls9gc\" (UID: \"a06fb867-32f0-4993-9bc1-bca2e03a953b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-85732c-54589c45dd-ls9gc"
Sep 09 04:10:43.475401 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:43.475384    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a06fb867-32f0-4993-9bc1-bca2e03a953b-maas-api-tls\") pod \"maas-api-e2e-worker-w4-85732c-54589c45dd-ls9gc\" (UID: \"a06fb867-32f0-4993-9bc1-bca2e03a953b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-85732c-54589c45dd-ls9gc"
Sep 09 04:10:43.574469 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:43.574432    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/01e315ac-1215-48b4-87fe-7a1108db4cc1-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-85732c-6d5cd7c4b6-ffzdb\" (UID: \"01e315ac-1215-48b4-87fe-7a1108db4cc1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-85732c-6d5cd7c4b6-ffzdb"
Sep 09 04:10:43.574640 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:43.574482    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-659wf\" (UniqueName: \"kubernetes.io/projected/20d1cfab-32f0-4f34-a789-b27e0c009583-kube-api-access-659wf\") pod \"payload-processing-e2e-worker-w4-85732c-65749c6df9-c844k\" (UID: \"20d1cfab-32f0-4f34-a789-b27e0c009583\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-85732c-65749c6df9-c844k"
Sep 09 04:10:43.574640 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:43.574524    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltf9l\" (UniqueName: \"kubernetes.io/projected/01e315ac-1215-48b4-87fe-7a1108db4cc1-kube-api-access-ltf9l\") pod \"payload-pre-processing-e2e-worker-w4-85732c-6d5cd7c4b6-ffzdb\" (UID: \"01e315ac-1215-48b4-87fe-7a1108db4cc1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-85732c-6d5cd7c4b6-ffzdb"
Sep 09 04:10:43.574640 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:43.574550    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20d1cfab-32f0-4f34-a789-b27e0c009583-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-85732c-65749c6df9-c844k\" (UID: \"20d1cfab-32f0-4f34-a789-b27e0c009583\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-85732c-65749c6df9-c844k"
Sep 09 04:10:43.575136 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:43.575098    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/01e315ac-1215-48b4-87fe-7a1108db4cc1-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-85732c-6d5cd7c4b6-ffzdb\" (UID: \"01e315ac-1215-48b4-87fe-7a1108db4cc1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-85732c-6d5cd7c4b6-ffzdb"
Sep 09 04:10:43.575227 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:43.575211    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20d1cfab-32f0-4f34-a789-b27e0c009583-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-85732c-65749c6df9-c844k\" (UID: \"20d1cfab-32f0-4f34-a789-b27e0c009583\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-85732c-65749c6df9-c844k"
Sep 09 04:10:43.582464 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:43.582445    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-659wf\" (UniqueName: \"kubernetes.io/projected/20d1cfab-32f0-4f34-a789-b27e0c009583-kube-api-access-659wf\") pod \"payload-processing-e2e-worker-w4-85732c-65749c6df9-c844k\" (UID: \"20d1cfab-32f0-4f34-a789-b27e0c009583\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-85732c-65749c6df9-c844k"
Sep 09 04:10:43.582563 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:43.582521    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltf9l\" (UniqueName: \"kubernetes.io/projected/01e315ac-1215-48b4-87fe-7a1108db4cc1-kube-api-access-ltf9l\") pod \"payload-pre-processing-e2e-worker-w4-85732c-6d5cd7c4b6-ffzdb\" (UID: \"01e315ac-1215-48b4-87fe-7a1108db4cc1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-85732c-6d5cd7c4b6-ffzdb"
Sep 09 04:10:43.696652 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:43.696580    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-85732c-54589c45dd-ls9gc"
Sep 09 04:10:43.748528 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:43.748294    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-85732c-6d5cd7c4b6-ffzdb"
Sep 09 04:10:43.762150 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:43.761864    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-85732c-65749c6df9-c844k"
Sep 09 04:10:43.967817 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:43.967790    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-85732c-54589c45dd-ls9gc"]
Sep 09 04:10:43.970045 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:10:43.970025    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda06fb867_32f0_4993_9bc1_bca2e03a953b.slice/crio-04ae9f59e6c8d3ec9413a2397e38fff391fb71c16ed067c2ccf095c20d633b70 WatchSource:0}: Error finding container 04ae9f59e6c8d3ec9413a2397e38fff391fb71c16ed067c2ccf095c20d633b70: Status 404 returned error can't find the container with id 04ae9f59e6c8d3ec9413a2397e38fff391fb71c16ed067c2ccf095c20d633b70
Sep 09 04:10:44.034536 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:44.034507    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-85732c-54589c45dd-ls9gc" event={"ID":"a06fb867-32f0-4993-9bc1-bca2e03a953b","Type":"ContainerStarted","Data":"04ae9f59e6c8d3ec9413a2397e38fff391fb71c16ed067c2ccf095c20d633b70"}
Sep 09 04:10:44.039544 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:44.039523    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-85732c-6d5cd7c4b6-ffzdb"]
Sep 09 04:10:44.041521 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:10:44.041495    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01e315ac_1215_48b4_87fe_7a1108db4cc1.slice/crio-b1587a279da951084cad9e9a66286f10053c68e68d99a174aeb3c6d4e56c8e09 WatchSource:0}: Error finding container b1587a279da951084cad9e9a66286f10053c68e68d99a174aeb3c6d4e56c8e09: Status 404 returned error can't find the container with id b1587a279da951084cad9e9a66286f10053c68e68d99a174aeb3c6d4e56c8e09
Sep 09 04:10:44.044373 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:44.044356    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-85732c-65749c6df9-c844k"]
Sep 09 04:10:44.046361 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:10:44.046325    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20d1cfab_32f0_4f34_a789_b27e0c009583.slice/crio-9faadd1e6da4bfaeec944fb8ef208c340e748b652667ad8f5fe0e0577a7dbd56 WatchSource:0}: Error finding container 9faadd1e6da4bfaeec944fb8ef208c340e748b652667ad8f5fe0e0577a7dbd56: Status 404 returned error can't find the container with id 9faadd1e6da4bfaeec944fb8ef208c340e748b652667ad8f5fe0e0577a7dbd56
Sep 09 04:10:44.485348 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:44.485309    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-359adf-d7487ddc4-8q8rb"]
Sep 09 04:10:44.488765 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:44.488740    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-359adf-d7487ddc4-8q8rb"
Sep 09 04:10:44.491081 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:44.491061    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-359adf\""
Sep 09 04:10:44.491200 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:44.491063    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-359adf-dockercfg-wnsxd\""
Sep 09 04:10:44.499388 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:44.499367    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-359adf-d7487ddc4-8q8rb"]
Sep 09 04:10:44.519099 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:44.519069    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-359adf-546f7f8c97-gxn5v"]
Sep 09 04:10:44.522755 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:44.522732    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-359adf-546f7f8c97-gxn5v"
Sep 09 04:10:44.532657 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:44.532635    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-359adf-546f7f8c97-gxn5v"]
Sep 09 04:10:44.582277 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:44.582239    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/add487c3-333d-482b-bdc8-fefdb027ca88-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-359adf-546f7f8c97-gxn5v\" (UID: \"add487c3-333d-482b-bdc8-fefdb027ca88\") " pod="openshift-ingress/payload-processing-e2e-inf-a-359adf-546f7f8c97-gxn5v"
Sep 09 04:10:44.582466 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:44.582304    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ddbb781-2f2f-4b6a-888d-feedf9fbd159-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-359adf-d7487ddc4-8q8rb\" (UID: \"4ddbb781-2f2f-4b6a-888d-feedf9fbd159\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-359adf-d7487ddc4-8q8rb"
Sep 09 04:10:44.582466 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:44.582433    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmpmv\" (UniqueName: \"kubernetes.io/projected/4ddbb781-2f2f-4b6a-888d-feedf9fbd159-kube-api-access-bmpmv\") pod \"payload-pre-processing-e2e-inf-a-359adf-d7487ddc4-8q8rb\" (UID: \"4ddbb781-2f2f-4b6a-888d-feedf9fbd159\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-359adf-d7487ddc4-8q8rb"
Sep 09 04:10:44.582586 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:44.582486    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56lrr\" (UniqueName: \"kubernetes.io/projected/add487c3-333d-482b-bdc8-fefdb027ca88-kube-api-access-56lrr\") pod \"payload-processing-e2e-inf-a-359adf-546f7f8c97-gxn5v\" (UID: \"add487c3-333d-482b-bdc8-fefdb027ca88\") " pod="openshift-ingress/payload-processing-e2e-inf-a-359adf-546f7f8c97-gxn5v"
Sep 09 04:10:44.683680 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:44.683637    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmpmv\" (UniqueName: \"kubernetes.io/projected/4ddbb781-2f2f-4b6a-888d-feedf9fbd159-kube-api-access-bmpmv\") pod \"payload-pre-processing-e2e-inf-a-359adf-d7487ddc4-8q8rb\" (UID: \"4ddbb781-2f2f-4b6a-888d-feedf9fbd159\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-359adf-d7487ddc4-8q8rb"
Sep 09 04:10:44.683894 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:44.683710    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56lrr\" (UniqueName: \"kubernetes.io/projected/add487c3-333d-482b-bdc8-fefdb027ca88-kube-api-access-56lrr\") pod \"payload-processing-e2e-inf-a-359adf-546f7f8c97-gxn5v\" (UID: \"add487c3-333d-482b-bdc8-fefdb027ca88\") " pod="openshift-ingress/payload-processing-e2e-inf-a-359adf-546f7f8c97-gxn5v"
Sep 09 04:10:44.683894 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:44.683743    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/add487c3-333d-482b-bdc8-fefdb027ca88-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-359adf-546f7f8c97-gxn5v\" (UID: \"add487c3-333d-482b-bdc8-fefdb027ca88\") " pod="openshift-ingress/payload-processing-e2e-inf-a-359adf-546f7f8c97-gxn5v"
Sep 09 04:10:44.683894 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:44.683781    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ddbb781-2f2f-4b6a-888d-feedf9fbd159-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-359adf-d7487ddc4-8q8rb\" (UID: \"4ddbb781-2f2f-4b6a-888d-feedf9fbd159\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-359adf-d7487ddc4-8q8rb"
Sep 09 04:10:44.684797 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:44.684750    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ddbb781-2f2f-4b6a-888d-feedf9fbd159-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-359adf-d7487ddc4-8q8rb\" (UID: \"4ddbb781-2f2f-4b6a-888d-feedf9fbd159\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-359adf-d7487ddc4-8q8rb"
Sep 09 04:10:44.684892 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:44.684874    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/add487c3-333d-482b-bdc8-fefdb027ca88-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-359adf-546f7f8c97-gxn5v\" (UID: \"add487c3-333d-482b-bdc8-fefdb027ca88\") " pod="openshift-ingress/payload-processing-e2e-inf-a-359adf-546f7f8c97-gxn5v"
Sep 09 04:10:44.694139 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:44.694098    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56lrr\" (UniqueName: \"kubernetes.io/projected/add487c3-333d-482b-bdc8-fefdb027ca88-kube-api-access-56lrr\") pod \"payload-processing-e2e-inf-a-359adf-546f7f8c97-gxn5v\" (UID: \"add487c3-333d-482b-bdc8-fefdb027ca88\") " pod="openshift-ingress/payload-processing-e2e-inf-a-359adf-546f7f8c97-gxn5v"
Sep 09 04:10:44.696697 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:44.696650    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmpmv\" (UniqueName: \"kubernetes.io/projected/4ddbb781-2f2f-4b6a-888d-feedf9fbd159-kube-api-access-bmpmv\") pod \"payload-pre-processing-e2e-inf-a-359adf-d7487ddc4-8q8rb\" (UID: \"4ddbb781-2f2f-4b6a-888d-feedf9fbd159\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-359adf-d7487ddc4-8q8rb"
Sep 09 04:10:44.800592 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:44.800499    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-359adf-d7487ddc4-8q8rb"
Sep 09 04:10:44.836035 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:44.835952    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-359adf-546f7f8c97-gxn5v"
Sep 09 04:10:45.042745 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:45.042286    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-85732c-65749c6df9-c844k" event={"ID":"20d1cfab-32f0-4f34-a789-b27e0c009583","Type":"ContainerStarted","Data":"79f196c71e9cbbdb103683659221a8f18053ddd565de9272d7f22d5a8a4e59c4"}
Sep 09 04:10:45.042745 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:45.042355    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-85732c-65749c6df9-c844k" event={"ID":"20d1cfab-32f0-4f34-a789-b27e0c009583","Type":"ContainerStarted","Data":"9faadd1e6da4bfaeec944fb8ef208c340e748b652667ad8f5fe0e0577a7dbd56"}
Sep 09 04:10:45.042745 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:45.042606    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-85732c-65749c6df9-c844k"
Sep 09 04:10:45.045418 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:45.045383    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-85732c-6d5cd7c4b6-ffzdb" event={"ID":"01e315ac-1215-48b4-87fe-7a1108db4cc1","Type":"ContainerStarted","Data":"4ccce5697efe18706bbd99d7606668e866f38e327b2216c8b00c77e11ac88ff3"}
Sep 09 04:10:45.045544 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:45.045431    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-85732c-6d5cd7c4b6-ffzdb" event={"ID":"01e315ac-1215-48b4-87fe-7a1108db4cc1","Type":"ContainerStarted","Data":"b1587a279da951084cad9e9a66286f10053c68e68d99a174aeb3c6d4e56c8e09"}
Sep 09 04:10:45.045544 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:45.045530    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-85732c-6d5cd7c4b6-ffzdb"
Sep 09 04:10:45.066774 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:45.066685    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-85732c-65749c6df9-c844k" podStartSLOduration=2.066664928 podStartE2EDuration="2.066664928s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:10:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:10:45.065887122 +0000 UTC m=+769.985058031" watchObservedRunningTime="2026-09-09 04:10:45.066664928 +0000 UTC m=+769.985835839"
Sep 09 04:10:45.085897 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:45.085846    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-85732c-6d5cd7c4b6-ffzdb" podStartSLOduration=2.085827156 podStartE2EDuration="2.085827156s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:10:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:10:45.083385097 +0000 UTC m=+770.002556008" watchObservedRunningTime="2026-09-09 04:10:45.085827156 +0000 UTC m=+770.004998070"
Sep 09 04:10:45.090659 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:45.090392    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-359adf-d7487ddc4-8q8rb"]
Sep 09 04:10:45.093275 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:10:45.093245    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ddbb781_2f2f_4b6a_888d_feedf9fbd159.slice/crio-16a5abf04b77c8a63e63312f2ef98e9e9931e7d5eb26175d6fb064964efa1843 WatchSource:0}: Error finding container 16a5abf04b77c8a63e63312f2ef98e9e9931e7d5eb26175d6fb064964efa1843: Status 404 returned error can't find the container with id 16a5abf04b77c8a63e63312f2ef98e9e9931e7d5eb26175d6fb064964efa1843
Sep 09 04:10:45.140843 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:45.140819    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-359adf-546f7f8c97-gxn5v"]
Sep 09 04:10:45.147764 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:10:45.147735    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadd487c3_333d_482b_bdc8_fefdb027ca88.slice/crio-1b8d7721bc222df3afac8c32e8a1aa5b0a964f8e6057362380850d7bc1044922 WatchSource:0}: Error finding container 1b8d7721bc222df3afac8c32e8a1aa5b0a964f8e6057362380850d7bc1044922: Status 404 returned error can't find the container with id 1b8d7721bc222df3afac8c32e8a1aa5b0a964f8e6057362380850d7bc1044922
Sep 09 04:10:46.054449 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:46.054413    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-359adf-546f7f8c97-gxn5v" event={"ID":"add487c3-333d-482b-bdc8-fefdb027ca88","Type":"ContainerStarted","Data":"43e550d24ba63bb482aa1d4ec805d2a152bac2cf40322cc0ac1576ee6bd5c2f6"}
Sep 09 04:10:46.054905 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:46.054545    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-359adf-546f7f8c97-gxn5v" event={"ID":"add487c3-333d-482b-bdc8-fefdb027ca88","Type":"ContainerStarted","Data":"1b8d7721bc222df3afac8c32e8a1aa5b0a964f8e6057362380850d7bc1044922"}
Sep 09 04:10:46.054905 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:46.054805    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-359adf-546f7f8c97-gxn5v"
Sep 09 04:10:46.057288 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:46.057210    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-359adf-d7487ddc4-8q8rb" event={"ID":"4ddbb781-2f2f-4b6a-888d-feedf9fbd159","Type":"ContainerStarted","Data":"8f125f0de2a1425f25361be4dfa3cf68f0c3e2a1446a30805fb4cb15962a081f"}
Sep 09 04:10:46.057288 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:46.057263    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-359adf-d7487ddc4-8q8rb" event={"ID":"4ddbb781-2f2f-4b6a-888d-feedf9fbd159","Type":"ContainerStarted","Data":"16a5abf04b77c8a63e63312f2ef98e9e9931e7d5eb26175d6fb064964efa1843"}
Sep 09 04:10:46.057679 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:46.057659    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-359adf-d7487ddc4-8q8rb"
Sep 09 04:10:46.076201 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:46.076095    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-359adf-546f7f8c97-gxn5v" podStartSLOduration=2.076079186 podStartE2EDuration="2.076079186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:10:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:10:46.074968157 +0000 UTC m=+770.994139078" watchObservedRunningTime="2026-09-09 04:10:46.076079186 +0000 UTC m=+770.995250097"
Sep 09 04:10:46.097525 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:46.097483    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-359adf-d7487ddc4-8q8rb" podStartSLOduration=2.097467493 podStartE2EDuration="2.097467493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:10:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:10:46.096068338 +0000 UTC m=+771.015239250" watchObservedRunningTime="2026-09-09 04:10:46.097467493 +0000 UTC m=+771.016638402"
Sep 09 04:10:47.063867 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:47.063772    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-85732c-54589c45dd-ls9gc" event={"ID":"a06fb867-32f0-4993-9bc1-bca2e03a953b","Type":"ContainerStarted","Data":"3e271a2c417842a7eedb9ab4ec5c6d3f44ae94d7922e4a1ccd6025407438f759"}
Sep 09 04:10:47.064292 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:47.064180    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-85732c-54589c45dd-ls9gc"
Sep 09 04:10:47.080880 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:47.080836    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-85732c-54589c45dd-ls9gc" podStartSLOduration=2.374581723 podStartE2EDuration="5.080823266s" totalImagesPullingTime="2.706241543s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:10:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:10:47.079893732 +0000 UTC m=+771.999064641" watchObservedRunningTime="2026-09-09 04:10:47.080823266 +0000 UTC m=+771.999994174"
Sep 09 04:10:49.586374 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:49.586327    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-4ad5fa-797d7dd8cd-mlw76"]
Sep 09 04:10:49.590504 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:49.590478    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-4ad5fa-797d7dd8cd-mlw76"
Sep 09 04:10:49.593066 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:49.593035    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-4ad5fa\""
Sep 09 04:10:49.598482 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:49.598459    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-4ad5fa-797d7dd8cd-mlw76"]
Sep 09 04:10:49.604074 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:49.604053    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-4ad5fa-9464748d7-98ntv"]
Sep 09 04:10:49.607403 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:49.607383    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-4ad5fa-9464748d7-98ntv"
Sep 09 04:10:49.609594 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:49.609576    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-4ad5fa-dockercfg-8b997\""
Sep 09 04:10:49.620689 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:49.620666    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-4ad5fa-9464748d7-98ntv"]
Sep 09 04:10:49.632992 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:49.632967    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nvjt\" (UniqueName: \"kubernetes.io/projected/83e7ffea-fc29-4812-9dcd-4ca3a83d8dba-kube-api-access-7nvjt\") pod \"payload-pre-processing-e2e-autores-4ad5fa-797d7dd8cd-mlw76\" (UID: \"83e7ffea-fc29-4812-9dcd-4ca3a83d8dba\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-4ad5fa-797d7dd8cd-mlw76"
Sep 09 04:10:49.633092 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:49.633026    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83e7ffea-fc29-4812-9dcd-4ca3a83d8dba-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-4ad5fa-797d7dd8cd-mlw76\" (UID: \"83e7ffea-fc29-4812-9dcd-4ca3a83d8dba\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-4ad5fa-797d7dd8cd-mlw76"
Sep 09 04:10:49.633175 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:49.633152    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pg8kq\" (UniqueName: \"kubernetes.io/projected/96564240-e02e-40ca-a3a4-11615480686c-kube-api-access-pg8kq\") pod \"payload-processing-e2e-autores-4ad5fa-9464748d7-98ntv\" (UID: \"96564240-e02e-40ca-a3a4-11615480686c\") " pod="openshift-ingress/payload-processing-e2e-autores-4ad5fa-9464748d7-98ntv"
Sep 09 04:10:49.633246 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:49.633220    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96564240-e02e-40ca-a3a4-11615480686c-plugins-config-volume\") pod \"payload-processing-e2e-autores-4ad5fa-9464748d7-98ntv\" (UID: \"96564240-e02e-40ca-a3a4-11615480686c\") " pod="openshift-ingress/payload-processing-e2e-autores-4ad5fa-9464748d7-98ntv"
Sep 09 04:10:49.734137 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:49.734092    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pg8kq\" (UniqueName: \"kubernetes.io/projected/96564240-e02e-40ca-a3a4-11615480686c-kube-api-access-pg8kq\") pod \"payload-processing-e2e-autores-4ad5fa-9464748d7-98ntv\" (UID: \"96564240-e02e-40ca-a3a4-11615480686c\") " pod="openshift-ingress/payload-processing-e2e-autores-4ad5fa-9464748d7-98ntv"
Sep 09 04:10:49.734285 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:49.734177    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96564240-e02e-40ca-a3a4-11615480686c-plugins-config-volume\") pod \"payload-processing-e2e-autores-4ad5fa-9464748d7-98ntv\" (UID: \"96564240-e02e-40ca-a3a4-11615480686c\") " pod="openshift-ingress/payload-processing-e2e-autores-4ad5fa-9464748d7-98ntv"
Sep 09 04:10:49.734285 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:49.734230    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nvjt\" (UniqueName: \"kubernetes.io/projected/83e7ffea-fc29-4812-9dcd-4ca3a83d8dba-kube-api-access-7nvjt\") pod \"payload-pre-processing-e2e-autores-4ad5fa-797d7dd8cd-mlw76\" (UID: \"83e7ffea-fc29-4812-9dcd-4ca3a83d8dba\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-4ad5fa-797d7dd8cd-mlw76"
Sep 09 04:10:49.734285 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:49.734273    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83e7ffea-fc29-4812-9dcd-4ca3a83d8dba-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-4ad5fa-797d7dd8cd-mlw76\" (UID: \"83e7ffea-fc29-4812-9dcd-4ca3a83d8dba\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-4ad5fa-797d7dd8cd-mlw76"
Sep 09 04:10:49.734895 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:49.734875    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96564240-e02e-40ca-a3a4-11615480686c-plugins-config-volume\") pod \"payload-processing-e2e-autores-4ad5fa-9464748d7-98ntv\" (UID: \"96564240-e02e-40ca-a3a4-11615480686c\") " pod="openshift-ingress/payload-processing-e2e-autores-4ad5fa-9464748d7-98ntv"
Sep 09 04:10:49.735006 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:49.734989    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83e7ffea-fc29-4812-9dcd-4ca3a83d8dba-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-4ad5fa-797d7dd8cd-mlw76\" (UID: \"83e7ffea-fc29-4812-9dcd-4ca3a83d8dba\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-4ad5fa-797d7dd8cd-mlw76"
Sep 09 04:10:49.741713 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:49.741691    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pg8kq\" (UniqueName: \"kubernetes.io/projected/96564240-e02e-40ca-a3a4-11615480686c-kube-api-access-pg8kq\") pod \"payload-processing-e2e-autores-4ad5fa-9464748d7-98ntv\" (UID: \"96564240-e02e-40ca-a3a4-11615480686c\") " pod="openshift-ingress/payload-processing-e2e-autores-4ad5fa-9464748d7-98ntv"
Sep 09 04:10:49.741814 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:49.741790    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nvjt\" (UniqueName: \"kubernetes.io/projected/83e7ffea-fc29-4812-9dcd-4ca3a83d8dba-kube-api-access-7nvjt\") pod \"payload-pre-processing-e2e-autores-4ad5fa-797d7dd8cd-mlw76\" (UID: \"83e7ffea-fc29-4812-9dcd-4ca3a83d8dba\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-4ad5fa-797d7dd8cd-mlw76"
Sep 09 04:10:49.901567 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:49.901531    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-4ad5fa-797d7dd8cd-mlw76"
Sep 09 04:10:49.920224 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:49.920194    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-4ad5fa-9464748d7-98ntv"
Sep 09 04:10:50.198213 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:50.198182    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-4ad5fa-797d7dd8cd-mlw76"]
Sep 09 04:10:50.199937 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:10:50.199900    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83e7ffea_fc29_4812_9dcd_4ca3a83d8dba.slice/crio-da2585d0f7d072219ad682ec3df2d4d59049d9d7265b1a9b16dde9d02b1fe542 WatchSource:0}: Error finding container da2585d0f7d072219ad682ec3df2d4d59049d9d7265b1a9b16dde9d02b1fe542: Status 404 returned error can't find the container with id da2585d0f7d072219ad682ec3df2d4d59049d9d7265b1a9b16dde9d02b1fe542
Sep 09 04:10:50.220507 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:50.220486    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-4ad5fa-9464748d7-98ntv"]
Sep 09 04:10:50.222159 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:10:50.222135    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96564240_e02e_40ca_a3a4_11615480686c.slice/crio-4790830cba8e04b518c3b01bfdea1402d670121b39eb2f2c2848e2f6319994e3 WatchSource:0}: Error finding container 4790830cba8e04b518c3b01bfdea1402d670121b39eb2f2c2848e2f6319994e3: Status 404 returned error can't find the container with id 4790830cba8e04b518c3b01bfdea1402d670121b39eb2f2c2848e2f6319994e3
Sep 09 04:10:51.105585 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:51.105541    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-4ad5fa-797d7dd8cd-mlw76" event={"ID":"83e7ffea-fc29-4812-9dcd-4ca3a83d8dba","Type":"ContainerStarted","Data":"5df30da36d5c634a49d8cba8b2a6f11cf8fab1f93ad9eb81639f50fd1534e417"}
Sep 09 04:10:51.106021 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:51.105680    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-4ad5fa-797d7dd8cd-mlw76"
Sep 09 04:10:51.106021 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:51.105707    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-4ad5fa-797d7dd8cd-mlw76" event={"ID":"83e7ffea-fc29-4812-9dcd-4ca3a83d8dba","Type":"ContainerStarted","Data":"da2585d0f7d072219ad682ec3df2d4d59049d9d7265b1a9b16dde9d02b1fe542"}
Sep 09 04:10:51.107720 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:51.107679    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-4ad5fa-9464748d7-98ntv" event={"ID":"96564240-e02e-40ca-a3a4-11615480686c","Type":"ContainerStarted","Data":"d3b52744d6c372b35f646655e37c630685f1f87881a3cbb087f1b2319def1afc"}
Sep 09 04:10:51.107854 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:51.107741    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-4ad5fa-9464748d7-98ntv" event={"ID":"96564240-e02e-40ca-a3a4-11615480686c","Type":"ContainerStarted","Data":"4790830cba8e04b518c3b01bfdea1402d670121b39eb2f2c2848e2f6319994e3"}
Sep 09 04:10:51.107854 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:51.107797    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-4ad5fa-9464748d7-98ntv"
Sep 09 04:10:51.123968 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:51.123927    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-4ad5fa-797d7dd8cd-mlw76" podStartSLOduration=2.123913103 podStartE2EDuration="2.123913103s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:10:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:10:51.122175223 +0000 UTC m=+776.041346133" watchObservedRunningTime="2026-09-09 04:10:51.123913103 +0000 UTC m=+776.043084011"
Sep 09 04:10:51.138249 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:51.138204    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-4ad5fa-9464748d7-98ntv" podStartSLOduration=2.138194245 podStartE2EDuration="2.138194245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:10:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:10:51.137255296 +0000 UTC m=+776.056426199" watchObservedRunningTime="2026-09-09 04:10:51.138194245 +0000 UTC m=+776.057365153"
Sep 09 04:10:51.179131 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:51.178643    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-45a82a13-8466c878f-fdxzh"]
Sep 09 04:10:51.183140 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:51.182603    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-45a82a13-8466c878f-fdxzh"
Sep 09 04:10:51.186741 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:51.186621    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-45a82a13\""
Sep 09 04:10:51.187037 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:51.187018    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-45a82a13-dockercfg-cswt5\""
Sep 09 04:10:51.196129 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:51.194646    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-45a82a13-8466c878f-fdxzh"]
Sep 09 04:10:51.250168 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:51.250134    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e92cc6d5-ef50-4517-b244-a7fff1779f19-plugins-config-volume\") pod \"payload-processing-e2e-ait-45a82a13-8466c878f-fdxzh\" (UID: \"e92cc6d5-ef50-4517-b244-a7fff1779f19\") " pod="openshift-ingress/payload-processing-e2e-ait-45a82a13-8466c878f-fdxzh"
Sep 09 04:10:51.250347 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:51.250309    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9jcz\" (UniqueName: \"kubernetes.io/projected/e92cc6d5-ef50-4517-b244-a7fff1779f19-kube-api-access-b9jcz\") pod \"payload-processing-e2e-ait-45a82a13-8466c878f-fdxzh\" (UID: \"e92cc6d5-ef50-4517-b244-a7fff1779f19\") " pod="openshift-ingress/payload-processing-e2e-ait-45a82a13-8466c878f-fdxzh"
Sep 09 04:10:51.350904 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:51.350870    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9jcz\" (UniqueName: \"kubernetes.io/projected/e92cc6d5-ef50-4517-b244-a7fff1779f19-kube-api-access-b9jcz\") pod \"payload-processing-e2e-ait-45a82a13-8466c878f-fdxzh\" (UID: \"e92cc6d5-ef50-4517-b244-a7fff1779f19\") " pod="openshift-ingress/payload-processing-e2e-ait-45a82a13-8466c878f-fdxzh"
Sep 09 04:10:51.351066 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:51.350915    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e92cc6d5-ef50-4517-b244-a7fff1779f19-plugins-config-volume\") pod \"payload-processing-e2e-ait-45a82a13-8466c878f-fdxzh\" (UID: \"e92cc6d5-ef50-4517-b244-a7fff1779f19\") " pod="openshift-ingress/payload-processing-e2e-ait-45a82a13-8466c878f-fdxzh"
Sep 09 04:10:51.351574 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:51.351555    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e92cc6d5-ef50-4517-b244-a7fff1779f19-plugins-config-volume\") pod \"payload-processing-e2e-ait-45a82a13-8466c878f-fdxzh\" (UID: \"e92cc6d5-ef50-4517-b244-a7fff1779f19\") " pod="openshift-ingress/payload-processing-e2e-ait-45a82a13-8466c878f-fdxzh"
Sep 09 04:10:51.372133 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:51.370730    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9jcz\" (UniqueName: \"kubernetes.io/projected/e92cc6d5-ef50-4517-b244-a7fff1779f19-kube-api-access-b9jcz\") pod \"payload-processing-e2e-ait-45a82a13-8466c878f-fdxzh\" (UID: \"e92cc6d5-ef50-4517-b244-a7fff1779f19\") " pod="openshift-ingress/payload-processing-e2e-ait-45a82a13-8466c878f-fdxzh"
Sep 09 04:10:51.506513 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:51.506469    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-45a82a13-8466c878f-fdxzh"
Sep 09 04:10:51.807767 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:51.807658    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-45a82a13-8466c878f-fdxzh"]
Sep 09 04:10:51.810527 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:10:51.810495    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode92cc6d5_ef50_4517_b244_a7fff1779f19.slice/crio-bc77c9d1e7f651380e43e37f0feca7cf179c989e4d9917bb5f311e8871dfaad2 WatchSource:0}: Error finding container bc77c9d1e7f651380e43e37f0feca7cf179c989e4d9917bb5f311e8871dfaad2: Status 404 returned error can't find the container with id bc77c9d1e7f651380e43e37f0feca7cf179c989e4d9917bb5f311e8871dfaad2
Sep 09 04:10:52.115863 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:52.115696    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-45a82a13-8466c878f-fdxzh" event={"ID":"e92cc6d5-ef50-4517-b244-a7fff1779f19","Type":"ContainerStarted","Data":"7452831cf2c8c9ddf42a22a935c854aab1a6f3c70b9dad95dd9435e6cba596c1"}
Sep 09 04:10:52.115863 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:52.115762    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-45a82a13-8466c878f-fdxzh" event={"ID":"e92cc6d5-ef50-4517-b244-a7fff1779f19","Type":"ContainerStarted","Data":"bc77c9d1e7f651380e43e37f0feca7cf179c989e4d9917bb5f311e8871dfaad2"}
Sep 09 04:10:52.115863 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:52.115814    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-45a82a13-8466c878f-fdxzh"
Sep 09 04:10:52.133743 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:52.133682    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-45a82a13-8466c878f-fdxzh" podStartSLOduration=1.133663183 podStartE2EDuration="1.133663183s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:10:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:10:52.132042552 +0000 UTC m=+777.051213455" watchObservedRunningTime="2026-09-09 04:10:52.133663183 +0000 UTC m=+777.052834093"
Sep 09 04:10:53.085391 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:53.085361    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-85732c-54589c45dd-ls9gc"
Sep 09 04:10:56.059050 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:56.059016    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-85732c-6d5cd7c4b6-ffzdb"
Sep 09 04:10:56.059566 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:56.059100    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-85732c-65749c6df9-c844k"
Sep 09 04:10:57.062303 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:57.062268    2557 patch_prober.go:28] interesting pod/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.28:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 09 04:10:57.062678 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:57.062349    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6" podUID="b0cd2205-e41e-4495-83a3-29d3da0c034f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.28:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 09 04:10:57.066316 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:57.066288    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-359adf-d7487ddc4-8q8rb"
Sep 09 04:10:57.066457 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:57.066391    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-359adf-546f7f8c97-gxn5v"
Sep 09 04:10:58.145505 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:58.145474    2557 generic.go:356] "Generic (PLEG): container finished" podID="b0cd2205-e41e-4495-83a3-29d3da0c034f" containerID="25edb2ff4fd72705f3109dbf6c4c986ee9ad91b716e654c84dc416b0490e7bc9" exitCode=0
Sep 09 04:10:58.145912 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:58.145549    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6" event={"ID":"b0cd2205-e41e-4495-83a3-29d3da0c034f","Type":"ContainerDied","Data":"25edb2ff4fd72705f3109dbf6c4c986ee9ad91b716e654c84dc416b0490e7bc9"}
Sep 09 04:10:58.145953 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:58.145931    2557 scope.go:117] "RemoveContainer" containerID="25edb2ff4fd72705f3109dbf6c4c986ee9ad91b716e654c84dc416b0490e7bc9"
Sep 09 04:10:58.147076 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:58.147045    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 09 04:10:58.147161 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:58.147145    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 09 04:10:58.147200 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:58.147184    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 09 04:10:59.151931 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:10:59.151897    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6" event={"ID":"b0cd2205-e41e-4495-83a3-29d3da0c034f","Type":"ContainerStarted","Data":"c5448f9d3361a10899c8fbb00130a733b64c922abfb9a4a269a62a67362fe298"}
Sep 09 04:11:00.105524 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:00.105486    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:11:00.105829 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:00.105804    2557 patch_prober.go:28] interesting pod/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.28:15021/healthz/ready\": dial tcp 10.132.0.28:15021: connect: connection refused" start-of-body=
Sep 09 04:11:00.105910 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:00.105865    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6" podUID="b0cd2205-e41e-4495-83a3-29d3da0c034f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.28:15021/healthz/ready\": dial tcp 10.132.0.28:15021: connect: connection refused"
Sep 09 04:11:01.105454 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:01.105421    2557 patch_prober.go:28] interesting pod/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.28:15021/healthz/ready\": dial tcp 10.132.0.28:15021: connect: connection refused" start-of-body=
Sep 09 04:11:01.105841 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:01.105472    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6" podUID="b0cd2205-e41e-4495-83a3-29d3da0c034f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.28:15021/healthz/ready\": dial tcp 10.132.0.28:15021: connect: connection refused"
Sep 09 04:11:02.105551 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:02.105510    2557 patch_prober.go:28] interesting pod/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.28:15021/healthz/ready\": dial tcp 10.132.0.28:15021: connect: connection refused" start-of-body=
Sep 09 04:11:02.105923 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:02.105571    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6" podUID="b0cd2205-e41e-4495-83a3-29d3da0c034f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.28:15021/healthz/ready\": dial tcp 10.132.0.28:15021: connect: connection refused"
Sep 09 04:11:02.117846 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:02.117822    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-4ad5fa-797d7dd8cd-mlw76"
Sep 09 04:11:02.117974 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:02.117887    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-4ad5fa-9464748d7-98ntv"
Sep 09 04:11:03.105763 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:03.105726    2557 patch_prober.go:28] interesting pod/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.28:15021/healthz/ready\": dial tcp 10.132.0.28:15021: connect: connection refused" start-of-body=
Sep 09 04:11:03.106264 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:03.105777    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6" podUID="b0cd2205-e41e-4495-83a3-29d3da0c034f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.28:15021/healthz/ready\": dial tcp 10.132.0.28:15021: connect: connection refused"
Sep 09 04:11:03.123292 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:03.123271    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-45a82a13-8466c878f-fdxzh"
Sep 09 04:11:04.105582 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:04.105550    2557 patch_prober.go:28] interesting pod/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.28:15021/healthz/ready\": dial tcp 10.132.0.28:15021: connect: connection refused" start-of-body=
Sep 09 04:11:04.105819 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:04.105603    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6" podUID="b0cd2205-e41e-4495-83a3-29d3da0c034f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.28:15021/healthz/ready\": dial tcp 10.132.0.28:15021: connect: connection refused"
Sep 09 04:11:05.109237 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:05.109208    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:11:05.109587 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:05.109534    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:11:05.110301 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:05.110284    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:11:12.764464 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:12.764426    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-629522-6d5c87fb6c-qwrlh"]
Sep 09 04:11:12.770699 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:12.770678    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-629522-6d5c87fb6c-qwrlh"
Sep 09 04:11:12.772907 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:12.772873    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-629522-dockercfg-tbw4s\""
Sep 09 04:11:12.773133 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:12.773098    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-629522\""
Sep 09 04:11:12.779230 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:12.779204    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-629522-6d5c87fb6c-qwrlh"]
Sep 09 04:11:12.870065 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:12.870000    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22942\" (UniqueName: \"kubernetes.io/projected/f2a15779-643f-4369-8470-f986d81ec1af-kube-api-access-22942\") pod \"payload-processing-e2e-inf-b-629522-6d5c87fb6c-qwrlh\" (UID: \"f2a15779-643f-4369-8470-f986d81ec1af\") " pod="openshift-ingress/payload-processing-e2e-inf-b-629522-6d5c87fb6c-qwrlh"
Sep 09 04:11:12.870281 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:12.870169    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2a15779-643f-4369-8470-f986d81ec1af-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-629522-6d5c87fb6c-qwrlh\" (UID: \"f2a15779-643f-4369-8470-f986d81ec1af\") " pod="openshift-ingress/payload-processing-e2e-inf-b-629522-6d5c87fb6c-qwrlh"
Sep 09 04:11:12.970803 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:12.970765    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2a15779-643f-4369-8470-f986d81ec1af-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-629522-6d5c87fb6c-qwrlh\" (UID: \"f2a15779-643f-4369-8470-f986d81ec1af\") " pod="openshift-ingress/payload-processing-e2e-inf-b-629522-6d5c87fb6c-qwrlh"
Sep 09 04:11:12.970979 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:12.970952    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22942\" (UniqueName: \"kubernetes.io/projected/f2a15779-643f-4369-8470-f986d81ec1af-kube-api-access-22942\") pod \"payload-processing-e2e-inf-b-629522-6d5c87fb6c-qwrlh\" (UID: \"f2a15779-643f-4369-8470-f986d81ec1af\") " pod="openshift-ingress/payload-processing-e2e-inf-b-629522-6d5c87fb6c-qwrlh"
Sep 09 04:11:12.971549 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:12.971531    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2a15779-643f-4369-8470-f986d81ec1af-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-629522-6d5c87fb6c-qwrlh\" (UID: \"f2a15779-643f-4369-8470-f986d81ec1af\") " pod="openshift-ingress/payload-processing-e2e-inf-b-629522-6d5c87fb6c-qwrlh"
Sep 09 04:11:12.978727 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:12.978700    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22942\" (UniqueName: \"kubernetes.io/projected/f2a15779-643f-4369-8470-f986d81ec1af-kube-api-access-22942\") pod \"payload-processing-e2e-inf-b-629522-6d5c87fb6c-qwrlh\" (UID: \"f2a15779-643f-4369-8470-f986d81ec1af\") " pod="openshift-ingress/payload-processing-e2e-inf-b-629522-6d5c87fb6c-qwrlh"
Sep 09 04:11:13.095511 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:13.095433    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-629522-6d5c87fb6c-qwrlh"
Sep 09 04:11:13.365069 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:13.365044    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-629522-6d5c87fb6c-qwrlh"]
Sep 09 04:11:13.367281 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:11:13.367253    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2a15779_643f_4369_8470_f986d81ec1af.slice/crio-fc868bfd2092f8a2c43c0a1baccdcf92f8e07075ed35cebd7c0a1f1148edd814 WatchSource:0}: Error finding container fc868bfd2092f8a2c43c0a1baccdcf92f8e07075ed35cebd7c0a1f1148edd814: Status 404 returned error can't find the container with id fc868bfd2092f8a2c43c0a1baccdcf92f8e07075ed35cebd7c0a1f1148edd814
Sep 09 04:11:14.247627 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:14.247592    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-629522-6d5c87fb6c-qwrlh" event={"ID":"f2a15779-643f-4369-8470-f986d81ec1af","Type":"ContainerStarted","Data":"44fb37541dbf9cf6d733cad3534e774942f6c94474062cf4cc051b257651c58f"}
Sep 09 04:11:14.248002 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:14.247651    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-629522-6d5c87fb6c-qwrlh" event={"ID":"f2a15779-643f-4369-8470-f986d81ec1af","Type":"ContainerStarted","Data":"fc868bfd2092f8a2c43c0a1baccdcf92f8e07075ed35cebd7c0a1f1148edd814"}
Sep 09 04:11:14.248002 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:14.247699    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-629522-6d5c87fb6c-qwrlh"
Sep 09 04:11:14.265327 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:14.265278    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-629522-6d5c87fb6c-qwrlh" podStartSLOduration=2.265264074 podStartE2EDuration="2.265264074s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:11:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:11:14.2632187 +0000 UTC m=+799.182389610" watchObservedRunningTime="2026-09-09 04:11:14.265264074 +0000 UTC m=+799.184434983"
Sep 09 04:11:25.254533 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:25.254459    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-629522-6d5c87fb6c-qwrlh"
Sep 09 04:11:30.988556 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:30.988520    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-45a82a13-8466c878f-fdxzh"]
Sep 09 04:11:30.988969 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:30.988787    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-45a82a13-8466c878f-fdxzh" podUID="e92cc6d5-ef50-4517-b244-a7fff1779f19" containerName="payload-processing" containerID="cri-o://7452831cf2c8c9ddf42a22a935c854aab1a6f3c70b9dad95dd9435e6cba596c1" gracePeriod=30
Sep 09 04:11:31.252698 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:31.252636    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-45a82a13-8466c878f-fdxzh"
Sep 09 04:11:31.330829 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:31.330795    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e92cc6d5-ef50-4517-b244-a7fff1779f19-plugins-config-volume\") pod \"e92cc6d5-ef50-4517-b244-a7fff1779f19\" (UID: \"e92cc6d5-ef50-4517-b244-a7fff1779f19\") "
Sep 09 04:11:31.331008 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:31.330839    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b9jcz\" (UniqueName: \"kubernetes.io/projected/e92cc6d5-ef50-4517-b244-a7fff1779f19-kube-api-access-b9jcz\") pod \"e92cc6d5-ef50-4517-b244-a7fff1779f19\" (UID: \"e92cc6d5-ef50-4517-b244-a7fff1779f19\") "
Sep 09 04:11:31.331392 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:31.331350    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e92cc6d5-ef50-4517-b244-a7fff1779f19-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e92cc6d5-ef50-4517-b244-a7fff1779f19" (UID: "e92cc6d5-ef50-4517-b244-a7fff1779f19"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 04:11:31.332375 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:31.332345    2557 generic.go:356] "Generic (PLEG): container finished" podID="e92cc6d5-ef50-4517-b244-a7fff1779f19" containerID="7452831cf2c8c9ddf42a22a935c854aab1a6f3c70b9dad95dd9435e6cba596c1" exitCode=0
Sep 09 04:11:31.332533 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:31.332412    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-45a82a13-8466c878f-fdxzh"
Sep 09 04:11:31.332533 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:31.332442    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-45a82a13-8466c878f-fdxzh" event={"ID":"e92cc6d5-ef50-4517-b244-a7fff1779f19","Type":"ContainerDied","Data":"7452831cf2c8c9ddf42a22a935c854aab1a6f3c70b9dad95dd9435e6cba596c1"}
Sep 09 04:11:31.332533 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:31.332512    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-45a82a13-8466c878f-fdxzh" event={"ID":"e92cc6d5-ef50-4517-b244-a7fff1779f19","Type":"ContainerDied","Data":"bc77c9d1e7f651380e43e37f0feca7cf179c989e4d9917bb5f311e8871dfaad2"}
Sep 09 04:11:31.332735 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:31.332562    2557 scope.go:117] "RemoveContainer" containerID="7452831cf2c8c9ddf42a22a935c854aab1a6f3c70b9dad95dd9435e6cba596c1"
Sep 09 04:11:31.334267 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:31.334243    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e92cc6d5-ef50-4517-b244-a7fff1779f19-kube-api-access-b9jcz" (OuterVolumeSpecName: "kube-api-access-b9jcz") pod "e92cc6d5-ef50-4517-b244-a7fff1779f19" (UID: "e92cc6d5-ef50-4517-b244-a7fff1779f19"). InnerVolumeSpecName "kube-api-access-b9jcz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 04:11:31.345198 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:31.345177    2557 scope.go:117] "RemoveContainer" containerID="7452831cf2c8c9ddf42a22a935c854aab1a6f3c70b9dad95dd9435e6cba596c1"
Sep 09 04:11:31.345547 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:11:31.345519    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7452831cf2c8c9ddf42a22a935c854aab1a6f3c70b9dad95dd9435e6cba596c1\": container with ID starting with 7452831cf2c8c9ddf42a22a935c854aab1a6f3c70b9dad95dd9435e6cba596c1 not found: ID does not exist" containerID="7452831cf2c8c9ddf42a22a935c854aab1a6f3c70b9dad95dd9435e6cba596c1"
Sep 09 04:11:31.345620 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:31.345561    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7452831cf2c8c9ddf42a22a935c854aab1a6f3c70b9dad95dd9435e6cba596c1"} err="failed to get container status \"7452831cf2c8c9ddf42a22a935c854aab1a6f3c70b9dad95dd9435e6cba596c1\": rpc error: code = NotFound desc = could not find container \"7452831cf2c8c9ddf42a22a935c854aab1a6f3c70b9dad95dd9435e6cba596c1\": container with ID starting with 7452831cf2c8c9ddf42a22a935c854aab1a6f3c70b9dad95dd9435e6cba596c1 not found: ID does not exist"
Sep 09 04:11:31.432238 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:31.432212    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e92cc6d5-ef50-4517-b244-a7fff1779f19-plugins-config-volume\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:11:31.432238 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:31.432235    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9jcz\" (UniqueName: \"kubernetes.io/projected/e92cc6d5-ef50-4517-b244-a7fff1779f19-kube-api-access-b9jcz\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:11:31.647494 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:31.647462    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-45a82a13-8466c878f-fdxzh"]
Sep 09 04:11:31.651151 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:31.651106    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-45a82a13-8466c878f-fdxzh"]
Sep 09 04:11:33.619870 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:33.619832    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e92cc6d5-ef50-4517-b244-a7fff1779f19" path="/var/lib/kubelet/pods/e92cc6d5-ef50-4517-b244-a7fff1779f19/volumes"
Sep 09 04:11:36.258018 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:36.257987    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-3748ac1b-848cd5766b-l55cs"]
Sep 09 04:11:36.258533 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:36.258390    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e92cc6d5-ef50-4517-b244-a7fff1779f19" containerName="payload-processing"
Sep 09 04:11:36.258533 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:36.258403    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="e92cc6d5-ef50-4517-b244-a7fff1779f19" containerName="payload-processing"
Sep 09 04:11:36.258533 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:36.258466    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="e92cc6d5-ef50-4517-b244-a7fff1779f19" containerName="payload-processing"
Sep 09 04:11:36.263247 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:36.263224    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3748ac1b-848cd5766b-l55cs"
Sep 09 04:11:36.265536 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:36.265515    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-3748ac1b-dockercfg-dknqf\""
Sep 09 04:11:36.265645 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:36.265523    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-3748ac1b\""
Sep 09 04:11:36.271487 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:36.271464    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-3748ac1b-848cd5766b-l55cs"]
Sep 09 04:11:36.383762 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:36.383720    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1bf8eea7-f2a7-49b0-8288-168ad7a430c2-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-3748ac1b-848cd5766b-l55cs\" (UID: \"1bf8eea7-f2a7-49b0-8288-168ad7a430c2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-3748ac1b-848cd5766b-l55cs"
Sep 09 04:11:36.383920 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:36.383835    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sm2r\" (UniqueName: \"kubernetes.io/projected/1bf8eea7-f2a7-49b0-8288-168ad7a430c2-kube-api-access-9sm2r\") pod \"payload-pre-processing-e2e-migrate-3748ac1b-848cd5766b-l55cs\" (UID: \"1bf8eea7-f2a7-49b0-8288-168ad7a430c2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-3748ac1b-848cd5766b-l55cs"
Sep 09 04:11:36.484692 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:36.484654    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sm2r\" (UniqueName: \"kubernetes.io/projected/1bf8eea7-f2a7-49b0-8288-168ad7a430c2-kube-api-access-9sm2r\") pod \"payload-pre-processing-e2e-migrate-3748ac1b-848cd5766b-l55cs\" (UID: \"1bf8eea7-f2a7-49b0-8288-168ad7a430c2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-3748ac1b-848cd5766b-l55cs"
Sep 09 04:11:36.484859 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:36.484751    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1bf8eea7-f2a7-49b0-8288-168ad7a430c2-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-3748ac1b-848cd5766b-l55cs\" (UID: \"1bf8eea7-f2a7-49b0-8288-168ad7a430c2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-3748ac1b-848cd5766b-l55cs"
Sep 09 04:11:36.485561 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:36.485539    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1bf8eea7-f2a7-49b0-8288-168ad7a430c2-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-3748ac1b-848cd5766b-l55cs\" (UID: \"1bf8eea7-f2a7-49b0-8288-168ad7a430c2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-3748ac1b-848cd5766b-l55cs"
Sep 09 04:11:36.493104 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:36.493072    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sm2r\" (UniqueName: \"kubernetes.io/projected/1bf8eea7-f2a7-49b0-8288-168ad7a430c2-kube-api-access-9sm2r\") pod \"payload-pre-processing-e2e-migrate-3748ac1b-848cd5766b-l55cs\" (UID: \"1bf8eea7-f2a7-49b0-8288-168ad7a430c2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-3748ac1b-848cd5766b-l55cs"
Sep 09 04:11:36.574952 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:36.574876    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3748ac1b-848cd5766b-l55cs"
Sep 09 04:11:36.860258 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:36.860229    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-3748ac1b-848cd5766b-l55cs"]
Sep 09 04:11:36.862157 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:11:36.862108    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bf8eea7_f2a7_49b0_8288_168ad7a430c2.slice/crio-64ec53914032336b168f14014e4d0c734d819cb2988f69ef3133d80a5030c033 WatchSource:0}: Error finding container 64ec53914032336b168f14014e4d0c734d819cb2988f69ef3133d80a5030c033: Status 404 returned error can't find the container with id 64ec53914032336b168f14014e4d0c734d819cb2988f69ef3133d80a5030c033
Sep 09 04:11:37.368830 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:37.368798    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3748ac1b-848cd5766b-l55cs" event={"ID":"1bf8eea7-f2a7-49b0-8288-168ad7a430c2","Type":"ContainerStarted","Data":"17f5e2687098dfee7a363e780d1d68b8ef166a6af12f1761920d06cae5d955fa"}
Sep 09 04:11:37.369265 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:37.368850    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3748ac1b-848cd5766b-l55cs" event={"ID":"1bf8eea7-f2a7-49b0-8288-168ad7a430c2","Type":"ContainerStarted","Data":"64ec53914032336b168f14014e4d0c734d819cb2988f69ef3133d80a5030c033"}
Sep 09 04:11:37.369265 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:37.368886    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3748ac1b-848cd5766b-l55cs"
Sep 09 04:11:37.385992 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:37.385944    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3748ac1b-848cd5766b-l55cs" podStartSLOduration=1.385928265 podStartE2EDuration="1.385928265s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:11:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:11:37.383587376 +0000 UTC m=+822.302758285" watchObservedRunningTime="2026-09-09 04:11:37.385928265 +0000 UTC m=+822.305099209"
Sep 09 04:11:48.376571 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:11:48.376539    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3748ac1b-848cd5766b-l55cs"
Sep 09 04:12:06.946356 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:06.946314    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-3748ac1b-848cd5766b-l55cs"]
Sep 09 04:12:06.946830 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:06.946586    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3748ac1b-848cd5766b-l55cs" podUID="1bf8eea7-f2a7-49b0-8288-168ad7a430c2" containerName="payload-pre-processing" containerID="cri-o://17f5e2687098dfee7a363e780d1d68b8ef166a6af12f1761920d06cae5d955fa" gracePeriod=30
Sep 09 04:12:07.217364 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:07.217310    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3748ac1b-848cd5766b-l55cs"
Sep 09 04:12:07.255520 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:07.255489    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9sm2r\" (UniqueName: \"kubernetes.io/projected/1bf8eea7-f2a7-49b0-8288-168ad7a430c2-kube-api-access-9sm2r\") pod \"1bf8eea7-f2a7-49b0-8288-168ad7a430c2\" (UID: \"1bf8eea7-f2a7-49b0-8288-168ad7a430c2\") "
Sep 09 04:12:07.255692 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:07.255568    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1bf8eea7-f2a7-49b0-8288-168ad7a430c2-plugins-config-volume\") pod \"1bf8eea7-f2a7-49b0-8288-168ad7a430c2\" (UID: \"1bf8eea7-f2a7-49b0-8288-168ad7a430c2\") "
Sep 09 04:12:07.255974 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:07.255946    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bf8eea7-f2a7-49b0-8288-168ad7a430c2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1bf8eea7-f2a7-49b0-8288-168ad7a430c2" (UID: "1bf8eea7-f2a7-49b0-8288-168ad7a430c2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 04:12:07.258295 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:07.258258    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bf8eea7-f2a7-49b0-8288-168ad7a430c2-kube-api-access-9sm2r" (OuterVolumeSpecName: "kube-api-access-9sm2r") pod "1bf8eea7-f2a7-49b0-8288-168ad7a430c2" (UID: "1bf8eea7-f2a7-49b0-8288-168ad7a430c2"). InnerVolumeSpecName "kube-api-access-9sm2r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 04:12:07.356257 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:07.356223    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1bf8eea7-f2a7-49b0-8288-168ad7a430c2-plugins-config-volume\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:12:07.356257 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:07.356263    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9sm2r\" (UniqueName: \"kubernetes.io/projected/1bf8eea7-f2a7-49b0-8288-168ad7a430c2-kube-api-access-9sm2r\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:12:07.519744 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:07.519660    2557 generic.go:356] "Generic (PLEG): container finished" podID="1bf8eea7-f2a7-49b0-8288-168ad7a430c2" containerID="17f5e2687098dfee7a363e780d1d68b8ef166a6af12f1761920d06cae5d955fa" exitCode=0
Sep 09 04:12:07.519744 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:07.519709    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3748ac1b-848cd5766b-l55cs"
Sep 09 04:12:07.519927 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:07.519745    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3748ac1b-848cd5766b-l55cs" event={"ID":"1bf8eea7-f2a7-49b0-8288-168ad7a430c2","Type":"ContainerDied","Data":"17f5e2687098dfee7a363e780d1d68b8ef166a6af12f1761920d06cae5d955fa"}
Sep 09 04:12:07.519927 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:07.519798    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-3748ac1b-848cd5766b-l55cs" event={"ID":"1bf8eea7-f2a7-49b0-8288-168ad7a430c2","Type":"ContainerDied","Data":"64ec53914032336b168f14014e4d0c734d819cb2988f69ef3133d80a5030c033"}
Sep 09 04:12:07.519927 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:07.519820    2557 scope.go:117] "RemoveContainer" containerID="17f5e2687098dfee7a363e780d1d68b8ef166a6af12f1761920d06cae5d955fa"
Sep 09 04:12:07.529374 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:07.529358    2557 scope.go:117] "RemoveContainer" containerID="17f5e2687098dfee7a363e780d1d68b8ef166a6af12f1761920d06cae5d955fa"
Sep 09 04:12:07.529632 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:12:07.529611    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17f5e2687098dfee7a363e780d1d68b8ef166a6af12f1761920d06cae5d955fa\": container with ID starting with 17f5e2687098dfee7a363e780d1d68b8ef166a6af12f1761920d06cae5d955fa not found: ID does not exist" containerID="17f5e2687098dfee7a363e780d1d68b8ef166a6af12f1761920d06cae5d955fa"
Sep 09 04:12:07.529677 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:07.529640    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17f5e2687098dfee7a363e780d1d68b8ef166a6af12f1761920d06cae5d955fa"} err="failed to get container status \"17f5e2687098dfee7a363e780d1d68b8ef166a6af12f1761920d06cae5d955fa\": rpc error: code = NotFound desc = could not find container \"17f5e2687098dfee7a363e780d1d68b8ef166a6af12f1761920d06cae5d955fa\": container with ID starting with 17f5e2687098dfee7a363e780d1d68b8ef166a6af12f1761920d06cae5d955fa not found: ID does not exist"
Sep 09 04:12:07.542234 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:07.542208    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-3748ac1b-848cd5766b-l55cs"]
Sep 09 04:12:07.545827 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:07.545805    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-3748ac1b-848cd5766b-l55cs"]
Sep 09 04:12:07.618875 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:07.618848    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1bf8eea7-f2a7-49b0-8288-168ad7a430c2" path="/var/lib/kubelet/pods/1bf8eea7-f2a7-49b0-8288-168ad7a430c2/volumes"
Sep 09 04:12:12.248598 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:12.248564    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3dbd54be-dbd48464f-jqjml"]
Sep 09 04:12:12.248984 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:12.248968    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bf8eea7-f2a7-49b0-8288-168ad7a430c2" containerName="payload-pre-processing"
Sep 09 04:12:12.249027 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:12.248986    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bf8eea7-f2a7-49b0-8288-168ad7a430c2" containerName="payload-pre-processing"
Sep 09 04:12:12.249075 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:12.249065    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="1bf8eea7-f2a7-49b0-8288-168ad7a430c2" containerName="payload-pre-processing"
Sep 09 04:12:12.253486 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:12.253470    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3dbd54be-dbd48464f-jqjml"
Sep 09 04:12:12.255986 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:12.255957    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-3dbd54be-dockercfg-w8c78\""
Sep 09 04:12:12.256204 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:12.255959    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-3dbd54be\""
Sep 09 04:12:12.260802 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:12.260780    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3dbd54be-dbd48464f-jqjml"]
Sep 09 04:12:12.291667 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:12.291189    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ece25b8-3f9d-4764-8995-287409fe9d62-plugins-config-volume\") pod \"payload-processing-e2e-ait-3dbd54be-dbd48464f-jqjml\" (UID: \"4ece25b8-3f9d-4764-8995-287409fe9d62\") " pod="openshift-ingress/payload-processing-e2e-ait-3dbd54be-dbd48464f-jqjml"
Sep 09 04:12:12.291803 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:12.291745    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8xmm\" (UniqueName: \"kubernetes.io/projected/4ece25b8-3f9d-4764-8995-287409fe9d62-kube-api-access-k8xmm\") pod \"payload-processing-e2e-ait-3dbd54be-dbd48464f-jqjml\" (UID: \"4ece25b8-3f9d-4764-8995-287409fe9d62\") " pod="openshift-ingress/payload-processing-e2e-ait-3dbd54be-dbd48464f-jqjml"
Sep 09 04:12:12.393280 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:12.393243    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ece25b8-3f9d-4764-8995-287409fe9d62-plugins-config-volume\") pod \"payload-processing-e2e-ait-3dbd54be-dbd48464f-jqjml\" (UID: \"4ece25b8-3f9d-4764-8995-287409fe9d62\") " pod="openshift-ingress/payload-processing-e2e-ait-3dbd54be-dbd48464f-jqjml"
Sep 09 04:12:12.393447 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:12.393305    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8xmm\" (UniqueName: \"kubernetes.io/projected/4ece25b8-3f9d-4764-8995-287409fe9d62-kube-api-access-k8xmm\") pod \"payload-processing-e2e-ait-3dbd54be-dbd48464f-jqjml\" (UID: \"4ece25b8-3f9d-4764-8995-287409fe9d62\") " pod="openshift-ingress/payload-processing-e2e-ait-3dbd54be-dbd48464f-jqjml"
Sep 09 04:12:12.393917 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:12.393895    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ece25b8-3f9d-4764-8995-287409fe9d62-plugins-config-volume\") pod \"payload-processing-e2e-ait-3dbd54be-dbd48464f-jqjml\" (UID: \"4ece25b8-3f9d-4764-8995-287409fe9d62\") " pod="openshift-ingress/payload-processing-e2e-ait-3dbd54be-dbd48464f-jqjml"
Sep 09 04:12:12.401254 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:12.401231    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8xmm\" (UniqueName: \"kubernetes.io/projected/4ece25b8-3f9d-4764-8995-287409fe9d62-kube-api-access-k8xmm\") pod \"payload-processing-e2e-ait-3dbd54be-dbd48464f-jqjml\" (UID: \"4ece25b8-3f9d-4764-8995-287409fe9d62\") " pod="openshift-ingress/payload-processing-e2e-ait-3dbd54be-dbd48464f-jqjml"
Sep 09 04:12:12.565447 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:12.565371    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3dbd54be-dbd48464f-jqjml"
Sep 09 04:12:12.835064 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:12.834991    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3dbd54be-dbd48464f-jqjml"]
Sep 09 04:12:12.838532 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:12:12.838495    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ece25b8_3f9d_4764_8995_287409fe9d62.slice/crio-ed3e8d37998f9074b30991790e3827331a32acd1378edef0b7e95a891ca33a5d WatchSource:0}: Error finding container ed3e8d37998f9074b30991790e3827331a32acd1378edef0b7e95a891ca33a5d: Status 404 returned error can't find the container with id ed3e8d37998f9074b30991790e3827331a32acd1378edef0b7e95a891ca33a5d
Sep 09 04:12:13.555104 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:13.555073    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3dbd54be-dbd48464f-jqjml" event={"ID":"4ece25b8-3f9d-4764-8995-287409fe9d62","Type":"ContainerStarted","Data":"79e758257ecd1f3fdb7ce705086626a5283fd7f56633f97cd24369de4aabc98a"}
Sep 09 04:12:13.555489 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:13.555144    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3dbd54be-dbd48464f-jqjml" event={"ID":"4ece25b8-3f9d-4764-8995-287409fe9d62","Type":"ContainerStarted","Data":"ed3e8d37998f9074b30991790e3827331a32acd1378edef0b7e95a891ca33a5d"}
Sep 09 04:12:13.555489 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:13.555168    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-3dbd54be-dbd48464f-jqjml"
Sep 09 04:12:13.571224 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:13.571174    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-3dbd54be-dbd48464f-jqjml" podStartSLOduration=1.571155818 podStartE2EDuration="1.571155818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:12:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:12:13.570389324 +0000 UTC m=+858.489560235" watchObservedRunningTime="2026-09-09 04:12:13.571155818 +0000 UTC m=+858.490326729"
Sep 09 04:12:24.562793 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:24.562763    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-3dbd54be-dbd48464f-jqjml"
Sep 09 04:12:42.185809 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:42.185775    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3dbd54be-dbd48464f-jqjml"]
Sep 09 04:12:42.186410 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:42.186013    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-3dbd54be-dbd48464f-jqjml" podUID="4ece25b8-3f9d-4764-8995-287409fe9d62" containerName="payload-processing" containerID="cri-o://79e758257ecd1f3fdb7ce705086626a5283fd7f56633f97cd24369de4aabc98a" gracePeriod=30
Sep 09 04:12:42.449873 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:42.449821    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3dbd54be-dbd48464f-jqjml"
Sep 09 04:12:42.560961 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:42.560929    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8xmm\" (UniqueName: \"kubernetes.io/projected/4ece25b8-3f9d-4764-8995-287409fe9d62-kube-api-access-k8xmm\") pod \"4ece25b8-3f9d-4764-8995-287409fe9d62\" (UID: \"4ece25b8-3f9d-4764-8995-287409fe9d62\") "
Sep 09 04:12:42.561160 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:42.560977    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ece25b8-3f9d-4764-8995-287409fe9d62-plugins-config-volume\") pod \"4ece25b8-3f9d-4764-8995-287409fe9d62\" (UID: \"4ece25b8-3f9d-4764-8995-287409fe9d62\") "
Sep 09 04:12:42.561418 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:42.561395    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ece25b8-3f9d-4764-8995-287409fe9d62-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4ece25b8-3f9d-4764-8995-287409fe9d62" (UID: "4ece25b8-3f9d-4764-8995-287409fe9d62"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 04:12:42.563074 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:42.563054    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ece25b8-3f9d-4764-8995-287409fe9d62-kube-api-access-k8xmm" (OuterVolumeSpecName: "kube-api-access-k8xmm") pod "4ece25b8-3f9d-4764-8995-287409fe9d62" (UID: "4ece25b8-3f9d-4764-8995-287409fe9d62"). InnerVolumeSpecName "kube-api-access-k8xmm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 04:12:42.662164 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:42.662134    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8xmm\" (UniqueName: \"kubernetes.io/projected/4ece25b8-3f9d-4764-8995-287409fe9d62-kube-api-access-k8xmm\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:12:42.662164 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:42.662159    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ece25b8-3f9d-4764-8995-287409fe9d62-plugins-config-volume\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:12:42.693644 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:42.693614    2557 generic.go:356] "Generic (PLEG): container finished" podID="4ece25b8-3f9d-4764-8995-287409fe9d62" containerID="79e758257ecd1f3fdb7ce705086626a5283fd7f56633f97cd24369de4aabc98a" exitCode=0
Sep 09 04:12:42.693779 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:42.693661    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3dbd54be-dbd48464f-jqjml"
Sep 09 04:12:42.693779 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:42.693695    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3dbd54be-dbd48464f-jqjml" event={"ID":"4ece25b8-3f9d-4764-8995-287409fe9d62","Type":"ContainerDied","Data":"79e758257ecd1f3fdb7ce705086626a5283fd7f56633f97cd24369de4aabc98a"}
Sep 09 04:12:42.693779 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:42.693750    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3dbd54be-dbd48464f-jqjml" event={"ID":"4ece25b8-3f9d-4764-8995-287409fe9d62","Type":"ContainerDied","Data":"ed3e8d37998f9074b30991790e3827331a32acd1378edef0b7e95a891ca33a5d"}
Sep 09 04:12:42.693779 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:42.693772    2557 scope.go:117] "RemoveContainer" containerID="79e758257ecd1f3fdb7ce705086626a5283fd7f56633f97cd24369de4aabc98a"
Sep 09 04:12:42.704131 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:42.704074    2557 scope.go:117] "RemoveContainer" containerID="79e758257ecd1f3fdb7ce705086626a5283fd7f56633f97cd24369de4aabc98a"
Sep 09 04:12:42.705993 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:12:42.705960    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79e758257ecd1f3fdb7ce705086626a5283fd7f56633f97cd24369de4aabc98a\": container with ID starting with 79e758257ecd1f3fdb7ce705086626a5283fd7f56633f97cd24369de4aabc98a not found: ID does not exist" containerID="79e758257ecd1f3fdb7ce705086626a5283fd7f56633f97cd24369de4aabc98a"
Sep 09 04:12:42.706164 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:42.706139    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79e758257ecd1f3fdb7ce705086626a5283fd7f56633f97cd24369de4aabc98a"} err="failed to get container status \"79e758257ecd1f3fdb7ce705086626a5283fd7f56633f97cd24369de4aabc98a\": rpc error: code = NotFound desc = could not find container \"79e758257ecd1f3fdb7ce705086626a5283fd7f56633f97cd24369de4aabc98a\": container with ID starting with 79e758257ecd1f3fdb7ce705086626a5283fd7f56633f97cd24369de4aabc98a not found: ID does not exist"
Sep 09 04:12:42.720241 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:42.720219    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3dbd54be-dbd48464f-jqjml"]
Sep 09 04:12:42.724213 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:42.724191    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3dbd54be-dbd48464f-jqjml"]
Sep 09 04:12:43.619212 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:43.619175    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ece25b8-3f9d-4764-8995-287409fe9d62" path="/var/lib/kubelet/pods/4ece25b8-3f9d-4764-8995-287409fe9d62/volumes"
Sep 09 04:12:48.080055 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:48.079965    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj"]
Sep 09 04:12:48.080566 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:48.080533    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ece25b8-3f9d-4764-8995-287409fe9d62" containerName="payload-processing"
Sep 09 04:12:48.080566 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:48.080559    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ece25b8-3f9d-4764-8995-287409fe9d62" containerName="payload-processing"
Sep 09 04:12:48.080745 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:48.080658    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ece25b8-3f9d-4764-8995-287409fe9d62" containerName="payload-processing"
Sep 09 04:12:48.084077 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:48.084057    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj"
Sep 09 04:12:48.086284 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:48.086267    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-1be20a5f-serving-cert\""
Sep 09 04:12:48.093147 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:48.092799    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj"]
Sep 09 04:12:48.215080 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:48.215049    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06473528-93fe-433b-b327-abef6a484305-openshift-service-ca\") pod \"maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj\" (UID: \"06473528-93fe-433b-b327-abef6a484305\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj"
Sep 09 04:12:48.215281 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:48.215143    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-756xb\" (UniqueName: \"kubernetes.io/projected/06473528-93fe-433b-b327-abef6a484305-kube-api-access-756xb\") pod \"maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj\" (UID: \"06473528-93fe-433b-b327-abef6a484305\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj"
Sep 09 04:12:48.215281 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:48.215173    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06473528-93fe-433b-b327-abef6a484305-metrics-certs\") pod \"maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj\" (UID: \"06473528-93fe-433b-b327-abef6a484305\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj"
Sep 09 04:12:48.215281 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:48.215188    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06473528-93fe-433b-b327-abef6a484305-maas-api-tls\") pod \"maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj\" (UID: \"06473528-93fe-433b-b327-abef6a484305\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj"
Sep 09 04:12:48.316462 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:48.316427    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-756xb\" (UniqueName: \"kubernetes.io/projected/06473528-93fe-433b-b327-abef6a484305-kube-api-access-756xb\") pod \"maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj\" (UID: \"06473528-93fe-433b-b327-abef6a484305\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj"
Sep 09 04:12:48.316623 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:48.316476    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06473528-93fe-433b-b327-abef6a484305-metrics-certs\") pod \"maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj\" (UID: \"06473528-93fe-433b-b327-abef6a484305\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj"
Sep 09 04:12:48.316623 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:48.316599    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06473528-93fe-433b-b327-abef6a484305-maas-api-tls\") pod \"maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj\" (UID: \"06473528-93fe-433b-b327-abef6a484305\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj"
Sep 09 04:12:48.316701 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:48.316666    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06473528-93fe-433b-b327-abef6a484305-openshift-service-ca\") pod \"maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj\" (UID: \"06473528-93fe-433b-b327-abef6a484305\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj"
Sep 09 04:12:48.317375 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:48.317356    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06473528-93fe-433b-b327-abef6a484305-openshift-service-ca\") pod \"maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj\" (UID: \"06473528-93fe-433b-b327-abef6a484305\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj"
Sep 09 04:12:48.318943 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:48.318919    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06473528-93fe-433b-b327-abef6a484305-maas-api-tls\") pod \"maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj\" (UID: \"06473528-93fe-433b-b327-abef6a484305\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj"
Sep 09 04:12:48.319054 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:48.318992    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06473528-93fe-433b-b327-abef6a484305-metrics-certs\") pod \"maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj\" (UID: \"06473528-93fe-433b-b327-abef6a484305\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj"
Sep 09 04:12:48.324426 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:48.324403    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-756xb\" (UniqueName: \"kubernetes.io/projected/06473528-93fe-433b-b327-abef6a484305-kube-api-access-756xb\") pod \"maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj\" (UID: \"06473528-93fe-433b-b327-abef6a484305\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj"
Sep 09 04:12:48.397456 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:48.397426    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj"
Sep 09 04:12:48.674945 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:48.674850    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj"]
Sep 09 04:12:48.677885 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:12:48.677857    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06473528_93fe_433b_b327_abef6a484305.slice/crio-5e6e3126850821d044c80b66108a480a7a185d94f0b50eb8708eb0d52a4e5386 WatchSource:0}: Error finding container 5e6e3126850821d044c80b66108a480a7a185d94f0b50eb8708eb0d52a4e5386: Status 404 returned error can't find the container with id 5e6e3126850821d044c80b66108a480a7a185d94f0b50eb8708eb0d52a4e5386
Sep 09 04:12:48.725863 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:48.725831    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj" event={"ID":"06473528-93fe-433b-b327-abef6a484305","Type":"ContainerStarted","Data":"5e6e3126850821d044c80b66108a480a7a185d94f0b50eb8708eb0d52a4e5386"}
Sep 09 04:12:49.733063 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:49.733026    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj" event={"ID":"06473528-93fe-433b-b327-abef6a484305","Type":"ContainerStarted","Data":"4ecfe08aa1836cedf62bd0933466c97ddd5ef11b56dedea2e585b26591ce6374"}
Sep 09 04:12:49.733467 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:49.733156    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj"
Sep 09 04:12:49.751318 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:49.751260    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj" podStartSLOduration=1.293085821 podStartE2EDuration="1.751242687s" totalImagesPullingTime="458.156866ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:12:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:12:49.748834754 +0000 UTC m=+894.668005664" watchObservedRunningTime="2026-09-09 04:12:49.751242687 +0000 UTC m=+894.670413594"
Sep 09 04:12:55.555088 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:55.555059    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gj7h7_7816cd49-e9c2-4f7b-b8ec-4e5711b675a5/console-operator/2.log"
Sep 09 04:12:55.557062 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:55.557030    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gj7h7_7816cd49-e9c2-4f7b-b8ec-4e5711b675a5/console-operator/2.log"
Sep 09 04:12:55.560591 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:55.560564    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rfvrc_1828d4d8-0d24-4e4d-a502-aee013a3b116/ovn-acl-logging/0.log"
Sep 09 04:12:55.562507 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:55.562490    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rfvrc_1828d4d8-0d24-4e4d-a502-aee013a3b116/ovn-acl-logging/0.log"
Sep 09 04:12:55.744152 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:12:55.744101    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj"
Sep 09 04:13:18.870809 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:18.870776    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj"]
Sep 09 04:13:18.871430 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:18.871057    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj" podUID="06473528-93fe-433b-b327-abef6a484305" containerName="maas-api" containerID="cri-o://4ecfe08aa1836cedf62bd0933466c97ddd5ef11b56dedea2e585b26591ce6374" gracePeriod=30
Sep 09 04:13:19.124951 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:19.124901    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj"
Sep 09 04:13:19.187296 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:19.187267    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-756xb\" (UniqueName: \"kubernetes.io/projected/06473528-93fe-433b-b327-abef6a484305-kube-api-access-756xb\") pod \"06473528-93fe-433b-b327-abef6a484305\" (UID: \"06473528-93fe-433b-b327-abef6a484305\") "
Sep 09 04:13:19.187427 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:19.187322    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06473528-93fe-433b-b327-abef6a484305-maas-api-tls\") pod \"06473528-93fe-433b-b327-abef6a484305\" (UID: \"06473528-93fe-433b-b327-abef6a484305\") "
Sep 09 04:13:19.187427 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:19.187346    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06473528-93fe-433b-b327-abef6a484305-openshift-service-ca\") pod \"06473528-93fe-433b-b327-abef6a484305\" (UID: \"06473528-93fe-433b-b327-abef6a484305\") "
Sep 09 04:13:19.187427 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:19.187393    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06473528-93fe-433b-b327-abef6a484305-metrics-certs\") pod \"06473528-93fe-433b-b327-abef6a484305\" (UID: \"06473528-93fe-433b-b327-abef6a484305\") "
Sep 09 04:13:19.187732 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:19.187705    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06473528-93fe-433b-b327-abef6a484305-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "06473528-93fe-433b-b327-abef6a484305" (UID: "06473528-93fe-433b-b327-abef6a484305"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 04:13:19.189430 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:19.189388    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06473528-93fe-433b-b327-abef6a484305-kube-api-access-756xb" (OuterVolumeSpecName: "kube-api-access-756xb") pod "06473528-93fe-433b-b327-abef6a484305" (UID: "06473528-93fe-433b-b327-abef6a484305"). InnerVolumeSpecName "kube-api-access-756xb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 04:13:19.189543 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:19.189471    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06473528-93fe-433b-b327-abef6a484305-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "06473528-93fe-433b-b327-abef6a484305" (UID: "06473528-93fe-433b-b327-abef6a484305"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 04:13:19.189543 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:19.189483    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06473528-93fe-433b-b327-abef6a484305-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "06473528-93fe-433b-b327-abef6a484305" (UID: "06473528-93fe-433b-b327-abef6a484305"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 04:13:19.289012 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:19.288975    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-756xb\" (UniqueName: \"kubernetes.io/projected/06473528-93fe-433b-b327-abef6a484305-kube-api-access-756xb\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:13:19.289012 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:19.289006    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06473528-93fe-433b-b327-abef6a484305-maas-api-tls\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:13:19.289012 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:19.289016    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06473528-93fe-433b-b327-abef6a484305-openshift-service-ca\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:13:19.289253 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:19.289025    2557 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06473528-93fe-433b-b327-abef6a484305-metrics-certs\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:13:19.883228 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:19.883140    2557 generic.go:356] "Generic (PLEG): container finished" podID="06473528-93fe-433b-b327-abef6a484305" containerID="4ecfe08aa1836cedf62bd0933466c97ddd5ef11b56dedea2e585b26591ce6374" exitCode=0
Sep 09 04:13:19.883595 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:19.883232    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj"
Sep 09 04:13:19.883595 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:19.883276    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj" event={"ID":"06473528-93fe-433b-b327-abef6a484305","Type":"ContainerDied","Data":"4ecfe08aa1836cedf62bd0933466c97ddd5ef11b56dedea2e585b26591ce6374"}
Sep 09 04:13:19.883595 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:19.883324    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj" event={"ID":"06473528-93fe-433b-b327-abef6a484305","Type":"ContainerDied","Data":"5e6e3126850821d044c80b66108a480a7a185d94f0b50eb8708eb0d52a4e5386"}
Sep 09 04:13:19.883595 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:19.883345    2557 scope.go:117] "RemoveContainer" containerID="4ecfe08aa1836cedf62bd0933466c97ddd5ef11b56dedea2e585b26591ce6374"
Sep 09 04:13:19.892605 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:19.892280    2557 scope.go:117] "RemoveContainer" containerID="4ecfe08aa1836cedf62bd0933466c97ddd5ef11b56dedea2e585b26591ce6374"
Sep 09 04:13:19.892729 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:13:19.892615    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ecfe08aa1836cedf62bd0933466c97ddd5ef11b56dedea2e585b26591ce6374\": container with ID starting with 4ecfe08aa1836cedf62bd0933466c97ddd5ef11b56dedea2e585b26591ce6374 not found: ID does not exist" containerID="4ecfe08aa1836cedf62bd0933466c97ddd5ef11b56dedea2e585b26591ce6374"
Sep 09 04:13:19.892729 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:19.892648    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ecfe08aa1836cedf62bd0933466c97ddd5ef11b56dedea2e585b26591ce6374"} err="failed to get container status \"4ecfe08aa1836cedf62bd0933466c97ddd5ef11b56dedea2e585b26591ce6374\": rpc error: code = NotFound desc = could not find container \"4ecfe08aa1836cedf62bd0933466c97ddd5ef11b56dedea2e585b26591ce6374\": container with ID starting with 4ecfe08aa1836cedf62bd0933466c97ddd5ef11b56dedea2e585b26591ce6374 not found: ID does not exist"
Sep 09 04:13:19.900359 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:19.900334    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj"]
Sep 09 04:13:19.911733 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:19.911712    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1be20a5f-6694dcc865-skxfj"]
Sep 09 04:13:21.619171 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:21.619141    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06473528-93fe-433b-b327-abef6a484305" path="/var/lib/kubelet/pods/06473528-93fe-433b-b327-abef6a484305/volumes"
Sep 09 04:13:22.166752 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:22.166716    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-4ad5fa-9464748d7-98ntv"]
Sep 09 04:13:22.167027 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:22.166991    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-4ad5fa-9464748d7-98ntv" podUID="96564240-e02e-40ca-a3a4-11615480686c" containerName="payload-processing" containerID="cri-o://d3b52744d6c372b35f646655e37c630685f1f87881a3cbb087f1b2319def1afc" gracePeriod=30
Sep 09 04:13:22.175766 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:22.175390    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-4ad5fa-797d7dd8cd-mlw76"]
Sep 09 04:13:22.175766 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:22.175702    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-4ad5fa-797d7dd8cd-mlw76" podUID="83e7ffea-fc29-4812-9dcd-4ca3a83d8dba" containerName="payload-pre-processing" containerID="cri-o://5df30da36d5c634a49d8cba8b2a6f11cf8fab1f93ad9eb81639f50fd1534e417" gracePeriod=30
Sep 09 04:13:22.462053 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:22.462027    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-4ad5fa-9464748d7-98ntv"
Sep 09 04:13:22.464926 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:22.464899    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-4ad5fa-797d7dd8cd-mlw76"
Sep 09 04:13:22.518918 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:22.518889    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pg8kq\" (UniqueName: \"kubernetes.io/projected/96564240-e02e-40ca-a3a4-11615480686c-kube-api-access-pg8kq\") pod \"96564240-e02e-40ca-a3a4-11615480686c\" (UID: \"96564240-e02e-40ca-a3a4-11615480686c\") "
Sep 09 04:13:22.519104 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:22.518961    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96564240-e02e-40ca-a3a4-11615480686c-plugins-config-volume\") pod \"96564240-e02e-40ca-a3a4-11615480686c\" (UID: \"96564240-e02e-40ca-a3a4-11615480686c\") "
Sep 09 04:13:22.519391 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:22.519359    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96564240-e02e-40ca-a3a4-11615480686c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "96564240-e02e-40ca-a3a4-11615480686c" (UID: "96564240-e02e-40ca-a3a4-11615480686c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 04:13:22.521014 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:22.520994    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96564240-e02e-40ca-a3a4-11615480686c-kube-api-access-pg8kq" (OuterVolumeSpecName: "kube-api-access-pg8kq") pod "96564240-e02e-40ca-a3a4-11615480686c" (UID: "96564240-e02e-40ca-a3a4-11615480686c"). InnerVolumeSpecName "kube-api-access-pg8kq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 04:13:22.619631 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:22.619596    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7nvjt\" (UniqueName: \"kubernetes.io/projected/83e7ffea-fc29-4812-9dcd-4ca3a83d8dba-kube-api-access-7nvjt\") pod \"83e7ffea-fc29-4812-9dcd-4ca3a83d8dba\" (UID: \"83e7ffea-fc29-4812-9dcd-4ca3a83d8dba\") "
Sep 09 04:13:22.620086 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:22.619642    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83e7ffea-fc29-4812-9dcd-4ca3a83d8dba-plugins-config-volume\") pod \"83e7ffea-fc29-4812-9dcd-4ca3a83d8dba\" (UID: \"83e7ffea-fc29-4812-9dcd-4ca3a83d8dba\") "
Sep 09 04:13:22.620086 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:22.619816    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96564240-e02e-40ca-a3a4-11615480686c-plugins-config-volume\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:13:22.620086 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:22.619830    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pg8kq\" (UniqueName: \"kubernetes.io/projected/96564240-e02e-40ca-a3a4-11615480686c-kube-api-access-pg8kq\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:13:22.620266 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:22.620106    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83e7ffea-fc29-4812-9dcd-4ca3a83d8dba-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "83e7ffea-fc29-4812-9dcd-4ca3a83d8dba" (UID: "83e7ffea-fc29-4812-9dcd-4ca3a83d8dba"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 04:13:22.621704 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:22.621684    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83e7ffea-fc29-4812-9dcd-4ca3a83d8dba-kube-api-access-7nvjt" (OuterVolumeSpecName: "kube-api-access-7nvjt") pod "83e7ffea-fc29-4812-9dcd-4ca3a83d8dba" (UID: "83e7ffea-fc29-4812-9dcd-4ca3a83d8dba"). InnerVolumeSpecName "kube-api-access-7nvjt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 04:13:22.725237 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:22.720973    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7nvjt\" (UniqueName: \"kubernetes.io/projected/83e7ffea-fc29-4812-9dcd-4ca3a83d8dba-kube-api-access-7nvjt\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:13:22.725237 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:22.721005    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83e7ffea-fc29-4812-9dcd-4ca3a83d8dba-plugins-config-volume\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:13:22.899418 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:22.899378    2557 generic.go:356] "Generic (PLEG): container finished" podID="96564240-e02e-40ca-a3a4-11615480686c" containerID="d3b52744d6c372b35f646655e37c630685f1f87881a3cbb087f1b2319def1afc" exitCode=0
Sep 09 04:13:22.899576 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:22.899440    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-4ad5fa-9464748d7-98ntv"
Sep 09 04:13:22.899576 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:22.899470    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-4ad5fa-9464748d7-98ntv" event={"ID":"96564240-e02e-40ca-a3a4-11615480686c","Type":"ContainerDied","Data":"d3b52744d6c372b35f646655e37c630685f1f87881a3cbb087f1b2319def1afc"}
Sep 09 04:13:22.899576 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:22.899558    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-4ad5fa-9464748d7-98ntv" event={"ID":"96564240-e02e-40ca-a3a4-11615480686c","Type":"ContainerDied","Data":"4790830cba8e04b518c3b01bfdea1402d670121b39eb2f2c2848e2f6319994e3"}
Sep 09 04:13:22.899708 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:22.899583    2557 scope.go:117] "RemoveContainer" containerID="d3b52744d6c372b35f646655e37c630685f1f87881a3cbb087f1b2319def1afc"
Sep 09 04:13:22.900907 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:22.900881    2557 generic.go:356] "Generic (PLEG): container finished" podID="83e7ffea-fc29-4812-9dcd-4ca3a83d8dba" containerID="5df30da36d5c634a49d8cba8b2a6f11cf8fab1f93ad9eb81639f50fd1534e417" exitCode=0
Sep 09 04:13:22.901031 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:22.900929    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-4ad5fa-797d7dd8cd-mlw76" event={"ID":"83e7ffea-fc29-4812-9dcd-4ca3a83d8dba","Type":"ContainerDied","Data":"5df30da36d5c634a49d8cba8b2a6f11cf8fab1f93ad9eb81639f50fd1534e417"}
Sep 09 04:13:22.901031 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:22.900964    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-4ad5fa-797d7dd8cd-mlw76" event={"ID":"83e7ffea-fc29-4812-9dcd-4ca3a83d8dba","Type":"ContainerDied","Data":"da2585d0f7d072219ad682ec3df2d4d59049d9d7265b1a9b16dde9d02b1fe542"}
Sep 09 04:13:22.901031 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:22.901008    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-4ad5fa-797d7dd8cd-mlw76"
Sep 09 04:13:22.910212 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:22.910192    2557 scope.go:117] "RemoveContainer" containerID="d3b52744d6c372b35f646655e37c630685f1f87881a3cbb087f1b2319def1afc"
Sep 09 04:13:22.910538 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:13:22.910511    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3b52744d6c372b35f646655e37c630685f1f87881a3cbb087f1b2319def1afc\": container with ID starting with d3b52744d6c372b35f646655e37c630685f1f87881a3cbb087f1b2319def1afc not found: ID does not exist" containerID="d3b52744d6c372b35f646655e37c630685f1f87881a3cbb087f1b2319def1afc"
Sep 09 04:13:22.910624 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:22.910548    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3b52744d6c372b35f646655e37c630685f1f87881a3cbb087f1b2319def1afc"} err="failed to get container status \"d3b52744d6c372b35f646655e37c630685f1f87881a3cbb087f1b2319def1afc\": rpc error: code = NotFound desc = could not find container \"d3b52744d6c372b35f646655e37c630685f1f87881a3cbb087f1b2319def1afc\": container with ID starting with d3b52744d6c372b35f646655e37c630685f1f87881a3cbb087f1b2319def1afc not found: ID does not exist"
Sep 09 04:13:22.910624 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:22.910573    2557 scope.go:117] "RemoveContainer" containerID="5df30da36d5c634a49d8cba8b2a6f11cf8fab1f93ad9eb81639f50fd1534e417"
Sep 09 04:13:22.918089 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:22.918074    2557 scope.go:117] "RemoveContainer" containerID="5df30da36d5c634a49d8cba8b2a6f11cf8fab1f93ad9eb81639f50fd1534e417"
Sep 09 04:13:22.918365 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:13:22.918346    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5df30da36d5c634a49d8cba8b2a6f11cf8fab1f93ad9eb81639f50fd1534e417\": container with ID starting with 5df30da36d5c634a49d8cba8b2a6f11cf8fab1f93ad9eb81639f50fd1534e417 not found: ID does not exist" containerID="5df30da36d5c634a49d8cba8b2a6f11cf8fab1f93ad9eb81639f50fd1534e417"
Sep 09 04:13:22.918430 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:22.918371    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5df30da36d5c634a49d8cba8b2a6f11cf8fab1f93ad9eb81639f50fd1534e417"} err="failed to get container status \"5df30da36d5c634a49d8cba8b2a6f11cf8fab1f93ad9eb81639f50fd1534e417\": rpc error: code = NotFound desc = could not find container \"5df30da36d5c634a49d8cba8b2a6f11cf8fab1f93ad9eb81639f50fd1534e417\": container with ID starting with 5df30da36d5c634a49d8cba8b2a6f11cf8fab1f93ad9eb81639f50fd1534e417 not found: ID does not exist"
Sep 09 04:13:22.924433 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:22.924413    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-4ad5fa-9464748d7-98ntv"]
Sep 09 04:13:22.927679 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:22.927659    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-4ad5fa-9464748d7-98ntv"]
Sep 09 04:13:22.938044 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:22.938025    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-4ad5fa-797d7dd8cd-mlw76"]
Sep 09 04:13:22.942132 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:22.942098    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-4ad5fa-797d7dd8cd-mlw76"]
Sep 09 04:13:23.619375 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:23.619342    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83e7ffea-fc29-4812-9dcd-4ca3a83d8dba" path="/var/lib/kubelet/pods/83e7ffea-fc29-4812-9dcd-4ca3a83d8dba/volumes"
Sep 09 04:13:23.619702 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:13:23.619688    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96564240-e02e-40ca-a3a4-11615480686c" path="/var/lib/kubelet/pods/96564240-e02e-40ca-a3a4-11615480686c/volumes"
Sep 09 04:14:10.105248 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:14:10.105205    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-b80654-57d9964785-tz4mc"]
Sep 09 04:14:10.105815 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:14:10.105786    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83e7ffea-fc29-4812-9dcd-4ca3a83d8dba" containerName="payload-pre-processing"
Sep 09 04:14:10.105815 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:14:10.105806    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="83e7ffea-fc29-4812-9dcd-4ca3a83d8dba" containerName="payload-pre-processing"
Sep 09 04:14:10.105935 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:14:10.105823    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06473528-93fe-433b-b327-abef6a484305" containerName="maas-api"
Sep 09 04:14:10.105935 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:14:10.105832    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="06473528-93fe-433b-b327-abef6a484305" containerName="maas-api"
Sep 09 04:14:10.105935 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:14:10.105845    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96564240-e02e-40ca-a3a4-11615480686c" containerName="payload-processing"
Sep 09 04:14:10.105935 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:14:10.105854    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="96564240-e02e-40ca-a3a4-11615480686c" containerName="payload-processing"
Sep 09 04:14:10.106156 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:14:10.105956    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="83e7ffea-fc29-4812-9dcd-4ca3a83d8dba" containerName="payload-pre-processing"
Sep 09 04:14:10.106156 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:14:10.105970    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="96564240-e02e-40ca-a3a4-11615480686c" containerName="payload-processing"
Sep 09 04:14:10.106156 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:14:10.105980    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="06473528-93fe-433b-b327-abef6a484305" containerName="maas-api"
Sep 09 04:14:10.110686 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:14:10.110656    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b80654-57d9964785-tz4mc"
Sep 09 04:14:10.113397 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:14:10.113373    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-b80654-serving-cert\""
Sep 09 04:14:10.120410 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:14:10.120381    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-b80654-57d9964785-tz4mc"]
Sep 09 04:14:10.226608 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:14:10.226573    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfdebe0a-b4c5-4cd5-abd4-5c79a622f935-openshift-service-ca\") pod \"maas-api-e2e-explicit-b80654-57d9964785-tz4mc\" (UID: \"cfdebe0a-b4c5-4cd5-abd4-5c79a622f935\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b80654-57d9964785-tz4mc"
Sep 09 04:14:10.226608 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:14:10.226615    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-246qz\" (UniqueName: \"kubernetes.io/projected/cfdebe0a-b4c5-4cd5-abd4-5c79a622f935-kube-api-access-246qz\") pod \"maas-api-e2e-explicit-b80654-57d9964785-tz4mc\" (UID: \"cfdebe0a-b4c5-4cd5-abd4-5c79a622f935\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b80654-57d9964785-tz4mc"
Sep 09 04:14:10.226850 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:14:10.226662    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfdebe0a-b4c5-4cd5-abd4-5c79a622f935-metrics-certs\") pod \"maas-api-e2e-explicit-b80654-57d9964785-tz4mc\" (UID: \"cfdebe0a-b4c5-4cd5-abd4-5c79a622f935\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b80654-57d9964785-tz4mc"
Sep 09 04:14:10.226850 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:14:10.226694    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfdebe0a-b4c5-4cd5-abd4-5c79a622f935-maas-api-tls\") pod \"maas-api-e2e-explicit-b80654-57d9964785-tz4mc\" (UID: \"cfdebe0a-b4c5-4cd5-abd4-5c79a622f935\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b80654-57d9964785-tz4mc"
Sep 09 04:14:10.328232 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:14:10.328200    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfdebe0a-b4c5-4cd5-abd4-5c79a622f935-openshift-service-ca\") pod \"maas-api-e2e-explicit-b80654-57d9964785-tz4mc\" (UID: \"cfdebe0a-b4c5-4cd5-abd4-5c79a622f935\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b80654-57d9964785-tz4mc"
Sep 09 04:14:10.328232 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:14:10.328235    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-246qz\" (UniqueName: \"kubernetes.io/projected/cfdebe0a-b4c5-4cd5-abd4-5c79a622f935-kube-api-access-246qz\") pod \"maas-api-e2e-explicit-b80654-57d9964785-tz4mc\" (UID: \"cfdebe0a-b4c5-4cd5-abd4-5c79a622f935\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b80654-57d9964785-tz4mc"
Sep 09 04:14:10.328478 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:14:10.328260    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfdebe0a-b4c5-4cd5-abd4-5c79a622f935-metrics-certs\") pod \"maas-api-e2e-explicit-b80654-57d9964785-tz4mc\" (UID: \"cfdebe0a-b4c5-4cd5-abd4-5c79a622f935\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b80654-57d9964785-tz4mc"
Sep 09 04:14:10.328478 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:14:10.328281    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfdebe0a-b4c5-4cd5-abd4-5c79a622f935-maas-api-tls\") pod \"maas-api-e2e-explicit-b80654-57d9964785-tz4mc\" (UID: \"cfdebe0a-b4c5-4cd5-abd4-5c79a622f935\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b80654-57d9964785-tz4mc"
Sep 09 04:14:10.328952 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:14:10.328927    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfdebe0a-b4c5-4cd5-abd4-5c79a622f935-openshift-service-ca\") pod \"maas-api-e2e-explicit-b80654-57d9964785-tz4mc\" (UID: \"cfdebe0a-b4c5-4cd5-abd4-5c79a622f935\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b80654-57d9964785-tz4mc"
Sep 09 04:14:10.330850 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:14:10.330830    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfdebe0a-b4c5-4cd5-abd4-5c79a622f935-metrics-certs\") pod \"maas-api-e2e-explicit-b80654-57d9964785-tz4mc\" (UID: \"cfdebe0a-b4c5-4cd5-abd4-5c79a622f935\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b80654-57d9964785-tz4mc"
Sep 09 04:14:10.330973 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:14:10.330951    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfdebe0a-b4c5-4cd5-abd4-5c79a622f935-maas-api-tls\") pod \"maas-api-e2e-explicit-b80654-57d9964785-tz4mc\" (UID: \"cfdebe0a-b4c5-4cd5-abd4-5c79a622f935\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b80654-57d9964785-tz4mc"
Sep 09 04:14:10.337583 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:14:10.337559    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-246qz\" (UniqueName: \"kubernetes.io/projected/cfdebe0a-b4c5-4cd5-abd4-5c79a622f935-kube-api-access-246qz\") pod \"maas-api-e2e-explicit-b80654-57d9964785-tz4mc\" (UID: \"cfdebe0a-b4c5-4cd5-abd4-5c79a622f935\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b80654-57d9964785-tz4mc"
Sep 09 04:14:10.422154 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:14:10.422118    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b80654-57d9964785-tz4mc"
Sep 09 04:14:10.696281 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:14:10.696163    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-b80654-57d9964785-tz4mc"]
Sep 09 04:14:10.698984 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:14:10.698956    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfdebe0a_b4c5_4cd5_abd4_5c79a622f935.slice/crio-3bfb5facfaa5b973930571e6f2c07eadc5b664c759bc7e0a1aa8bd96be8d0f92 WatchSource:0}: Error finding container 3bfb5facfaa5b973930571e6f2c07eadc5b664c759bc7e0a1aa8bd96be8d0f92: Status 404 returned error can't find the container with id 3bfb5facfaa5b973930571e6f2c07eadc5b664c759bc7e0a1aa8bd96be8d0f92
Sep 09 04:14:11.139573 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:14:11.139542    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b80654-57d9964785-tz4mc" event={"ID":"cfdebe0a-b4c5-4cd5-abd4-5c79a622f935","Type":"ContainerStarted","Data":"3bfb5facfaa5b973930571e6f2c07eadc5b664c759bc7e0a1aa8bd96be8d0f92"}
Sep 09 04:14:12.145656 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:14:12.145618    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b80654-57d9964785-tz4mc" event={"ID":"cfdebe0a-b4c5-4cd5-abd4-5c79a622f935","Type":"ContainerStarted","Data":"78c690533fdb72b37210919e09981708f701137516f2feeefc6dcabd5431a872"}
Sep 09 04:14:12.146153 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:14:12.145747    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b80654-57d9964785-tz4mc"
Sep 09 04:14:12.162938 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:14:12.162894    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b80654-57d9964785-tz4mc" podStartSLOduration=1.590068982 podStartE2EDuration="2.162879436s" totalImagesPullingTime="572.810454ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:14:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:14:12.161627398 +0000 UTC m=+977.080798308" watchObservedRunningTime="2026-09-09 04:14:12.162879436 +0000 UTC m=+977.082050345"
Sep 09 04:14:18.156056 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:14:18.155980    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b80654-57d9964785-tz4mc"
Sep 09 04:15:00.143690 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:00.143650    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-359adf-29815455-v5cpr"]
Sep 09 04:15:00.147845 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:00.147816    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-359adf-29815455-v5cpr"
Sep 09 04:15:00.150433 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:00.150403    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qlbvk\""
Sep 09 04:15:00.158004 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:00.157873    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-359adf-29815455-v5cpr"]
Sep 09 04:15:00.168626 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:00.168587    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-359adf-29815455-v5cpr"
Sep 09 04:15:00.459320 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:00.459208    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-359adf-29815455-v5cpr"]
Sep 09 04:15:00.463629 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:15:00.463589    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd831303c_5971_4dff_9238_0eb3b19e0ea9.slice/crio-369bdaccc9ea6a28c3f5cb4c1fed7fe4576f3a5bf91cbf941ceec99f06a3f130 WatchSource:0}: Error finding container 369bdaccc9ea6a28c3f5cb4c1fed7fe4576f3a5bf91cbf941ceec99f06a3f130: Status 404 returned error can't find the container with id 369bdaccc9ea6a28c3f5cb4c1fed7fe4576f3a5bf91cbf941ceec99f06a3f130
Sep 09 04:15:00.471746 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:00.471720    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk"]
Sep 09 04:15:00.478954 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:00.478928    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk"
Sep 09 04:15:00.481765 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:00.481738    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-f0f5e1c0-serving-cert\""
Sep 09 04:15:00.490605 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:00.490576    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk"]
Sep 09 04:15:00.585001 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:00.584969    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52adaa08-2e9a-4890-b0d1-8fff93149720-maas-api-tls\") pod \"maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk\" (UID: \"52adaa08-2e9a-4890-b0d1-8fff93149720\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk"
Sep 09 04:15:00.585320 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:00.585295    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52adaa08-2e9a-4890-b0d1-8fff93149720-metrics-certs\") pod \"maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk\" (UID: \"52adaa08-2e9a-4890-b0d1-8fff93149720\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk"
Sep 09 04:15:00.585534 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:00.585518    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52adaa08-2e9a-4890-b0d1-8fff93149720-openshift-service-ca\") pod \"maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk\" (UID: \"52adaa08-2e9a-4890-b0d1-8fff93149720\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk"
Sep 09 04:15:00.585638 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:00.585625    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgv8q\" (UniqueName: \"kubernetes.io/projected/52adaa08-2e9a-4890-b0d1-8fff93149720-kube-api-access-hgv8q\") pod \"maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk\" (UID: \"52adaa08-2e9a-4890-b0d1-8fff93149720\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk"
Sep 09 04:15:00.686583 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:00.686544    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52adaa08-2e9a-4890-b0d1-8fff93149720-maas-api-tls\") pod \"maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk\" (UID: \"52adaa08-2e9a-4890-b0d1-8fff93149720\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk"
Sep 09 04:15:00.686754 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:00.686601    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52adaa08-2e9a-4890-b0d1-8fff93149720-metrics-certs\") pod \"maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk\" (UID: \"52adaa08-2e9a-4890-b0d1-8fff93149720\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk"
Sep 09 04:15:00.686754 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:00.686660    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52adaa08-2e9a-4890-b0d1-8fff93149720-openshift-service-ca\") pod \"maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk\" (UID: \"52adaa08-2e9a-4890-b0d1-8fff93149720\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk"
Sep 09 04:15:00.686754 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:00.686680    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgv8q\" (UniqueName: \"kubernetes.io/projected/52adaa08-2e9a-4890-b0d1-8fff93149720-kube-api-access-hgv8q\") pod \"maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk\" (UID: \"52adaa08-2e9a-4890-b0d1-8fff93149720\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk"
Sep 09 04:15:00.686754 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:15:00.686715    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-f0f5e1c0-serving-cert: secret "maas-api-e2e-mt-f0f5e1c0-serving-cert" not found
Sep 09 04:15:00.686890 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:15:00.686802    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52adaa08-2e9a-4890-b0d1-8fff93149720-maas-api-tls podName:52adaa08-2e9a-4890-b0d1-8fff93149720 nodeName:}" failed. No retries permitted until 2026-09-09 04:15:01.186778643 +0000 UTC m=+1026.105949554 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/52adaa08-2e9a-4890-b0d1-8fff93149720-maas-api-tls") pod "maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk" (UID: "52adaa08-2e9a-4890-b0d1-8fff93149720") : secret "maas-api-e2e-mt-f0f5e1c0-serving-cert" not found
Sep 09 04:15:00.687399 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:00.687379    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52adaa08-2e9a-4890-b0d1-8fff93149720-openshift-service-ca\") pod \"maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk\" (UID: \"52adaa08-2e9a-4890-b0d1-8fff93149720\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk"
Sep 09 04:15:00.689026 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:00.689009    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52adaa08-2e9a-4890-b0d1-8fff93149720-metrics-certs\") pod \"maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk\" (UID: \"52adaa08-2e9a-4890-b0d1-8fff93149720\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk"
Sep 09 04:15:00.696225 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:00.696198    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgv8q\" (UniqueName: \"kubernetes.io/projected/52adaa08-2e9a-4890-b0d1-8fff93149720-kube-api-access-hgv8q\") pod \"maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk\" (UID: \"52adaa08-2e9a-4890-b0d1-8fff93149720\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk"
Sep 09 04:15:01.095075 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:01.095042    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f0f5e1c0-648bfb754-xwxnl"]
Sep 09 04:15:01.098661 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:01.098643    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f0f5e1c0-648bfb754-xwxnl"
Sep 09 04:15:01.100891 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:01.100872    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-f0f5e1c0\""
Sep 09 04:15:01.108352 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:01.108329    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f0f5e1c0-648bfb754-xwxnl"]
Sep 09 04:15:01.191364 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:01.191329    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5p8p\" (UniqueName: \"kubernetes.io/projected/41e6feab-ec4e-4e86-af9c-f4b285a2de9c-kube-api-access-z5p8p\") pod \"payload-pre-processing-e2e-mt-f0f5e1c0-648bfb754-xwxnl\" (UID: \"41e6feab-ec4e-4e86-af9c-f4b285a2de9c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f0f5e1c0-648bfb754-xwxnl"
Sep 09 04:15:01.191727 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:01.191391    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41e6feab-ec4e-4e86-af9c-f4b285a2de9c-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f0f5e1c0-648bfb754-xwxnl\" (UID: \"41e6feab-ec4e-4e86-af9c-f4b285a2de9c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f0f5e1c0-648bfb754-xwxnl"
Sep 09 04:15:01.191727 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:01.191473    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52adaa08-2e9a-4890-b0d1-8fff93149720-maas-api-tls\") pod \"maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk\" (UID: \"52adaa08-2e9a-4890-b0d1-8fff93149720\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk"
Sep 09 04:15:01.193792 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:01.193771    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52adaa08-2e9a-4890-b0d1-8fff93149720-maas-api-tls\") pod \"maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk\" (UID: \"52adaa08-2e9a-4890-b0d1-8fff93149720\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk"
Sep 09 04:15:01.292334 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:01.292291    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5p8p\" (UniqueName: \"kubernetes.io/projected/41e6feab-ec4e-4e86-af9c-f4b285a2de9c-kube-api-access-z5p8p\") pod \"payload-pre-processing-e2e-mt-f0f5e1c0-648bfb754-xwxnl\" (UID: \"41e6feab-ec4e-4e86-af9c-f4b285a2de9c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f0f5e1c0-648bfb754-xwxnl"
Sep 09 04:15:01.292506 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:01.292381    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41e6feab-ec4e-4e86-af9c-f4b285a2de9c-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f0f5e1c0-648bfb754-xwxnl\" (UID: \"41e6feab-ec4e-4e86-af9c-f4b285a2de9c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f0f5e1c0-648bfb754-xwxnl"
Sep 09 04:15:01.293090 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:01.293070    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41e6feab-ec4e-4e86-af9c-f4b285a2de9c-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f0f5e1c0-648bfb754-xwxnl\" (UID: \"41e6feab-ec4e-4e86-af9c-f4b285a2de9c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f0f5e1c0-648bfb754-xwxnl"
Sep 09 04:15:01.300066 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:01.300041    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5p8p\" (UniqueName: \"kubernetes.io/projected/41e6feab-ec4e-4e86-af9c-f4b285a2de9c-kube-api-access-z5p8p\") pod \"payload-pre-processing-e2e-mt-f0f5e1c0-648bfb754-xwxnl\" (UID: \"41e6feab-ec4e-4e86-af9c-f4b285a2de9c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f0f5e1c0-648bfb754-xwxnl"
Sep 09 04:15:01.385700 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:01.385668    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-359adf-29815455-v5cpr" event={"ID":"d831303c-5971-4dff-9238-0eb3b19e0ea9","Type":"ContainerStarted","Data":"369bdaccc9ea6a28c3f5cb4c1fed7fe4576f3a5bf91cbf941ceec99f06a3f130"}
Sep 09 04:15:01.391856 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:01.391830    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk"
Sep 09 04:15:01.409517 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:01.409496    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f0f5e1c0-648bfb754-xwxnl"
Sep 09 04:15:01.670087 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:01.669963    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk"]
Sep 09 04:15:01.672376 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:15:01.672344    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52adaa08_2e9a_4890_b0d1_8fff93149720.slice/crio-911a119b2f479a6776cf129f590260667d4375fdcecafb74a8209b33cb0508fe WatchSource:0}: Error finding container 911a119b2f479a6776cf129f590260667d4375fdcecafb74a8209b33cb0508fe: Status 404 returned error can't find the container with id 911a119b2f479a6776cf129f590260667d4375fdcecafb74a8209b33cb0508fe
Sep 09 04:15:01.691747 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:01.691724    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f0f5e1c0-648bfb754-xwxnl"]
Sep 09 04:15:01.693887 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:15:01.693857    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41e6feab_ec4e_4e86_af9c_f4b285a2de9c.slice/crio-f4ec655fe6a8bb5c42f0b642360173a1a93428a093839cda64624141822b5e46 WatchSource:0}: Error finding container f4ec655fe6a8bb5c42f0b642360173a1a93428a093839cda64624141822b5e46: Status 404 returned error can't find the container with id f4ec655fe6a8bb5c42f0b642360173a1a93428a093839cda64624141822b5e46
Sep 09 04:15:02.398351 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:02.397773    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f0f5e1c0-648bfb754-xwxnl" event={"ID":"41e6feab-ec4e-4e86-af9c-f4b285a2de9c","Type":"ContainerStarted","Data":"80aa6d047151dac27fa21335f293fafab283c2394bd0aa238d4696738f38b4c0"}
Sep 09 04:15:02.398351 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:02.397837    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f0f5e1c0-648bfb754-xwxnl" event={"ID":"41e6feab-ec4e-4e86-af9c-f4b285a2de9c","Type":"ContainerStarted","Data":"f4ec655fe6a8bb5c42f0b642360173a1a93428a093839cda64624141822b5e46"}
Sep 09 04:15:02.398351 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:02.397981    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-f0f5e1c0-648bfb754-xwxnl"
Sep 09 04:15:02.399898 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:02.399870    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk" event={"ID":"52adaa08-2e9a-4890-b0d1-8fff93149720","Type":"ContainerStarted","Data":"4497f8ec5ddda97ad3f405470189646000226a4a1ea1cd7c0cfca1183b563411"}
Sep 09 04:15:02.400005 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:02.399924    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk" event={"ID":"52adaa08-2e9a-4890-b0d1-8fff93149720","Type":"ContainerStarted","Data":"911a119b2f479a6776cf129f590260667d4375fdcecafb74a8209b33cb0508fe"}
Sep 09 04:15:02.400005 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:02.399958    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk"
Sep 09 04:15:02.417212 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:02.417153    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-f0f5e1c0-648bfb754-xwxnl" podStartSLOduration=1.417132834 podStartE2EDuration="1.417132834s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:15:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:15:02.413987012 +0000 UTC m=+1027.333157924" watchObservedRunningTime="2026-09-09 04:15:02.417132834 +0000 UTC m=+1027.336303746"
Sep 09 04:15:02.430054 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:02.429967    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk" podStartSLOduration=1.811501007 podStartE2EDuration="2.429808894s" totalImagesPullingTime="618.307887ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:15:02.429325656 +0000 UTC m=+1027.348496569" watchObservedRunningTime="2026-09-09 04:15:02.429808894 +0000 UTC m=+1027.348979806"
Sep 09 04:15:04.412447 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:04.412411    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-359adf-29815455-v5cpr" event={"ID":"d831303c-5971-4dff-9238-0eb3b19e0ea9","Type":"ContainerStarted","Data":"594102e46a7e49e1fcace5363e78fd593701267f3e69f6a7f6af936798cbe679"}
Sep 09 04:15:04.428260 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:04.428207    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-359adf-29815455-v5cpr" podStartSLOduration=1.19279015 podStartE2EDuration="4.428192464s" totalImagesPullingTime="3.235402314s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:15:04.426612367 +0000 UTC m=+1029.345783278" watchObservedRunningTime="2026-09-09 04:15:04.428192464 +0000 UTC m=+1029.347363374"
Sep 09 04:15:05.418820 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:05.418781    2557 generic.go:356] "Generic (PLEG): container finished" podID="d831303c-5971-4dff-9238-0eb3b19e0ea9" containerID="594102e46a7e49e1fcace5363e78fd593701267f3e69f6a7f6af936798cbe679" exitCode=0
Sep 09 04:15:05.419216 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:05.418825    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-359adf-29815455-v5cpr" event={"ID":"d831303c-5971-4dff-9238-0eb3b19e0ea9","Type":"ContainerDied","Data":"594102e46a7e49e1fcace5363e78fd593701267f3e69f6a7f6af936798cbe679"}
Sep 09 04:15:06.559226 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:06.559196    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-359adf-29815455-v5cpr"
Sep 09 04:15:07.429403 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:07.429368    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-359adf-29815455-v5cpr" event={"ID":"d831303c-5971-4dff-9238-0eb3b19e0ea9","Type":"ContainerDied","Data":"369bdaccc9ea6a28c3f5cb4c1fed7fe4576f3a5bf91cbf941ceec99f06a3f130"}
Sep 09 04:15:07.429576 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:07.429417    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="369bdaccc9ea6a28c3f5cb4c1fed7fe4576f3a5bf91cbf941ceec99f06a3f130"
Sep 09 04:15:07.429576 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:07.429380    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-359adf-29815455-v5cpr"
Sep 09 04:15:08.411346 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:08.411319    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk"
Sep 09 04:15:13.408456 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:13.408424    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-f0f5e1c0-648bfb754-xwxnl"
Sep 09 04:15:42.254038 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.254004    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk"]
Sep 09 04:15:42.254651 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.254225    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk" podUID="52adaa08-2e9a-4890-b0d1-8fff93149720" containerName="maas-api" containerID="cri-o://4497f8ec5ddda97ad3f405470189646000226a4a1ea1cd7c0cfca1183b563411" gracePeriod=30
Sep 09 04:15:42.308570 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.308517    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f0f5e1c0-648bfb754-xwxnl"]
Sep 09 04:15:42.308893 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.308773    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-f0f5e1c0-648bfb754-xwxnl" podUID="41e6feab-ec4e-4e86-af9c-f4b285a2de9c" containerName="payload-pre-processing" containerID="cri-o://80aa6d047151dac27fa21335f293fafab283c2394bd0aa238d4696738f38b4c0" gracePeriod=30
Sep 09 04:15:42.558664 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.558636    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk"
Sep 09 04:15:42.589980 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.589954    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f0f5e1c0-648bfb754-xwxnl"
Sep 09 04:15:42.610986 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.610959    2557 generic.go:356] "Generic (PLEG): container finished" podID="41e6feab-ec4e-4e86-af9c-f4b285a2de9c" containerID="80aa6d047151dac27fa21335f293fafab283c2394bd0aa238d4696738f38b4c0" exitCode=0
Sep 09 04:15:42.611151 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.611025    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f0f5e1c0-648bfb754-xwxnl"
Sep 09 04:15:42.611151 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.611063    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f0f5e1c0-648bfb754-xwxnl" event={"ID":"41e6feab-ec4e-4e86-af9c-f4b285a2de9c","Type":"ContainerDied","Data":"80aa6d047151dac27fa21335f293fafab283c2394bd0aa238d4696738f38b4c0"}
Sep 09 04:15:42.611310 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.611292    2557 scope.go:117] "RemoveContainer" containerID="80aa6d047151dac27fa21335f293fafab283c2394bd0aa238d4696738f38b4c0"
Sep 09 04:15:42.611397 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.611375    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f0f5e1c0-648bfb754-xwxnl" event={"ID":"41e6feab-ec4e-4e86-af9c-f4b285a2de9c","Type":"ContainerDied","Data":"f4ec655fe6a8bb5c42f0b642360173a1a93428a093839cda64624141822b5e46"}
Sep 09 04:15:42.613225 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.613200    2557 generic.go:356] "Generic (PLEG): container finished" podID="52adaa08-2e9a-4890-b0d1-8fff93149720" containerID="4497f8ec5ddda97ad3f405470189646000226a4a1ea1cd7c0cfca1183b563411" exitCode=0
Sep 09 04:15:42.613329 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.613245    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk" event={"ID":"52adaa08-2e9a-4890-b0d1-8fff93149720","Type":"ContainerDied","Data":"4497f8ec5ddda97ad3f405470189646000226a4a1ea1cd7c0cfca1183b563411"}
Sep 09 04:15:42.613329 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.613276    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk"
Sep 09 04:15:42.613329 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.613284    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk" event={"ID":"52adaa08-2e9a-4890-b0d1-8fff93149720","Type":"ContainerDied","Data":"911a119b2f479a6776cf129f590260667d4375fdcecafb74a8209b33cb0508fe"}
Sep 09 04:15:42.620439 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.620422    2557 scope.go:117] "RemoveContainer" containerID="80aa6d047151dac27fa21335f293fafab283c2394bd0aa238d4696738f38b4c0"
Sep 09 04:15:42.620706 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:15:42.620690    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80aa6d047151dac27fa21335f293fafab283c2394bd0aa238d4696738f38b4c0\": container with ID starting with 80aa6d047151dac27fa21335f293fafab283c2394bd0aa238d4696738f38b4c0 not found: ID does not exist" containerID="80aa6d047151dac27fa21335f293fafab283c2394bd0aa238d4696738f38b4c0"
Sep 09 04:15:42.620790 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.620717    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80aa6d047151dac27fa21335f293fafab283c2394bd0aa238d4696738f38b4c0"} err="failed to get container status \"80aa6d047151dac27fa21335f293fafab283c2394bd0aa238d4696738f38b4c0\": rpc error: code = NotFound desc = could not find container \"80aa6d047151dac27fa21335f293fafab283c2394bd0aa238d4696738f38b4c0\": container with ID starting with 80aa6d047151dac27fa21335f293fafab283c2394bd0aa238d4696738f38b4c0 not found: ID does not exist"
Sep 09 04:15:42.620790 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.620740    2557 scope.go:117] "RemoveContainer" containerID="4497f8ec5ddda97ad3f405470189646000226a4a1ea1cd7c0cfca1183b563411"
Sep 09 04:15:42.628061 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.628046    2557 scope.go:117] "RemoveContainer" containerID="4497f8ec5ddda97ad3f405470189646000226a4a1ea1cd7c0cfca1183b563411"
Sep 09 04:15:42.628406 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:15:42.628389    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4497f8ec5ddda97ad3f405470189646000226a4a1ea1cd7c0cfca1183b563411\": container with ID starting with 4497f8ec5ddda97ad3f405470189646000226a4a1ea1cd7c0cfca1183b563411 not found: ID does not exist" containerID="4497f8ec5ddda97ad3f405470189646000226a4a1ea1cd7c0cfca1183b563411"
Sep 09 04:15:42.628472 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.628413    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4497f8ec5ddda97ad3f405470189646000226a4a1ea1cd7c0cfca1183b563411"} err="failed to get container status \"4497f8ec5ddda97ad3f405470189646000226a4a1ea1cd7c0cfca1183b563411\": rpc error: code = NotFound desc = could not find container \"4497f8ec5ddda97ad3f405470189646000226a4a1ea1cd7c0cfca1183b563411\": container with ID starting with 4497f8ec5ddda97ad3f405470189646000226a4a1ea1cd7c0cfca1183b563411 not found: ID does not exist"
Sep 09 04:15:42.658854 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.658834    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52adaa08-2e9a-4890-b0d1-8fff93149720-maas-api-tls\") pod \"52adaa08-2e9a-4890-b0d1-8fff93149720\" (UID: \"52adaa08-2e9a-4890-b0d1-8fff93149720\") "
Sep 09 04:15:42.658935 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.658899    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgv8q\" (UniqueName: \"kubernetes.io/projected/52adaa08-2e9a-4890-b0d1-8fff93149720-kube-api-access-hgv8q\") pod \"52adaa08-2e9a-4890-b0d1-8fff93149720\" (UID: \"52adaa08-2e9a-4890-b0d1-8fff93149720\") "
Sep 09 04:15:42.658935 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.658924    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41e6feab-ec4e-4e86-af9c-f4b285a2de9c-plugins-config-volume\") pod \"41e6feab-ec4e-4e86-af9c-f4b285a2de9c\" (UID: \"41e6feab-ec4e-4e86-af9c-f4b285a2de9c\") "
Sep 09 04:15:42.659051 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.658954    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5p8p\" (UniqueName: \"kubernetes.io/projected/41e6feab-ec4e-4e86-af9c-f4b285a2de9c-kube-api-access-z5p8p\") pod \"41e6feab-ec4e-4e86-af9c-f4b285a2de9c\" (UID: \"41e6feab-ec4e-4e86-af9c-f4b285a2de9c\") "
Sep 09 04:15:42.659051 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.658996    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52adaa08-2e9a-4890-b0d1-8fff93149720-metrics-certs\") pod \"52adaa08-2e9a-4890-b0d1-8fff93149720\" (UID: \"52adaa08-2e9a-4890-b0d1-8fff93149720\") "
Sep 09 04:15:42.659051 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.659022    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52adaa08-2e9a-4890-b0d1-8fff93149720-openshift-service-ca\") pod \"52adaa08-2e9a-4890-b0d1-8fff93149720\" (UID: \"52adaa08-2e9a-4890-b0d1-8fff93149720\") "
Sep 09 04:15:42.659431 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.659410    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41e6feab-ec4e-4e86-af9c-f4b285a2de9c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "41e6feab-ec4e-4e86-af9c-f4b285a2de9c" (UID: "41e6feab-ec4e-4e86-af9c-f4b285a2de9c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 04:15:42.659512 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.659467    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52adaa08-2e9a-4890-b0d1-8fff93149720-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "52adaa08-2e9a-4890-b0d1-8fff93149720" (UID: "52adaa08-2e9a-4890-b0d1-8fff93149720"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 04:15:42.660815 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.660789    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52adaa08-2e9a-4890-b0d1-8fff93149720-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "52adaa08-2e9a-4890-b0d1-8fff93149720" (UID: "52adaa08-2e9a-4890-b0d1-8fff93149720"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 04:15:42.661023 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.661010    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52adaa08-2e9a-4890-b0d1-8fff93149720-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "52adaa08-2e9a-4890-b0d1-8fff93149720" (UID: "52adaa08-2e9a-4890-b0d1-8fff93149720"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 04:15:42.661440 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.661423    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41e6feab-ec4e-4e86-af9c-f4b285a2de9c-kube-api-access-z5p8p" (OuterVolumeSpecName: "kube-api-access-z5p8p") pod "41e6feab-ec4e-4e86-af9c-f4b285a2de9c" (UID: "41e6feab-ec4e-4e86-af9c-f4b285a2de9c"). InnerVolumeSpecName "kube-api-access-z5p8p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 04:15:42.661494 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.661434    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52adaa08-2e9a-4890-b0d1-8fff93149720-kube-api-access-hgv8q" (OuterVolumeSpecName: "kube-api-access-hgv8q") pod "52adaa08-2e9a-4890-b0d1-8fff93149720" (UID: "52adaa08-2e9a-4890-b0d1-8fff93149720"). InnerVolumeSpecName "kube-api-access-hgv8q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 04:15:42.759686 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.759654    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/52adaa08-2e9a-4890-b0d1-8fff93149720-maas-api-tls\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:15:42.759686 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.759681    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgv8q\" (UniqueName: \"kubernetes.io/projected/52adaa08-2e9a-4890-b0d1-8fff93149720-kube-api-access-hgv8q\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:15:42.759686 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.759691    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41e6feab-ec4e-4e86-af9c-f4b285a2de9c-plugins-config-volume\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:15:42.759885 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.759700    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5p8p\" (UniqueName: \"kubernetes.io/projected/41e6feab-ec4e-4e86-af9c-f4b285a2de9c-kube-api-access-z5p8p\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:15:42.759885 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.759708    2557 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52adaa08-2e9a-4890-b0d1-8fff93149720-metrics-certs\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:15:42.759885 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.759716    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/52adaa08-2e9a-4890-b0d1-8fff93149720-openshift-service-ca\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:15:42.934061 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.934032    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f0f5e1c0-648bfb754-xwxnl"]
Sep 09 04:15:42.937766 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.937740    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f0f5e1c0-648bfb754-xwxnl"]
Sep 09 04:15:42.947715 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.947690    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk"]
Sep 09 04:15:42.951220 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:42.951199    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f0f5e1c0-688bd44857-pkjsk"]
Sep 09 04:15:43.620078 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:43.620044    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41e6feab-ec4e-4e86-af9c-f4b285a2de9c" path="/var/lib/kubelet/pods/41e6feab-ec4e-4e86-af9c-f4b285a2de9c/volumes"
Sep 09 04:15:43.620460 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:43.620444    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52adaa08-2e9a-4890-b0d1-8fff93149720" path="/var/lib/kubelet/pods/52adaa08-2e9a-4890-b0d1-8fff93149720/volumes"
Sep 09 04:15:52.763923 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:52.763891    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-85732c/distinct-w4-85732c-kserve-654fb7bb89-cfhns"]
Sep 09 04:15:52.764304 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:52.764278    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41e6feab-ec4e-4e86-af9c-f4b285a2de9c" containerName="payload-pre-processing"
Sep 09 04:15:52.764304 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:52.764291    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="41e6feab-ec4e-4e86-af9c-f4b285a2de9c" containerName="payload-pre-processing"
Sep 09 04:15:52.764304 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:52.764298    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d831303c-5971-4dff-9238-0eb3b19e0ea9" containerName="cleanup"
Sep 09 04:15:52.764304 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:52.764304    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d831303c-5971-4dff-9238-0eb3b19e0ea9" containerName="cleanup"
Sep 09 04:15:52.764433 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:52.764317    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52adaa08-2e9a-4890-b0d1-8fff93149720" containerName="maas-api"
Sep 09 04:15:52.764433 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:52.764323    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="52adaa08-2e9a-4890-b0d1-8fff93149720" containerName="maas-api"
Sep 09 04:15:52.764433 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:52.764397    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d831303c-5971-4dff-9238-0eb3b19e0ea9" containerName="cleanup"
Sep 09 04:15:52.764433 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:52.764407    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="41e6feab-ec4e-4e86-af9c-f4b285a2de9c" containerName="payload-pre-processing"
Sep 09 04:15:52.764433 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:52.764415    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="52adaa08-2e9a-4890-b0d1-8fff93149720" containerName="maas-api"
Sep 09 04:15:52.767213 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:52.767191    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-85732c/distinct-w4-85732c-kserve-654fb7bb89-cfhns"
Sep 09 04:15:52.769675 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:52.769647    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-85732c\"/\"openshift-service-ca.crt\""
Sep 09 04:15:52.769675 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:52.769667    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-85732c\"/\"kube-root-ca.crt\""
Sep 09 04:15:52.770615 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:52.770583    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-85732c\"/\"distinct-w4-85732c-kserve-self-signed-certs\""
Sep 09 04:15:52.770718 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:52.770629    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-85732c\"/\"default-dockercfg-scmzk\""
Sep 09 04:15:52.777214 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:52.777187    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-85732c/distinct-w4-85732c-kserve-654fb7bb89-cfhns"]
Sep 09 04:15:52.852319 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:52.852290    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b009df2d-e263-42b8-a652-e96fda4bb254-dshm\") pod \"distinct-w4-85732c-kserve-654fb7bb89-cfhns\" (UID: \"b009df2d-e263-42b8-a652-e96fda4bb254\") " pod="e2e-models-e2e-worker-w4-85732c/distinct-w4-85732c-kserve-654fb7bb89-cfhns"
Sep 09 04:15:52.852460 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:52.852326    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b009df2d-e263-42b8-a652-e96fda4bb254-home\") pod \"distinct-w4-85732c-kserve-654fb7bb89-cfhns\" (UID: \"b009df2d-e263-42b8-a652-e96fda4bb254\") " pod="e2e-models-e2e-worker-w4-85732c/distinct-w4-85732c-kserve-654fb7bb89-cfhns"
Sep 09 04:15:52.852460 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:52.852345    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b009df2d-e263-42b8-a652-e96fda4bb254-tmp-dir\") pod \"distinct-w4-85732c-kserve-654fb7bb89-cfhns\" (UID: \"b009df2d-e263-42b8-a652-e96fda4bb254\") " pod="e2e-models-e2e-worker-w4-85732c/distinct-w4-85732c-kserve-654fb7bb89-cfhns"
Sep 09 04:15:52.852546 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:52.852460    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzmpw\" (UniqueName: \"kubernetes.io/projected/b009df2d-e263-42b8-a652-e96fda4bb254-kube-api-access-xzmpw\") pod \"distinct-w4-85732c-kserve-654fb7bb89-cfhns\" (UID: \"b009df2d-e263-42b8-a652-e96fda4bb254\") " pod="e2e-models-e2e-worker-w4-85732c/distinct-w4-85732c-kserve-654fb7bb89-cfhns"
Sep 09 04:15:52.852546 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:52.852510    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b009df2d-e263-42b8-a652-e96fda4bb254-model-cache\") pod \"distinct-w4-85732c-kserve-654fb7bb89-cfhns\" (UID: \"b009df2d-e263-42b8-a652-e96fda4bb254\") " pod="e2e-models-e2e-worker-w4-85732c/distinct-w4-85732c-kserve-654fb7bb89-cfhns"
Sep 09 04:15:52.852625 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:52.852543    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b009df2d-e263-42b8-a652-e96fda4bb254-tls-certs\") pod \"distinct-w4-85732c-kserve-654fb7bb89-cfhns\" (UID: \"b009df2d-e263-42b8-a652-e96fda4bb254\") " pod="e2e-models-e2e-worker-w4-85732c/distinct-w4-85732c-kserve-654fb7bb89-cfhns"
Sep 09 04:15:52.953240 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:52.953209    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzmpw\" (UniqueName: \"kubernetes.io/projected/b009df2d-e263-42b8-a652-e96fda4bb254-kube-api-access-xzmpw\") pod \"distinct-w4-85732c-kserve-654fb7bb89-cfhns\" (UID: \"b009df2d-e263-42b8-a652-e96fda4bb254\") " pod="e2e-models-e2e-worker-w4-85732c/distinct-w4-85732c-kserve-654fb7bb89-cfhns"
Sep 09 04:15:52.953398 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:52.953249    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b009df2d-e263-42b8-a652-e96fda4bb254-model-cache\") pod \"distinct-w4-85732c-kserve-654fb7bb89-cfhns\" (UID: \"b009df2d-e263-42b8-a652-e96fda4bb254\") " pod="e2e-models-e2e-worker-w4-85732c/distinct-w4-85732c-kserve-654fb7bb89-cfhns"
Sep 09 04:15:52.953398 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:52.953266    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b009df2d-e263-42b8-a652-e96fda4bb254-tls-certs\") pod \"distinct-w4-85732c-kserve-654fb7bb89-cfhns\" (UID: \"b009df2d-e263-42b8-a652-e96fda4bb254\") " pod="e2e-models-e2e-worker-w4-85732c/distinct-w4-85732c-kserve-654fb7bb89-cfhns"
Sep 09 04:15:52.953398 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:52.953327    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b009df2d-e263-42b8-a652-e96fda4bb254-dshm\") pod \"distinct-w4-85732c-kserve-654fb7bb89-cfhns\" (UID: \"b009df2d-e263-42b8-a652-e96fda4bb254\") " pod="e2e-models-e2e-worker-w4-85732c/distinct-w4-85732c-kserve-654fb7bb89-cfhns"
Sep 09 04:15:52.953398 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:52.953359    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b009df2d-e263-42b8-a652-e96fda4bb254-home\") pod \"distinct-w4-85732c-kserve-654fb7bb89-cfhns\" (UID: \"b009df2d-e263-42b8-a652-e96fda4bb254\") " pod="e2e-models-e2e-worker-w4-85732c/distinct-w4-85732c-kserve-654fb7bb89-cfhns"
Sep 09 04:15:52.953398 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:52.953381    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b009df2d-e263-42b8-a652-e96fda4bb254-tmp-dir\") pod \"distinct-w4-85732c-kserve-654fb7bb89-cfhns\" (UID: \"b009df2d-e263-42b8-a652-e96fda4bb254\") " pod="e2e-models-e2e-worker-w4-85732c/distinct-w4-85732c-kserve-654fb7bb89-cfhns"
Sep 09 04:15:52.953691 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:52.953658    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b009df2d-e263-42b8-a652-e96fda4bb254-model-cache\") pod \"distinct-w4-85732c-kserve-654fb7bb89-cfhns\" (UID: \"b009df2d-e263-42b8-a652-e96fda4bb254\") " pod="e2e-models-e2e-worker-w4-85732c/distinct-w4-85732c-kserve-654fb7bb89-cfhns"
Sep 09 04:15:52.953827 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:52.953676    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b009df2d-e263-42b8-a652-e96fda4bb254-home\") pod \"distinct-w4-85732c-kserve-654fb7bb89-cfhns\" (UID: \"b009df2d-e263-42b8-a652-e96fda4bb254\") " pod="e2e-models-e2e-worker-w4-85732c/distinct-w4-85732c-kserve-654fb7bb89-cfhns"
Sep 09 04:15:52.953827 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:52.953750    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b009df2d-e263-42b8-a652-e96fda4bb254-tmp-dir\") pod \"distinct-w4-85732c-kserve-654fb7bb89-cfhns\" (UID: \"b009df2d-e263-42b8-a652-e96fda4bb254\") " pod="e2e-models-e2e-worker-w4-85732c/distinct-w4-85732c-kserve-654fb7bb89-cfhns"
Sep 09 04:15:52.955576 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:52.955559    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b009df2d-e263-42b8-a652-e96fda4bb254-dshm\") pod \"distinct-w4-85732c-kserve-654fb7bb89-cfhns\" (UID: \"b009df2d-e263-42b8-a652-e96fda4bb254\") " pod="e2e-models-e2e-worker-w4-85732c/distinct-w4-85732c-kserve-654fb7bb89-cfhns"
Sep 09 04:15:52.955960 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:52.955939    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b009df2d-e263-42b8-a652-e96fda4bb254-tls-certs\") pod \"distinct-w4-85732c-kserve-654fb7bb89-cfhns\" (UID: \"b009df2d-e263-42b8-a652-e96fda4bb254\") " pod="e2e-models-e2e-worker-w4-85732c/distinct-w4-85732c-kserve-654fb7bb89-cfhns"
Sep 09 04:15:52.961723 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:52.961704    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzmpw\" (UniqueName: \"kubernetes.io/projected/b009df2d-e263-42b8-a652-e96fda4bb254-kube-api-access-xzmpw\") pod \"distinct-w4-85732c-kserve-654fb7bb89-cfhns\" (UID: \"b009df2d-e263-42b8-a652-e96fda4bb254\") " pod="e2e-models-e2e-worker-w4-85732c/distinct-w4-85732c-kserve-654fb7bb89-cfhns"
Sep 09 04:15:53.078788 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:53.078694    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-85732c/distinct-w4-85732c-kserve-654fb7bb89-cfhns"
Sep 09 04:15:53.352394 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:53.352370    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-85732c/distinct-w4-85732c-kserve-654fb7bb89-cfhns"]
Sep 09 04:15:53.354741 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:15:53.354712    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb009df2d_e263_42b8_a652_e96fda4bb254.slice/crio-2220b042c0ded2022fffe34d401049894eaa926a08523f1e37f365d00bcd706a WatchSource:0}: Error finding container 2220b042c0ded2022fffe34d401049894eaa926a08523f1e37f365d00bcd706a: Status 404 returned error can't find the container with id 2220b042c0ded2022fffe34d401049894eaa926a08523f1e37f365d00bcd706a
Sep 09 04:15:53.666962 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:53.666928    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-85732c/distinct-w4-85732c-kserve-654fb7bb89-cfhns" event={"ID":"b009df2d-e263-42b8-a652-e96fda4bb254","Type":"ContainerStarted","Data":"cef1ca00b1fdcab3551bf17124e54520f7f2afbf9771cb9d2cb9ebe559aaa479"}
Sep 09 04:15:53.666962 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:53.666969    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-85732c/distinct-w4-85732c-kserve-654fb7bb89-cfhns" event={"ID":"b009df2d-e263-42b8-a652-e96fda4bb254","Type":"ContainerStarted","Data":"2220b042c0ded2022fffe34d401049894eaa926a08523f1e37f365d00bcd706a"}
Sep 09 04:15:53.685173 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:15:53.685126    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-85732c/distinct-w4-85732c-kserve-654fb7bb89-cfhns" podStartSLOduration=1.685097206 podStartE2EDuration="1.685097206s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:15:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:15:53.682930715 +0000 UTC m=+1078.602101623" watchObservedRunningTime="2026-09-09 04:15:53.685097206 +0000 UTC m=+1078.604268116"
Sep 09 04:16:03.079179 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:03.079141    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-85732c/distinct-w4-85732c-kserve-654fb7bb89-cfhns"
Sep 09 04:16:03.079179 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:03.079187    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-85732c/distinct-w4-85732c-kserve-654fb7bb89-cfhns"
Sep 09 04:16:03.095885 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:03.095858    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-85732c/distinct-w4-85732c-kserve-654fb7bb89-cfhns"
Sep 09 04:16:03.729970 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:03.729944    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-85732c/distinct-w4-85732c-kserve-654fb7bb89-cfhns"
Sep 09 04:16:06.673861 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:06.673826    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-b80654-57d9964785-tz4mc"]
Sep 09 04:16:06.674312 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:06.674084    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b80654-57d9964785-tz4mc" podUID="cfdebe0a-b4c5-4cd5-abd4-5c79a622f935" containerName="maas-api" containerID="cri-o://78c690533fdb72b37210919e09981708f701137516f2feeefc6dcabd5431a872" gracePeriod=30
Sep 09 04:16:06.932181 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:06.932106    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b80654-57d9964785-tz4mc"
Sep 09 04:16:07.075292 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:07.075257    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfdebe0a-b4c5-4cd5-abd4-5c79a622f935-openshift-service-ca\") pod \"cfdebe0a-b4c5-4cd5-abd4-5c79a622f935\" (UID: \"cfdebe0a-b4c5-4cd5-abd4-5c79a622f935\") "
Sep 09 04:16:07.075292 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:07.075300    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfdebe0a-b4c5-4cd5-abd4-5c79a622f935-metrics-certs\") pod \"cfdebe0a-b4c5-4cd5-abd4-5c79a622f935\" (UID: \"cfdebe0a-b4c5-4cd5-abd4-5c79a622f935\") "
Sep 09 04:16:07.075527 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:07.075316    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfdebe0a-b4c5-4cd5-abd4-5c79a622f935-maas-api-tls\") pod \"cfdebe0a-b4c5-4cd5-abd4-5c79a622f935\" (UID: \"cfdebe0a-b4c5-4cd5-abd4-5c79a622f935\") "
Sep 09 04:16:07.075527 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:07.075335    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-246qz\" (UniqueName: \"kubernetes.io/projected/cfdebe0a-b4c5-4cd5-abd4-5c79a622f935-kube-api-access-246qz\") pod \"cfdebe0a-b4c5-4cd5-abd4-5c79a622f935\" (UID: \"cfdebe0a-b4c5-4cd5-abd4-5c79a622f935\") "
Sep 09 04:16:07.075763 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:07.075728    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfdebe0a-b4c5-4cd5-abd4-5c79a622f935-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cfdebe0a-b4c5-4cd5-abd4-5c79a622f935" (UID: "cfdebe0a-b4c5-4cd5-abd4-5c79a622f935"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 04:16:07.077871 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:07.077835    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfdebe0a-b4c5-4cd5-abd4-5c79a622f935-kube-api-access-246qz" (OuterVolumeSpecName: "kube-api-access-246qz") pod "cfdebe0a-b4c5-4cd5-abd4-5c79a622f935" (UID: "cfdebe0a-b4c5-4cd5-abd4-5c79a622f935"). InnerVolumeSpecName "kube-api-access-246qz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 04:16:07.078082 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:07.078053    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfdebe0a-b4c5-4cd5-abd4-5c79a622f935-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cfdebe0a-b4c5-4cd5-abd4-5c79a622f935" (UID: "cfdebe0a-b4c5-4cd5-abd4-5c79a622f935"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 04:16:07.078082 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:07.078070    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfdebe0a-b4c5-4cd5-abd4-5c79a622f935-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "cfdebe0a-b4c5-4cd5-abd4-5c79a622f935" (UID: "cfdebe0a-b4c5-4cd5-abd4-5c79a622f935"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 04:16:07.176480 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:07.176455    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfdebe0a-b4c5-4cd5-abd4-5c79a622f935-openshift-service-ca\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:16:07.176480 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:07.176479    2557 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfdebe0a-b4c5-4cd5-abd4-5c79a622f935-metrics-certs\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:16:07.176612 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:07.176489    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfdebe0a-b4c5-4cd5-abd4-5c79a622f935-maas-api-tls\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:16:07.176612 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:07.176498    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-246qz\" (UniqueName: \"kubernetes.io/projected/cfdebe0a-b4c5-4cd5-abd4-5c79a622f935-kube-api-access-246qz\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:16:07.742966 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:07.742927    2557 generic.go:356] "Generic (PLEG): container finished" podID="cfdebe0a-b4c5-4cd5-abd4-5c79a622f935" containerID="78c690533fdb72b37210919e09981708f701137516f2feeefc6dcabd5431a872" exitCode=0
Sep 09 04:16:07.743361 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:07.742975    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b80654-57d9964785-tz4mc"
Sep 09 04:16:07.743361 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:07.743011    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b80654-57d9964785-tz4mc" event={"ID":"cfdebe0a-b4c5-4cd5-abd4-5c79a622f935","Type":"ContainerDied","Data":"78c690533fdb72b37210919e09981708f701137516f2feeefc6dcabd5431a872"}
Sep 09 04:16:07.743361 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:07.743065    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b80654-57d9964785-tz4mc" event={"ID":"cfdebe0a-b4c5-4cd5-abd4-5c79a622f935","Type":"ContainerDied","Data":"3bfb5facfaa5b973930571e6f2c07eadc5b664c759bc7e0a1aa8bd96be8d0f92"}
Sep 09 04:16:07.743361 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:07.743087    2557 scope.go:117] "RemoveContainer" containerID="78c690533fdb72b37210919e09981708f701137516f2feeefc6dcabd5431a872"
Sep 09 04:16:07.751675 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:07.751657    2557 scope.go:117] "RemoveContainer" containerID="78c690533fdb72b37210919e09981708f701137516f2feeefc6dcabd5431a872"
Sep 09 04:16:07.751930 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:16:07.751915    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78c690533fdb72b37210919e09981708f701137516f2feeefc6dcabd5431a872\": container with ID starting with 78c690533fdb72b37210919e09981708f701137516f2feeefc6dcabd5431a872 not found: ID does not exist" containerID="78c690533fdb72b37210919e09981708f701137516f2feeefc6dcabd5431a872"
Sep 09 04:16:07.751987 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:07.751938    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78c690533fdb72b37210919e09981708f701137516f2feeefc6dcabd5431a872"} err="failed to get container status \"78c690533fdb72b37210919e09981708f701137516f2feeefc6dcabd5431a872\": rpc error: code = NotFound desc = could not find container \"78c690533fdb72b37210919e09981708f701137516f2feeefc6dcabd5431a872\": container with ID starting with 78c690533fdb72b37210919e09981708f701137516f2feeefc6dcabd5431a872 not found: ID does not exist"
Sep 09 04:16:07.759554 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:07.759530    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-b80654-57d9964785-tz4mc"]
Sep 09 04:16:07.764207 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:07.764186    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-b80654-57d9964785-tz4mc"]
Sep 09 04:16:09.483550 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:09.483517    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-46ddac-77d99dd7cc-lt9mh"]
Sep 09 04:16:09.483952 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:09.483932    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfdebe0a-b4c5-4cd5-abd4-5c79a622f935" containerName="maas-api"
Sep 09 04:16:09.483952 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:09.483944    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfdebe0a-b4c5-4cd5-abd4-5c79a622f935" containerName="maas-api"
Sep 09 04:16:09.484072 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:09.484017    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfdebe0a-b4c5-4cd5-abd4-5c79a622f935" containerName="maas-api"
Sep 09 04:16:09.486771 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:09.486737    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-46ddac-77d99dd7cc-lt9mh"
Sep 09 04:16:09.489063 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:09.489044    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-46ddac-dockercfg-2z9nl\""
Sep 09 04:16:09.489188 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:09.489166    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-46ddac\""
Sep 09 04:16:09.496657 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:09.496633    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-46ddac-77d99dd7cc-lt9mh"]
Sep 09 04:16:09.600389 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:09.600356    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfhz8\" (UniqueName: \"kubernetes.io/projected/b655b0f5-4908-4030-9a69-a4f5aaa83d0d-kube-api-access-nfhz8\") pod \"payload-processing-e2e-shared-a-w2-46ddac-77d99dd7cc-lt9mh\" (UID: \"b655b0f5-4908-4030-9a69-a4f5aaa83d0d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-46ddac-77d99dd7cc-lt9mh"
Sep 09 04:16:09.600555 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:09.600421    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b655b0f5-4908-4030-9a69-a4f5aaa83d0d-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-46ddac-77d99dd7cc-lt9mh\" (UID: \"b655b0f5-4908-4030-9a69-a4f5aaa83d0d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-46ddac-77d99dd7cc-lt9mh"
Sep 09 04:16:09.620324 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:09.620296    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfdebe0a-b4c5-4cd5-abd4-5c79a622f935" path="/var/lib/kubelet/pods/cfdebe0a-b4c5-4cd5-abd4-5c79a622f935/volumes"
Sep 09 04:16:09.701396 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:09.701358    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfhz8\" (UniqueName: \"kubernetes.io/projected/b655b0f5-4908-4030-9a69-a4f5aaa83d0d-kube-api-access-nfhz8\") pod \"payload-processing-e2e-shared-a-w2-46ddac-77d99dd7cc-lt9mh\" (UID: \"b655b0f5-4908-4030-9a69-a4f5aaa83d0d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-46ddac-77d99dd7cc-lt9mh"
Sep 09 04:16:09.701576 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:09.701472    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b655b0f5-4908-4030-9a69-a4f5aaa83d0d-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-46ddac-77d99dd7cc-lt9mh\" (UID: \"b655b0f5-4908-4030-9a69-a4f5aaa83d0d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-46ddac-77d99dd7cc-lt9mh"
Sep 09 04:16:09.702156 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:09.702135    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b655b0f5-4908-4030-9a69-a4f5aaa83d0d-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-46ddac-77d99dd7cc-lt9mh\" (UID: \"b655b0f5-4908-4030-9a69-a4f5aaa83d0d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-46ddac-77d99dd7cc-lt9mh"
Sep 09 04:16:09.710042 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:09.710020    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfhz8\" (UniqueName: \"kubernetes.io/projected/b655b0f5-4908-4030-9a69-a4f5aaa83d0d-kube-api-access-nfhz8\") pod \"payload-processing-e2e-shared-a-w2-46ddac-77d99dd7cc-lt9mh\" (UID: \"b655b0f5-4908-4030-9a69-a4f5aaa83d0d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-46ddac-77d99dd7cc-lt9mh"
Sep 09 04:16:09.799364 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:09.799296    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-46ddac-77d99dd7cc-lt9mh"
Sep 09 04:16:10.068019 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:10.067997    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-46ddac-77d99dd7cc-lt9mh"]
Sep 09 04:16:10.070227 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:16:10.070199    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb655b0f5_4908_4030_9a69_a4f5aaa83d0d.slice/crio-efeca21438dce3d28e6881ee0aaa848b9a93699ba87f21909045520a56e7b53a WatchSource:0}: Error finding container efeca21438dce3d28e6881ee0aaa848b9a93699ba87f21909045520a56e7b53a: Status 404 returned error can't find the container with id efeca21438dce3d28e6881ee0aaa848b9a93699ba87f21909045520a56e7b53a
Sep 09 04:16:10.759094 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:10.759054    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-46ddac-77d99dd7cc-lt9mh" event={"ID":"b655b0f5-4908-4030-9a69-a4f5aaa83d0d","Type":"ContainerStarted","Data":"dfc06199d84f4aeb6f9399f4c9651b38bb110f59ddbd9e49089a944eb4d3f5ea"}
Sep 09 04:16:10.759485 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:10.759131    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-46ddac-77d99dd7cc-lt9mh" event={"ID":"b655b0f5-4908-4030-9a69-a4f5aaa83d0d","Type":"ContainerStarted","Data":"efeca21438dce3d28e6881ee0aaa848b9a93699ba87f21909045520a56e7b53a"}
Sep 09 04:16:10.759485 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:10.759158    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-46ddac-77d99dd7cc-lt9mh"
Sep 09 04:16:10.776126 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:10.776067    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-46ddac-77d99dd7cc-lt9mh" podStartSLOduration=1.776051735 podStartE2EDuration="1.776051735s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:16:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:16:10.774560835 +0000 UTC m=+1095.693731769" watchObservedRunningTime="2026-09-09 04:16:10.776051735 +0000 UTC m=+1095.695222645"
Sep 09 04:16:21.766308 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:21.766282    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-46ddac-77d99dd7cc-lt9mh"
Sep 09 04:16:37.263868 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:37.263831    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf"]
Sep 09 04:16:37.267021 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:37.266998    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf"
Sep 09 04:16:37.269562 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:37.269531    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-9f2be5-serving-cert\""
Sep 09 04:16:37.278595 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:37.278569    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf"]
Sep 09 04:16:37.329600 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:37.329572    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce66729c-0dda-43cb-9365-98bc06b1e6da-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf\" (UID: \"ce66729c-0dda-43cb-9365-98bc06b1e6da\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf"
Sep 09 04:16:37.329761 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:37.329612    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbkf8\" (UniqueName: \"kubernetes.io/projected/ce66729c-0dda-43cb-9365-98bc06b1e6da-kube-api-access-pbkf8\") pod \"maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf\" (UID: \"ce66729c-0dda-43cb-9365-98bc06b1e6da\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf"
Sep 09 04:16:37.329761 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:37.329722    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce66729c-0dda-43cb-9365-98bc06b1e6da-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf\" (UID: \"ce66729c-0dda-43cb-9365-98bc06b1e6da\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf"
Sep 09 04:16:37.329761 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:37.329757    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce66729c-0dda-43cb-9365-98bc06b1e6da-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf\" (UID: \"ce66729c-0dda-43cb-9365-98bc06b1e6da\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf"
Sep 09 04:16:37.431067 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:37.431030    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce66729c-0dda-43cb-9365-98bc06b1e6da-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf\" (UID: \"ce66729c-0dda-43cb-9365-98bc06b1e6da\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf"
Sep 09 04:16:37.431249 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:37.431075    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce66729c-0dda-43cb-9365-98bc06b1e6da-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf\" (UID: \"ce66729c-0dda-43cb-9365-98bc06b1e6da\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf"
Sep 09 04:16:37.431249 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:37.431136    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce66729c-0dda-43cb-9365-98bc06b1e6da-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf\" (UID: \"ce66729c-0dda-43cb-9365-98bc06b1e6da\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf"
Sep 09 04:16:37.431249 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:37.431163    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbkf8\" (UniqueName: \"kubernetes.io/projected/ce66729c-0dda-43cb-9365-98bc06b1e6da-kube-api-access-pbkf8\") pod \"maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf\" (UID: \"ce66729c-0dda-43cb-9365-98bc06b1e6da\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf"
Sep 09 04:16:37.431249 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:16:37.431224    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f2be5-serving-cert: secret "maas-api-e2e-shared-b-w2-9f2be5-serving-cert" not found
Sep 09 04:16:37.431401 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:16:37.431289    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce66729c-0dda-43cb-9365-98bc06b1e6da-maas-api-tls podName:ce66729c-0dda-43cb-9365-98bc06b1e6da nodeName:}" failed. No retries permitted until 2026-09-09 04:16:37.931273086 +0000 UTC m=+1122.850443973 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ce66729c-0dda-43cb-9365-98bc06b1e6da-maas-api-tls") pod "maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf" (UID: "ce66729c-0dda-43cb-9365-98bc06b1e6da") : secret "maas-api-e2e-shared-b-w2-9f2be5-serving-cert" not found
Sep 09 04:16:37.431796 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:37.431778    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce66729c-0dda-43cb-9365-98bc06b1e6da-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf\" (UID: \"ce66729c-0dda-43cb-9365-98bc06b1e6da\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf"
Sep 09 04:16:37.433594 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:37.433579    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce66729c-0dda-43cb-9365-98bc06b1e6da-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf\" (UID: \"ce66729c-0dda-43cb-9365-98bc06b1e6da\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf"
Sep 09 04:16:37.441500 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:37.441480    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbkf8\" (UniqueName: \"kubernetes.io/projected/ce66729c-0dda-43cb-9365-98bc06b1e6da-kube-api-access-pbkf8\") pod \"maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf\" (UID: \"ce66729c-0dda-43cb-9365-98bc06b1e6da\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf"
Sep 09 04:16:37.936192 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:37.936159    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce66729c-0dda-43cb-9365-98bc06b1e6da-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf\" (UID: \"ce66729c-0dda-43cb-9365-98bc06b1e6da\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf"
Sep 09 04:16:37.938491 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:37.938463    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce66729c-0dda-43cb-9365-98bc06b1e6da-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf\" (UID: \"ce66729c-0dda-43cb-9365-98bc06b1e6da\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf"
Sep 09 04:16:38.018696 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:38.018657    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f2be5-7bf7bf47b6-wpzh4"]
Sep 09 04:16:38.022312 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:38.022296    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f2be5-7bf7bf47b6-wpzh4"
Sep 09 04:16:38.024768 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:38.024746    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-9f2be5\""
Sep 09 04:16:38.024984 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:38.024971    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-9f2be5-dockercfg-zs8qq\""
Sep 09 04:16:38.037163 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:38.032833    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f2be5-7bf7bf47b6-wpzh4"]
Sep 09 04:16:38.039682 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:38.039650    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03fc36b8-e935-438e-a6d8-b7ffe4c07a5f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-9f2be5-7bf7bf47b6-wpzh4\" (UID: \"03fc36b8-e935-438e-a6d8-b7ffe4c07a5f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f2be5-7bf7bf47b6-wpzh4"
Sep 09 04:16:38.039785 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:38.039740    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bxhs\" (UniqueName: \"kubernetes.io/projected/03fc36b8-e935-438e-a6d8-b7ffe4c07a5f-kube-api-access-2bxhs\") pod \"payload-pre-processing-e2e-shared-b-w2-9f2be5-7bf7bf47b6-wpzh4\" (UID: \"03fc36b8-e935-438e-a6d8-b7ffe4c07a5f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f2be5-7bf7bf47b6-wpzh4"
Sep 09 04:16:38.140355 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:38.140320    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bxhs\" (UniqueName: \"kubernetes.io/projected/03fc36b8-e935-438e-a6d8-b7ffe4c07a5f-kube-api-access-2bxhs\") pod \"payload-pre-processing-e2e-shared-b-w2-9f2be5-7bf7bf47b6-wpzh4\" (UID: \"03fc36b8-e935-438e-a6d8-b7ffe4c07a5f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f2be5-7bf7bf47b6-wpzh4"
Sep 09 04:16:38.140525 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:38.140406    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03fc36b8-e935-438e-a6d8-b7ffe4c07a5f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-9f2be5-7bf7bf47b6-wpzh4\" (UID: \"03fc36b8-e935-438e-a6d8-b7ffe4c07a5f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f2be5-7bf7bf47b6-wpzh4"
Sep 09 04:16:38.141017 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:38.140997    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03fc36b8-e935-438e-a6d8-b7ffe4c07a5f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-9f2be5-7bf7bf47b6-wpzh4\" (UID: \"03fc36b8-e935-438e-a6d8-b7ffe4c07a5f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f2be5-7bf7bf47b6-wpzh4"
Sep 09 04:16:38.151976 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:38.151953    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bxhs\" (UniqueName: \"kubernetes.io/projected/03fc36b8-e935-438e-a6d8-b7ffe4c07a5f-kube-api-access-2bxhs\") pod \"payload-pre-processing-e2e-shared-b-w2-9f2be5-7bf7bf47b6-wpzh4\" (UID: \"03fc36b8-e935-438e-a6d8-b7ffe4c07a5f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f2be5-7bf7bf47b6-wpzh4"
Sep 09 04:16:38.179492 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:38.179456    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf"
Sep 09 04:16:38.334725 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:38.334691    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f2be5-7bf7bf47b6-wpzh4"
Sep 09 04:16:38.454952 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:38.454929    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf"]
Sep 09 04:16:38.457063 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:16:38.457033    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce66729c_0dda_43cb_9365_98bc06b1e6da.slice/crio-ee0047b7124dea8d2aebd838ceebca4247028b7f793a34337266073875509af5 WatchSource:0}: Error finding container ee0047b7124dea8d2aebd838ceebca4247028b7f793a34337266073875509af5: Status 404 returned error can't find the container with id ee0047b7124dea8d2aebd838ceebca4247028b7f793a34337266073875509af5
Sep 09 04:16:38.458386 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:38.458365    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 04:16:38.607148 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:38.607073    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f2be5-7bf7bf47b6-wpzh4"]
Sep 09 04:16:38.609263 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:16:38.609236    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03fc36b8_e935_438e_a6d8_b7ffe4c07a5f.slice/crio-09691ab4719a7ee8ec48d935e3ba5f3e14418e1d7e972159cc4f677556c1f8c0 WatchSource:0}: Error finding container 09691ab4719a7ee8ec48d935e3ba5f3e14418e1d7e972159cc4f677556c1f8c0: Status 404 returned error can't find the container with id 09691ab4719a7ee8ec48d935e3ba5f3e14418e1d7e972159cc4f677556c1f8c0
Sep 09 04:16:38.899846 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:38.899811    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf" event={"ID":"ce66729c-0dda-43cb-9365-98bc06b1e6da","Type":"ContainerStarted","Data":"ee0047b7124dea8d2aebd838ceebca4247028b7f793a34337266073875509af5"}
Sep 09 04:16:38.901392 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:38.901363    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f2be5-7bf7bf47b6-wpzh4" event={"ID":"03fc36b8-e935-438e-a6d8-b7ffe4c07a5f","Type":"ContainerStarted","Data":"382c4a7aaf6ca6a5a4da6235307dac61173fe62b1584c72e7fd00a436900bbd0"}
Sep 09 04:16:38.901525 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:38.901411    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f2be5-7bf7bf47b6-wpzh4" event={"ID":"03fc36b8-e935-438e-a6d8-b7ffe4c07a5f","Type":"ContainerStarted","Data":"09691ab4719a7ee8ec48d935e3ba5f3e14418e1d7e972159cc4f677556c1f8c0"}
Sep 09 04:16:38.901525 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:38.901496    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f2be5-7bf7bf47b6-wpzh4"
Sep 09 04:16:38.921393 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:38.921350    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f2be5-7bf7bf47b6-wpzh4" podStartSLOduration=0.921336171 podStartE2EDuration="921.336171ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:16:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:16:38.919633388 +0000 UTC m=+1123.838804346" watchObservedRunningTime="2026-09-09 04:16:38.921336171 +0000 UTC m=+1123.840507144"
Sep 09 04:16:39.907364 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:39.907324    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf" event={"ID":"ce66729c-0dda-43cb-9365-98bc06b1e6da","Type":"ContainerStarted","Data":"8e4686487bb368a84a521ef54e8a1b02c4197a08c65e1537d165603aeafb30fb"}
Sep 09 04:16:39.907778 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:39.907603    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf"
Sep 09 04:16:39.924489 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:39.924359    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf" podStartSLOduration=2.091640982 podStartE2EDuration="2.924340174s" totalImagesPullingTime="832.699192ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:16:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:16:39.923759027 +0000 UTC m=+1124.842929936" watchObservedRunningTime="2026-09-09 04:16:39.924340174 +0000 UTC m=+1124.843511085"
Sep 09 04:16:45.917394 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:45.917356    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf"
Sep 09 04:16:49.910958 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:16:49.910920    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f2be5-7bf7bf47b6-wpzh4"
Sep 09 04:17:23.188441 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:23.188359    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-429d71ed-7fd46c9dcc-fwtqg"]
Sep 09 04:17:23.193471 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:23.193450    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-429d71ed-7fd46c9dcc-fwtqg"
Sep 09 04:17:23.200916 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:23.200886    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-429d71ed\""
Sep 09 04:17:23.206457 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:23.206430    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-429d71ed-7fd46c9dcc-fwtqg"]
Sep 09 04:17:23.242967 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:23.242939    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a08b0bc8-027b-4ac8-8f79-b516cfee3d09-plugins-config-volume\") pod \"payload-processing-e2e-mt-429d71ed-7fd46c9dcc-fwtqg\" (UID: \"a08b0bc8-027b-4ac8-8f79-b516cfee3d09\") " pod="openshift-ingress/payload-processing-e2e-mt-429d71ed-7fd46c9dcc-fwtqg"
Sep 09 04:17:23.242967 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:23.242970    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jcvg\" (UniqueName: \"kubernetes.io/projected/a08b0bc8-027b-4ac8-8f79-b516cfee3d09-kube-api-access-6jcvg\") pod \"payload-processing-e2e-mt-429d71ed-7fd46c9dcc-fwtqg\" (UID: \"a08b0bc8-027b-4ac8-8f79-b516cfee3d09\") " pod="openshift-ingress/payload-processing-e2e-mt-429d71ed-7fd46c9dcc-fwtqg"
Sep 09 04:17:23.344079 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:23.344027    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a08b0bc8-027b-4ac8-8f79-b516cfee3d09-plugins-config-volume\") pod \"payload-processing-e2e-mt-429d71ed-7fd46c9dcc-fwtqg\" (UID: \"a08b0bc8-027b-4ac8-8f79-b516cfee3d09\") " pod="openshift-ingress/payload-processing-e2e-mt-429d71ed-7fd46c9dcc-fwtqg"
Sep 09 04:17:23.344079 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:23.344062    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jcvg\" (UniqueName: \"kubernetes.io/projected/a08b0bc8-027b-4ac8-8f79-b516cfee3d09-kube-api-access-6jcvg\") pod \"payload-processing-e2e-mt-429d71ed-7fd46c9dcc-fwtqg\" (UID: \"a08b0bc8-027b-4ac8-8f79-b516cfee3d09\") " pod="openshift-ingress/payload-processing-e2e-mt-429d71ed-7fd46c9dcc-fwtqg"
Sep 09 04:17:23.344830 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:23.344809    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a08b0bc8-027b-4ac8-8f79-b516cfee3d09-plugins-config-volume\") pod \"payload-processing-e2e-mt-429d71ed-7fd46c9dcc-fwtqg\" (UID: \"a08b0bc8-027b-4ac8-8f79-b516cfee3d09\") " pod="openshift-ingress/payload-processing-e2e-mt-429d71ed-7fd46c9dcc-fwtqg"
Sep 09 04:17:23.357467 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:23.357442    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jcvg\" (UniqueName: \"kubernetes.io/projected/a08b0bc8-027b-4ac8-8f79-b516cfee3d09-kube-api-access-6jcvg\") pod \"payload-processing-e2e-mt-429d71ed-7fd46c9dcc-fwtqg\" (UID: \"a08b0bc8-027b-4ac8-8f79-b516cfee3d09\") " pod="openshift-ingress/payload-processing-e2e-mt-429d71ed-7fd46c9dcc-fwtqg"
Sep 09 04:17:23.512606 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:23.512512    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-429d71ed-7fd46c9dcc-fwtqg"
Sep 09 04:17:23.788578 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:23.788552    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-429d71ed-7fd46c9dcc-fwtqg"]
Sep 09 04:17:23.790138 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:17:23.790090    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda08b0bc8_027b_4ac8_8f79_b516cfee3d09.slice/crio-2ee4196544e6003fccaa76f5dec073e667cce7c4958e40f500dac9dc3cf4af1c WatchSource:0}: Error finding container 2ee4196544e6003fccaa76f5dec073e667cce7c4958e40f500dac9dc3cf4af1c: Status 404 returned error can't find the container with id 2ee4196544e6003fccaa76f5dec073e667cce7c4958e40f500dac9dc3cf4af1c
Sep 09 04:17:24.134699 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:24.134667    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-429d71ed-7fd46c9dcc-fwtqg" event={"ID":"a08b0bc8-027b-4ac8-8f79-b516cfee3d09","Type":"ContainerStarted","Data":"a28aadfa5879abe4c1c73b0bb34aa51c1467caecf55f50a71fc258a5e8aeecb9"}
Sep 09 04:17:24.134871 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:24.134720    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-429d71ed-7fd46c9dcc-fwtqg" event={"ID":"a08b0bc8-027b-4ac8-8f79-b516cfee3d09","Type":"ContainerStarted","Data":"2ee4196544e6003fccaa76f5dec073e667cce7c4958e40f500dac9dc3cf4af1c"}
Sep 09 04:17:24.134871 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:24.134743    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-429d71ed-7fd46c9dcc-fwtqg"
Sep 09 04:17:24.164425 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:24.164378    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-429d71ed-7fd46c9dcc-fwtqg" podStartSLOduration=1.16436186 podStartE2EDuration="1.16436186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:17:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:17:24.162976823 +0000 UTC m=+1169.082147736" watchObservedRunningTime="2026-09-09 04:17:24.16436186 +0000 UTC m=+1169.083532770"
Sep 09 04:17:34.946684 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:34.946647    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-85732c/distinct-2-w4-85732c-kserve-7f96669d6b-764xk"]
Sep 09 04:17:34.950439 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:34.950412    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-85732c/distinct-2-w4-85732c-kserve-7f96669d6b-764xk"
Sep 09 04:17:34.952934 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:34.952912    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-85732c\"/\"distinct-2-w4-85732c-kserve-self-signed-certs\""
Sep 09 04:17:34.961611 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:34.961581    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-85732c/distinct-2-w4-85732c-kserve-7f96669d6b-764xk"]
Sep 09 04:17:35.050532 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:35.050491    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19f04fc8-0e98-46b9-a42f-cd57aa63b0ac-model-cache\") pod \"distinct-2-w4-85732c-kserve-7f96669d6b-764xk\" (UID: \"19f04fc8-0e98-46b9-a42f-cd57aa63b0ac\") " pod="e2e-models-e2e-worker-w4-85732c/distinct-2-w4-85732c-kserve-7f96669d6b-764xk"
Sep 09 04:17:35.050713 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:35.050547    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19f04fc8-0e98-46b9-a42f-cd57aa63b0ac-home\") pod \"distinct-2-w4-85732c-kserve-7f96669d6b-764xk\" (UID: \"19f04fc8-0e98-46b9-a42f-cd57aa63b0ac\") " pod="e2e-models-e2e-worker-w4-85732c/distinct-2-w4-85732c-kserve-7f96669d6b-764xk"
Sep 09 04:17:35.050713 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:35.050573    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19f04fc8-0e98-46b9-a42f-cd57aa63b0ac-tmp-dir\") pod \"distinct-2-w4-85732c-kserve-7f96669d6b-764xk\" (UID: \"19f04fc8-0e98-46b9-a42f-cd57aa63b0ac\") " pod="e2e-models-e2e-worker-w4-85732c/distinct-2-w4-85732c-kserve-7f96669d6b-764xk"
Sep 09 04:17:35.050713 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:35.050622    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19f04fc8-0e98-46b9-a42f-cd57aa63b0ac-dshm\") pod \"distinct-2-w4-85732c-kserve-7f96669d6b-764xk\" (UID: \"19f04fc8-0e98-46b9-a42f-cd57aa63b0ac\") " pod="e2e-models-e2e-worker-w4-85732c/distinct-2-w4-85732c-kserve-7f96669d6b-764xk"
Sep 09 04:17:35.050713 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:35.050690    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19f04fc8-0e98-46b9-a42f-cd57aa63b0ac-tls-certs\") pod \"distinct-2-w4-85732c-kserve-7f96669d6b-764xk\" (UID: \"19f04fc8-0e98-46b9-a42f-cd57aa63b0ac\") " pod="e2e-models-e2e-worker-w4-85732c/distinct-2-w4-85732c-kserve-7f96669d6b-764xk"
Sep 09 04:17:35.050913 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:35.050741    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bwjv\" (UniqueName: \"kubernetes.io/projected/19f04fc8-0e98-46b9-a42f-cd57aa63b0ac-kube-api-access-4bwjv\") pod \"distinct-2-w4-85732c-kserve-7f96669d6b-764xk\" (UID: \"19f04fc8-0e98-46b9-a42f-cd57aa63b0ac\") " pod="e2e-models-e2e-worker-w4-85732c/distinct-2-w4-85732c-kserve-7f96669d6b-764xk"
Sep 09 04:17:35.142938 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:35.142909    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-429d71ed-7fd46c9dcc-fwtqg"
Sep 09 04:17:35.151669 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:35.151635    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19f04fc8-0e98-46b9-a42f-cd57aa63b0ac-dshm\") pod \"distinct-2-w4-85732c-kserve-7f96669d6b-764xk\" (UID: \"19f04fc8-0e98-46b9-a42f-cd57aa63b0ac\") " pod="e2e-models-e2e-worker-w4-85732c/distinct-2-w4-85732c-kserve-7f96669d6b-764xk"
Sep 09 04:17:35.151825 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:35.151685    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19f04fc8-0e98-46b9-a42f-cd57aa63b0ac-tls-certs\") pod \"distinct-2-w4-85732c-kserve-7f96669d6b-764xk\" (UID: \"19f04fc8-0e98-46b9-a42f-cd57aa63b0ac\") " pod="e2e-models-e2e-worker-w4-85732c/distinct-2-w4-85732c-kserve-7f96669d6b-764xk"
Sep 09 04:17:35.151888 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:35.151857    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bwjv\" (UniqueName: \"kubernetes.io/projected/19f04fc8-0e98-46b9-a42f-cd57aa63b0ac-kube-api-access-4bwjv\") pod \"distinct-2-w4-85732c-kserve-7f96669d6b-764xk\" (UID: \"19f04fc8-0e98-46b9-a42f-cd57aa63b0ac\") " pod="e2e-models-e2e-worker-w4-85732c/distinct-2-w4-85732c-kserve-7f96669d6b-764xk"
Sep 09 04:17:35.151945 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:35.151931    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19f04fc8-0e98-46b9-a42f-cd57aa63b0ac-model-cache\") pod \"distinct-2-w4-85732c-kserve-7f96669d6b-764xk\" (UID: \"19f04fc8-0e98-46b9-a42f-cd57aa63b0ac\") " pod="e2e-models-e2e-worker-w4-85732c/distinct-2-w4-85732c-kserve-7f96669d6b-764xk"
Sep 09 04:17:35.152008 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:35.151967    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19f04fc8-0e98-46b9-a42f-cd57aa63b0ac-home\") pod \"distinct-2-w4-85732c-kserve-7f96669d6b-764xk\" (UID: \"19f04fc8-0e98-46b9-a42f-cd57aa63b0ac\") " pod="e2e-models-e2e-worker-w4-85732c/distinct-2-w4-85732c-kserve-7f96669d6b-764xk"
Sep 09 04:17:35.152008 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:35.151984    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19f04fc8-0e98-46b9-a42f-cd57aa63b0ac-tmp-dir\") pod \"distinct-2-w4-85732c-kserve-7f96669d6b-764xk\" (UID: \"19f04fc8-0e98-46b9-a42f-cd57aa63b0ac\") " pod="e2e-models-e2e-worker-w4-85732c/distinct-2-w4-85732c-kserve-7f96669d6b-764xk"
Sep 09 04:17:35.152299 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:35.152272    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19f04fc8-0e98-46b9-a42f-cd57aa63b0ac-model-cache\") pod \"distinct-2-w4-85732c-kserve-7f96669d6b-764xk\" (UID: \"19f04fc8-0e98-46b9-a42f-cd57aa63b0ac\") " pod="e2e-models-e2e-worker-w4-85732c/distinct-2-w4-85732c-kserve-7f96669d6b-764xk"
Sep 09 04:17:35.152299 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:35.152295    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19f04fc8-0e98-46b9-a42f-cd57aa63b0ac-home\") pod \"distinct-2-w4-85732c-kserve-7f96669d6b-764xk\" (UID: \"19f04fc8-0e98-46b9-a42f-cd57aa63b0ac\") " pod="e2e-models-e2e-worker-w4-85732c/distinct-2-w4-85732c-kserve-7f96669d6b-764xk"
Sep 09 04:17:35.152443 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:35.152342    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19f04fc8-0e98-46b9-a42f-cd57aa63b0ac-tmp-dir\") pod \"distinct-2-w4-85732c-kserve-7f96669d6b-764xk\" (UID: \"19f04fc8-0e98-46b9-a42f-cd57aa63b0ac\") " pod="e2e-models-e2e-worker-w4-85732c/distinct-2-w4-85732c-kserve-7f96669d6b-764xk"
Sep 09 04:17:35.154048 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:35.154024    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19f04fc8-0e98-46b9-a42f-cd57aa63b0ac-dshm\") pod \"distinct-2-w4-85732c-kserve-7f96669d6b-764xk\" (UID: \"19f04fc8-0e98-46b9-a42f-cd57aa63b0ac\") " pod="e2e-models-e2e-worker-w4-85732c/distinct-2-w4-85732c-kserve-7f96669d6b-764xk"
Sep 09 04:17:35.154263 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:35.154248    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19f04fc8-0e98-46b9-a42f-cd57aa63b0ac-tls-certs\") pod \"distinct-2-w4-85732c-kserve-7f96669d6b-764xk\" (UID: \"19f04fc8-0e98-46b9-a42f-cd57aa63b0ac\") " pod="e2e-models-e2e-worker-w4-85732c/distinct-2-w4-85732c-kserve-7f96669d6b-764xk"
Sep 09 04:17:35.164165 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:35.164136    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bwjv\" (UniqueName: \"kubernetes.io/projected/19f04fc8-0e98-46b9-a42f-cd57aa63b0ac-kube-api-access-4bwjv\") pod \"distinct-2-w4-85732c-kserve-7f96669d6b-764xk\" (UID: \"19f04fc8-0e98-46b9-a42f-cd57aa63b0ac\") " pod="e2e-models-e2e-worker-w4-85732c/distinct-2-w4-85732c-kserve-7f96669d6b-764xk"
Sep 09 04:17:35.259881 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:35.259784    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-85732c/distinct-2-w4-85732c-kserve-7f96669d6b-764xk"
Sep 09 04:17:35.544250 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:35.544183    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-85732c/distinct-2-w4-85732c-kserve-7f96669d6b-764xk"]
Sep 09 04:17:35.546647 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:17:35.546614    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod19f04fc8_0e98_46b9_a42f_cd57aa63b0ac.slice/crio-6996289f56fa5ce63a9d9bc93295f992e0c3fd28d06abef7ee5c06fa2c04320c WatchSource:0}: Error finding container 6996289f56fa5ce63a9d9bc93295f992e0c3fd28d06abef7ee5c06fa2c04320c: Status 404 returned error can't find the container with id 6996289f56fa5ce63a9d9bc93295f992e0c3fd28d06abef7ee5c06fa2c04320c
Sep 09 04:17:36.199649 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:36.199612    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-85732c/distinct-2-w4-85732c-kserve-7f96669d6b-764xk" event={"ID":"19f04fc8-0e98-46b9-a42f-cd57aa63b0ac","Type":"ContainerStarted","Data":"07f8ddaac6e74e316950823044c54e6af899661682635197d92e801ddfaca399"}
Sep 09 04:17:36.199649 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:36.199654    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-85732c/distinct-2-w4-85732c-kserve-7f96669d6b-764xk" event={"ID":"19f04fc8-0e98-46b9-a42f-cd57aa63b0ac","Type":"ContainerStarted","Data":"6996289f56fa5ce63a9d9bc93295f992e0c3fd28d06abef7ee5c06fa2c04320c"}
Sep 09 04:17:36.216989 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:36.216934    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-85732c/distinct-2-w4-85732c-kserve-7f96669d6b-764xk" podStartSLOduration=2.216918723 podStartE2EDuration="2.216918723s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:17:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:17:36.216351649 +0000 UTC m=+1181.135522559" watchObservedRunningTime="2026-09-09 04:17:36.216918723 +0000 UTC m=+1181.136089632"
Sep 09 04:17:45.260664 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:45.260630    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-85732c/distinct-2-w4-85732c-kserve-7f96669d6b-764xk"
Sep 09 04:17:45.261167 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:45.260673    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-85732c/distinct-2-w4-85732c-kserve-7f96669d6b-764xk"
Sep 09 04:17:45.273124 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:45.273083    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-85732c/distinct-2-w4-85732c-kserve-7f96669d6b-764xk"
Sep 09 04:17:45.821503 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:45.821465    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-359adf-29815455-v5cpr"]
Sep 09 04:17:45.825509 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:45.825484    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-359adf-29815455-v5cpr"]
Sep 09 04:17:45.867088 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:45.867058    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-359adf-546f7f8c97-gxn5v"]
Sep 09 04:17:45.867332 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:45.867304    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-359adf-546f7f8c97-gxn5v" podUID="add487c3-333d-482b-bdc8-fefdb027ca88" containerName="payload-processing" containerID="cri-o://43e550d24ba63bb482aa1d4ec805d2a152bac2cf40322cc0ac1576ee6bd5c2f6" gracePeriod=30
Sep 09 04:17:45.889209 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:45.889176    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-359adf-d7487ddc4-8q8rb"]
Sep 09 04:17:45.889405 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:45.889382    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-359adf-d7487ddc4-8q8rb" podUID="4ddbb781-2f2f-4b6a-888d-feedf9fbd159" containerName="payload-pre-processing" containerID="cri-o://8f125f0de2a1425f25361be4dfa3cf68f0c3e2a1446a30805fb4cb15962a081f" gracePeriod=30
Sep 09 04:17:46.181661 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:46.181640    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-359adf-546f7f8c97-gxn5v"
Sep 09 04:17:46.184467 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:46.184448    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-359adf-d7487ddc4-8q8rb"
Sep 09 04:17:46.256430 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:46.256393    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ddbb781-2f2f-4b6a-888d-feedf9fbd159-plugins-config-volume\") pod \"4ddbb781-2f2f-4b6a-888d-feedf9fbd159\" (UID: \"4ddbb781-2f2f-4b6a-888d-feedf9fbd159\") "
Sep 09 04:17:46.256430 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:46.256429    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bmpmv\" (UniqueName: \"kubernetes.io/projected/4ddbb781-2f2f-4b6a-888d-feedf9fbd159-kube-api-access-bmpmv\") pod \"4ddbb781-2f2f-4b6a-888d-feedf9fbd159\" (UID: \"4ddbb781-2f2f-4b6a-888d-feedf9fbd159\") "
Sep 09 04:17:46.256686 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:46.256451    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/add487c3-333d-482b-bdc8-fefdb027ca88-plugins-config-volume\") pod \"add487c3-333d-482b-bdc8-fefdb027ca88\" (UID: \"add487c3-333d-482b-bdc8-fefdb027ca88\") "
Sep 09 04:17:46.256686 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:46.256665    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-56lrr\" (UniqueName: \"kubernetes.io/projected/add487c3-333d-482b-bdc8-fefdb027ca88-kube-api-access-56lrr\") pod \"add487c3-333d-482b-bdc8-fefdb027ca88\" (UID: \"add487c3-333d-482b-bdc8-fefdb027ca88\") "
Sep 09 04:17:46.257048 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:46.257017    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ddbb781-2f2f-4b6a-888d-feedf9fbd159-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4ddbb781-2f2f-4b6a-888d-feedf9fbd159" (UID: "4ddbb781-2f2f-4b6a-888d-feedf9fbd159"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 04:17:46.257212 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:46.257050    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/add487c3-333d-482b-bdc8-fefdb027ca88-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "add487c3-333d-482b-bdc8-fefdb027ca88" (UID: "add487c3-333d-482b-bdc8-fefdb027ca88"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 04:17:46.257962 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:46.257932    2557 generic.go:356] "Generic (PLEG): container finished" podID="add487c3-333d-482b-bdc8-fefdb027ca88" containerID="43e550d24ba63bb482aa1d4ec805d2a152bac2cf40322cc0ac1576ee6bd5c2f6" exitCode=0
Sep 09 04:17:46.258079 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:46.257968    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-359adf-546f7f8c97-gxn5v" event={"ID":"add487c3-333d-482b-bdc8-fefdb027ca88","Type":"ContainerDied","Data":"43e550d24ba63bb482aa1d4ec805d2a152bac2cf40322cc0ac1576ee6bd5c2f6"}
Sep 09 04:17:46.258079 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:46.257993    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-359adf-546f7f8c97-gxn5v"
Sep 09 04:17:46.258079 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:46.258055    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-359adf-546f7f8c97-gxn5v" event={"ID":"add487c3-333d-482b-bdc8-fefdb027ca88","Type":"ContainerDied","Data":"1b8d7721bc222df3afac8c32e8a1aa5b0a964f8e6057362380850d7bc1044922"}
Sep 09 04:17:46.258257 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:46.258080    2557 scope.go:117] "RemoveContainer" containerID="43e550d24ba63bb482aa1d4ec805d2a152bac2cf40322cc0ac1576ee6bd5c2f6"
Sep 09 04:17:46.259375 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:46.259335    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/add487c3-333d-482b-bdc8-fefdb027ca88-kube-api-access-56lrr" (OuterVolumeSpecName: "kube-api-access-56lrr") pod "add487c3-333d-482b-bdc8-fefdb027ca88" (UID: "add487c3-333d-482b-bdc8-fefdb027ca88"). InnerVolumeSpecName "kube-api-access-56lrr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 04:17:46.259375 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:46.259352    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ddbb781-2f2f-4b6a-888d-feedf9fbd159-kube-api-access-bmpmv" (OuterVolumeSpecName: "kube-api-access-bmpmv") pod "4ddbb781-2f2f-4b6a-888d-feedf9fbd159" (UID: "4ddbb781-2f2f-4b6a-888d-feedf9fbd159"). InnerVolumeSpecName "kube-api-access-bmpmv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 04:17:46.259788 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:46.259764    2557 generic.go:356] "Generic (PLEG): container finished" podID="4ddbb781-2f2f-4b6a-888d-feedf9fbd159" containerID="8f125f0de2a1425f25361be4dfa3cf68f0c3e2a1446a30805fb4cb15962a081f" exitCode=0
Sep 09 04:17:46.259940 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:46.259811    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-359adf-d7487ddc4-8q8rb"
Sep 09 04:17:46.259940 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:46.259846    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-359adf-d7487ddc4-8q8rb" event={"ID":"4ddbb781-2f2f-4b6a-888d-feedf9fbd159","Type":"ContainerDied","Data":"8f125f0de2a1425f25361be4dfa3cf68f0c3e2a1446a30805fb4cb15962a081f"}
Sep 09 04:17:46.259940 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:46.259900    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-359adf-d7487ddc4-8q8rb" event={"ID":"4ddbb781-2f2f-4b6a-888d-feedf9fbd159","Type":"ContainerDied","Data":"16a5abf04b77c8a63e63312f2ef98e9e9931e7d5eb26175d6fb064964efa1843"}
Sep 09 04:17:46.271880 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:46.271860    2557 scope.go:117] "RemoveContainer" containerID="43e550d24ba63bb482aa1d4ec805d2a152bac2cf40322cc0ac1576ee6bd5c2f6"
Sep 09 04:17:46.272167 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:17:46.272133    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43e550d24ba63bb482aa1d4ec805d2a152bac2cf40322cc0ac1576ee6bd5c2f6\": container with ID starting with 43e550d24ba63bb482aa1d4ec805d2a152bac2cf40322cc0ac1576ee6bd5c2f6 not found: ID does not exist" containerID="43e550d24ba63bb482aa1d4ec805d2a152bac2cf40322cc0ac1576ee6bd5c2f6"
Sep 09 04:17:46.272167 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:46.272155    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43e550d24ba63bb482aa1d4ec805d2a152bac2cf40322cc0ac1576ee6bd5c2f6"} err="failed to get container status \"43e550d24ba63bb482aa1d4ec805d2a152bac2cf40322cc0ac1576ee6bd5c2f6\": rpc error: code = NotFound desc = could not find container \"43e550d24ba63bb482aa1d4ec805d2a152bac2cf40322cc0ac1576ee6bd5c2f6\": container with ID starting with 43e550d24ba63bb482aa1d4ec805d2a152bac2cf40322cc0ac1576ee6bd5c2f6 not found: ID does not exist"
Sep 09 04:17:46.272255 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:46.272171    2557 scope.go:117] "RemoveContainer" containerID="8f125f0de2a1425f25361be4dfa3cf68f0c3e2a1446a30805fb4cb15962a081f"
Sep 09 04:17:46.274771 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:46.274749    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-85732c/distinct-2-w4-85732c-kserve-7f96669d6b-764xk"
Sep 09 04:17:46.280364 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:46.280346    2557 scope.go:117] "RemoveContainer" containerID="8f125f0de2a1425f25361be4dfa3cf68f0c3e2a1446a30805fb4cb15962a081f"
Sep 09 04:17:46.280626 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:17:46.280608    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f125f0de2a1425f25361be4dfa3cf68f0c3e2a1446a30805fb4cb15962a081f\": container with ID starting with 8f125f0de2a1425f25361be4dfa3cf68f0c3e2a1446a30805fb4cb15962a081f not found: ID does not exist" containerID="8f125f0de2a1425f25361be4dfa3cf68f0c3e2a1446a30805fb4cb15962a081f"
Sep 09 04:17:46.280667 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:46.280635    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f125f0de2a1425f25361be4dfa3cf68f0c3e2a1446a30805fb4cb15962a081f"} err="failed to get container status \"8f125f0de2a1425f25361be4dfa3cf68f0c3e2a1446a30805fb4cb15962a081f\": rpc error: code = NotFound desc = could not find container \"8f125f0de2a1425f25361be4dfa3cf68f0c3e2a1446a30805fb4cb15962a081f\": container with ID starting with 8f125f0de2a1425f25361be4dfa3cf68f0c3e2a1446a30805fb4cb15962a081f not found: ID does not exist"
Sep 09 04:17:46.296644 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:46.296621    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-359adf-d7487ddc4-8q8rb"]
Sep 09 04:17:46.299330 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:46.299309    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-359adf-d7487ddc4-8q8rb"]
Sep 09 04:17:46.357715 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:46.357689    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ddbb781-2f2f-4b6a-888d-feedf9fbd159-plugins-config-volume\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:17:46.357865 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:46.357721    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmpmv\" (UniqueName: \"kubernetes.io/projected/4ddbb781-2f2f-4b6a-888d-feedf9fbd159-kube-api-access-bmpmv\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:17:46.357865 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:46.357739    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/add487c3-333d-482b-bdc8-fefdb027ca88-plugins-config-volume\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:17:46.357865 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:46.357753    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56lrr\" (UniqueName: \"kubernetes.io/projected/add487c3-333d-482b-bdc8-fefdb027ca88-kube-api-access-56lrr\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:17:46.588622 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:46.588593    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-359adf-546f7f8c97-gxn5v"]
Sep 09 04:17:46.591692 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:46.591665    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-359adf-546f7f8c97-gxn5v"]
Sep 09 04:17:47.620772 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:47.620733    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ddbb781-2f2f-4b6a-888d-feedf9fbd159" path="/var/lib/kubelet/pods/4ddbb781-2f2f-4b6a-888d-feedf9fbd159/volumes"
Sep 09 04:17:47.621171 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:47.621088    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="add487c3-333d-482b-bdc8-fefdb027ca88" path="/var/lib/kubelet/pods/add487c3-333d-482b-bdc8-fefdb027ca88/volumes"
Sep 09 04:17:47.621352 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:47.621339    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d831303c-5971-4dff-9238-0eb3b19e0ea9" path="/var/lib/kubelet/pods/d831303c-5971-4dff-9238-0eb3b19e0ea9/volumes"
Sep 09 04:17:55.587421 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:55.587383    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gj7h7_7816cd49-e9c2-4f7b-b8ec-4e5711b675a5/console-operator/2.log"
Sep 09 04:17:55.587819 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:55.587685    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gj7h7_7816cd49-e9c2-4f7b-b8ec-4e5711b675a5/console-operator/2.log"
Sep 09 04:17:55.593232 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:55.593212    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rfvrc_1828d4d8-0d24-4e4d-a502-aee013a3b116/ovn-acl-logging/0.log"
Sep 09 04:17:55.593684 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:55.593663    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rfvrc_1828d4d8-0d24-4e4d-a502-aee013a3b116/ovn-acl-logging/0.log"
Sep 09 04:17:58.238527 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:58.238495    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"]
Sep 09 04:17:58.238940 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:17:58.238716    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6" podUID="b0cd2205-e41e-4495-83a3-29d3da0c034f" containerName="istio-proxy" containerID="cri-o://c5448f9d3361a10899c8fbb00130a733b64c922abfb9a4a269a62a67362fe298" gracePeriod=30
Sep 09 04:18:03.496482 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:03.496455    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:18:03.617604 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:03.617580    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0cd2205-e41e-4495-83a3-29d3da0c034f-workload-certs\") pod \"b0cd2205-e41e-4495-83a3-29d3da0c034f\" (UID: \"b0cd2205-e41e-4495-83a3-29d3da0c034f\") "
Sep 09 04:18:03.617792 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:03.617618    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0cd2205-e41e-4495-83a3-29d3da0c034f-credential-socket\") pod \"b0cd2205-e41e-4495-83a3-29d3da0c034f\" (UID: \"b0cd2205-e41e-4495-83a3-29d3da0c034f\") "
Sep 09 04:18:03.617792 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:03.617643    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0cd2205-e41e-4495-83a3-29d3da0c034f-istiod-ca-cert\") pod \"b0cd2205-e41e-4495-83a3-29d3da0c034f\" (UID: \"b0cd2205-e41e-4495-83a3-29d3da0c034f\") "
Sep 09 04:18:03.617792 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:03.617711    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wstl2\" (UniqueName: \"kubernetes.io/projected/b0cd2205-e41e-4495-83a3-29d3da0c034f-kube-api-access-wstl2\") pod \"b0cd2205-e41e-4495-83a3-29d3da0c034f\" (UID: \"b0cd2205-e41e-4495-83a3-29d3da0c034f\") "
Sep 09 04:18:03.617792 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:03.617747    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0cd2205-e41e-4495-83a3-29d3da0c034f-istio-token\") pod \"b0cd2205-e41e-4495-83a3-29d3da0c034f\" (UID: \"b0cd2205-e41e-4495-83a3-29d3da0c034f\") "
Sep 09 04:18:03.617792 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:03.617777    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0cd2205-e41e-4495-83a3-29d3da0c034f-istio-envoy\") pod \"b0cd2205-e41e-4495-83a3-29d3da0c034f\" (UID: \"b0cd2205-e41e-4495-83a3-29d3da0c034f\") "
Sep 09 04:18:03.618059 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:03.617814    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0cd2205-e41e-4495-83a3-29d3da0c034f-workload-socket\") pod \"b0cd2205-e41e-4495-83a3-29d3da0c034f\" (UID: \"b0cd2205-e41e-4495-83a3-29d3da0c034f\") "
Sep 09 04:18:03.618059 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:03.617842    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0cd2205-e41e-4495-83a3-29d3da0c034f-istio-data\") pod \"b0cd2205-e41e-4495-83a3-29d3da0c034f\" (UID: \"b0cd2205-e41e-4495-83a3-29d3da0c034f\") "
Sep 09 04:18:03.618059 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:03.617866    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0cd2205-e41e-4495-83a3-29d3da0c034f-istio-podinfo\") pod \"b0cd2205-e41e-4495-83a3-29d3da0c034f\" (UID: \"b0cd2205-e41e-4495-83a3-29d3da0c034f\") "
Sep 09 04:18:03.618059 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:03.617890    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0cd2205-e41e-4495-83a3-29d3da0c034f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b0cd2205-e41e-4495-83a3-29d3da0c034f" (UID: "b0cd2205-e41e-4495-83a3-29d3da0c034f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 04:18:03.618059 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:03.617928    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0cd2205-e41e-4495-83a3-29d3da0c034f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b0cd2205-e41e-4495-83a3-29d3da0c034f" (UID: "b0cd2205-e41e-4495-83a3-29d3da0c034f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 04:18:03.618347 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:03.618063    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0cd2205-e41e-4495-83a3-29d3da0c034f-istio-data" (OuterVolumeSpecName: "istio-data") pod "b0cd2205-e41e-4495-83a3-29d3da0c034f" (UID: "b0cd2205-e41e-4495-83a3-29d3da0c034f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 04:18:03.618347 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:03.618196    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0cd2205-e41e-4495-83a3-29d3da0c034f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b0cd2205-e41e-4495-83a3-29d3da0c034f" (UID: "b0cd2205-e41e-4495-83a3-29d3da0c034f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 04:18:03.618347 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:03.618334    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0cd2205-e41e-4495-83a3-29d3da0c034f-workload-certs\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:18:03.618502 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:03.618362    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0cd2205-e41e-4495-83a3-29d3da0c034f-credential-socket\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:18:03.618502 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:03.618379    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0cd2205-e41e-4495-83a3-29d3da0c034f-istiod-ca-cert\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:18:03.618502 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:03.618393    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0cd2205-e41e-4495-83a3-29d3da0c034f-istio-data\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:18:03.619021 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:03.618992    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0cd2205-e41e-4495-83a3-29d3da0c034f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b0cd2205-e41e-4495-83a3-29d3da0c034f" (UID: "b0cd2205-e41e-4495-83a3-29d3da0c034f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 04:18:03.620182 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:03.620157    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0cd2205-e41e-4495-83a3-29d3da0c034f-istio-token" (OuterVolumeSpecName: "istio-token") pod "b0cd2205-e41e-4495-83a3-29d3da0c034f" (UID: "b0cd2205-e41e-4495-83a3-29d3da0c034f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 04:18:03.620284 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:03.620257    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0cd2205-e41e-4495-83a3-29d3da0c034f-kube-api-access-wstl2" (OuterVolumeSpecName: "kube-api-access-wstl2") pod "b0cd2205-e41e-4495-83a3-29d3da0c034f" (UID: "b0cd2205-e41e-4495-83a3-29d3da0c034f"). InnerVolumeSpecName "kube-api-access-wstl2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 04:18:03.620424 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:03.620405    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b0cd2205-e41e-4495-83a3-29d3da0c034f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b0cd2205-e41e-4495-83a3-29d3da0c034f" (UID: "b0cd2205-e41e-4495-83a3-29d3da0c034f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 04:18:03.620512 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:03.620484    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0cd2205-e41e-4495-83a3-29d3da0c034f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b0cd2205-e41e-4495-83a3-29d3da0c034f" (UID: "b0cd2205-e41e-4495-83a3-29d3da0c034f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 04:18:03.718957 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:03.718932    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wstl2\" (UniqueName: \"kubernetes.io/projected/b0cd2205-e41e-4495-83a3-29d3da0c034f-kube-api-access-wstl2\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:18:03.718957 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:03.718954    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0cd2205-e41e-4495-83a3-29d3da0c034f-istio-token\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:18:03.718957 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:03.718964    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0cd2205-e41e-4495-83a3-29d3da0c034f-istio-envoy\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:18:03.719266 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:03.718972    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0cd2205-e41e-4495-83a3-29d3da0c034f-workload-socket\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:18:03.719266 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:03.718980    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0cd2205-e41e-4495-83a3-29d3da0c034f-istio-podinfo\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:18:04.358055 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:04.358017    2557 generic.go:356] "Generic (PLEG): container finished" podID="b0cd2205-e41e-4495-83a3-29d3da0c034f" containerID="c5448f9d3361a10899c8fbb00130a733b64c922abfb9a4a269a62a67362fe298" exitCode=0
Sep 09 04:18:04.358055 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:04.358059    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6" event={"ID":"b0cd2205-e41e-4495-83a3-29d3da0c034f","Type":"ContainerDied","Data":"c5448f9d3361a10899c8fbb00130a733b64c922abfb9a4a269a62a67362fe298"}
Sep 09 04:18:04.358306 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:04.358088    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"
Sep 09 04:18:04.358306 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:04.358101    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6" event={"ID":"b0cd2205-e41e-4495-83a3-29d3da0c034f","Type":"ContainerDied","Data":"cabc0bcc751b06e392fdc90fe4081e742db6f2196a7cc3bf9516132c4b8e9232"}
Sep 09 04:18:04.358306 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:04.358137    2557 scope.go:117] "RemoveContainer" containerID="c5448f9d3361a10899c8fbb00130a733b64c922abfb9a4a269a62a67362fe298"
Sep 09 04:18:04.370445 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:04.370427    2557 scope.go:117] "RemoveContainer" containerID="25edb2ff4fd72705f3109dbf6c4c986ee9ad91b716e654c84dc416b0490e7bc9"
Sep 09 04:18:04.378643 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:04.378624    2557 scope.go:117] "RemoveContainer" containerID="c5448f9d3361a10899c8fbb00130a733b64c922abfb9a4a269a62a67362fe298"
Sep 09 04:18:04.378898 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:18:04.378875    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5448f9d3361a10899c8fbb00130a733b64c922abfb9a4a269a62a67362fe298\": container with ID starting with c5448f9d3361a10899c8fbb00130a733b64c922abfb9a4a269a62a67362fe298 not found: ID does not exist" containerID="c5448f9d3361a10899c8fbb00130a733b64c922abfb9a4a269a62a67362fe298"
Sep 09 04:18:04.378963 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:04.378906    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5448f9d3361a10899c8fbb00130a733b64c922abfb9a4a269a62a67362fe298"} err="failed to get container status \"c5448f9d3361a10899c8fbb00130a733b64c922abfb9a4a269a62a67362fe298\": rpc error: code = NotFound desc = could not find container \"c5448f9d3361a10899c8fbb00130a733b64c922abfb9a4a269a62a67362fe298\": container with ID starting with c5448f9d3361a10899c8fbb00130a733b64c922abfb9a4a269a62a67362fe298 not found: ID does not exist"
Sep 09 04:18:04.378963 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:04.378924    2557 scope.go:117] "RemoveContainer" containerID="25edb2ff4fd72705f3109dbf6c4c986ee9ad91b716e654c84dc416b0490e7bc9"
Sep 09 04:18:04.379192 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:18:04.379175    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25edb2ff4fd72705f3109dbf6c4c986ee9ad91b716e654c84dc416b0490e7bc9\": container with ID starting with 25edb2ff4fd72705f3109dbf6c4c986ee9ad91b716e654c84dc416b0490e7bc9 not found: ID does not exist" containerID="25edb2ff4fd72705f3109dbf6c4c986ee9ad91b716e654c84dc416b0490e7bc9"
Sep 09 04:18:04.379238 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:04.379200    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25edb2ff4fd72705f3109dbf6c4c986ee9ad91b716e654c84dc416b0490e7bc9"} err="failed to get container status \"25edb2ff4fd72705f3109dbf6c4c986ee9ad91b716e654c84dc416b0490e7bc9\": rpc error: code = NotFound desc = could not find container \"25edb2ff4fd72705f3109dbf6c4c986ee9ad91b716e654c84dc416b0490e7bc9\": container with ID starting with 25edb2ff4fd72705f3109dbf6c4c986ee9ad91b716e654c84dc416b0490e7bc9 not found: ID does not exist"
Sep 09 04:18:04.382013 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:04.381993    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"]
Sep 09 04:18:04.385815 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:04.385796    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-359adf-openshift-default-879dc896b-k5kg6"]
Sep 09 04:18:05.619447 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:05.619400    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0cd2205-e41e-4495-83a3-29d3da0c034f" path="/var/lib/kubelet/pods/b0cd2205-e41e-4495-83a3-29d3da0c034f/volumes"
Sep 09 04:18:09.252342 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:09.252310    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-629522-6d5c87fb6c-qwrlh"]
Sep 09 04:18:09.252788 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:09.252551    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-629522-6d5c87fb6c-qwrlh" podUID="f2a15779-643f-4369-8470-f986d81ec1af" containerName="payload-processing" containerID="cri-o://44fb37541dbf9cf6d733cad3534e774942f6c94474062cf4cc051b257651c58f" gracePeriod=30
Sep 09 04:18:09.386077 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:09.386044    2557 generic.go:356] "Generic (PLEG): container finished" podID="f2a15779-643f-4369-8470-f986d81ec1af" containerID="44fb37541dbf9cf6d733cad3534e774942f6c94474062cf4cc051b257651c58f" exitCode=0
Sep 09 04:18:09.386245 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:09.386129    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-629522-6d5c87fb6c-qwrlh" event={"ID":"f2a15779-643f-4369-8470-f986d81ec1af","Type":"ContainerDied","Data":"44fb37541dbf9cf6d733cad3534e774942f6c94474062cf4cc051b257651c58f"}
Sep 09 04:18:09.511299 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:09.511241    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-629522-6d5c87fb6c-qwrlh"
Sep 09 04:18:09.569215 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:09.569185    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2a15779-643f-4369-8470-f986d81ec1af-plugins-config-volume\") pod \"f2a15779-643f-4369-8470-f986d81ec1af\" (UID: \"f2a15779-643f-4369-8470-f986d81ec1af\") "
Sep 09 04:18:09.569367 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:09.569232    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-22942\" (UniqueName: \"kubernetes.io/projected/f2a15779-643f-4369-8470-f986d81ec1af-kube-api-access-22942\") pod \"f2a15779-643f-4369-8470-f986d81ec1af\" (UID: \"f2a15779-643f-4369-8470-f986d81ec1af\") "
Sep 09 04:18:09.569548 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:09.569527    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2a15779-643f-4369-8470-f986d81ec1af-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f2a15779-643f-4369-8470-f986d81ec1af" (UID: "f2a15779-643f-4369-8470-f986d81ec1af"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 04:18:09.571316 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:09.571289    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2a15779-643f-4369-8470-f986d81ec1af-kube-api-access-22942" (OuterVolumeSpecName: "kube-api-access-22942") pod "f2a15779-643f-4369-8470-f986d81ec1af" (UID: "f2a15779-643f-4369-8470-f986d81ec1af"). InnerVolumeSpecName "kube-api-access-22942". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 04:18:09.670333 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:09.670308    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2a15779-643f-4369-8470-f986d81ec1af-plugins-config-volume\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:18:09.670333 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:09.670331    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22942\" (UniqueName: \"kubernetes.io/projected/f2a15779-643f-4369-8470-f986d81ec1af-kube-api-access-22942\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:18:10.392492 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:10.392466    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-629522-6d5c87fb6c-qwrlh"
Sep 09 04:18:10.392945 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:10.392461    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-629522-6d5c87fb6c-qwrlh" event={"ID":"f2a15779-643f-4369-8470-f986d81ec1af","Type":"ContainerDied","Data":"fc868bfd2092f8a2c43c0a1baccdcf92f8e07075ed35cebd7c0a1f1148edd814"}
Sep 09 04:18:10.392945 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:10.392620    2557 scope.go:117] "RemoveContainer" containerID="44fb37541dbf9cf6d733cad3534e774942f6c94474062cf4cc051b257651c58f"
Sep 09 04:18:10.409523 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:10.409498    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-629522-6d5c87fb6c-qwrlh"]
Sep 09 04:18:10.416016 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:10.415993    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-629522-6d5c87fb6c-qwrlh"]
Sep 09 04:18:11.618995 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:11.618964    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2a15779-643f-4369-8470-f986d81ec1af" path="/var/lib/kubelet/pods/f2a15779-643f-4369-8470-f986d81ec1af/volumes"
Sep 09 04:18:23.087380 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:23.087347    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-426183-6b9f4c95f5-hgch5"]
Sep 09 04:18:23.087739 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:23.087700    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0cd2205-e41e-4495-83a3-29d3da0c034f" containerName="istio-proxy"
Sep 09 04:18:23.087739 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:23.087710    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0cd2205-e41e-4495-83a3-29d3da0c034f" containerName="istio-proxy"
Sep 09 04:18:23.087739 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:23.087722    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0cd2205-e41e-4495-83a3-29d3da0c034f" containerName="istio-proxy"
Sep 09 04:18:23.087739 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:23.087728    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0cd2205-e41e-4495-83a3-29d3da0c034f" containerName="istio-proxy"
Sep 09 04:18:23.087739 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:23.087738    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2a15779-643f-4369-8470-f986d81ec1af" containerName="payload-processing"
Sep 09 04:18:23.087930 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:23.087744    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2a15779-643f-4369-8470-f986d81ec1af" containerName="payload-processing"
Sep 09 04:18:23.087930 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:23.087753    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ddbb781-2f2f-4b6a-888d-feedf9fbd159" containerName="payload-pre-processing"
Sep 09 04:18:23.087930 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:23.087758    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ddbb781-2f2f-4b6a-888d-feedf9fbd159" containerName="payload-pre-processing"
Sep 09 04:18:23.087930 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:23.087776    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="add487c3-333d-482b-bdc8-fefdb027ca88" containerName="payload-processing"
Sep 09 04:18:23.087930 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:23.087781    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="add487c3-333d-482b-bdc8-fefdb027ca88" containerName="payload-processing"
Sep 09 04:18:23.087930 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:23.087838    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ddbb781-2f2f-4b6a-888d-feedf9fbd159" containerName="payload-pre-processing"
Sep 09 04:18:23.087930 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:23.087850    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0cd2205-e41e-4495-83a3-29d3da0c034f" containerName="istio-proxy"
Sep 09 04:18:23.087930 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:23.087855    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2a15779-643f-4369-8470-f986d81ec1af" containerName="payload-processing"
Sep 09 04:18:23.087930 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:23.087861    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="add487c3-333d-482b-bdc8-fefdb027ca88" containerName="payload-processing"
Sep 09 04:18:23.092219 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:23.092200    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-426183-6b9f4c95f5-hgch5"
Sep 09 04:18:23.095430 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:23.095411    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-426183-dockercfg-sct77\""
Sep 09 04:18:23.095857 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:23.095839    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-426183\""
Sep 09 04:18:23.107220 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:23.107191    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-426183-6b9f4c95f5-hgch5"]
Sep 09 04:18:23.184695 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:23.184652    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lbcl\" (UniqueName: \"kubernetes.io/projected/2cdc0724-54a2-424f-839d-203b761acd08-kube-api-access-6lbcl\") pod \"payload-processing-e2e-shared-a-w3-426183-6b9f4c95f5-hgch5\" (UID: \"2cdc0724-54a2-424f-839d-203b761acd08\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-426183-6b9f4c95f5-hgch5"
Sep 09 04:18:23.184845 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:23.184736    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2cdc0724-54a2-424f-839d-203b761acd08-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-426183-6b9f4c95f5-hgch5\" (UID: \"2cdc0724-54a2-424f-839d-203b761acd08\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-426183-6b9f4c95f5-hgch5"
Sep 09 04:18:23.286100 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:23.286067    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2cdc0724-54a2-424f-839d-203b761acd08-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-426183-6b9f4c95f5-hgch5\" (UID: \"2cdc0724-54a2-424f-839d-203b761acd08\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-426183-6b9f4c95f5-hgch5"
Sep 09 04:18:23.286275 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:23.286165    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lbcl\" (UniqueName: \"kubernetes.io/projected/2cdc0724-54a2-424f-839d-203b761acd08-kube-api-access-6lbcl\") pod \"payload-processing-e2e-shared-a-w3-426183-6b9f4c95f5-hgch5\" (UID: \"2cdc0724-54a2-424f-839d-203b761acd08\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-426183-6b9f4c95f5-hgch5"
Sep 09 04:18:23.286759 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:23.286739    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2cdc0724-54a2-424f-839d-203b761acd08-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-426183-6b9f4c95f5-hgch5\" (UID: \"2cdc0724-54a2-424f-839d-203b761acd08\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-426183-6b9f4c95f5-hgch5"
Sep 09 04:18:23.294775 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:23.294750    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lbcl\" (UniqueName: \"kubernetes.io/projected/2cdc0724-54a2-424f-839d-203b761acd08-kube-api-access-6lbcl\") pod \"payload-processing-e2e-shared-a-w3-426183-6b9f4c95f5-hgch5\" (UID: \"2cdc0724-54a2-424f-839d-203b761acd08\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-426183-6b9f4c95f5-hgch5"
Sep 09 04:18:23.401666 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:23.401632    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-426183-6b9f4c95f5-hgch5"
Sep 09 04:18:23.672256 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:23.672230    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-426183-6b9f4c95f5-hgch5"]
Sep 09 04:18:23.674094 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:18:23.674067    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cdc0724_54a2_424f_839d_203b761acd08.slice/crio-68e12c107099ed0492248409f2f7ae1fd48f68fcac87ccc27f84545ccd6112f1 WatchSource:0}: Error finding container 68e12c107099ed0492248409f2f7ae1fd48f68fcac87ccc27f84545ccd6112f1: Status 404 returned error can't find the container with id 68e12c107099ed0492248409f2f7ae1fd48f68fcac87ccc27f84545ccd6112f1
Sep 09 04:18:24.476670 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:24.476634    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-426183-6b9f4c95f5-hgch5" event={"ID":"2cdc0724-54a2-424f-839d-203b761acd08","Type":"ContainerStarted","Data":"f436252fb58e6e424526c31c375c524ce2f040531da40a5cf93df8c30f566198"}
Sep 09 04:18:24.477162 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:24.476692    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-426183-6b9f4c95f5-hgch5" event={"ID":"2cdc0724-54a2-424f-839d-203b761acd08","Type":"ContainerStarted","Data":"68e12c107099ed0492248409f2f7ae1fd48f68fcac87ccc27f84545ccd6112f1"}
Sep 09 04:18:24.477162 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:24.476759    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-426183-6b9f4c95f5-hgch5"
Sep 09 04:18:24.497679 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:24.497615    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-426183-6b9f4c95f5-hgch5" podStartSLOduration=1.497599492 podStartE2EDuration="1.497599492s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:18:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:18:24.496879699 +0000 UTC m=+1229.416050610" watchObservedRunningTime="2026-09-09 04:18:24.497599492 +0000 UTC m=+1229.416770401"
Sep 09 04:18:35.483156 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:35.483099    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-426183-6b9f4c95f5-hgch5"
Sep 09 04:18:51.058540 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:51.058143    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq"]
Sep 09 04:18:51.058994 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:51.058779    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0cd2205-e41e-4495-83a3-29d3da0c034f" containerName="istio-proxy"
Sep 09 04:18:51.062865 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:51.062842    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq"
Sep 09 04:18:51.065206 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:51.065182    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-9c44b5-serving-cert\""
Sep 09 04:18:51.073150 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:51.073125    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq"]
Sep 09 04:18:51.238692 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:51.238659    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de42a0ec-7e12-41cb-b2a2-29c04d7758a8-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq\" (UID: \"de42a0ec-7e12-41cb-b2a2-29c04d7758a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq"
Sep 09 04:18:51.238875 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:51.238702    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de42a0ec-7e12-41cb-b2a2-29c04d7758a8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq\" (UID: \"de42a0ec-7e12-41cb-b2a2-29c04d7758a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq"
Sep 09 04:18:51.238875 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:51.238747    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2qjv\" (UniqueName: \"kubernetes.io/projected/de42a0ec-7e12-41cb-b2a2-29c04d7758a8-kube-api-access-n2qjv\") pod \"maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq\" (UID: \"de42a0ec-7e12-41cb-b2a2-29c04d7758a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq"
Sep 09 04:18:51.238875 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:51.238792    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de42a0ec-7e12-41cb-b2a2-29c04d7758a8-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq\" (UID: \"de42a0ec-7e12-41cb-b2a2-29c04d7758a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq"
Sep 09 04:18:51.339607 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:51.339524    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de42a0ec-7e12-41cb-b2a2-29c04d7758a8-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq\" (UID: \"de42a0ec-7e12-41cb-b2a2-29c04d7758a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq"
Sep 09 04:18:51.339607 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:51.339565    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de42a0ec-7e12-41cb-b2a2-29c04d7758a8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq\" (UID: \"de42a0ec-7e12-41cb-b2a2-29c04d7758a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq"
Sep 09 04:18:51.339607 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:51.339598    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2qjv\" (UniqueName: \"kubernetes.io/projected/de42a0ec-7e12-41cb-b2a2-29c04d7758a8-kube-api-access-n2qjv\") pod \"maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq\" (UID: \"de42a0ec-7e12-41cb-b2a2-29c04d7758a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq"
Sep 09 04:18:51.339828 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:51.339624    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de42a0ec-7e12-41cb-b2a2-29c04d7758a8-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq\" (UID: \"de42a0ec-7e12-41cb-b2a2-29c04d7758a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq"
Sep 09 04:18:51.340369 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:51.340343    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de42a0ec-7e12-41cb-b2a2-29c04d7758a8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq\" (UID: \"de42a0ec-7e12-41cb-b2a2-29c04d7758a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq"
Sep 09 04:18:51.342021 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:51.341989    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de42a0ec-7e12-41cb-b2a2-29c04d7758a8-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq\" (UID: \"de42a0ec-7e12-41cb-b2a2-29c04d7758a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq"
Sep 09 04:18:51.342132 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:51.342077    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de42a0ec-7e12-41cb-b2a2-29c04d7758a8-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq\" (UID: \"de42a0ec-7e12-41cb-b2a2-29c04d7758a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq"
Sep 09 04:18:51.349898 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:51.349873    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2qjv\" (UniqueName: \"kubernetes.io/projected/de42a0ec-7e12-41cb-b2a2-29c04d7758a8-kube-api-access-n2qjv\") pod \"maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq\" (UID: \"de42a0ec-7e12-41cb-b2a2-29c04d7758a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq"
Sep 09 04:18:51.376218 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:51.376191    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq"
Sep 09 04:18:51.649094 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:51.649069    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq"]
Sep 09 04:18:51.650756 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:18:51.650732    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde42a0ec_7e12_41cb_b2a2_29c04d7758a8.slice/crio-ffa23f34de2979d06fc48e4d5949795595da27b475600ed72dd7e60fb33a734f WatchSource:0}: Error finding container ffa23f34de2979d06fc48e4d5949795595da27b475600ed72dd7e60fb33a734f: Status 404 returned error can't find the container with id ffa23f34de2979d06fc48e4d5949795595da27b475600ed72dd7e60fb33a734f
Sep 09 04:18:52.613810 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:52.613784    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq" event={"ID":"de42a0ec-7e12-41cb-b2a2-29c04d7758a8","Type":"ContainerStarted","Data":"764be7eaaecbaa32957a6711f5f3c1f0678907e807d447314eae27f922988098"}
Sep 09 04:18:52.614137 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:52.613828    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq" event={"ID":"de42a0ec-7e12-41cb-b2a2-29c04d7758a8","Type":"ContainerStarted","Data":"ffa23f34de2979d06fc48e4d5949795595da27b475600ed72dd7e60fb33a734f"}
Sep 09 04:18:52.614137 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:52.613910    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq"
Sep 09 04:18:52.642381 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:52.642319    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq" podStartSLOduration=0.756410402 podStartE2EDuration="1.642302346s" totalImagesPullingTime="885.891944ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:18:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:18:52.632564115 +0000 UTC m=+1257.551735025" watchObservedRunningTime="2026-09-09 04:18:52.642302346 +0000 UTC m=+1257.561473256"
Sep 09 04:18:58.625334 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:18:58.625305    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq"
Sep 09 04:19:48.981045 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:19:48.981003    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-429d71ed-7fd46c9dcc-fwtqg"]
Sep 09 04:19:48.981561 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:19:48.981299    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-429d71ed-7fd46c9dcc-fwtqg" podUID="a08b0bc8-027b-4ac8-8f79-b516cfee3d09" containerName="payload-processing" containerID="cri-o://a28aadfa5879abe4c1c73b0bb34aa51c1467caecf55f50a71fc258a5e8aeecb9" gracePeriod=30
Sep 09 04:19:49.232185 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:19:49.232107    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-429d71ed-7fd46c9dcc-fwtqg"
Sep 09 04:19:49.259641 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:19:49.259605    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6jcvg\" (UniqueName: \"kubernetes.io/projected/a08b0bc8-027b-4ac8-8f79-b516cfee3d09-kube-api-access-6jcvg\") pod \"a08b0bc8-027b-4ac8-8f79-b516cfee3d09\" (UID: \"a08b0bc8-027b-4ac8-8f79-b516cfee3d09\") "
Sep 09 04:19:49.259798 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:19:49.259676    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a08b0bc8-027b-4ac8-8f79-b516cfee3d09-plugins-config-volume\") pod \"a08b0bc8-027b-4ac8-8f79-b516cfee3d09\" (UID: \"a08b0bc8-027b-4ac8-8f79-b516cfee3d09\") "
Sep 09 04:19:49.260056 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:19:49.260034    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a08b0bc8-027b-4ac8-8f79-b516cfee3d09-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a08b0bc8-027b-4ac8-8f79-b516cfee3d09" (UID: "a08b0bc8-027b-4ac8-8f79-b516cfee3d09"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 04:19:49.261640 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:19:49.261618    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a08b0bc8-027b-4ac8-8f79-b516cfee3d09-kube-api-access-6jcvg" (OuterVolumeSpecName: "kube-api-access-6jcvg") pod "a08b0bc8-027b-4ac8-8f79-b516cfee3d09" (UID: "a08b0bc8-027b-4ac8-8f79-b516cfee3d09"). InnerVolumeSpecName "kube-api-access-6jcvg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 04:19:49.360717 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:19:49.360675    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a08b0bc8-027b-4ac8-8f79-b516cfee3d09-plugins-config-volume\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:19:49.360717 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:19:49.360709    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jcvg\" (UniqueName: \"kubernetes.io/projected/a08b0bc8-027b-4ac8-8f79-b516cfee3d09-kube-api-access-6jcvg\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:19:49.895503 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:19:49.895421    2557 generic.go:356] "Generic (PLEG): container finished" podID="a08b0bc8-027b-4ac8-8f79-b516cfee3d09" containerID="a28aadfa5879abe4c1c73b0bb34aa51c1467caecf55f50a71fc258a5e8aeecb9" exitCode=0
Sep 09 04:19:49.895503 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:19:49.895458    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-429d71ed-7fd46c9dcc-fwtqg"
Sep 09 04:19:49.895503 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:19:49.895495    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-429d71ed-7fd46c9dcc-fwtqg" event={"ID":"a08b0bc8-027b-4ac8-8f79-b516cfee3d09","Type":"ContainerDied","Data":"a28aadfa5879abe4c1c73b0bb34aa51c1467caecf55f50a71fc258a5e8aeecb9"}
Sep 09 04:19:49.895718 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:19:49.895550    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-429d71ed-7fd46c9dcc-fwtqg" event={"ID":"a08b0bc8-027b-4ac8-8f79-b516cfee3d09","Type":"ContainerDied","Data":"2ee4196544e6003fccaa76f5dec073e667cce7c4958e40f500dac9dc3cf4af1c"}
Sep 09 04:19:49.895718 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:19:49.895577    2557 scope.go:117] "RemoveContainer" containerID="a28aadfa5879abe4c1c73b0bb34aa51c1467caecf55f50a71fc258a5e8aeecb9"
Sep 09 04:19:49.904380 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:19:49.904367    2557 scope.go:117] "RemoveContainer" containerID="a28aadfa5879abe4c1c73b0bb34aa51c1467caecf55f50a71fc258a5e8aeecb9"
Sep 09 04:19:49.904608 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:19:49.904592    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a28aadfa5879abe4c1c73b0bb34aa51c1467caecf55f50a71fc258a5e8aeecb9\": container with ID starting with a28aadfa5879abe4c1c73b0bb34aa51c1467caecf55f50a71fc258a5e8aeecb9 not found: ID does not exist" containerID="a28aadfa5879abe4c1c73b0bb34aa51c1467caecf55f50a71fc258a5e8aeecb9"
Sep 09 04:19:49.904661 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:19:49.904622    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a28aadfa5879abe4c1c73b0bb34aa51c1467caecf55f50a71fc258a5e8aeecb9"} err="failed to get container status \"a28aadfa5879abe4c1c73b0bb34aa51c1467caecf55f50a71fc258a5e8aeecb9\": rpc error: code = NotFound desc = could not find container \"a28aadfa5879abe4c1c73b0bb34aa51c1467caecf55f50a71fc258a5e8aeecb9\": container with ID starting with a28aadfa5879abe4c1c73b0bb34aa51c1467caecf55f50a71fc258a5e8aeecb9 not found: ID does not exist"
Sep 09 04:19:49.912688 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:19:49.912666    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-429d71ed-7fd46c9dcc-fwtqg"]
Sep 09 04:19:49.917073 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:19:49.917052    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-429d71ed-7fd46c9dcc-fwtqg"]
Sep 09 04:19:51.619935 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:19:51.619897    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a08b0bc8-027b-4ac8-8f79-b516cfee3d09" path="/var/lib/kubelet/pods/a08b0bc8-027b-4ac8-8f79-b516cfee3d09/volumes"
Sep 09 04:20:18.865279 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:20:18.865203    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-85732c/e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd"]
Sep 09 04:20:18.865647 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:20:18.865605    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a08b0bc8-027b-4ac8-8f79-b516cfee3d09" containerName="payload-processing"
Sep 09 04:20:18.865647 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:20:18.865619    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a08b0bc8-027b-4ac8-8f79-b516cfee3d09" containerName="payload-processing"
Sep 09 04:20:18.865720 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:20:18.865702    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a08b0bc8-027b-4ac8-8f79-b516cfee3d09" containerName="payload-processing"
Sep 09 04:20:18.869776 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:20:18.869760    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-85732c/e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd"
Sep 09 04:20:18.872445 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:20:18.872424    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-85732c\"/\"e2e-conflict-a-4afe0483-kserve-self-signed-certs\""
Sep 09 04:20:18.877208 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:20:18.877181    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-85732c/e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd"]
Sep 09 04:20:19.031852 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:20:19.031818    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c91c0c6-790f-4623-859f-e783574935ed-tmp-dir\") pod \"e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd\" (UID: \"1c91c0c6-790f-4623-859f-e783574935ed\") " pod="e2e-models-e2e-worker-w4-85732c/e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd"
Sep 09 04:20:19.032031 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:20:19.031861    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-552h6\" (UniqueName: \"kubernetes.io/projected/1c91c0c6-790f-4623-859f-e783574935ed-kube-api-access-552h6\") pod \"e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd\" (UID: \"1c91c0c6-790f-4623-859f-e783574935ed\") " pod="e2e-models-e2e-worker-w4-85732c/e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd"
Sep 09 04:20:19.032031 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:20:19.031895    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c91c0c6-790f-4623-859f-e783574935ed-dshm\") pod \"e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd\" (UID: \"1c91c0c6-790f-4623-859f-e783574935ed\") " pod="e2e-models-e2e-worker-w4-85732c/e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd"
Sep 09 04:20:19.032031 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:20:19.031953    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c91c0c6-790f-4623-859f-e783574935ed-tls-certs\") pod \"e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd\" (UID: \"1c91c0c6-790f-4623-859f-e783574935ed\") " pod="e2e-models-e2e-worker-w4-85732c/e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd"
Sep 09 04:20:19.032031 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:20:19.032012    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c91c0c6-790f-4623-859f-e783574935ed-home\") pod \"e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd\" (UID: \"1c91c0c6-790f-4623-859f-e783574935ed\") " pod="e2e-models-e2e-worker-w4-85732c/e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd"
Sep 09 04:20:19.032224 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:20:19.032059    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c91c0c6-790f-4623-859f-e783574935ed-model-cache\") pod \"e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd\" (UID: \"1c91c0c6-790f-4623-859f-e783574935ed\") " pod="e2e-models-e2e-worker-w4-85732c/e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd"
Sep 09 04:20:19.132958 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:20:19.132875    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c91c0c6-790f-4623-859f-e783574935ed-tmp-dir\") pod \"e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd\" (UID: \"1c91c0c6-790f-4623-859f-e783574935ed\") " pod="e2e-models-e2e-worker-w4-85732c/e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd"
Sep 09 04:20:19.132958 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:20:19.132934    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-552h6\" (UniqueName: \"kubernetes.io/projected/1c91c0c6-790f-4623-859f-e783574935ed-kube-api-access-552h6\") pod \"e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd\" (UID: \"1c91c0c6-790f-4623-859f-e783574935ed\") " pod="e2e-models-e2e-worker-w4-85732c/e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd"
Sep 09 04:20:19.133201 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:20:19.132973    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c91c0c6-790f-4623-859f-e783574935ed-dshm\") pod \"e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd\" (UID: \"1c91c0c6-790f-4623-859f-e783574935ed\") " pod="e2e-models-e2e-worker-w4-85732c/e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd"
Sep 09 04:20:19.133201 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:20:19.133000    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c91c0c6-790f-4623-859f-e783574935ed-tls-certs\") pod \"e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd\" (UID: \"1c91c0c6-790f-4623-859f-e783574935ed\") " pod="e2e-models-e2e-worker-w4-85732c/e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd"
Sep 09 04:20:19.133201 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:20:19.133056    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c91c0c6-790f-4623-859f-e783574935ed-home\") pod \"e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd\" (UID: \"1c91c0c6-790f-4623-859f-e783574935ed\") " pod="e2e-models-e2e-worker-w4-85732c/e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd"
Sep 09 04:20:19.133201 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:20:19.133089    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c91c0c6-790f-4623-859f-e783574935ed-model-cache\") pod \"e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd\" (UID: \"1c91c0c6-790f-4623-859f-e783574935ed\") " pod="e2e-models-e2e-worker-w4-85732c/e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd"
Sep 09 04:20:19.133422 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:20:19.133363    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c91c0c6-790f-4623-859f-e783574935ed-tmp-dir\") pod \"e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd\" (UID: \"1c91c0c6-790f-4623-859f-e783574935ed\") " pod="e2e-models-e2e-worker-w4-85732c/e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd"
Sep 09 04:20:19.133601 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:20:19.133573    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c91c0c6-790f-4623-859f-e783574935ed-home\") pod \"e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd\" (UID: \"1c91c0c6-790f-4623-859f-e783574935ed\") " pod="e2e-models-e2e-worker-w4-85732c/e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd"
Sep 09 04:20:19.133601 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:20:19.133589    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c91c0c6-790f-4623-859f-e783574935ed-model-cache\") pod \"e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd\" (UID: \"1c91c0c6-790f-4623-859f-e783574935ed\") " pod="e2e-models-e2e-worker-w4-85732c/e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd"
Sep 09 04:20:19.135750 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:20:19.135723    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c91c0c6-790f-4623-859f-e783574935ed-dshm\") pod \"e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd\" (UID: \"1c91c0c6-790f-4623-859f-e783574935ed\") " pod="e2e-models-e2e-worker-w4-85732c/e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd"
Sep 09 04:20:19.135840 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:20:19.135754    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c91c0c6-790f-4623-859f-e783574935ed-tls-certs\") pod \"e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd\" (UID: \"1c91c0c6-790f-4623-859f-e783574935ed\") " pod="e2e-models-e2e-worker-w4-85732c/e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd"
Sep 09 04:20:19.141509 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:20:19.141485    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-552h6\" (UniqueName: \"kubernetes.io/projected/1c91c0c6-790f-4623-859f-e783574935ed-kube-api-access-552h6\") pod \"e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd\" (UID: \"1c91c0c6-790f-4623-859f-e783574935ed\") " pod="e2e-models-e2e-worker-w4-85732c/e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd"
Sep 09 04:20:19.179345 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:20:19.179316    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-85732c/e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd"
Sep 09 04:20:19.450557 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:20:19.450531    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-85732c/e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd"]
Sep 09 04:20:19.452434 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:20:19.452404    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1c91c0c6_790f_4623_859f_e783574935ed.slice/crio-b86406f4583e701c4ff2bb8fd238ea4e27e07b9702656cf4fb2b9bdcf2662f23 WatchSource:0}: Error finding container b86406f4583e701c4ff2bb8fd238ea4e27e07b9702656cf4fb2b9bdcf2662f23: Status 404 returned error can't find the container with id b86406f4583e701c4ff2bb8fd238ea4e27e07b9702656cf4fb2b9bdcf2662f23
Sep 09 04:20:20.051280 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:20:20.051245    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-85732c/e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd" event={"ID":"1c91c0c6-790f-4623-859f-e783574935ed","Type":"ContainerStarted","Data":"2ff91e61458704bb1e6d08cc93716f05b83914fc8d847348a51f0c5527d93655"}
Sep 09 04:20:20.051280 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:20:20.051287    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-85732c/e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd" event={"ID":"1c91c0c6-790f-4623-859f-e783574935ed","Type":"ContainerStarted","Data":"b86406f4583e701c4ff2bb8fd238ea4e27e07b9702656cf4fb2b9bdcf2662f23"}
Sep 09 04:20:20.070381 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:20:20.070326    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-85732c/e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd" podStartSLOduration=2.070310645 podStartE2EDuration="2.070310645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:20:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:20:20.067697677 +0000 UTC m=+1344.986868586" watchObservedRunningTime="2026-09-09 04:20:20.070310645 +0000 UTC m=+1344.989481555"
Sep 09 04:20:29.179876 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:20:29.179836    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-85732c/e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd"
Sep 09 04:20:29.180390 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:20:29.179896    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-85732c/e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd"
Sep 09 04:20:29.192259 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:20:29.192236    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-85732c/e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd"
Sep 09 04:20:30.112003 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:20:30.111976    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-85732c/e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd"
Sep 09 04:21:47.053363 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:47.053329    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-85732c/e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd"]
Sep 09 04:21:55.704310 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:55.704228    2557 scope.go:117] "RemoveContainer" containerID="594102e46a7e49e1fcace5363e78fd593701267f3e69f6a7f6af936798cbe679"
Sep 09 04:21:57.379269 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:57.379235    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5"]
Sep 09 04:21:57.383681 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:57.383657    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5"
Sep 09 04:21:57.386537 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:57.386511    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-f95c96-serving-cert\""
Sep 09 04:21:57.394037 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:57.394007    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5"]
Sep 09 04:21:57.435402 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:57.435364    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0883b07-bdbf-4e45-885a-0a2c2a3072f3-openshift-service-ca\") pod \"maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5\" (UID: \"b0883b07-bdbf-4e45-885a-0a2c2a3072f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5"
Sep 09 04:21:57.435402 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:57.435405    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhq6p\" (UniqueName: \"kubernetes.io/projected/b0883b07-bdbf-4e45-885a-0a2c2a3072f3-kube-api-access-zhq6p\") pod \"maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5\" (UID: \"b0883b07-bdbf-4e45-885a-0a2c2a3072f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5"
Sep 09 04:21:57.435654 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:57.435432    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0883b07-bdbf-4e45-885a-0a2c2a3072f3-maas-api-tls\") pod \"maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5\" (UID: \"b0883b07-bdbf-4e45-885a-0a2c2a3072f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5"
Sep 09 04:21:57.435654 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:57.435564    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0883b07-bdbf-4e45-885a-0a2c2a3072f3-metrics-certs\") pod \"maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5\" (UID: \"b0883b07-bdbf-4e45-885a-0a2c2a3072f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5"
Sep 09 04:21:57.536166 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:57.536101    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0883b07-bdbf-4e45-885a-0a2c2a3072f3-openshift-service-ca\") pod \"maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5\" (UID: \"b0883b07-bdbf-4e45-885a-0a2c2a3072f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5"
Sep 09 04:21:57.536166 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:57.536162    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhq6p\" (UniqueName: \"kubernetes.io/projected/b0883b07-bdbf-4e45-885a-0a2c2a3072f3-kube-api-access-zhq6p\") pod \"maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5\" (UID: \"b0883b07-bdbf-4e45-885a-0a2c2a3072f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5"
Sep 09 04:21:57.536428 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:57.536201    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0883b07-bdbf-4e45-885a-0a2c2a3072f3-maas-api-tls\") pod \"maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5\" (UID: \"b0883b07-bdbf-4e45-885a-0a2c2a3072f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5"
Sep 09 04:21:57.536428 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:57.536264    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0883b07-bdbf-4e45-885a-0a2c2a3072f3-metrics-certs\") pod \"maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5\" (UID: \"b0883b07-bdbf-4e45-885a-0a2c2a3072f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5"
Sep 09 04:21:57.536428 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:21:57.536334    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-f95c96-serving-cert: secret "maas-api-e2e-api-a-f95c96-serving-cert" not found
Sep 09 04:21:57.536428 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:21:57.536402    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0883b07-bdbf-4e45-885a-0a2c2a3072f3-maas-api-tls podName:b0883b07-bdbf-4e45-885a-0a2c2a3072f3 nodeName:}" failed. No retries permitted until 2026-09-09 04:21:58.036383546 +0000 UTC m=+1442.955554434 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b0883b07-bdbf-4e45-885a-0a2c2a3072f3-maas-api-tls") pod "maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5" (UID: "b0883b07-bdbf-4e45-885a-0a2c2a3072f3") : secret "maas-api-e2e-api-a-f95c96-serving-cert" not found
Sep 09 04:21:57.536890 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:57.536866    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0883b07-bdbf-4e45-885a-0a2c2a3072f3-openshift-service-ca\") pod \"maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5\" (UID: \"b0883b07-bdbf-4e45-885a-0a2c2a3072f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5"
Sep 09 04:21:57.538719 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:57.538700    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0883b07-bdbf-4e45-885a-0a2c2a3072f3-metrics-certs\") pod \"maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5\" (UID: \"b0883b07-bdbf-4e45-885a-0a2c2a3072f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5"
Sep 09 04:21:57.544592 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:57.544568    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhq6p\" (UniqueName: \"kubernetes.io/projected/b0883b07-bdbf-4e45-885a-0a2c2a3072f3-kube-api-access-zhq6p\") pod \"maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5\" (UID: \"b0883b07-bdbf-4e45-885a-0a2c2a3072f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5"
Sep 09 04:21:58.041317 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:58.041280    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0883b07-bdbf-4e45-885a-0a2c2a3072f3-maas-api-tls\") pod \"maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5\" (UID: \"b0883b07-bdbf-4e45-885a-0a2c2a3072f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5"
Sep 09 04:21:58.043731 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:58.043703    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0883b07-bdbf-4e45-885a-0a2c2a3072f3-maas-api-tls\") pod \"maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5\" (UID: \"b0883b07-bdbf-4e45-885a-0a2c2a3072f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5"
Sep 09 04:21:58.064046 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:58.064011    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f95c96-5c7fd88458-dwfbv"]
Sep 09 04:21:58.067857 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:58.067835    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f95c96-5c7fd88458-dwfbv"
Sep 09 04:21:58.070652 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:58.070623    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-f95c96\""
Sep 09 04:21:58.080580 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:58.080551    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f95c96-5c7fd88458-dwfbv"]
Sep 09 04:21:58.089632 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:58.089601    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f95c96-7b99d7d7c5-wmw54"]
Sep 09 04:21:58.093154 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:58.093128    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-f95c96-7b99d7d7c5-wmw54"
Sep 09 04:21:58.096190 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:58.096164    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-f95c96-dockercfg-r62v6\""
Sep 09 04:21:58.108853 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:58.108817    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f95c96-7b99d7d7c5-wmw54"]
Sep 09 04:21:58.142757 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:58.142710    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmhnq\" (UniqueName: \"kubernetes.io/projected/ab367e3f-b00f-48ff-8c17-7d213b162cbb-kube-api-access-rmhnq\") pod \"payload-pre-processing-e2e-api-a-f95c96-5c7fd88458-dwfbv\" (UID: \"ab367e3f-b00f-48ff-8c17-7d213b162cbb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f95c96-5c7fd88458-dwfbv"
Sep 09 04:21:58.142937 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:58.142805    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab367e3f-b00f-48ff-8c17-7d213b162cbb-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-f95c96-5c7fd88458-dwfbv\" (UID: \"ab367e3f-b00f-48ff-8c17-7d213b162cbb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f95c96-5c7fd88458-dwfbv"
Sep 09 04:21:58.142937 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:58.142892    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1043d03c-a962-4f87-b4e4-64bb25b4f00b-plugins-config-volume\") pod \"payload-processing-e2e-api-a-f95c96-7b99d7d7c5-wmw54\" (UID: \"1043d03c-a962-4f87-b4e4-64bb25b4f00b\") " pod="openshift-ingress/payload-processing-e2e-api-a-f95c96-7b99d7d7c5-wmw54"
Sep 09 04:21:58.143017 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:58.142949    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phr8d\" (UniqueName: \"kubernetes.io/projected/1043d03c-a962-4f87-b4e4-64bb25b4f00b-kube-api-access-phr8d\") pod \"payload-processing-e2e-api-a-f95c96-7b99d7d7c5-wmw54\" (UID: \"1043d03c-a962-4f87-b4e4-64bb25b4f00b\") " pod="openshift-ingress/payload-processing-e2e-api-a-f95c96-7b99d7d7c5-wmw54"
Sep 09 04:21:58.244391 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:58.244342    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmhnq\" (UniqueName: \"kubernetes.io/projected/ab367e3f-b00f-48ff-8c17-7d213b162cbb-kube-api-access-rmhnq\") pod \"payload-pre-processing-e2e-api-a-f95c96-5c7fd88458-dwfbv\" (UID: \"ab367e3f-b00f-48ff-8c17-7d213b162cbb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f95c96-5c7fd88458-dwfbv"
Sep 09 04:21:58.244391 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:58.244398    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab367e3f-b00f-48ff-8c17-7d213b162cbb-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-f95c96-5c7fd88458-dwfbv\" (UID: \"ab367e3f-b00f-48ff-8c17-7d213b162cbb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f95c96-5c7fd88458-dwfbv"
Sep 09 04:21:58.244641 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:58.244459    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1043d03c-a962-4f87-b4e4-64bb25b4f00b-plugins-config-volume\") pod \"payload-processing-e2e-api-a-f95c96-7b99d7d7c5-wmw54\" (UID: \"1043d03c-a962-4f87-b4e4-64bb25b4f00b\") " pod="openshift-ingress/payload-processing-e2e-api-a-f95c96-7b99d7d7c5-wmw54"
Sep 09 04:21:58.244641 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:58.244501    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phr8d\" (UniqueName: \"kubernetes.io/projected/1043d03c-a962-4f87-b4e4-64bb25b4f00b-kube-api-access-phr8d\") pod \"payload-processing-e2e-api-a-f95c96-7b99d7d7c5-wmw54\" (UID: \"1043d03c-a962-4f87-b4e4-64bb25b4f00b\") " pod="openshift-ingress/payload-processing-e2e-api-a-f95c96-7b99d7d7c5-wmw54"
Sep 09 04:21:58.245179 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:58.245153    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab367e3f-b00f-48ff-8c17-7d213b162cbb-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-f95c96-5c7fd88458-dwfbv\" (UID: \"ab367e3f-b00f-48ff-8c17-7d213b162cbb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f95c96-5c7fd88458-dwfbv"
Sep 09 04:21:58.245317 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:58.245295    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1043d03c-a962-4f87-b4e4-64bb25b4f00b-plugins-config-volume\") pod \"payload-processing-e2e-api-a-f95c96-7b99d7d7c5-wmw54\" (UID: \"1043d03c-a962-4f87-b4e4-64bb25b4f00b\") " pod="openshift-ingress/payload-processing-e2e-api-a-f95c96-7b99d7d7c5-wmw54"
Sep 09 04:21:58.257932 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:58.257898    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phr8d\" (UniqueName: \"kubernetes.io/projected/1043d03c-a962-4f87-b4e4-64bb25b4f00b-kube-api-access-phr8d\") pod \"payload-processing-e2e-api-a-f95c96-7b99d7d7c5-wmw54\" (UID: \"1043d03c-a962-4f87-b4e4-64bb25b4f00b\") " pod="openshift-ingress/payload-processing-e2e-api-a-f95c96-7b99d7d7c5-wmw54"
Sep 09 04:21:58.258964 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:58.258942    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmhnq\" (UniqueName: \"kubernetes.io/projected/ab367e3f-b00f-48ff-8c17-7d213b162cbb-kube-api-access-rmhnq\") pod \"payload-pre-processing-e2e-api-a-f95c96-5c7fd88458-dwfbv\" (UID: \"ab367e3f-b00f-48ff-8c17-7d213b162cbb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f95c96-5c7fd88458-dwfbv"
Sep 09 04:21:58.296889 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:58.296786    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5"
Sep 09 04:21:58.379744 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:58.379688    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f95c96-5c7fd88458-dwfbv"
Sep 09 04:21:58.404177 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:58.404088    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-f95c96-7b99d7d7c5-wmw54"
Sep 09 04:21:58.571004 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:58.570976    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5"]
Sep 09 04:21:58.573359 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:21:58.573319    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0883b07_bdbf_4e45_885a_0a2c2a3072f3.slice/crio-13255549f87862c855de35bf37853b59e535089d5fa851162077ffcbdbf83997 WatchSource:0}: Error finding container 13255549f87862c855de35bf37853b59e535089d5fa851162077ffcbdbf83997: Status 404 returned error can't find the container with id 13255549f87862c855de35bf37853b59e535089d5fa851162077ffcbdbf83997
Sep 09 04:21:58.575103 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:58.575082    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 04:21:58.659079 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:58.659044    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f95c96-5c7fd88458-dwfbv"]
Sep 09 04:21:58.662590 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:21:58.662563    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab367e3f_b00f_48ff_8c17_7d213b162cbb.slice/crio-cd6da9a09b5b24782eb30ff4cd96ee2c8b81745697ab43a0a4e7c7c79895c663 WatchSource:0}: Error finding container cd6da9a09b5b24782eb30ff4cd96ee2c8b81745697ab43a0a4e7c7c79895c663: Status 404 returned error can't find the container with id cd6da9a09b5b24782eb30ff4cd96ee2c8b81745697ab43a0a4e7c7c79895c663
Sep 09 04:21:58.690261 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:58.690228    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f95c96-7b99d7d7c5-wmw54"]
Sep 09 04:21:58.702414 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:21:58.702359    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1043d03c_a962_4f87_b4e4_64bb25b4f00b.slice/crio-d4a5a7597ce9d7ff79482dafec5ef0d73290886ca81d6531d802eaa2f7669676 WatchSource:0}: Error finding container d4a5a7597ce9d7ff79482dafec5ef0d73290886ca81d6531d802eaa2f7669676: Status 404 returned error can't find the container with id d4a5a7597ce9d7ff79482dafec5ef0d73290886ca81d6531d802eaa2f7669676
Sep 09 04:21:59.540545 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:59.540437    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f95c96-5c7fd88458-dwfbv" event={"ID":"ab367e3f-b00f-48ff-8c17-7d213b162cbb","Type":"ContainerStarted","Data":"c7050b5852e5f4e7df4575c18798bcc83d7aca6674f138edfbb0662458630806"}
Sep 09 04:21:59.540545 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:59.540509    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f95c96-5c7fd88458-dwfbv"
Sep 09 04:21:59.540545 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:59.540532    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f95c96-5c7fd88458-dwfbv" event={"ID":"ab367e3f-b00f-48ff-8c17-7d213b162cbb","Type":"ContainerStarted","Data":"cd6da9a09b5b24782eb30ff4cd96ee2c8b81745697ab43a0a4e7c7c79895c663"}
Sep 09 04:21:59.542276 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:59.542247    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5" event={"ID":"b0883b07-bdbf-4e45-885a-0a2c2a3072f3","Type":"ContainerStarted","Data":"587a3e309de212f09d336093d680a3a8ebcb8110c16e193634bab333f5fe1567"}
Sep 09 04:21:59.542435 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:59.542304    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5"
Sep 09 04:21:59.542435 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:59.542328    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5" event={"ID":"b0883b07-bdbf-4e45-885a-0a2c2a3072f3","Type":"ContainerStarted","Data":"13255549f87862c855de35bf37853b59e535089d5fa851162077ffcbdbf83997"}
Sep 09 04:21:59.544023 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:59.543997    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-f95c96-7b99d7d7c5-wmw54" event={"ID":"1043d03c-a962-4f87-b4e4-64bb25b4f00b","Type":"ContainerStarted","Data":"0db0dc1c988cd0727bf91863968ea2d4db2ca26a0f26078a89f83c342d4bfad6"}
Sep 09 04:21:59.544156 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:59.544046    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-f95c96-7b99d7d7c5-wmw54" event={"ID":"1043d03c-a962-4f87-b4e4-64bb25b4f00b","Type":"ContainerStarted","Data":"d4a5a7597ce9d7ff79482dafec5ef0d73290886ca81d6531d802eaa2f7669676"}
Sep 09 04:21:59.544156 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:59.544081    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-f95c96-7b99d7d7c5-wmw54"
Sep 09 04:21:59.560388 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:59.560279    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f95c96-5c7fd88458-dwfbv" podStartSLOduration=1.560262061 podStartE2EDuration="1.560262061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:21:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:21:59.557761848 +0000 UTC m=+1444.476932759" watchObservedRunningTime="2026-09-09 04:21:59.560262061 +0000 UTC m=+1444.479432970"
Sep 09 04:21:59.577993 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:59.577938    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5" podStartSLOduration=2.038760696 podStartE2EDuration="2.577922435s" totalImagesPullingTime="539.161739ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:21:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:21:59.575795932 +0000 UTC m=+1444.494966843" watchObservedRunningTime="2026-09-09 04:21:59.577922435 +0000 UTC m=+1444.497093344"
Sep 09 04:21:59.593956 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:21:59.593890    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-f95c96-7b99d7d7c5-wmw54" podStartSLOduration=1.5938713629999999 podStartE2EDuration="1.593871363s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:21:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:21:59.591314316 +0000 UTC m=+1444.510485226" watchObservedRunningTime="2026-09-09 04:21:59.593871363 +0000 UTC m=+1444.513042274"
Sep 09 04:22:02.073996 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:02.073950    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-85732c/e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd" podUID="1c91c0c6-790f-4623-859f-e783574935ed" containerName="main" containerID="cri-o://2ff91e61458704bb1e6d08cc93716f05b83914fc8d847348a51f0c5527d93655" gracePeriod=45
Sep 09 04:22:02.341389 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:02.341359    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-85732c/e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd"
Sep 09 04:22:02.383991 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:02.383946    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c91c0c6-790f-4623-859f-e783574935ed-tmp-dir\") pod \"1c91c0c6-790f-4623-859f-e783574935ed\" (UID: \"1c91c0c6-790f-4623-859f-e783574935ed\") "
Sep 09 04:22:02.384215 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:02.384046    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c91c0c6-790f-4623-859f-e783574935ed-model-cache\") pod \"1c91c0c6-790f-4623-859f-e783574935ed\" (UID: \"1c91c0c6-790f-4623-859f-e783574935ed\") "
Sep 09 04:22:02.384215 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:02.384073    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-552h6\" (UniqueName: \"kubernetes.io/projected/1c91c0c6-790f-4623-859f-e783574935ed-kube-api-access-552h6\") pod \"1c91c0c6-790f-4623-859f-e783574935ed\" (UID: \"1c91c0c6-790f-4623-859f-e783574935ed\") "
Sep 09 04:22:02.384215 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:02.384106    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c91c0c6-790f-4623-859f-e783574935ed-home\") pod \"1c91c0c6-790f-4623-859f-e783574935ed\" (UID: \"1c91c0c6-790f-4623-859f-e783574935ed\") "
Sep 09 04:22:02.384215 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:02.384158    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c91c0c6-790f-4623-859f-e783574935ed-dshm\") pod \"1c91c0c6-790f-4623-859f-e783574935ed\" (UID: \"1c91c0c6-790f-4623-859f-e783574935ed\") "
Sep 09 04:22:02.384215 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:02.384206    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c91c0c6-790f-4623-859f-e783574935ed-tls-certs\") pod \"1c91c0c6-790f-4623-859f-e783574935ed\" (UID: \"1c91c0c6-790f-4623-859f-e783574935ed\") "
Sep 09 04:22:02.384512 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:02.384280    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c91c0c6-790f-4623-859f-e783574935ed-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1c91c0c6-790f-4623-859f-e783574935ed" (UID: "1c91c0c6-790f-4623-859f-e783574935ed"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 04:22:02.384512 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:02.384304    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c91c0c6-790f-4623-859f-e783574935ed-model-cache" (OuterVolumeSpecName: "model-cache") pod "1c91c0c6-790f-4623-859f-e783574935ed" (UID: "1c91c0c6-790f-4623-859f-e783574935ed"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 04:22:02.384620 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:02.384579    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c91c0c6-790f-4623-859f-e783574935ed-home" (OuterVolumeSpecName: "home") pod "1c91c0c6-790f-4623-859f-e783574935ed" (UID: "1c91c0c6-790f-4623-859f-e783574935ed"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 04:22:02.384620 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:02.384591    2557 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c91c0c6-790f-4623-859f-e783574935ed-tmp-dir\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:22:02.384620 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:02.384609    2557 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c91c0c6-790f-4623-859f-e783574935ed-model-cache\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:22:02.386481 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:02.386445    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c91c0c6-790f-4623-859f-e783574935ed-dshm" (OuterVolumeSpecName: "dshm") pod "1c91c0c6-790f-4623-859f-e783574935ed" (UID: "1c91c0c6-790f-4623-859f-e783574935ed"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 04:22:02.386872 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:02.386851    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c91c0c6-790f-4623-859f-e783574935ed-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1c91c0c6-790f-4623-859f-e783574935ed" (UID: "1c91c0c6-790f-4623-859f-e783574935ed"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 04:22:02.386940 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:02.386870    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c91c0c6-790f-4623-859f-e783574935ed-kube-api-access-552h6" (OuterVolumeSpecName: "kube-api-access-552h6") pod "1c91c0c6-790f-4623-859f-e783574935ed" (UID: "1c91c0c6-790f-4623-859f-e783574935ed"). InnerVolumeSpecName "kube-api-access-552h6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 04:22:02.485847 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:02.485806    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-552h6\" (UniqueName: \"kubernetes.io/projected/1c91c0c6-790f-4623-859f-e783574935ed-kube-api-access-552h6\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:22:02.485847 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:02.485841    2557 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c91c0c6-790f-4623-859f-e783574935ed-home\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:22:02.485847 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:02.485850    2557 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c91c0c6-790f-4623-859f-e783574935ed-dshm\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:22:02.486096 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:02.485860    2557 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c91c0c6-790f-4623-859f-e783574935ed-tls-certs\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:22:02.561077 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:02.561046    2557 generic.go:356] "Generic (PLEG): container finished" podID="1c91c0c6-790f-4623-859f-e783574935ed" containerID="2ff91e61458704bb1e6d08cc93716f05b83914fc8d847348a51f0c5527d93655" exitCode=0
Sep 09 04:22:02.561280 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:02.561087    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-85732c/e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd"
Sep 09 04:22:02.561280 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:02.561140    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-85732c/e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd" event={"ID":"1c91c0c6-790f-4623-859f-e783574935ed","Type":"ContainerDied","Data":"2ff91e61458704bb1e6d08cc93716f05b83914fc8d847348a51f0c5527d93655"}
Sep 09 04:22:02.561280 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:02.561189    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-85732c/e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd" event={"ID":"1c91c0c6-790f-4623-859f-e783574935ed","Type":"ContainerDied","Data":"b86406f4583e701c4ff2bb8fd238ea4e27e07b9702656cf4fb2b9bdcf2662f23"}
Sep 09 04:22:02.561280 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:02.561208    2557 scope.go:117] "RemoveContainer" containerID="2ff91e61458704bb1e6d08cc93716f05b83914fc8d847348a51f0c5527d93655"
Sep 09 04:22:02.570524 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:02.570503    2557 scope.go:117] "RemoveContainer" containerID="2ff91e61458704bb1e6d08cc93716f05b83914fc8d847348a51f0c5527d93655"
Sep 09 04:22:02.570822 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:22:02.570804    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ff91e61458704bb1e6d08cc93716f05b83914fc8d847348a51f0c5527d93655\": container with ID starting with 2ff91e61458704bb1e6d08cc93716f05b83914fc8d847348a51f0c5527d93655 not found: ID does not exist" containerID="2ff91e61458704bb1e6d08cc93716f05b83914fc8d847348a51f0c5527d93655"
Sep 09 04:22:02.570865 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:02.570831    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ff91e61458704bb1e6d08cc93716f05b83914fc8d847348a51f0c5527d93655"} err="failed to get container status \"2ff91e61458704bb1e6d08cc93716f05b83914fc8d847348a51f0c5527d93655\": rpc error: code = NotFound desc = could not find container \"2ff91e61458704bb1e6d08cc93716f05b83914fc8d847348a51f0c5527d93655\": container with ID starting with 2ff91e61458704bb1e6d08cc93716f05b83914fc8d847348a51f0c5527d93655 not found: ID does not exist"
Sep 09 04:22:02.583490 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:02.583416    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-85732c/e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd"]
Sep 09 04:22:02.589180 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:02.589152    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-85732c/e2e-conflict-a-4afe0483-kserve-bf8cfcf48-q6mgd"]
Sep 09 04:22:03.625321 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:03.621882    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c91c0c6-790f-4623-859f-e783574935ed" path="/var/lib/kubelet/pods/1c91c0c6-790f-4623-859f-e783574935ed/volumes"
Sep 09 04:22:05.555174 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:05.555139    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5"
Sep 09 04:22:10.551307 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:10.551270    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-f95c96-7b99d7d7c5-wmw54"
Sep 09 04:22:10.551696 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:10.551348    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f95c96-5c7fd88458-dwfbv"
Sep 09 04:22:27.351758 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:27.351728    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-acfe5b-68df5c4445-rhdx8"]
Sep 09 04:22:27.352466 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:27.352166    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c91c0c6-790f-4623-859f-e783574935ed" containerName="main"
Sep 09 04:22:27.352466 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:27.352178    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c91c0c6-790f-4623-859f-e783574935ed" containerName="main"
Sep 09 04:22:27.352466 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:27.352251    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c91c0c6-790f-4623-859f-e783574935ed" containerName="main"
Sep 09 04:22:27.355327 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:27.355310    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-acfe5b-68df5c4445-rhdx8"
Sep 09 04:22:27.357769 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:27.357749    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-acfe5b\""
Sep 09 04:22:27.358418 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:27.358226    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-acfe5b-dockercfg-qht8m\""
Sep 09 04:22:27.365241 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:27.365222    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-acfe5b-68df5c4445-rhdx8"]
Sep 09 04:22:27.421929 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:27.421895    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9d5j\" (UniqueName: \"kubernetes.io/projected/6e37c2a1-3b49-4d10-bc98-07aa794ace0a-kube-api-access-d9d5j\") pod \"payload-processing-e2e-api-b-acfe5b-68df5c4445-rhdx8\" (UID: \"6e37c2a1-3b49-4d10-bc98-07aa794ace0a\") " pod="openshift-ingress/payload-processing-e2e-api-b-acfe5b-68df5c4445-rhdx8"
Sep 09 04:22:27.422077 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:27.421935    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e37c2a1-3b49-4d10-bc98-07aa794ace0a-plugins-config-volume\") pod \"payload-processing-e2e-api-b-acfe5b-68df5c4445-rhdx8\" (UID: \"6e37c2a1-3b49-4d10-bc98-07aa794ace0a\") " pod="openshift-ingress/payload-processing-e2e-api-b-acfe5b-68df5c4445-rhdx8"
Sep 09 04:22:27.522776 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:27.522731    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9d5j\" (UniqueName: \"kubernetes.io/projected/6e37c2a1-3b49-4d10-bc98-07aa794ace0a-kube-api-access-d9d5j\") pod \"payload-processing-e2e-api-b-acfe5b-68df5c4445-rhdx8\" (UID: \"6e37c2a1-3b49-4d10-bc98-07aa794ace0a\") " pod="openshift-ingress/payload-processing-e2e-api-b-acfe5b-68df5c4445-rhdx8"
Sep 09 04:22:27.522776 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:27.522774    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e37c2a1-3b49-4d10-bc98-07aa794ace0a-plugins-config-volume\") pod \"payload-processing-e2e-api-b-acfe5b-68df5c4445-rhdx8\" (UID: \"6e37c2a1-3b49-4d10-bc98-07aa794ace0a\") " pod="openshift-ingress/payload-processing-e2e-api-b-acfe5b-68df5c4445-rhdx8"
Sep 09 04:22:27.523542 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:27.523520    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e37c2a1-3b49-4d10-bc98-07aa794ace0a-plugins-config-volume\") pod \"payload-processing-e2e-api-b-acfe5b-68df5c4445-rhdx8\" (UID: \"6e37c2a1-3b49-4d10-bc98-07aa794ace0a\") " pod="openshift-ingress/payload-processing-e2e-api-b-acfe5b-68df5c4445-rhdx8"
Sep 09 04:22:27.531915 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:27.531888    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9d5j\" (UniqueName: \"kubernetes.io/projected/6e37c2a1-3b49-4d10-bc98-07aa794ace0a-kube-api-access-d9d5j\") pod \"payload-processing-e2e-api-b-acfe5b-68df5c4445-rhdx8\" (UID: \"6e37c2a1-3b49-4d10-bc98-07aa794ace0a\") " pod="openshift-ingress/payload-processing-e2e-api-b-acfe5b-68df5c4445-rhdx8"
Sep 09 04:22:27.669240 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:27.669209    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-acfe5b-68df5c4445-rhdx8"
Sep 09 04:22:27.949300 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:27.949273    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-acfe5b-68df5c4445-rhdx8"]
Sep 09 04:22:27.950890 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:22:27.950860    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e37c2a1_3b49_4d10_bc98_07aa794ace0a.slice/crio-49cd76d53282ce11f4184e327aac2609a52c165d7d2d86355afa39201cea437b WatchSource:0}: Error finding container 49cd76d53282ce11f4184e327aac2609a52c165d7d2d86355afa39201cea437b: Status 404 returned error can't find the container with id 49cd76d53282ce11f4184e327aac2609a52c165d7d2d86355afa39201cea437b
Sep 09 04:22:28.707799 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:28.707758    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-acfe5b-68df5c4445-rhdx8" event={"ID":"6e37c2a1-3b49-4d10-bc98-07aa794ace0a","Type":"ContainerStarted","Data":"9f2f978d3f8ac3e37c440bed0282018d18e95101163c12254ba11c38d0a16303"}
Sep 09 04:22:28.708202 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:28.707824    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-acfe5b-68df5c4445-rhdx8" event={"ID":"6e37c2a1-3b49-4d10-bc98-07aa794ace0a","Type":"ContainerStarted","Data":"49cd76d53282ce11f4184e327aac2609a52c165d7d2d86355afa39201cea437b"}
Sep 09 04:22:28.708202 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:28.707907    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-acfe5b-68df5c4445-rhdx8"
Sep 09 04:22:28.725678 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:28.725632    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-acfe5b-68df5c4445-rhdx8" podStartSLOduration=1.7256175489999999 podStartE2EDuration="1.725617549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:22:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:22:28.724622057 +0000 UTC m=+1473.643792967" watchObservedRunningTime="2026-09-09 04:22:28.725617549 +0000 UTC m=+1473.644788459"
Sep 09 04:22:39.714613 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:39.714584    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-acfe5b-68df5c4445-rhdx8"
Sep 09 04:22:55.620233 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:55.620203    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gj7h7_7816cd49-e9c2-4f7b-b8ec-4e5711b675a5/console-operator/2.log"
Sep 09 04:22:55.621493 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:55.621469    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gj7h7_7816cd49-e9c2-4f7b-b8ec-4e5711b675a5/console-operator/2.log"
Sep 09 04:22:55.626033 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:55.626011    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rfvrc_1828d4d8-0d24-4e4d-a502-aee013a3b116/ovn-acl-logging/0.log"
Sep 09 04:22:55.627043 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:22:55.627022    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rfvrc_1828d4d8-0d24-4e4d-a502-aee013a3b116/ovn-acl-logging/0.log"
Sep 09 04:23:08.769852 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:08.769809    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5"]
Sep 09 04:23:08.770442 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:08.770086    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5" podUID="b0883b07-bdbf-4e45-885a-0a2c2a3072f3" containerName="maas-api" containerID="cri-o://587a3e309de212f09d336093d680a3a8ebcb8110c16e193634bab333f5fe1567" gracePeriod=30
Sep 09 04:23:08.820081 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:08.820047    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f95c96-7b99d7d7c5-wmw54"]
Sep 09 04:23:08.820377 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:08.820325    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-f95c96-7b99d7d7c5-wmw54" podUID="1043d03c-a962-4f87-b4e4-64bb25b4f00b" containerName="payload-processing" containerID="cri-o://0db0dc1c988cd0727bf91863968ea2d4db2ca26a0f26078a89f83c342d4bfad6" gracePeriod=30
Sep 09 04:23:08.826179 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:08.826155    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f95c96-5c7fd88458-dwfbv"]
Sep 09 04:23:08.826538 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:08.826401    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f95c96-5c7fd88458-dwfbv" podUID="ab367e3f-b00f-48ff-8c17-7d213b162cbb" containerName="payload-pre-processing" containerID="cri-o://c7050b5852e5f4e7df4575c18798bcc83d7aca6674f138edfbb0662458630806" gracePeriod=30
Sep 09 04:23:08.907994 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:08.907963    2557 generic.go:356] "Generic (PLEG): container finished" podID="b0883b07-bdbf-4e45-885a-0a2c2a3072f3" containerID="587a3e309de212f09d336093d680a3a8ebcb8110c16e193634bab333f5fe1567" exitCode=0
Sep 09 04:23:08.908127 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:08.908037    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5" event={"ID":"b0883b07-bdbf-4e45-885a-0a2c2a3072f3","Type":"ContainerDied","Data":"587a3e309de212f09d336093d680a3a8ebcb8110c16e193634bab333f5fe1567"}
Sep 09 04:23:09.105014 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.104982    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5"
Sep 09 04:23:09.128527 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.128489    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-f95c96-7b99d7d7c5-wmw54"
Sep 09 04:23:09.137377 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.137353    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f95c96-5c7fd88458-dwfbv"
Sep 09 04:23:09.203908 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.203875    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab367e3f-b00f-48ff-8c17-7d213b162cbb-plugins-config-volume\") pod \"ab367e3f-b00f-48ff-8c17-7d213b162cbb\" (UID: \"ab367e3f-b00f-48ff-8c17-7d213b162cbb\") "
Sep 09 04:23:09.204079 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.203919    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1043d03c-a962-4f87-b4e4-64bb25b4f00b-plugins-config-volume\") pod \"1043d03c-a962-4f87-b4e4-64bb25b4f00b\" (UID: \"1043d03c-a962-4f87-b4e4-64bb25b4f00b\") "
Sep 09 04:23:09.204079 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.203960    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0883b07-bdbf-4e45-885a-0a2c2a3072f3-openshift-service-ca\") pod \"b0883b07-bdbf-4e45-885a-0a2c2a3072f3\" (UID: \"b0883b07-bdbf-4e45-885a-0a2c2a3072f3\") "
Sep 09 04:23:09.204079 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.203978    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0883b07-bdbf-4e45-885a-0a2c2a3072f3-metrics-certs\") pod \"b0883b07-bdbf-4e45-885a-0a2c2a3072f3\" (UID: \"b0883b07-bdbf-4e45-885a-0a2c2a3072f3\") "
Sep 09 04:23:09.204275 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.204101    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phr8d\" (UniqueName: \"kubernetes.io/projected/1043d03c-a962-4f87-b4e4-64bb25b4f00b-kube-api-access-phr8d\") pod \"1043d03c-a962-4f87-b4e4-64bb25b4f00b\" (UID: \"1043d03c-a962-4f87-b4e4-64bb25b4f00b\") "
Sep 09 04:23:09.204275 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.204156    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rmhnq\" (UniqueName: \"kubernetes.io/projected/ab367e3f-b00f-48ff-8c17-7d213b162cbb-kube-api-access-rmhnq\") pod \"ab367e3f-b00f-48ff-8c17-7d213b162cbb\" (UID: \"ab367e3f-b00f-48ff-8c17-7d213b162cbb\") "
Sep 09 04:23:09.204275 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.204225    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0883b07-bdbf-4e45-885a-0a2c2a3072f3-maas-api-tls\") pod \"b0883b07-bdbf-4e45-885a-0a2c2a3072f3\" (UID: \"b0883b07-bdbf-4e45-885a-0a2c2a3072f3\") "
Sep 09 04:23:09.204275 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.204261    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zhq6p\" (UniqueName: \"kubernetes.io/projected/b0883b07-bdbf-4e45-885a-0a2c2a3072f3-kube-api-access-zhq6p\") pod \"b0883b07-bdbf-4e45-885a-0a2c2a3072f3\" (UID: \"b0883b07-bdbf-4e45-885a-0a2c2a3072f3\") "
Sep 09 04:23:09.204485 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.204359    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0883b07-bdbf-4e45-885a-0a2c2a3072f3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b0883b07-bdbf-4e45-885a-0a2c2a3072f3" (UID: "b0883b07-bdbf-4e45-885a-0a2c2a3072f3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 04:23:09.204485 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.204383    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1043d03c-a962-4f87-b4e4-64bb25b4f00b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1043d03c-a962-4f87-b4e4-64bb25b4f00b" (UID: "1043d03c-a962-4f87-b4e4-64bb25b4f00b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 04:23:09.204485 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.204405    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab367e3f-b00f-48ff-8c17-7d213b162cbb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ab367e3f-b00f-48ff-8c17-7d213b162cbb" (UID: "ab367e3f-b00f-48ff-8c17-7d213b162cbb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 04:23:09.204727 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.204666    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab367e3f-b00f-48ff-8c17-7d213b162cbb-plugins-config-volume\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:23:09.204727 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.204692    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1043d03c-a962-4f87-b4e4-64bb25b4f00b-plugins-config-volume\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:23:09.204727 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.204706    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0883b07-bdbf-4e45-885a-0a2c2a3072f3-openshift-service-ca\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:23:09.206763 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.206735    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0883b07-bdbf-4e45-885a-0a2c2a3072f3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b0883b07-bdbf-4e45-885a-0a2c2a3072f3" (UID: "b0883b07-bdbf-4e45-885a-0a2c2a3072f3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 04:23:09.206856 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.206737    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1043d03c-a962-4f87-b4e4-64bb25b4f00b-kube-api-access-phr8d" (OuterVolumeSpecName: "kube-api-access-phr8d") pod "1043d03c-a962-4f87-b4e4-64bb25b4f00b" (UID: "1043d03c-a962-4f87-b4e4-64bb25b4f00b"). InnerVolumeSpecName "kube-api-access-phr8d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 04:23:09.206856 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.206803    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0883b07-bdbf-4e45-885a-0a2c2a3072f3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b0883b07-bdbf-4e45-885a-0a2c2a3072f3" (UID: "b0883b07-bdbf-4e45-885a-0a2c2a3072f3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 04:23:09.206972 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.206878    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0883b07-bdbf-4e45-885a-0a2c2a3072f3-kube-api-access-zhq6p" (OuterVolumeSpecName: "kube-api-access-zhq6p") pod "b0883b07-bdbf-4e45-885a-0a2c2a3072f3" (UID: "b0883b07-bdbf-4e45-885a-0a2c2a3072f3"). InnerVolumeSpecName "kube-api-access-zhq6p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 04:23:09.207167 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.207142    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab367e3f-b00f-48ff-8c17-7d213b162cbb-kube-api-access-rmhnq" (OuterVolumeSpecName: "kube-api-access-rmhnq") pod "ab367e3f-b00f-48ff-8c17-7d213b162cbb" (UID: "ab367e3f-b00f-48ff-8c17-7d213b162cbb"). InnerVolumeSpecName "kube-api-access-rmhnq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 04:23:09.305464 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.305382    2557 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0883b07-bdbf-4e45-885a-0a2c2a3072f3-metrics-certs\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:23:09.305464 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.305414    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phr8d\" (UniqueName: \"kubernetes.io/projected/1043d03c-a962-4f87-b4e4-64bb25b4f00b-kube-api-access-phr8d\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:23:09.305464 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.305430    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmhnq\" (UniqueName: \"kubernetes.io/projected/ab367e3f-b00f-48ff-8c17-7d213b162cbb-kube-api-access-rmhnq\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:23:09.305464 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.305442    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0883b07-bdbf-4e45-885a-0a2c2a3072f3-maas-api-tls\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:23:09.305464 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.305453    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhq6p\" (UniqueName: \"kubernetes.io/projected/b0883b07-bdbf-4e45-885a-0a2c2a3072f3-kube-api-access-zhq6p\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:23:09.919999 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.919911    2557 generic.go:356] "Generic (PLEG): container finished" podID="ab367e3f-b00f-48ff-8c17-7d213b162cbb" containerID="c7050b5852e5f4e7df4575c18798bcc83d7aca6674f138edfbb0662458630806" exitCode=0
Sep 09 04:23:09.919999 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.919964    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f95c96-5c7fd88458-dwfbv"
Sep 09 04:23:09.920526 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.919992    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f95c96-5c7fd88458-dwfbv" event={"ID":"ab367e3f-b00f-48ff-8c17-7d213b162cbb","Type":"ContainerDied","Data":"c7050b5852e5f4e7df4575c18798bcc83d7aca6674f138edfbb0662458630806"}
Sep 09 04:23:09.920526 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.920073    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f95c96-5c7fd88458-dwfbv" event={"ID":"ab367e3f-b00f-48ff-8c17-7d213b162cbb","Type":"ContainerDied","Data":"cd6da9a09b5b24782eb30ff4cd96ee2c8b81745697ab43a0a4e7c7c79895c663"}
Sep 09 04:23:09.920526 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.920097    2557 scope.go:117] "RemoveContainer" containerID="c7050b5852e5f4e7df4575c18798bcc83d7aca6674f138edfbb0662458630806"
Sep 09 04:23:09.921687 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.921663    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5" event={"ID":"b0883b07-bdbf-4e45-885a-0a2c2a3072f3","Type":"ContainerDied","Data":"13255549f87862c855de35bf37853b59e535089d5fa851162077ffcbdbf83997"}
Sep 09 04:23:09.921687 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.921677    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5"
Sep 09 04:23:09.923164 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.923138    2557 generic.go:356] "Generic (PLEG): container finished" podID="1043d03c-a962-4f87-b4e4-64bb25b4f00b" containerID="0db0dc1c988cd0727bf91863968ea2d4db2ca26a0f26078a89f83c342d4bfad6" exitCode=0
Sep 09 04:23:09.923272 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.923188    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-f95c96-7b99d7d7c5-wmw54"
Sep 09 04:23:09.923272 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.923212    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-f95c96-7b99d7d7c5-wmw54" event={"ID":"1043d03c-a962-4f87-b4e4-64bb25b4f00b","Type":"ContainerDied","Data":"0db0dc1c988cd0727bf91863968ea2d4db2ca26a0f26078a89f83c342d4bfad6"}
Sep 09 04:23:09.923272 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.923248    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-f95c96-7b99d7d7c5-wmw54" event={"ID":"1043d03c-a962-4f87-b4e4-64bb25b4f00b","Type":"ContainerDied","Data":"d4a5a7597ce9d7ff79482dafec5ef0d73290886ca81d6531d802eaa2f7669676"}
Sep 09 04:23:09.929766 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.929747    2557 scope.go:117] "RemoveContainer" containerID="c7050b5852e5f4e7df4575c18798bcc83d7aca6674f138edfbb0662458630806"
Sep 09 04:23:09.930040 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:23:09.930020    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7050b5852e5f4e7df4575c18798bcc83d7aca6674f138edfbb0662458630806\": container with ID starting with c7050b5852e5f4e7df4575c18798bcc83d7aca6674f138edfbb0662458630806 not found: ID does not exist" containerID="c7050b5852e5f4e7df4575c18798bcc83d7aca6674f138edfbb0662458630806"
Sep 09 04:23:09.930244 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.930047    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7050b5852e5f4e7df4575c18798bcc83d7aca6674f138edfbb0662458630806"} err="failed to get container status \"c7050b5852e5f4e7df4575c18798bcc83d7aca6674f138edfbb0662458630806\": rpc error: code = NotFound desc = could not find container \"c7050b5852e5f4e7df4575c18798bcc83d7aca6674f138edfbb0662458630806\": container with ID starting with c7050b5852e5f4e7df4575c18798bcc83d7aca6674f138edfbb0662458630806 not found: ID does not exist"
Sep 09 04:23:09.930244 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.930062    2557 scope.go:117] "RemoveContainer" containerID="587a3e309de212f09d336093d680a3a8ebcb8110c16e193634bab333f5fe1567"
Sep 09 04:23:09.937919 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.937898    2557 scope.go:117] "RemoveContainer" containerID="0db0dc1c988cd0727bf91863968ea2d4db2ca26a0f26078a89f83c342d4bfad6"
Sep 09 04:23:09.944432 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.944409    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5"]
Sep 09 04:23:09.946483 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.946462    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f95c96-55bd5b8f9b-kvlw5"]
Sep 09 04:23:09.947047 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.947028    2557 scope.go:117] "RemoveContainer" containerID="0db0dc1c988cd0727bf91863968ea2d4db2ca26a0f26078a89f83c342d4bfad6"
Sep 09 04:23:09.947326 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:23:09.947309    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0db0dc1c988cd0727bf91863968ea2d4db2ca26a0f26078a89f83c342d4bfad6\": container with ID starting with 0db0dc1c988cd0727bf91863968ea2d4db2ca26a0f26078a89f83c342d4bfad6 not found: ID does not exist" containerID="0db0dc1c988cd0727bf91863968ea2d4db2ca26a0f26078a89f83c342d4bfad6"
Sep 09 04:23:09.947390 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.947335    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0db0dc1c988cd0727bf91863968ea2d4db2ca26a0f26078a89f83c342d4bfad6"} err="failed to get container status \"0db0dc1c988cd0727bf91863968ea2d4db2ca26a0f26078a89f83c342d4bfad6\": rpc error: code = NotFound desc = could not find container \"0db0dc1c988cd0727bf91863968ea2d4db2ca26a0f26078a89f83c342d4bfad6\": container with ID starting with 0db0dc1c988cd0727bf91863968ea2d4db2ca26a0f26078a89f83c342d4bfad6 not found: ID does not exist"
Sep 09 04:23:09.955976 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.955953    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f95c96-5c7fd88458-dwfbv"]
Sep 09 04:23:09.962658 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.962639    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f95c96-5c7fd88458-dwfbv"]
Sep 09 04:23:09.977690 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.977672    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f95c96-7b99d7d7c5-wmw54"]
Sep 09 04:23:09.981609 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:09.981589    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f95c96-7b99d7d7c5-wmw54"]
Sep 09 04:23:11.619883 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:11.619850    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1043d03c-a962-4f87-b4e4-64bb25b4f00b" path="/var/lib/kubelet/pods/1043d03c-a962-4f87-b4e4-64bb25b4f00b/volumes"
Sep 09 04:23:11.620279 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:11.620222    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab367e3f-b00f-48ff-8c17-7d213b162cbb" path="/var/lib/kubelet/pods/ab367e3f-b00f-48ff-8c17-7d213b162cbb/volumes"
Sep 09 04:23:11.620579 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:11.620566    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0883b07-bdbf-4e45-885a-0a2c2a3072f3" path="/var/lib/kubelet/pods/b0883b07-bdbf-4e45-885a-0a2c2a3072f3/volumes"
Sep 09 04:23:31.707650 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:31.707569    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-acfe5b-68df5c4445-rhdx8"]
Sep 09 04:23:31.708610 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:31.707781    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-acfe5b-68df5c4445-rhdx8" podUID="6e37c2a1-3b49-4d10-bc98-07aa794ace0a" containerName="payload-processing" containerID="cri-o://9f2f978d3f8ac3e37c440bed0282018d18e95101163c12254ba11c38d0a16303" gracePeriod=30
Sep 09 04:23:31.961587 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:31.961533    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-acfe5b-68df5c4445-rhdx8"
Sep 09 04:23:32.006915 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:32.006885    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e37c2a1-3b49-4d10-bc98-07aa794ace0a-plugins-config-volume\") pod \"6e37c2a1-3b49-4d10-bc98-07aa794ace0a\" (UID: \"6e37c2a1-3b49-4d10-bc98-07aa794ace0a\") "
Sep 09 04:23:32.007048 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:32.006954    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9d5j\" (UniqueName: \"kubernetes.io/projected/6e37c2a1-3b49-4d10-bc98-07aa794ace0a-kube-api-access-d9d5j\") pod \"6e37c2a1-3b49-4d10-bc98-07aa794ace0a\" (UID: \"6e37c2a1-3b49-4d10-bc98-07aa794ace0a\") "
Sep 09 04:23:32.007345 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:32.007312    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e37c2a1-3b49-4d10-bc98-07aa794ace0a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6e37c2a1-3b49-4d10-bc98-07aa794ace0a" (UID: "6e37c2a1-3b49-4d10-bc98-07aa794ace0a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 04:23:32.008970 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:32.008947    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e37c2a1-3b49-4d10-bc98-07aa794ace0a-kube-api-access-d9d5j" (OuterVolumeSpecName: "kube-api-access-d9d5j") pod "6e37c2a1-3b49-4d10-bc98-07aa794ace0a" (UID: "6e37c2a1-3b49-4d10-bc98-07aa794ace0a"). InnerVolumeSpecName "kube-api-access-d9d5j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 04:23:32.039596 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:32.039565    2557 generic.go:356] "Generic (PLEG): container finished" podID="6e37c2a1-3b49-4d10-bc98-07aa794ace0a" containerID="9f2f978d3f8ac3e37c440bed0282018d18e95101163c12254ba11c38d0a16303" exitCode=0
Sep 09 04:23:32.039762 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:32.039614    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-acfe5b-68df5c4445-rhdx8"
Sep 09 04:23:32.039762 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:32.039643    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-acfe5b-68df5c4445-rhdx8" event={"ID":"6e37c2a1-3b49-4d10-bc98-07aa794ace0a","Type":"ContainerDied","Data":"9f2f978d3f8ac3e37c440bed0282018d18e95101163c12254ba11c38d0a16303"}
Sep 09 04:23:32.039762 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:32.039697    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-acfe5b-68df5c4445-rhdx8" event={"ID":"6e37c2a1-3b49-4d10-bc98-07aa794ace0a","Type":"ContainerDied","Data":"49cd76d53282ce11f4184e327aac2609a52c165d7d2d86355afa39201cea437b"}
Sep 09 04:23:32.039762 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:32.039719    2557 scope.go:117] "RemoveContainer" containerID="9f2f978d3f8ac3e37c440bed0282018d18e95101163c12254ba11c38d0a16303"
Sep 09 04:23:32.049217 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:32.049200    2557 scope.go:117] "RemoveContainer" containerID="9f2f978d3f8ac3e37c440bed0282018d18e95101163c12254ba11c38d0a16303"
Sep 09 04:23:32.049456 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:23:32.049439    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f2f978d3f8ac3e37c440bed0282018d18e95101163c12254ba11c38d0a16303\": container with ID starting with 9f2f978d3f8ac3e37c440bed0282018d18e95101163c12254ba11c38d0a16303 not found: ID does not exist" containerID="9f2f978d3f8ac3e37c440bed0282018d18e95101163c12254ba11c38d0a16303"
Sep 09 04:23:32.049506 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:32.049466    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f2f978d3f8ac3e37c440bed0282018d18e95101163c12254ba11c38d0a16303"} err="failed to get container status \"9f2f978d3f8ac3e37c440bed0282018d18e95101163c12254ba11c38d0a16303\": rpc error: code = NotFound desc = could not find container \"9f2f978d3f8ac3e37c440bed0282018d18e95101163c12254ba11c38d0a16303\": container with ID starting with 9f2f978d3f8ac3e37c440bed0282018d18e95101163c12254ba11c38d0a16303 not found: ID does not exist"
Sep 09 04:23:32.061850 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:32.061816    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-acfe5b-68df5c4445-rhdx8"]
Sep 09 04:23:32.065821 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:32.065799    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-acfe5b-68df5c4445-rhdx8"]
Sep 09 04:23:32.107768 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:32.107742    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e37c2a1-3b49-4d10-bc98-07aa794ace0a-plugins-config-volume\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:23:32.107768 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:32.107764    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9d5j\" (UniqueName: \"kubernetes.io/projected/6e37c2a1-3b49-4d10-bc98-07aa794ace0a-kube-api-access-d9d5j\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:23:33.620236 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:33.620205    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e37c2a1-3b49-4d10-bc98-07aa794ace0a" path="/var/lib/kubelet/pods/6e37c2a1-3b49-4d10-bc98-07aa794ace0a/volumes"
Sep 09 04:23:50.582279 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:50.582223    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-46ddac-77d99dd7cc-lt9mh"]
Sep 09 04:23:50.583424 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:50.582484    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-46ddac-77d99dd7cc-lt9mh" podUID="b655b0f5-4908-4030-9a69-a4f5aaa83d0d" containerName="payload-processing" containerID="cri-o://dfc06199d84f4aeb6f9399f4c9651b38bb110f59ddbd9e49089a944eb4d3f5ea" gracePeriod=30
Sep 09 04:23:50.848837 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:50.848815    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-46ddac-77d99dd7cc-lt9mh"
Sep 09 04:23:50.974401 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:50.974371    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nfhz8\" (UniqueName: \"kubernetes.io/projected/b655b0f5-4908-4030-9a69-a4f5aaa83d0d-kube-api-access-nfhz8\") pod \"b655b0f5-4908-4030-9a69-a4f5aaa83d0d\" (UID: \"b655b0f5-4908-4030-9a69-a4f5aaa83d0d\") "
Sep 09 04:23:50.974578 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:50.974433    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b655b0f5-4908-4030-9a69-a4f5aaa83d0d-plugins-config-volume\") pod \"b655b0f5-4908-4030-9a69-a4f5aaa83d0d\" (UID: \"b655b0f5-4908-4030-9a69-a4f5aaa83d0d\") "
Sep 09 04:23:50.974800 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:50.974773    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b655b0f5-4908-4030-9a69-a4f5aaa83d0d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b655b0f5-4908-4030-9a69-a4f5aaa83d0d" (UID: "b655b0f5-4908-4030-9a69-a4f5aaa83d0d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 04:23:50.976475 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:50.976450    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b655b0f5-4908-4030-9a69-a4f5aaa83d0d-kube-api-access-nfhz8" (OuterVolumeSpecName: "kube-api-access-nfhz8") pod "b655b0f5-4908-4030-9a69-a4f5aaa83d0d" (UID: "b655b0f5-4908-4030-9a69-a4f5aaa83d0d"). InnerVolumeSpecName "kube-api-access-nfhz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 04:23:51.076046 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:51.076021    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nfhz8\" (UniqueName: \"kubernetes.io/projected/b655b0f5-4908-4030-9a69-a4f5aaa83d0d-kube-api-access-nfhz8\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:23:51.076046 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:51.076043    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b655b0f5-4908-4030-9a69-a4f5aaa83d0d-plugins-config-volume\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:23:51.134589 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:51.134556    2557 generic.go:356] "Generic (PLEG): container finished" podID="b655b0f5-4908-4030-9a69-a4f5aaa83d0d" containerID="dfc06199d84f4aeb6f9399f4c9651b38bb110f59ddbd9e49089a944eb4d3f5ea" exitCode=0
Sep 09 04:23:51.134817 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:51.134599    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-46ddac-77d99dd7cc-lt9mh"
Sep 09 04:23:51.134817 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:51.134648    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-46ddac-77d99dd7cc-lt9mh" event={"ID":"b655b0f5-4908-4030-9a69-a4f5aaa83d0d","Type":"ContainerDied","Data":"dfc06199d84f4aeb6f9399f4c9651b38bb110f59ddbd9e49089a944eb4d3f5ea"}
Sep 09 04:23:51.134817 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:51.134695    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-46ddac-77d99dd7cc-lt9mh" event={"ID":"b655b0f5-4908-4030-9a69-a4f5aaa83d0d","Type":"ContainerDied","Data":"efeca21438dce3d28e6881ee0aaa848b9a93699ba87f21909045520a56e7b53a"}
Sep 09 04:23:51.134817 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:51.134723    2557 scope.go:117] "RemoveContainer" containerID="dfc06199d84f4aeb6f9399f4c9651b38bb110f59ddbd9e49089a944eb4d3f5ea"
Sep 09 04:23:51.143965 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:51.143945    2557 scope.go:117] "RemoveContainer" containerID="dfc06199d84f4aeb6f9399f4c9651b38bb110f59ddbd9e49089a944eb4d3f5ea"
Sep 09 04:23:51.144223 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:23:51.144205    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfc06199d84f4aeb6f9399f4c9651b38bb110f59ddbd9e49089a944eb4d3f5ea\": container with ID starting with dfc06199d84f4aeb6f9399f4c9651b38bb110f59ddbd9e49089a944eb4d3f5ea not found: ID does not exist" containerID="dfc06199d84f4aeb6f9399f4c9651b38bb110f59ddbd9e49089a944eb4d3f5ea"
Sep 09 04:23:51.144275 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:51.144232    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfc06199d84f4aeb6f9399f4c9651b38bb110f59ddbd9e49089a944eb4d3f5ea"} err="failed to get container status \"dfc06199d84f4aeb6f9399f4c9651b38bb110f59ddbd9e49089a944eb4d3f5ea\": rpc error: code = NotFound desc = could not find container \"dfc06199d84f4aeb6f9399f4c9651b38bb110f59ddbd9e49089a944eb4d3f5ea\": container with ID starting with dfc06199d84f4aeb6f9399f4c9651b38bb110f59ddbd9e49089a944eb4d3f5ea not found: ID does not exist"
Sep 09 04:23:51.155863 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:51.155843    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-46ddac-77d99dd7cc-lt9mh"]
Sep 09 04:23:51.159355 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:51.159334    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-46ddac-77d99dd7cc-lt9mh"]
Sep 09 04:23:51.620226 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:23:51.620196    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b655b0f5-4908-4030-9a69-a4f5aaa83d0d" path="/var/lib/kubelet/pods/b655b0f5-4908-4030-9a69-a4f5aaa83d0d/volumes"
Sep 09 04:24:06.645845 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:06.645809    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-85732c/distinct-2-w4-85732c-kserve-7f96669d6b-764xk"]
Sep 09 04:24:06.656123 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:06.654319    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-85732c/distinct-w4-85732c-kserve-654fb7bb89-cfhns"]
Sep 09 04:24:08.931203 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:08.931171    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf"]
Sep 09 04:24:08.931586 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:08.931383    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf" podUID="ce66729c-0dda-43cb-9365-98bc06b1e6da" containerName="maas-api" containerID="cri-o://8e4686487bb368a84a521ef54e8a1b02c4197a08c65e1537d165603aeafb30fb" gracePeriod=30
Sep 09 04:24:08.991146 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:08.991053    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f2be5-7bf7bf47b6-wpzh4"]
Sep 09 04:24:08.991322 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:08.991302    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f2be5-7bf7bf47b6-wpzh4" podUID="03fc36b8-e935-438e-a6d8-b7ffe4c07a5f" containerName="payload-pre-processing" containerID="cri-o://382c4a7aaf6ca6a5a4da6235307dac61173fe62b1584c72e7fd00a436900bbd0" gracePeriod=30
Sep 09 04:24:09.224253 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:09.224218    2557 generic.go:356] "Generic (PLEG): container finished" podID="03fc36b8-e935-438e-a6d8-b7ffe4c07a5f" containerID="382c4a7aaf6ca6a5a4da6235307dac61173fe62b1584c72e7fd00a436900bbd0" exitCode=0
Sep 09 04:24:09.224444 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:09.224391    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f2be5-7bf7bf47b6-wpzh4" event={"ID":"03fc36b8-e935-438e-a6d8-b7ffe4c07a5f","Type":"ContainerDied","Data":"382c4a7aaf6ca6a5a4da6235307dac61173fe62b1584c72e7fd00a436900bbd0"}
Sep 09 04:24:09.228429 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:09.227574    2557 generic.go:356] "Generic (PLEG): container finished" podID="ce66729c-0dda-43cb-9365-98bc06b1e6da" containerID="8e4686487bb368a84a521ef54e8a1b02c4197a08c65e1537d165603aeafb30fb" exitCode=0
Sep 09 04:24:09.228429 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:09.227648    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf" event={"ID":"ce66729c-0dda-43cb-9365-98bc06b1e6da","Type":"ContainerDied","Data":"8e4686487bb368a84a521ef54e8a1b02c4197a08c65e1537d165603aeafb30fb"}
Sep 09 04:24:09.248882 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:09.248822    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf"
Sep 09 04:24:09.279072 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:09.279044    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f2be5-7bf7bf47b6-wpzh4"
Sep 09 04:24:09.334157 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:09.334104    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce66729c-0dda-43cb-9365-98bc06b1e6da-openshift-service-ca\") pod \"ce66729c-0dda-43cb-9365-98bc06b1e6da\" (UID: \"ce66729c-0dda-43cb-9365-98bc06b1e6da\") "
Sep 09 04:24:09.334339 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:09.334167    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce66729c-0dda-43cb-9365-98bc06b1e6da-maas-api-tls\") pod \"ce66729c-0dda-43cb-9365-98bc06b1e6da\" (UID: \"ce66729c-0dda-43cb-9365-98bc06b1e6da\") "
Sep 09 04:24:09.334339 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:09.334190    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pbkf8\" (UniqueName: \"kubernetes.io/projected/ce66729c-0dda-43cb-9365-98bc06b1e6da-kube-api-access-pbkf8\") pod \"ce66729c-0dda-43cb-9365-98bc06b1e6da\" (UID: \"ce66729c-0dda-43cb-9365-98bc06b1e6da\") "
Sep 09 04:24:09.334339 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:09.334247    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce66729c-0dda-43cb-9365-98bc06b1e6da-metrics-certs\") pod \"ce66729c-0dda-43cb-9365-98bc06b1e6da\" (UID: \"ce66729c-0dda-43cb-9365-98bc06b1e6da\") "
Sep 09 04:24:09.334525 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:09.334495    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce66729c-0dda-43cb-9365-98bc06b1e6da-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ce66729c-0dda-43cb-9365-98bc06b1e6da" (UID: "ce66729c-0dda-43cb-9365-98bc06b1e6da"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 04:24:09.336345 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:09.336316    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce66729c-0dda-43cb-9365-98bc06b1e6da-kube-api-access-pbkf8" (OuterVolumeSpecName: "kube-api-access-pbkf8") pod "ce66729c-0dda-43cb-9365-98bc06b1e6da" (UID: "ce66729c-0dda-43cb-9365-98bc06b1e6da"). InnerVolumeSpecName "kube-api-access-pbkf8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 04:24:09.336438 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:09.336358    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce66729c-0dda-43cb-9365-98bc06b1e6da-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ce66729c-0dda-43cb-9365-98bc06b1e6da" (UID: "ce66729c-0dda-43cb-9365-98bc06b1e6da"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 04:24:09.336438 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:09.336374    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce66729c-0dda-43cb-9365-98bc06b1e6da-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ce66729c-0dda-43cb-9365-98bc06b1e6da" (UID: "ce66729c-0dda-43cb-9365-98bc06b1e6da"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 04:24:09.435514 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:09.435484    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03fc36b8-e935-438e-a6d8-b7ffe4c07a5f-plugins-config-volume\") pod \"03fc36b8-e935-438e-a6d8-b7ffe4c07a5f\" (UID: \"03fc36b8-e935-438e-a6d8-b7ffe4c07a5f\") "
Sep 09 04:24:09.435689 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:09.435536    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2bxhs\" (UniqueName: \"kubernetes.io/projected/03fc36b8-e935-438e-a6d8-b7ffe4c07a5f-kube-api-access-2bxhs\") pod \"03fc36b8-e935-438e-a6d8-b7ffe4c07a5f\" (UID: \"03fc36b8-e935-438e-a6d8-b7ffe4c07a5f\") "
Sep 09 04:24:09.435771 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:09.435753    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce66729c-0dda-43cb-9365-98bc06b1e6da-openshift-service-ca\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:24:09.435829 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:09.435777    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce66729c-0dda-43cb-9365-98bc06b1e6da-maas-api-tls\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:24:09.435829 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:09.435792    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pbkf8\" (UniqueName: \"kubernetes.io/projected/ce66729c-0dda-43cb-9365-98bc06b1e6da-kube-api-access-pbkf8\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:24:09.435829 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:09.435807    2557 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce66729c-0dda-43cb-9365-98bc06b1e6da-metrics-certs\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:24:09.435829 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:09.435807    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03fc36b8-e935-438e-a6d8-b7ffe4c07a5f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "03fc36b8-e935-438e-a6d8-b7ffe4c07a5f" (UID: "03fc36b8-e935-438e-a6d8-b7ffe4c07a5f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 04:24:09.437685 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:09.437662    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03fc36b8-e935-438e-a6d8-b7ffe4c07a5f-kube-api-access-2bxhs" (OuterVolumeSpecName: "kube-api-access-2bxhs") pod "03fc36b8-e935-438e-a6d8-b7ffe4c07a5f" (UID: "03fc36b8-e935-438e-a6d8-b7ffe4c07a5f"). InnerVolumeSpecName "kube-api-access-2bxhs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 04:24:09.536253 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:09.536181    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03fc36b8-e935-438e-a6d8-b7ffe4c07a5f-plugins-config-volume\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:24:09.536253 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:09.536205    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2bxhs\" (UniqueName: \"kubernetes.io/projected/03fc36b8-e935-438e-a6d8-b7ffe4c07a5f-kube-api-access-2bxhs\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:24:10.233880 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:10.233796    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f2be5-7bf7bf47b6-wpzh4"
Sep 09 04:24:10.233880 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:10.233821    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f2be5-7bf7bf47b6-wpzh4" event={"ID":"03fc36b8-e935-438e-a6d8-b7ffe4c07a5f","Type":"ContainerDied","Data":"09691ab4719a7ee8ec48d935e3ba5f3e14418e1d7e972159cc4f677556c1f8c0"}
Sep 09 04:24:10.234397 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:10.233883    2557 scope.go:117] "RemoveContainer" containerID="382c4a7aaf6ca6a5a4da6235307dac61173fe62b1584c72e7fd00a436900bbd0"
Sep 09 04:24:10.235305 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:10.235284    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf" event={"ID":"ce66729c-0dda-43cb-9365-98bc06b1e6da","Type":"ContainerDied","Data":"ee0047b7124dea8d2aebd838ceebca4247028b7f793a34337266073875509af5"}
Sep 09 04:24:10.235359 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:10.235308    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf"
Sep 09 04:24:10.242511 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:10.242494    2557 scope.go:117] "RemoveContainer" containerID="8e4686487bb368a84a521ef54e8a1b02c4197a08c65e1537d165603aeafb30fb"
Sep 09 04:24:10.251480 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:10.251454    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf"]
Sep 09 04:24:10.253425 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:10.253404    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f2be5-65bc5595f5-tpnhf"]
Sep 09 04:24:10.264456 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:10.264437    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f2be5-7bf7bf47b6-wpzh4"]
Sep 09 04:24:10.268391 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:10.268372    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f2be5-7bf7bf47b6-wpzh4"]
Sep 09 04:24:11.619806 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:11.619772    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03fc36b8-e935-438e-a6d8-b7ffe4c07a5f" path="/var/lib/kubelet/pods/03fc36b8-e935-438e-a6d8-b7ffe4c07a5f/volumes"
Sep 09 04:24:11.620204 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:11.620189    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce66729c-0dda-43cb-9365-98bc06b1e6da" path="/var/lib/kubelet/pods/ce66729c-0dda-43cb-9365-98bc06b1e6da/volumes"
Sep 09 04:24:21.667728 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:21.667687    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-85732c/distinct-2-w4-85732c-kserve-7f96669d6b-764xk" podUID="19f04fc8-0e98-46b9-a42f-cd57aa63b0ac" containerName="main" containerID="cri-o://07f8ddaac6e74e316950823044c54e6af899661682635197d92e801ddfaca399" gracePeriod=45
Sep 09 04:24:21.676487 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:21.676454    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-85732c/distinct-w4-85732c-kserve-654fb7bb89-cfhns" podUID="b009df2d-e263-42b8-a652-e96fda4bb254" containerName="main" containerID="cri-o://cef1ca00b1fdcab3551bf17124e54520f7f2afbf9771cb9d2cb9ebe559aaa479" gracePeriod=45
Sep 09 04:24:21.948824 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:21.948796    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-85732c/distinct-2-w4-85732c-kserve-7f96669d6b-764xk"
Sep 09 04:24:21.951622 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:21.951598    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-85732c/distinct-w4-85732c-kserve-654fb7bb89-cfhns"
Sep 09 04:24:22.035483 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.035455    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b009df2d-e263-42b8-a652-e96fda4bb254-home\") pod \"b009df2d-e263-42b8-a652-e96fda4bb254\" (UID: \"b009df2d-e263-42b8-a652-e96fda4bb254\") "
Sep 09 04:24:22.035483 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.035487    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19f04fc8-0e98-46b9-a42f-cd57aa63b0ac-dshm\") pod \"19f04fc8-0e98-46b9-a42f-cd57aa63b0ac\" (UID: \"19f04fc8-0e98-46b9-a42f-cd57aa63b0ac\") "
Sep 09 04:24:22.035733 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.035512    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19f04fc8-0e98-46b9-a42f-cd57aa63b0ac-model-cache\") pod \"19f04fc8-0e98-46b9-a42f-cd57aa63b0ac\" (UID: \"19f04fc8-0e98-46b9-a42f-cd57aa63b0ac\") "
Sep 09 04:24:22.035733 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.035533    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19f04fc8-0e98-46b9-a42f-cd57aa63b0ac-home\") pod \"19f04fc8-0e98-46b9-a42f-cd57aa63b0ac\" (UID: \"19f04fc8-0e98-46b9-a42f-cd57aa63b0ac\") "
Sep 09 04:24:22.035733 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.035558    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19f04fc8-0e98-46b9-a42f-cd57aa63b0ac-tmp-dir\") pod \"19f04fc8-0e98-46b9-a42f-cd57aa63b0ac\" (UID: \"19f04fc8-0e98-46b9-a42f-cd57aa63b0ac\") "
Sep 09 04:24:22.035733 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.035572    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b009df2d-e263-42b8-a652-e96fda4bb254-model-cache\") pod \"b009df2d-e263-42b8-a652-e96fda4bb254\" (UID: \"b009df2d-e263-42b8-a652-e96fda4bb254\") "
Sep 09 04:24:22.035733 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.035606    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xzmpw\" (UniqueName: \"kubernetes.io/projected/b009df2d-e263-42b8-a652-e96fda4bb254-kube-api-access-xzmpw\") pod \"b009df2d-e263-42b8-a652-e96fda4bb254\" (UID: \"b009df2d-e263-42b8-a652-e96fda4bb254\") "
Sep 09 04:24:22.035733 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.035629    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b009df2d-e263-42b8-a652-e96fda4bb254-tls-certs\") pod \"b009df2d-e263-42b8-a652-e96fda4bb254\" (UID: \"b009df2d-e263-42b8-a652-e96fda4bb254\") "
Sep 09 04:24:22.035733 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.035656    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b009df2d-e263-42b8-a652-e96fda4bb254-tmp-dir\") pod \"b009df2d-e263-42b8-a652-e96fda4bb254\" (UID: \"b009df2d-e263-42b8-a652-e96fda4bb254\") "
Sep 09 04:24:22.035733 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.035701    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19f04fc8-0e98-46b9-a42f-cd57aa63b0ac-tls-certs\") pod \"19f04fc8-0e98-46b9-a42f-cd57aa63b0ac\" (UID: \"19f04fc8-0e98-46b9-a42f-cd57aa63b0ac\") "
Sep 09 04:24:22.036228 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.035757    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4bwjv\" (UniqueName: \"kubernetes.io/projected/19f04fc8-0e98-46b9-a42f-cd57aa63b0ac-kube-api-access-4bwjv\") pod \"19f04fc8-0e98-46b9-a42f-cd57aa63b0ac\" (UID: \"19f04fc8-0e98-46b9-a42f-cd57aa63b0ac\") "
Sep 09 04:24:22.036228 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.035755    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b009df2d-e263-42b8-a652-e96fda4bb254-home" (OuterVolumeSpecName: "home") pod "b009df2d-e263-42b8-a652-e96fda4bb254" (UID: "b009df2d-e263-42b8-a652-e96fda4bb254"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 04:24:22.036228 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.035781    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b009df2d-e263-42b8-a652-e96fda4bb254-dshm\") pod \"b009df2d-e263-42b8-a652-e96fda4bb254\" (UID: \"b009df2d-e263-42b8-a652-e96fda4bb254\") "
Sep 09 04:24:22.036228 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.035801    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19f04fc8-0e98-46b9-a42f-cd57aa63b0ac-model-cache" (OuterVolumeSpecName: "model-cache") pod "19f04fc8-0e98-46b9-a42f-cd57aa63b0ac" (UID: "19f04fc8-0e98-46b9-a42f-cd57aa63b0ac"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 04:24:22.036228 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.035836    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19f04fc8-0e98-46b9-a42f-cd57aa63b0ac-home" (OuterVolumeSpecName: "home") pod "19f04fc8-0e98-46b9-a42f-cd57aa63b0ac" (UID: "19f04fc8-0e98-46b9-a42f-cd57aa63b0ac"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 04:24:22.036228 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.036127    2557 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b009df2d-e263-42b8-a652-e96fda4bb254-home\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:24:22.036228 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.036148    2557 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19f04fc8-0e98-46b9-a42f-cd57aa63b0ac-model-cache\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:24:22.036228 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.036163    2557 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19f04fc8-0e98-46b9-a42f-cd57aa63b0ac-home\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:24:22.036634 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.036325    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b009df2d-e263-42b8-a652-e96fda4bb254-model-cache" (OuterVolumeSpecName: "model-cache") pod "b009df2d-e263-42b8-a652-e96fda4bb254" (UID: "b009df2d-e263-42b8-a652-e96fda4bb254"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 04:24:22.036634 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.036484    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19f04fc8-0e98-46b9-a42f-cd57aa63b0ac-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "19f04fc8-0e98-46b9-a42f-cd57aa63b0ac" (UID: "19f04fc8-0e98-46b9-a42f-cd57aa63b0ac"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 04:24:22.037370 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.037332    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b009df2d-e263-42b8-a652-e96fda4bb254-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b009df2d-e263-42b8-a652-e96fda4bb254" (UID: "b009df2d-e263-42b8-a652-e96fda4bb254"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 04:24:22.038073 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.038001    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19f04fc8-0e98-46b9-a42f-cd57aa63b0ac-dshm" (OuterVolumeSpecName: "dshm") pod "19f04fc8-0e98-46b9-a42f-cd57aa63b0ac" (UID: "19f04fc8-0e98-46b9-a42f-cd57aa63b0ac"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 04:24:22.038203 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.038179    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b009df2d-e263-42b8-a652-e96fda4bb254-dshm" (OuterVolumeSpecName: "dshm") pod "b009df2d-e263-42b8-a652-e96fda4bb254" (UID: "b009df2d-e263-42b8-a652-e96fda4bb254"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 04:24:22.038758 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.038728    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b009df2d-e263-42b8-a652-e96fda4bb254-kube-api-access-xzmpw" (OuterVolumeSpecName: "kube-api-access-xzmpw") pod "b009df2d-e263-42b8-a652-e96fda4bb254" (UID: "b009df2d-e263-42b8-a652-e96fda4bb254"). InnerVolumeSpecName "kube-api-access-xzmpw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 04:24:22.038910 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.038873    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19f04fc8-0e98-46b9-a42f-cd57aa63b0ac-kube-api-access-4bwjv" (OuterVolumeSpecName: "kube-api-access-4bwjv") pod "19f04fc8-0e98-46b9-a42f-cd57aa63b0ac" (UID: "19f04fc8-0e98-46b9-a42f-cd57aa63b0ac"). InnerVolumeSpecName "kube-api-access-4bwjv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 04:24:22.039278 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.039255    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19f04fc8-0e98-46b9-a42f-cd57aa63b0ac-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "19f04fc8-0e98-46b9-a42f-cd57aa63b0ac" (UID: "19f04fc8-0e98-46b9-a42f-cd57aa63b0ac"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 04:24:22.039731 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.039710    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b009df2d-e263-42b8-a652-e96fda4bb254-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b009df2d-e263-42b8-a652-e96fda4bb254" (UID: "b009df2d-e263-42b8-a652-e96fda4bb254"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 04:24:22.136722 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.136695    2557 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19f04fc8-0e98-46b9-a42f-cd57aa63b0ac-tls-certs\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:24:22.136722 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.136722    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bwjv\" (UniqueName: \"kubernetes.io/projected/19f04fc8-0e98-46b9-a42f-cd57aa63b0ac-kube-api-access-4bwjv\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:24:22.136923 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.136736    2557 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b009df2d-e263-42b8-a652-e96fda4bb254-dshm\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:24:22.136923 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.136747    2557 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19f04fc8-0e98-46b9-a42f-cd57aa63b0ac-dshm\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:24:22.136923 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.136758    2557 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19f04fc8-0e98-46b9-a42f-cd57aa63b0ac-tmp-dir\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:24:22.136923 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.136768    2557 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b009df2d-e263-42b8-a652-e96fda4bb254-model-cache\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:24:22.136923 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.136779    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzmpw\" (UniqueName: \"kubernetes.io/projected/b009df2d-e263-42b8-a652-e96fda4bb254-kube-api-access-xzmpw\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:24:22.136923 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.136789    2557 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b009df2d-e263-42b8-a652-e96fda4bb254-tls-certs\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:24:22.136923 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.136800    2557 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b009df2d-e263-42b8-a652-e96fda4bb254-tmp-dir\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:24:22.296071 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.295983    2557 generic.go:356] "Generic (PLEG): container finished" podID="b009df2d-e263-42b8-a652-e96fda4bb254" containerID="cef1ca00b1fdcab3551bf17124e54520f7f2afbf9771cb9d2cb9ebe559aaa479" exitCode=0
Sep 09 04:24:22.296071 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.296030    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-85732c/distinct-w4-85732c-kserve-654fb7bb89-cfhns"
Sep 09 04:24:22.296309 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.296070    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-85732c/distinct-w4-85732c-kserve-654fb7bb89-cfhns" event={"ID":"b009df2d-e263-42b8-a652-e96fda4bb254","Type":"ContainerDied","Data":"cef1ca00b1fdcab3551bf17124e54520f7f2afbf9771cb9d2cb9ebe559aaa479"}
Sep 09 04:24:22.296309 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.296126    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-85732c/distinct-w4-85732c-kserve-654fb7bb89-cfhns" event={"ID":"b009df2d-e263-42b8-a652-e96fda4bb254","Type":"ContainerDied","Data":"2220b042c0ded2022fffe34d401049894eaa926a08523f1e37f365d00bcd706a"}
Sep 09 04:24:22.296309 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.296147    2557 scope.go:117] "RemoveContainer" containerID="cef1ca00b1fdcab3551bf17124e54520f7f2afbf9771cb9d2cb9ebe559aaa479"
Sep 09 04:24:22.297476 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.297451    2557 generic.go:356] "Generic (PLEG): container finished" podID="19f04fc8-0e98-46b9-a42f-cd57aa63b0ac" containerID="07f8ddaac6e74e316950823044c54e6af899661682635197d92e801ddfaca399" exitCode=0
Sep 09 04:24:22.297569 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.297484    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-85732c/distinct-2-w4-85732c-kserve-7f96669d6b-764xk" event={"ID":"19f04fc8-0e98-46b9-a42f-cd57aa63b0ac","Type":"ContainerDied","Data":"07f8ddaac6e74e316950823044c54e6af899661682635197d92e801ddfaca399"}
Sep 09 04:24:22.297569 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.297503    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-85732c/distinct-2-w4-85732c-kserve-7f96669d6b-764xk"
Sep 09 04:24:22.297569 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.297523    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-85732c/distinct-2-w4-85732c-kserve-7f96669d6b-764xk" event={"ID":"19f04fc8-0e98-46b9-a42f-cd57aa63b0ac","Type":"ContainerDied","Data":"6996289f56fa5ce63a9d9bc93295f992e0c3fd28d06abef7ee5c06fa2c04320c"}
Sep 09 04:24:22.304854 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.304769    2557 scope.go:117] "RemoveContainer" containerID="cef1ca00b1fdcab3551bf17124e54520f7f2afbf9771cb9d2cb9ebe559aaa479"
Sep 09 04:24:22.305106 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:24:22.305086    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cef1ca00b1fdcab3551bf17124e54520f7f2afbf9771cb9d2cb9ebe559aaa479\": container with ID starting with cef1ca00b1fdcab3551bf17124e54520f7f2afbf9771cb9d2cb9ebe559aaa479 not found: ID does not exist" containerID="cef1ca00b1fdcab3551bf17124e54520f7f2afbf9771cb9d2cb9ebe559aaa479"
Sep 09 04:24:22.305187 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.305139    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cef1ca00b1fdcab3551bf17124e54520f7f2afbf9771cb9d2cb9ebe559aaa479"} err="failed to get container status \"cef1ca00b1fdcab3551bf17124e54520f7f2afbf9771cb9d2cb9ebe559aaa479\": rpc error: code = NotFound desc = could not find container \"cef1ca00b1fdcab3551bf17124e54520f7f2afbf9771cb9d2cb9ebe559aaa479\": container with ID starting with cef1ca00b1fdcab3551bf17124e54520f7f2afbf9771cb9d2cb9ebe559aaa479 not found: ID does not exist"
Sep 09 04:24:22.305187 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.305156    2557 scope.go:117] "RemoveContainer" containerID="07f8ddaac6e74e316950823044c54e6af899661682635197d92e801ddfaca399"
Sep 09 04:24:22.312795 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.312779    2557 scope.go:117] "RemoveContainer" containerID="07f8ddaac6e74e316950823044c54e6af899661682635197d92e801ddfaca399"
Sep 09 04:24:22.313030 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:24:22.313012    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07f8ddaac6e74e316950823044c54e6af899661682635197d92e801ddfaca399\": container with ID starting with 07f8ddaac6e74e316950823044c54e6af899661682635197d92e801ddfaca399 not found: ID does not exist" containerID="07f8ddaac6e74e316950823044c54e6af899661682635197d92e801ddfaca399"
Sep 09 04:24:22.313091 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.313034    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07f8ddaac6e74e316950823044c54e6af899661682635197d92e801ddfaca399"} err="failed to get container status \"07f8ddaac6e74e316950823044c54e6af899661682635197d92e801ddfaca399\": rpc error: code = NotFound desc = could not find container \"07f8ddaac6e74e316950823044c54e6af899661682635197d92e801ddfaca399\": container with ID starting with 07f8ddaac6e74e316950823044c54e6af899661682635197d92e801ddfaca399 not found: ID does not exist"
Sep 09 04:24:22.319009 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.318988    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-85732c/distinct-2-w4-85732c-kserve-7f96669d6b-764xk"]
Sep 09 04:24:22.325926 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.325907    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-85732c/distinct-2-w4-85732c-kserve-7f96669d6b-764xk"]
Sep 09 04:24:22.334942 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.334923    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-85732c/distinct-w4-85732c-kserve-654fb7bb89-cfhns"]
Sep 09 04:24:22.338861 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:22.338844    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-85732c/distinct-w4-85732c-kserve-654fb7bb89-cfhns"]
Sep 09 04:24:23.619541 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:23.619507    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19f04fc8-0e98-46b9-a42f-cd57aa63b0ac" path="/var/lib/kubelet/pods/19f04fc8-0e98-46b9-a42f-cd57aa63b0ac/volumes"
Sep 09 04:24:23.619954 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:24:23.619939    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b009df2d-e263-42b8-a652-e96fda4bb254" path="/var/lib/kubelet/pods/b009df2d-e263-42b8-a652-e96fda4bb254/volumes"
Sep 09 04:26:26.165029 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.164936    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-426183/rate-test-model-426183-kserve-c684d867-mtjhn"]
Sep 09 04:26:26.165623 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.165470    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19f04fc8-0e98-46b9-a42f-cd57aa63b0ac" containerName="main"
Sep 09 04:26:26.165623 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.165486    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="19f04fc8-0e98-46b9-a42f-cd57aa63b0ac" containerName="main"
Sep 09 04:26:26.165623 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.165496    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e37c2a1-3b49-4d10-bc98-07aa794ace0a" containerName="payload-processing"
Sep 09 04:26:26.165623 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.165505    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e37c2a1-3b49-4d10-bc98-07aa794ace0a" containerName="payload-processing"
Sep 09 04:26:26.165623 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.165522    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b655b0f5-4908-4030-9a69-a4f5aaa83d0d" containerName="payload-processing"
Sep 09 04:26:26.165623 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.165530    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b655b0f5-4908-4030-9a69-a4f5aaa83d0d" containerName="payload-processing"
Sep 09 04:26:26.165623 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.165545    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03fc36b8-e935-438e-a6d8-b7ffe4c07a5f" containerName="payload-pre-processing"
Sep 09 04:26:26.165623 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.165554    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="03fc36b8-e935-438e-a6d8-b7ffe4c07a5f" containerName="payload-pre-processing"
Sep 09 04:26:26.165623 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.165568    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab367e3f-b00f-48ff-8c17-7d213b162cbb" containerName="payload-pre-processing"
Sep 09 04:26:26.165623 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.165580    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab367e3f-b00f-48ff-8c17-7d213b162cbb" containerName="payload-pre-processing"
Sep 09 04:26:26.165623 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.165597    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce66729c-0dda-43cb-9365-98bc06b1e6da" containerName="maas-api"
Sep 09 04:26:26.165623 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.165607    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce66729c-0dda-43cb-9365-98bc06b1e6da" containerName="maas-api"
Sep 09 04:26:26.165623 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.165621    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b009df2d-e263-42b8-a652-e96fda4bb254" containerName="main"
Sep 09 04:26:26.165623 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.165629    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b009df2d-e263-42b8-a652-e96fda4bb254" containerName="main"
Sep 09 04:26:26.166409 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.165648    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1043d03c-a962-4f87-b4e4-64bb25b4f00b" containerName="payload-processing"
Sep 09 04:26:26.166409 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.165656    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="1043d03c-a962-4f87-b4e4-64bb25b4f00b" containerName="payload-processing"
Sep 09 04:26:26.166409 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.165678    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0883b07-bdbf-4e45-885a-0a2c2a3072f3" containerName="maas-api"
Sep 09 04:26:26.166409 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.165686    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0883b07-bdbf-4e45-885a-0a2c2a3072f3" containerName="maas-api"
Sep 09 04:26:26.166409 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.165759    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e37c2a1-3b49-4d10-bc98-07aa794ace0a" containerName="payload-processing"
Sep 09 04:26:26.166409 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.165768    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="19f04fc8-0e98-46b9-a42f-cd57aa63b0ac" containerName="main"
Sep 09 04:26:26.166409 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.165774    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="1043d03c-a962-4f87-b4e4-64bb25b4f00b" containerName="payload-processing"
Sep 09 04:26:26.166409 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.165781    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b009df2d-e263-42b8-a652-e96fda4bb254" containerName="main"
Sep 09 04:26:26.166409 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.165788    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce66729c-0dda-43cb-9365-98bc06b1e6da" containerName="maas-api"
Sep 09 04:26:26.166409 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.165795    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="03fc36b8-e935-438e-a6d8-b7ffe4c07a5f" containerName="payload-pre-processing"
Sep 09 04:26:26.166409 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.165800    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab367e3f-b00f-48ff-8c17-7d213b162cbb" containerName="payload-pre-processing"
Sep 09 04:26:26.166409 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.165806    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b655b0f5-4908-4030-9a69-a4f5aaa83d0d" containerName="payload-processing"
Sep 09 04:26:26.166409 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.165810    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0883b07-bdbf-4e45-885a-0a2c2a3072f3" containerName="maas-api"
Sep 09 04:26:26.168760 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.168735    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-426183/rate-test-model-426183-kserve-c684d867-mtjhn"
Sep 09 04:26:26.170935 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.170917    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-426183\"/\"kube-root-ca.crt\""
Sep 09 04:26:26.172002 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.171972    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-426183\"/\"default-dockercfg-q6z8l\""
Sep 09 04:26:26.172175 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.171981    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-426183\"/\"rate-test-model-426183-kserve-self-signed-certs\""
Sep 09 04:26:26.172175 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.172013    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-426183\"/\"openshift-service-ca.crt\""
Sep 09 04:26:26.178177 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.178126    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-426183/rate-test-model-426183-kserve-c684d867-mtjhn"]
Sep 09 04:26:26.277383 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.277348    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/983a929c-9b20-4c6a-a559-e6e03d4f4f63-tls-certs\") pod \"rate-test-model-426183-kserve-c684d867-mtjhn\" (UID: \"983a929c-9b20-4c6a-a559-e6e03d4f4f63\") " pod="ai-tenant-e2e-shared-a-w3-426183/rate-test-model-426183-kserve-c684d867-mtjhn"
Sep 09 04:26:26.277526 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.277404    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/983a929c-9b20-4c6a-a559-e6e03d4f4f63-home\") pod \"rate-test-model-426183-kserve-c684d867-mtjhn\" (UID: \"983a929c-9b20-4c6a-a559-e6e03d4f4f63\") " pod="ai-tenant-e2e-shared-a-w3-426183/rate-test-model-426183-kserve-c684d867-mtjhn"
Sep 09 04:26:26.277526 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.277422    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/983a929c-9b20-4c6a-a559-e6e03d4f4f63-model-cache\") pod \"rate-test-model-426183-kserve-c684d867-mtjhn\" (UID: \"983a929c-9b20-4c6a-a559-e6e03d4f4f63\") " pod="ai-tenant-e2e-shared-a-w3-426183/rate-test-model-426183-kserve-c684d867-mtjhn"
Sep 09 04:26:26.277526 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.277449    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkhgm\" (UniqueName: \"kubernetes.io/projected/983a929c-9b20-4c6a-a559-e6e03d4f4f63-kube-api-access-lkhgm\") pod \"rate-test-model-426183-kserve-c684d867-mtjhn\" (UID: \"983a929c-9b20-4c6a-a559-e6e03d4f4f63\") " pod="ai-tenant-e2e-shared-a-w3-426183/rate-test-model-426183-kserve-c684d867-mtjhn"
Sep 09 04:26:26.277526 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.277467    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/983a929c-9b20-4c6a-a559-e6e03d4f4f63-tmp-dir\") pod \"rate-test-model-426183-kserve-c684d867-mtjhn\" (UID: \"983a929c-9b20-4c6a-a559-e6e03d4f4f63\") " pod="ai-tenant-e2e-shared-a-w3-426183/rate-test-model-426183-kserve-c684d867-mtjhn"
Sep 09 04:26:26.277680 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.277567    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/983a929c-9b20-4c6a-a559-e6e03d4f4f63-dshm\") pod \"rate-test-model-426183-kserve-c684d867-mtjhn\" (UID: \"983a929c-9b20-4c6a-a559-e6e03d4f4f63\") " pod="ai-tenant-e2e-shared-a-w3-426183/rate-test-model-426183-kserve-c684d867-mtjhn"
Sep 09 04:26:26.378314 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.378267    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/983a929c-9b20-4c6a-a559-e6e03d4f4f63-dshm\") pod \"rate-test-model-426183-kserve-c684d867-mtjhn\" (UID: \"983a929c-9b20-4c6a-a559-e6e03d4f4f63\") " pod="ai-tenant-e2e-shared-a-w3-426183/rate-test-model-426183-kserve-c684d867-mtjhn"
Sep 09 04:26:26.378314 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.378319    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/983a929c-9b20-4c6a-a559-e6e03d4f4f63-tls-certs\") pod \"rate-test-model-426183-kserve-c684d867-mtjhn\" (UID: \"983a929c-9b20-4c6a-a559-e6e03d4f4f63\") " pod="ai-tenant-e2e-shared-a-w3-426183/rate-test-model-426183-kserve-c684d867-mtjhn"
Sep 09 04:26:26.378573 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.378380    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/983a929c-9b20-4c6a-a559-e6e03d4f4f63-home\") pod \"rate-test-model-426183-kserve-c684d867-mtjhn\" (UID: \"983a929c-9b20-4c6a-a559-e6e03d4f4f63\") " pod="ai-tenant-e2e-shared-a-w3-426183/rate-test-model-426183-kserve-c684d867-mtjhn"
Sep 09 04:26:26.378573 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.378400    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/983a929c-9b20-4c6a-a559-e6e03d4f4f63-model-cache\") pod \"rate-test-model-426183-kserve-c684d867-mtjhn\" (UID: \"983a929c-9b20-4c6a-a559-e6e03d4f4f63\") " pod="ai-tenant-e2e-shared-a-w3-426183/rate-test-model-426183-kserve-c684d867-mtjhn"
Sep 09 04:26:26.378573 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.378438    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkhgm\" (UniqueName: \"kubernetes.io/projected/983a929c-9b20-4c6a-a559-e6e03d4f4f63-kube-api-access-lkhgm\") pod \"rate-test-model-426183-kserve-c684d867-mtjhn\" (UID: \"983a929c-9b20-4c6a-a559-e6e03d4f4f63\") " pod="ai-tenant-e2e-shared-a-w3-426183/rate-test-model-426183-kserve-c684d867-mtjhn"
Sep 09 04:26:26.378573 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.378465    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/983a929c-9b20-4c6a-a559-e6e03d4f4f63-tmp-dir\") pod \"rate-test-model-426183-kserve-c684d867-mtjhn\" (UID: \"983a929c-9b20-4c6a-a559-e6e03d4f4f63\") " pod="ai-tenant-e2e-shared-a-w3-426183/rate-test-model-426183-kserve-c684d867-mtjhn"
Sep 09 04:26:26.378802 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.378777    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/983a929c-9b20-4c6a-a559-e6e03d4f4f63-home\") pod \"rate-test-model-426183-kserve-c684d867-mtjhn\" (UID: \"983a929c-9b20-4c6a-a559-e6e03d4f4f63\") " pod="ai-tenant-e2e-shared-a-w3-426183/rate-test-model-426183-kserve-c684d867-mtjhn"
Sep 09 04:26:26.378861 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.378814    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/983a929c-9b20-4c6a-a559-e6e03d4f4f63-tmp-dir\") pod \"rate-test-model-426183-kserve-c684d867-mtjhn\" (UID: \"983a929c-9b20-4c6a-a559-e6e03d4f4f63\") " pod="ai-tenant-e2e-shared-a-w3-426183/rate-test-model-426183-kserve-c684d867-mtjhn"
Sep 09 04:26:26.378899 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.378860    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/983a929c-9b20-4c6a-a559-e6e03d4f4f63-model-cache\") pod \"rate-test-model-426183-kserve-c684d867-mtjhn\" (UID: \"983a929c-9b20-4c6a-a559-e6e03d4f4f63\") " pod="ai-tenant-e2e-shared-a-w3-426183/rate-test-model-426183-kserve-c684d867-mtjhn"
Sep 09 04:26:26.380591 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.380569    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/983a929c-9b20-4c6a-a559-e6e03d4f4f63-dshm\") pod \"rate-test-model-426183-kserve-c684d867-mtjhn\" (UID: \"983a929c-9b20-4c6a-a559-e6e03d4f4f63\") " pod="ai-tenant-e2e-shared-a-w3-426183/rate-test-model-426183-kserve-c684d867-mtjhn"
Sep 09 04:26:26.380755 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.380738    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/983a929c-9b20-4c6a-a559-e6e03d4f4f63-tls-certs\") pod \"rate-test-model-426183-kserve-c684d867-mtjhn\" (UID: \"983a929c-9b20-4c6a-a559-e6e03d4f4f63\") " pod="ai-tenant-e2e-shared-a-w3-426183/rate-test-model-426183-kserve-c684d867-mtjhn"
Sep 09 04:26:26.387436 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.387409    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkhgm\" (UniqueName: \"kubernetes.io/projected/983a929c-9b20-4c6a-a559-e6e03d4f4f63-kube-api-access-lkhgm\") pod \"rate-test-model-426183-kserve-c684d867-mtjhn\" (UID: \"983a929c-9b20-4c6a-a559-e6e03d4f4f63\") " pod="ai-tenant-e2e-shared-a-w3-426183/rate-test-model-426183-kserve-c684d867-mtjhn"
Sep 09 04:26:26.480766 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.480677    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-426183/rate-test-model-426183-kserve-c684d867-mtjhn"
Sep 09 04:26:26.755207 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.755178    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-426183/rate-test-model-426183-kserve-c684d867-mtjhn"]
Sep 09 04:26:26.757063 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:26:26.757025    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod983a929c_9b20_4c6a_a559_e6e03d4f4f63.slice/crio-151c47a41e73121630e7480e7141e50a69642e3fc1ae84d606e139b19201d24b WatchSource:0}: Error finding container 151c47a41e73121630e7480e7141e50a69642e3fc1ae84d606e139b19201d24b: Status 404 returned error can't find the container with id 151c47a41e73121630e7480e7141e50a69642e3fc1ae84d606e139b19201d24b
Sep 09 04:26:26.872998 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.872962    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-426183/rate-test-model-426183-kserve-c684d867-mtjhn" event={"ID":"983a929c-9b20-4c6a-a559-e6e03d4f4f63","Type":"ContainerStarted","Data":"76fb80ba8de4b04fd5031dce4cf73354f441354336db6dc1c4a69ed3ada42d1f"}
Sep 09 04:26:26.873229 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.873009    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-426183/rate-test-model-426183-kserve-c684d867-mtjhn" event={"ID":"983a929c-9b20-4c6a-a559-e6e03d4f4f63","Type":"ContainerStarted","Data":"151c47a41e73121630e7480e7141e50a69642e3fc1ae84d606e139b19201d24b"}
Sep 09 04:26:26.897541 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:26.897497    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-426183/rate-test-model-426183-kserve-c684d867-mtjhn" podStartSLOduration=0.897481818 podStartE2EDuration="897.481818ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:26:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:26:26.894241438 +0000 UTC m=+1711.813412360" watchObservedRunningTime="2026-09-09 04:26:26.897481818 +0000 UTC m=+1711.816652726"
Sep 09 04:26:36.481352 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:36.481308    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-426183/rate-test-model-426183-kserve-c684d867-mtjhn"
Sep 09 04:26:36.483528 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:36.481370    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-426183/rate-test-model-426183-kserve-c684d867-mtjhn"
Sep 09 04:26:36.493539 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:36.493515    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-426183/rate-test-model-426183-kserve-c684d867-mtjhn"
Sep 09 04:26:36.929582 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:26:36.929554    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-426183/rate-test-model-426183-kserve-c684d867-mtjhn"
Sep 09 04:27:50.282745 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:50.282664    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-426183/rate-test-model-426183-kserve-c684d867-mtjhn"]
Sep 09 04:27:55.648686 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:55.648649    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gj7h7_7816cd49-e9c2-4f7b-b8ec-4e5711b675a5/console-operator/2.log"
Sep 09 04:27:55.650265 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:55.650243    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gj7h7_7816cd49-e9c2-4f7b-b8ec-4e5711b675a5/console-operator/2.log"
Sep 09 04:27:55.654274 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:55.654257    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rfvrc_1828d4d8-0d24-4e4d-a502-aee013a3b116/ovn-acl-logging/0.log"
Sep 09 04:27:55.655819 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:55.655802    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rfvrc_1828d4d8-0d24-4e4d-a502-aee013a3b116/ovn-acl-logging/0.log"
Sep 09 04:27:56.773518 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:56.773485    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw"]
Sep 09 04:27:56.776978 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:56.776865    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw"
Sep 09 04:27:56.779218 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:56.779197    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-a3ce90-serving-cert\""
Sep 09 04:27:56.785271 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:56.785246    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw"]
Sep 09 04:27:56.955890 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:56.955841    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hktsk\" (UniqueName: \"kubernetes.io/projected/57960f27-5f23-417b-82be-883363e51719-kube-api-access-hktsk\") pod \"maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw\" (UID: \"57960f27-5f23-417b-82be-883363e51719\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw"
Sep 09 04:27:56.956093 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:56.955993    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57960f27-5f23-417b-82be-883363e51719-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw\" (UID: \"57960f27-5f23-417b-82be-883363e51719\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw"
Sep 09 04:27:56.956093 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:56.956039    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57960f27-5f23-417b-82be-883363e51719-maas-api-tls\") pod \"maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw\" (UID: \"57960f27-5f23-417b-82be-883363e51719\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw"
Sep 09 04:27:56.956093 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:56.956073    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57960f27-5f23-417b-82be-883363e51719-metrics-certs\") pod \"maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw\" (UID: \"57960f27-5f23-417b-82be-883363e51719\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw"
Sep 09 04:27:57.057013 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:57.056932    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57960f27-5f23-417b-82be-883363e51719-metrics-certs\") pod \"maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw\" (UID: \"57960f27-5f23-417b-82be-883363e51719\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw"
Sep 09 04:27:57.057013 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:57.056990    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hktsk\" (UniqueName: \"kubernetes.io/projected/57960f27-5f23-417b-82be-883363e51719-kube-api-access-hktsk\") pod \"maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw\" (UID: \"57960f27-5f23-417b-82be-883363e51719\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw"
Sep 09 04:27:57.057221 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:57.057067    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57960f27-5f23-417b-82be-883363e51719-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw\" (UID: \"57960f27-5f23-417b-82be-883363e51719\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw"
Sep 09 04:27:57.057221 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:57.057098    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57960f27-5f23-417b-82be-883363e51719-maas-api-tls\") pod \"maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw\" (UID: \"57960f27-5f23-417b-82be-883363e51719\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw"
Sep 09 04:27:57.057791 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:57.057770    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57960f27-5f23-417b-82be-883363e51719-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw\" (UID: \"57960f27-5f23-417b-82be-883363e51719\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw"
Sep 09 04:27:57.059444 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:57.059416    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57960f27-5f23-417b-82be-883363e51719-metrics-certs\") pod \"maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw\" (UID: \"57960f27-5f23-417b-82be-883363e51719\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw"
Sep 09 04:27:57.059444 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:57.059441    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57960f27-5f23-417b-82be-883363e51719-maas-api-tls\") pod \"maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw\" (UID: \"57960f27-5f23-417b-82be-883363e51719\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw"
Sep 09 04:27:57.075630 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:57.075608    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hktsk\" (UniqueName: \"kubernetes.io/projected/57960f27-5f23-417b-82be-883363e51719-kube-api-access-hktsk\") pod \"maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw\" (UID: \"57960f27-5f23-417b-82be-883363e51719\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw"
Sep 09 04:27:57.088631 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:57.088609    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw"
Sep 09 04:27:57.365733 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:57.365706    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw"]
Sep 09 04:27:57.368656 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:27:57.368627    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57960f27_5f23_417b_82be_883363e51719.slice/crio-6742d4ea765efebd5120c89047b086c4f95c7c565b49854c7f4d1cd886411cbb WatchSource:0}: Error finding container 6742d4ea765efebd5120c89047b086c4f95c7c565b49854c7f4d1cd886411cbb: Status 404 returned error can't find the container with id 6742d4ea765efebd5120c89047b086c4f95c7c565b49854c7f4d1cd886411cbb
Sep 09 04:27:57.369923 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:57.369904    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 04:27:57.452335 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:57.452301    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a3ce90-66c9f5b478-p8z7j"]
Sep 09 04:27:57.457142 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:57.457103    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a3ce90-66c9f5b478-p8z7j"
Sep 09 04:27:57.459417 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:57.459391    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-a3ce90\""
Sep 09 04:27:57.459534 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:57.459491    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-a3ce90-dockercfg-ggcz5\""
Sep 09 04:27:57.469847 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:57.469823    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a3ce90-66c9f5b478-p8z7j"]
Sep 09 04:27:57.561043 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:57.561008    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nj98\" (UniqueName: \"kubernetes.io/projected/44fc4aac-f5ff-4c2d-af27-b2624fc70366-kube-api-access-9nj98\") pod \"payload-pre-processing-e2e-ipp-a-a3ce90-66c9f5b478-p8z7j\" (UID: \"44fc4aac-f5ff-4c2d-af27-b2624fc70366\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a3ce90-66c9f5b478-p8z7j"
Sep 09 04:27:57.561043 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:57.561054    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44fc4aac-f5ff-4c2d-af27-b2624fc70366-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-a3ce90-66c9f5b478-p8z7j\" (UID: \"44fc4aac-f5ff-4c2d-af27-b2624fc70366\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a3ce90-66c9f5b478-p8z7j"
Sep 09 04:27:57.661983 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:57.661895    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nj98\" (UniqueName: \"kubernetes.io/projected/44fc4aac-f5ff-4c2d-af27-b2624fc70366-kube-api-access-9nj98\") pod \"payload-pre-processing-e2e-ipp-a-a3ce90-66c9f5b478-p8z7j\" (UID: \"44fc4aac-f5ff-4c2d-af27-b2624fc70366\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a3ce90-66c9f5b478-p8z7j"
Sep 09 04:27:57.661983 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:57.661936    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44fc4aac-f5ff-4c2d-af27-b2624fc70366-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-a3ce90-66c9f5b478-p8z7j\" (UID: \"44fc4aac-f5ff-4c2d-af27-b2624fc70366\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a3ce90-66c9f5b478-p8z7j"
Sep 09 04:27:57.662677 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:57.662656    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44fc4aac-f5ff-4c2d-af27-b2624fc70366-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-a3ce90-66c9f5b478-p8z7j\" (UID: \"44fc4aac-f5ff-4c2d-af27-b2624fc70366\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a3ce90-66c9f5b478-p8z7j"
Sep 09 04:27:57.674643 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:57.674622    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nj98\" (UniqueName: \"kubernetes.io/projected/44fc4aac-f5ff-4c2d-af27-b2624fc70366-kube-api-access-9nj98\") pod \"payload-pre-processing-e2e-ipp-a-a3ce90-66c9f5b478-p8z7j\" (UID: \"44fc4aac-f5ff-4c2d-af27-b2624fc70366\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a3ce90-66c9f5b478-p8z7j"
Sep 09 04:27:57.767517 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:57.767485    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a3ce90-66c9f5b478-p8z7j"
Sep 09 04:27:58.038105 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:58.038077    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a3ce90-66c9f5b478-p8z7j"]
Sep 09 04:27:58.040513 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:27:58.040487    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44fc4aac_f5ff_4c2d_af27_b2624fc70366.slice/crio-4d07b285d7ab545f95a04e65806ab403b66c1aa7a6a7cf9473df4edf2b8b7ff3 WatchSource:0}: Error finding container 4d07b285d7ab545f95a04e65806ab403b66c1aa7a6a7cf9473df4edf2b8b7ff3: Status 404 returned error can't find the container with id 4d07b285d7ab545f95a04e65806ab403b66c1aa7a6a7cf9473df4edf2b8b7ff3
Sep 09 04:27:58.300869 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:58.300772    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a3ce90-66c9f5b478-p8z7j" event={"ID":"44fc4aac-f5ff-4c2d-af27-b2624fc70366","Type":"ContainerStarted","Data":"fa79ff5964794dfdb2aea1fabf68787be21e101633bc11ecb9539f3ae7800d44"}
Sep 09 04:27:58.300869 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:58.300832    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a3ce90-66c9f5b478-p8z7j" event={"ID":"44fc4aac-f5ff-4c2d-af27-b2624fc70366","Type":"ContainerStarted","Data":"4d07b285d7ab545f95a04e65806ab403b66c1aa7a6a7cf9473df4edf2b8b7ff3"}
Sep 09 04:27:58.301089 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:58.300966    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a3ce90-66c9f5b478-p8z7j"
Sep 09 04:27:58.302336 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:58.302311    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw" event={"ID":"57960f27-5f23-417b-82be-883363e51719","Type":"ContainerStarted","Data":"82acc72b3fe6255ead8cd7139ec18aba7182982bd10519b34254097bdbcd2942"}
Sep 09 04:27:58.302420 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:58.302351    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw" event={"ID":"57960f27-5f23-417b-82be-883363e51719","Type":"ContainerStarted","Data":"6742d4ea765efebd5120c89047b086c4f95c7c565b49854c7f4d1cd886411cbb"}
Sep 09 04:27:58.302420 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:58.302386    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw"
Sep 09 04:27:58.322061 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:58.322014    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a3ce90-66c9f5b478-p8z7j" podStartSLOduration=1.3220000459999999 podStartE2EDuration="1.322000046s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:27:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:27:58.321344773 +0000 UTC m=+1803.240515682" watchObservedRunningTime="2026-09-09 04:27:58.322000046 +0000 UTC m=+1803.241170953"
Sep 09 04:27:58.339070 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:27:58.338986    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw" podStartSLOduration=1.616062349 podStartE2EDuration="2.338973338s" totalImagesPullingTime="722.910989ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:27:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:27:58.338544309 +0000 UTC m=+1803.257715212" watchObservedRunningTime="2026-09-09 04:27:58.338973338 +0000 UTC m=+1803.258144247"
Sep 09 04:28:04.312517 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:04.312434    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw"
Sep 09 04:28:05.301948 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:05.301900    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-426183/rate-test-model-426183-kserve-c684d867-mtjhn" podUID="983a929c-9b20-4c6a-a559-e6e03d4f4f63" containerName="main" containerID="cri-o://76fb80ba8de4b04fd5031dce4cf73354f441354336db6dc1c4a69ed3ada42d1f" gracePeriod=45
Sep 09 04:28:05.545636 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:05.545612    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-426183/rate-test-model-426183-kserve-c684d867-mtjhn"
Sep 09 04:28:05.624514 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:05.624487    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/983a929c-9b20-4c6a-a559-e6e03d4f4f63-tls-certs\") pod \"983a929c-9b20-4c6a-a559-e6e03d4f4f63\" (UID: \"983a929c-9b20-4c6a-a559-e6e03d4f4f63\") "
Sep 09 04:28:05.624687 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:05.624526    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/983a929c-9b20-4c6a-a559-e6e03d4f4f63-tmp-dir\") pod \"983a929c-9b20-4c6a-a559-e6e03d4f4f63\" (UID: \"983a929c-9b20-4c6a-a559-e6e03d4f4f63\") "
Sep 09 04:28:05.624687 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:05.624556    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/983a929c-9b20-4c6a-a559-e6e03d4f4f63-home\") pod \"983a929c-9b20-4c6a-a559-e6e03d4f4f63\" (UID: \"983a929c-9b20-4c6a-a559-e6e03d4f4f63\") "
Sep 09 04:28:05.624687 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:05.624589    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/983a929c-9b20-4c6a-a559-e6e03d4f4f63-model-cache\") pod \"983a929c-9b20-4c6a-a559-e6e03d4f4f63\" (UID: \"983a929c-9b20-4c6a-a559-e6e03d4f4f63\") "
Sep 09 04:28:05.624687 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:05.624630    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lkhgm\" (UniqueName: \"kubernetes.io/projected/983a929c-9b20-4c6a-a559-e6e03d4f4f63-kube-api-access-lkhgm\") pod \"983a929c-9b20-4c6a-a559-e6e03d4f4f63\" (UID: \"983a929c-9b20-4c6a-a559-e6e03d4f4f63\") "
Sep 09 04:28:05.624687 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:05.624682    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/983a929c-9b20-4c6a-a559-e6e03d4f4f63-dshm\") pod \"983a929c-9b20-4c6a-a559-e6e03d4f4f63\" (UID: \"983a929c-9b20-4c6a-a559-e6e03d4f4f63\") "
Sep 09 04:28:05.624938 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:05.624783    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/983a929c-9b20-4c6a-a559-e6e03d4f4f63-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "983a929c-9b20-4c6a-a559-e6e03d4f4f63" (UID: "983a929c-9b20-4c6a-a559-e6e03d4f4f63"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 04:28:05.624938 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:05.624844    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/983a929c-9b20-4c6a-a559-e6e03d4f4f63-model-cache" (OuterVolumeSpecName: "model-cache") pod "983a929c-9b20-4c6a-a559-e6e03d4f4f63" (UID: "983a929c-9b20-4c6a-a559-e6e03d4f4f63"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 04:28:05.624938 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:05.624863    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/983a929c-9b20-4c6a-a559-e6e03d4f4f63-home" (OuterVolumeSpecName: "home") pod "983a929c-9b20-4c6a-a559-e6e03d4f4f63" (UID: "983a929c-9b20-4c6a-a559-e6e03d4f4f63"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 04:28:05.624938 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:05.624938    2557 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/983a929c-9b20-4c6a-a559-e6e03d4f4f63-tmp-dir\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:28:05.625070 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:05.624949    2557 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/983a929c-9b20-4c6a-a559-e6e03d4f4f63-home\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:28:05.625070 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:05.624958    2557 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/983a929c-9b20-4c6a-a559-e6e03d4f4f63-model-cache\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:28:05.626813 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:05.626782    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/983a929c-9b20-4c6a-a559-e6e03d4f4f63-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "983a929c-9b20-4c6a-a559-e6e03d4f4f63" (UID: "983a929c-9b20-4c6a-a559-e6e03d4f4f63"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 04:28:05.626920 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:05.626813    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/983a929c-9b20-4c6a-a559-e6e03d4f4f63-kube-api-access-lkhgm" (OuterVolumeSpecName: "kube-api-access-lkhgm") pod "983a929c-9b20-4c6a-a559-e6e03d4f4f63" (UID: "983a929c-9b20-4c6a-a559-e6e03d4f4f63"). InnerVolumeSpecName "kube-api-access-lkhgm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 04:28:05.626920 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:05.626855    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/983a929c-9b20-4c6a-a559-e6e03d4f4f63-dshm" (OuterVolumeSpecName: "dshm") pod "983a929c-9b20-4c6a-a559-e6e03d4f4f63" (UID: "983a929c-9b20-4c6a-a559-e6e03d4f4f63"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 04:28:05.726158 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:05.726108    2557 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/983a929c-9b20-4c6a-a559-e6e03d4f4f63-tls-certs\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:28:05.726158 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:05.726153    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lkhgm\" (UniqueName: \"kubernetes.io/projected/983a929c-9b20-4c6a-a559-e6e03d4f4f63-kube-api-access-lkhgm\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:28:05.726158 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:05.726163    2557 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/983a929c-9b20-4c6a-a559-e6e03d4f4f63-dshm\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:28:06.340927 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:06.340889    2557 generic.go:356] "Generic (PLEG): container finished" podID="983a929c-9b20-4c6a-a559-e6e03d4f4f63" containerID="76fb80ba8de4b04fd5031dce4cf73354f441354336db6dc1c4a69ed3ada42d1f" exitCode=0
Sep 09 04:28:06.340927 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:06.340930    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-426183/rate-test-model-426183-kserve-c684d867-mtjhn"
Sep 09 04:28:06.341170 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:06.340939    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-426183/rate-test-model-426183-kserve-c684d867-mtjhn" event={"ID":"983a929c-9b20-4c6a-a559-e6e03d4f4f63","Type":"ContainerDied","Data":"76fb80ba8de4b04fd5031dce4cf73354f441354336db6dc1c4a69ed3ada42d1f"}
Sep 09 04:28:06.341170 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:06.340977    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-426183/rate-test-model-426183-kserve-c684d867-mtjhn" event={"ID":"983a929c-9b20-4c6a-a559-e6e03d4f4f63","Type":"ContainerDied","Data":"151c47a41e73121630e7480e7141e50a69642e3fc1ae84d606e139b19201d24b"}
Sep 09 04:28:06.341170 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:06.340997    2557 scope.go:117] "RemoveContainer" containerID="76fb80ba8de4b04fd5031dce4cf73354f441354336db6dc1c4a69ed3ada42d1f"
Sep 09 04:28:06.349841 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:06.349824    2557 scope.go:117] "RemoveContainer" containerID="76fb80ba8de4b04fd5031dce4cf73354f441354336db6dc1c4a69ed3ada42d1f"
Sep 09 04:28:06.350068 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:28:06.350047    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76fb80ba8de4b04fd5031dce4cf73354f441354336db6dc1c4a69ed3ada42d1f\": container with ID starting with 76fb80ba8de4b04fd5031dce4cf73354f441354336db6dc1c4a69ed3ada42d1f not found: ID does not exist" containerID="76fb80ba8de4b04fd5031dce4cf73354f441354336db6dc1c4a69ed3ada42d1f"
Sep 09 04:28:06.350148 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:06.350080    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76fb80ba8de4b04fd5031dce4cf73354f441354336db6dc1c4a69ed3ada42d1f"} err="failed to get container status \"76fb80ba8de4b04fd5031dce4cf73354f441354336db6dc1c4a69ed3ada42d1f\": rpc error: code = NotFound desc = could not find container \"76fb80ba8de4b04fd5031dce4cf73354f441354336db6dc1c4a69ed3ada42d1f\": container with ID starting with 76fb80ba8de4b04fd5031dce4cf73354f441354336db6dc1c4a69ed3ada42d1f not found: ID does not exist"
Sep 09 04:28:06.363693 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:06.363670    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-426183/rate-test-model-426183-kserve-c684d867-mtjhn"]
Sep 09 04:28:06.367741 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:06.367720    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-426183/rate-test-model-426183-kserve-c684d867-mtjhn"]
Sep 09 04:28:07.619581 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:07.619548    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="983a929c-9b20-4c6a-a559-e6e03d4f4f63" path="/var/lib/kubelet/pods/983a929c-9b20-4c6a-a559-e6e03d4f4f63/volumes"
Sep 09 04:28:09.309545 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:09.309515    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a3ce90-66c9f5b478-p8z7j"
Sep 09 04:28:21.507021 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:21.506983    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"]
Sep 09 04:28:21.508313 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:21.508277    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="983a929c-9b20-4c6a-a559-e6e03d4f4f63" containerName="main"
Sep 09 04:28:21.508475 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:21.508458    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="983a929c-9b20-4c6a-a559-e6e03d4f4f63" containerName="main"
Sep 09 04:28:21.508906 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:21.508888    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="983a929c-9b20-4c6a-a559-e6e03d4f4f63" containerName="main"
Sep 09 04:28:21.514617 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:21.514589    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:28:21.517324 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:21.517302    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-0db87d-openshift-default-dockercfg-bxnw4\""
Sep 09 04:28:21.521368 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:21.521343    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"]
Sep 09 04:28:21.563833 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:21.563809    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-credential-socket\") pod \"e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f\" (UID: \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\") " pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:28:21.563961 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:21.563849    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-workload-certs\") pod \"e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f\" (UID: \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\") " pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:28:21.563961 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:21.563871    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-workload-socket\") pod \"e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f\" (UID: \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\") " pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:28:21.564075 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:21.563970    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-istiod-ca-cert\") pod \"e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f\" (UID: \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\") " pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:28:21.564075 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:21.564021    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69cvw\" (UniqueName: \"kubernetes.io/projected/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-kube-api-access-69cvw\") pod \"e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f\" (UID: \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\") " pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:28:21.564172 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:21.564077    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-istio-podinfo\") pod \"e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f\" (UID: \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\") " pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:28:21.564172 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:21.564132    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-istio-token\") pod \"e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f\" (UID: \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\") " pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:28:21.564172 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:21.564154    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-istio-data\") pod \"e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f\" (UID: \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\") " pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:28:21.564282 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:21.564186    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-istio-envoy\") pod \"e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f\" (UID: \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\") " pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:28:21.665388 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:21.665355    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-credential-socket\") pod \"e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f\" (UID: \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\") " pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:28:21.665567 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:21.665400    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-workload-certs\") pod \"e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f\" (UID: \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\") " pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:28:21.665567 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:21.665416    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-workload-socket\") pod \"e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f\" (UID: \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\") " pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:28:21.665567 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:21.665467    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-istiod-ca-cert\") pod \"e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f\" (UID: \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\") " pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:28:21.665744 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:21.665641    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69cvw\" (UniqueName: \"kubernetes.io/projected/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-kube-api-access-69cvw\") pod \"e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f\" (UID: \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\") " pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:28:21.665822 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:21.665794    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-workload-certs\") pod \"e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f\" (UID: \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\") " pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:28:21.665822 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:21.665808    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-credential-socket\") pod \"e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f\" (UID: \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\") " pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:28:21.666003 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:21.665984    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-istio-podinfo\") pod \"e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f\" (UID: \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\") " pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:28:21.666079 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:21.666006    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-workload-socket\") pod \"e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f\" (UID: \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\") " pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:28:21.666151 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:21.666094    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-istio-token\") pod \"e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f\" (UID: \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\") " pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:28:21.666212 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:21.666160    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-istio-data\") pod \"e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f\" (UID: \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\") " pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:28:21.666212 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:21.666203    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-istiod-ca-cert\") pod \"e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f\" (UID: \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\") " pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:28:21.666318 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:21.666213    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-istio-envoy\") pod \"e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f\" (UID: \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\") " pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:28:21.666519 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:21.666492    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-istio-data\") pod \"e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f\" (UID: \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\") " pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:28:21.668228 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:21.668211    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-istio-envoy\") pod \"e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f\" (UID: \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\") " pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:28:21.668431 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:21.668413    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-istio-podinfo\") pod \"e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f\" (UID: \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\") " pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:28:21.681827 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:21.681798    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-istio-token\") pod \"e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f\" (UID: \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\") " pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:28:21.681998 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:21.681981    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69cvw\" (UniqueName: \"kubernetes.io/projected/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-kube-api-access-69cvw\") pod \"e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f\" (UID: \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\") " pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:28:21.827641 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:21.827556    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:28:22.104242 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:22.104217    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"]
Sep 09 04:28:22.106156 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:28:22.106128    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod835c5d7a_9dd8_47d6_9ab9_f92a150a1ecf.slice/crio-7739b300fbaa051791a57c8e39a6a4f2e9337f76781fb85fcb1e87de9eb66366 WatchSource:0}: Error finding container 7739b300fbaa051791a57c8e39a6a4f2e9337f76781fb85fcb1e87de9eb66366: Status 404 returned error can't find the container with id 7739b300fbaa051791a57c8e39a6a4f2e9337f76781fb85fcb1e87de9eb66366
Sep 09 04:28:22.108323 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:22.108292    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 09 04:28:22.108401 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:22.108382    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 09 04:28:22.108441 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:22.108409    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 09 04:28:22.418460 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:22.418423    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f" event={"ID":"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf","Type":"ContainerStarted","Data":"d50498dc4b939d38325c94a2f0ab34e34c5e69e4cf0e831c54c00781b7c810c3"}
Sep 09 04:28:22.418634 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:22.418474    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f" event={"ID":"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf","Type":"ContainerStarted","Data":"7739b300fbaa051791a57c8e39a6a4f2e9337f76781fb85fcb1e87de9eb66366"}
Sep 09 04:28:22.442770 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:22.442715    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f" podStartSLOduration=1.442700014 podStartE2EDuration="1.442700014s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:28:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:28:22.440981108 +0000 UTC m=+1827.360152016" watchObservedRunningTime="2026-09-09 04:28:22.442700014 +0000 UTC m=+1827.361870922"
Sep 09 04:28:22.828483 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:22.828405    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:28:23.832960 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:23.832931    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:28:24.428233 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:24.428200    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:28:24.428974 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:24.428953    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:28:26.301564 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:26.301531    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0db87d-8c8cd5cc-9l756"]
Sep 09 04:28:26.304765 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:26.304748    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-0db87d-8c8cd5cc-9l756"
Sep 09 04:28:26.306995 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:26.306967    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-0db87d\""
Sep 09 04:28:26.307143 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:26.307045    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-0db87d-dockercfg-76pkh\""
Sep 09 04:28:26.317002 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:26.316979    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0db87d-8c8cd5cc-9l756"]
Sep 09 04:28:26.412108 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:26.412070    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rptn7\" (UniqueName: \"kubernetes.io/projected/c106ec1a-0d7e-43ce-b371-22ff3fda3baa-kube-api-access-rptn7\") pod \"payload-processing-e2e-ipp-b-0db87d-8c8cd5cc-9l756\" (UID: \"c106ec1a-0d7e-43ce-b371-22ff3fda3baa\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0db87d-8c8cd5cc-9l756"
Sep 09 04:28:26.412319 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:26.412149    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c106ec1a-0d7e-43ce-b371-22ff3fda3baa-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-0db87d-8c8cd5cc-9l756\" (UID: \"c106ec1a-0d7e-43ce-b371-22ff3fda3baa\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0db87d-8c8cd5cc-9l756"
Sep 09 04:28:26.513581 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:26.513549    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c106ec1a-0d7e-43ce-b371-22ff3fda3baa-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-0db87d-8c8cd5cc-9l756\" (UID: \"c106ec1a-0d7e-43ce-b371-22ff3fda3baa\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0db87d-8c8cd5cc-9l756"
Sep 09 04:28:26.513765 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:26.513739    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rptn7\" (UniqueName: \"kubernetes.io/projected/c106ec1a-0d7e-43ce-b371-22ff3fda3baa-kube-api-access-rptn7\") pod \"payload-processing-e2e-ipp-b-0db87d-8c8cd5cc-9l756\" (UID: \"c106ec1a-0d7e-43ce-b371-22ff3fda3baa\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0db87d-8c8cd5cc-9l756"
Sep 09 04:28:26.514327 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:26.514305    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c106ec1a-0d7e-43ce-b371-22ff3fda3baa-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-0db87d-8c8cd5cc-9l756\" (UID: \"c106ec1a-0d7e-43ce-b371-22ff3fda3baa\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0db87d-8c8cd5cc-9l756"
Sep 09 04:28:26.532431 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:26.532403    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rptn7\" (UniqueName: \"kubernetes.io/projected/c106ec1a-0d7e-43ce-b371-22ff3fda3baa-kube-api-access-rptn7\") pod \"payload-processing-e2e-ipp-b-0db87d-8c8cd5cc-9l756\" (UID: \"c106ec1a-0d7e-43ce-b371-22ff3fda3baa\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0db87d-8c8cd5cc-9l756"
Sep 09 04:28:26.615408 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:26.615380    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-0db87d-8c8cd5cc-9l756"
Sep 09 04:28:26.888191 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:26.888164    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0db87d-8c8cd5cc-9l756"]
Sep 09 04:28:26.890323 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:28:26.890291    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc106ec1a_0d7e_43ce_b371_22ff3fda3baa.slice/crio-4f640bc1ad8f01a2d7d8db80da4b8abb46fcda37ecc561aa09ccd4bfde38d1bb WatchSource:0}: Error finding container 4f640bc1ad8f01a2d7d8db80da4b8abb46fcda37ecc561aa09ccd4bfde38d1bb: Status 404 returned error can't find the container with id 4f640bc1ad8f01a2d7d8db80da4b8abb46fcda37ecc561aa09ccd4bfde38d1bb
Sep 09 04:28:27.443469 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:27.443428    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-0db87d-8c8cd5cc-9l756" event={"ID":"c106ec1a-0d7e-43ce-b371-22ff3fda3baa","Type":"ContainerStarted","Data":"cb2de555c7096abaeafb2ea5fa37ea636d718060a70906675616adeb4ba25220"}
Sep 09 04:28:27.443874 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:27.443479    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-0db87d-8c8cd5cc-9l756" event={"ID":"c106ec1a-0d7e-43ce-b371-22ff3fda3baa","Type":"ContainerStarted","Data":"4f640bc1ad8f01a2d7d8db80da4b8abb46fcda37ecc561aa09ccd4bfde38d1bb"}
Sep 09 04:28:27.443874 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:27.443572    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-0db87d-8c8cd5cc-9l756"
Sep 09 04:28:27.461859 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:27.461820    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-0db87d-8c8cd5cc-9l756" podStartSLOduration=1.461807826 podStartE2EDuration="1.461807826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:28:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:28:27.461643048 +0000 UTC m=+1832.380813959" watchObservedRunningTime="2026-09-09 04:28:27.461807826 +0000 UTC m=+1832.380978735"
Sep 09 04:28:38.450975 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:38.450944    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-0db87d-8c8cd5cc-9l756"
Sep 09 04:28:43.524476 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:43.524440    2557 generic.go:356] "Generic (PLEG): container finished" podID="835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf" containerID="d50498dc4b939d38325c94a2f0ab34e34c5e69e4cf0e831c54c00781b7c810c3" exitCode=0
Sep 09 04:28:43.524878 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:43.524496    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f" event={"ID":"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf","Type":"ContainerDied","Data":"d50498dc4b939d38325c94a2f0ab34e34c5e69e4cf0e831c54c00781b7c810c3"}
Sep 09 04:28:43.524878 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:43.524857    2557 scope.go:117] "RemoveContainer" containerID="d50498dc4b939d38325c94a2f0ab34e34c5e69e4cf0e831c54c00781b7c810c3"
Sep 09 04:28:43.525883 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:43.525848    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 09 04:28:43.525933 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:43.525920    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 09 04:28:43.525971 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:43.525952    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 09 04:28:44.531866 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:44.531832    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f" event={"ID":"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf","Type":"ContainerStarted","Data":"97450b70be4c1608858870cadbe87bcdaddf23f1e217076ac78a298ebdf0c28a"}
Sep 09 04:28:44.828826 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:44.828737    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:28:44.828995 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:44.828973    2557 patch_prober.go:28] interesting pod/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Sep 09 04:28:44.829054 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:44.829034    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f" podUID="835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Sep 09 04:28:45.828030 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:45.827989    2557 patch_prober.go:28] interesting pod/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Sep 09 04:28:45.828430 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:45.828049    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f" podUID="835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Sep 09 04:28:46.828710 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:46.828670    2557 patch_prober.go:28] interesting pod/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Sep 09 04:28:46.829073 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:46.828743    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f" podUID="835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Sep 09 04:28:47.828015 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:47.827978    2557 patch_prober.go:28] interesting pod/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Sep 09 04:28:47.828207 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:47.828049    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f" podUID="835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Sep 09 04:28:48.828344 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:48.828310    2557 patch_prober.go:28] interesting pod/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Sep 09 04:28:48.828727 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:48.828363    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f" podUID="835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Sep 09 04:28:49.832550 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:49.832521    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:28:49.832978 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:49.832821    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:28:49.833514 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:49.833494    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:28:54.411730 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:54.411703    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-q4xpl_2f2c11af-63a3-41d0-8bab-12654e7b98be/payload-processing/0.log"
Sep 09 04:28:54.530201 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:28:54.530168    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-0db87d-8c8cd5cc-9l756_c106ec1a-0d7e-43ce-b371-22ff3fda3baa/payload-processing/0.log"
Sep 09 04:29:30.466910 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:29:30.466823    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-q4xpl_2f2c11af-63a3-41d0-8bab-12654e7b98be/payload-processing/0.log"
Sep 09 04:29:30.595369 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:29:30.595338    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-0db87d-8c8cd5cc-9l756_c106ec1a-0d7e-43ce-b371-22ff3fda3baa/payload-processing/0.log"
Sep 09 04:29:36.022307 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:29:36.022274    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e787bf-57f697768b-z2bfs"]
Sep 09 04:29:36.025811 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:29:36.025794    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e787bf-57f697768b-z2bfs"
Sep 09 04:29:36.027953 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:29:36.027928    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-e787bf-dockercfg-d8d6s\""
Sep 09 04:29:36.028069 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:29:36.027969    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-e787bf\""
Sep 09 04:29:36.036650 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:29:36.036627    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e787bf-57f697768b-z2bfs"]
Sep 09 04:29:36.137087 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:29:36.137044    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ml2wv\" (UniqueName: \"kubernetes.io/projected/9e053347-0cd2-4d99-b1f1-b057aaf20523-kube-api-access-ml2wv\") pod \"payload-pre-processing-e2e-ipp-cleanup-e787bf-57f697768b-z2bfs\" (UID: \"9e053347-0cd2-4d99-b1f1-b057aaf20523\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e787bf-57f697768b-z2bfs"
Sep 09 04:29:36.137301 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:29:36.137194    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e053347-0cd2-4d99-b1f1-b057aaf20523-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-e787bf-57f697768b-z2bfs\" (UID: \"9e053347-0cd2-4d99-b1f1-b057aaf20523\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e787bf-57f697768b-z2bfs"
Sep 09 04:29:36.238376 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:29:36.238344    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ml2wv\" (UniqueName: \"kubernetes.io/projected/9e053347-0cd2-4d99-b1f1-b057aaf20523-kube-api-access-ml2wv\") pod \"payload-pre-processing-e2e-ipp-cleanup-e787bf-57f697768b-z2bfs\" (UID: \"9e053347-0cd2-4d99-b1f1-b057aaf20523\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e787bf-57f697768b-z2bfs"
Sep 09 04:29:36.238576 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:29:36.238458    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e053347-0cd2-4d99-b1f1-b057aaf20523-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-e787bf-57f697768b-z2bfs\" (UID: \"9e053347-0cd2-4d99-b1f1-b057aaf20523\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e787bf-57f697768b-z2bfs"
Sep 09 04:29:36.239156 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:29:36.239108    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e053347-0cd2-4d99-b1f1-b057aaf20523-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-e787bf-57f697768b-z2bfs\" (UID: \"9e053347-0cd2-4d99-b1f1-b057aaf20523\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e787bf-57f697768b-z2bfs"
Sep 09 04:29:36.249608 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:29:36.249577    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ml2wv\" (UniqueName: \"kubernetes.io/projected/9e053347-0cd2-4d99-b1f1-b057aaf20523-kube-api-access-ml2wv\") pod \"payload-pre-processing-e2e-ipp-cleanup-e787bf-57f697768b-z2bfs\" (UID: \"9e053347-0cd2-4d99-b1f1-b057aaf20523\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e787bf-57f697768b-z2bfs"
Sep 09 04:29:36.337183 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:29:36.337099    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e787bf-57f697768b-z2bfs"
Sep 09 04:29:36.606522 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:29:36.606498    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e787bf-57f697768b-z2bfs"]
Sep 09 04:29:36.608937 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:29:36.608908    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e053347_0cd2_4d99_b1f1_b057aaf20523.slice/crio-8b0d083be65a4a1f7e4cf8e8d46b04897cea65da8b086298f035721e6bbd2b45 WatchSource:0}: Error finding container 8b0d083be65a4a1f7e4cf8e8d46b04897cea65da8b086298f035721e6bbd2b45: Status 404 returned error can't find the container with id 8b0d083be65a4a1f7e4cf8e8d46b04897cea65da8b086298f035721e6bbd2b45
Sep 09 04:29:36.778824 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:29:36.778776    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e787bf-57f697768b-z2bfs" event={"ID":"9e053347-0cd2-4d99-b1f1-b057aaf20523","Type":"ContainerStarted","Data":"8dd331651076a7c4dc27167644b2b1c20627282f60319998ee89de9207dcb61a"}
Sep 09 04:29:36.779006 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:29:36.778849    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e787bf-57f697768b-z2bfs" event={"ID":"9e053347-0cd2-4d99-b1f1-b057aaf20523","Type":"ContainerStarted","Data":"8b0d083be65a4a1f7e4cf8e8d46b04897cea65da8b086298f035721e6bbd2b45"}
Sep 09 04:29:36.779006 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:29:36.778911    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e787bf-57f697768b-z2bfs"
Sep 09 04:29:36.795133 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:29:36.795064    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e787bf-57f697768b-z2bfs" podStartSLOduration=0.79504732 podStartE2EDuration="795.04732ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:29:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:29:36.793470065 +0000 UTC m=+1901.712640974" watchObservedRunningTime="2026-09-09 04:29:36.79504732 +0000 UTC m=+1901.714218228"
Sep 09 04:29:47.787285 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:29:47.787256    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e787bf-57f697768b-z2bfs"
Sep 09 04:30:00.143374 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:00.143338    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-426183-29815470-hkc87"]
Sep 09 04:30:00.147307 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:00.147282    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a3ce90-29815470-w22mx"]
Sep 09 04:30:00.147456 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:00.147403    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-426183-29815470-hkc87"
Sep 09 04:30:00.150030 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:00.150007    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qlbvk\""
Sep 09 04:30:00.150727 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:00.150706    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a3ce90-29815470-w22mx"
Sep 09 04:30:00.158467 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:00.158361    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-426183-29815470-hkc87"]
Sep 09 04:30:00.160237 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:00.160216    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a3ce90-29815470-w22mx"]
Sep 09 04:30:00.165464 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:00.165448    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-426183-29815470-hkc87"
Sep 09 04:30:00.173399 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:00.173381    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a3ce90-29815470-w22mx"
Sep 09 04:30:00.449138 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:00.449096    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-426183-29815470-hkc87"]
Sep 09 04:30:00.451101 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:30:00.451077    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17dcf909_82c7_4ac9_b9df_ef773a2f4681.slice/crio-30b1ed60c92fa2fc920a301e0645d8b6d557636c14fafc929f26fe0c8618435b WatchSource:0}: Error finding container 30b1ed60c92fa2fc920a301e0645d8b6d557636c14fafc929f26fe0c8618435b: Status 404 returned error can't find the container with id 30b1ed60c92fa2fc920a301e0645d8b6d557636c14fafc929f26fe0c8618435b
Sep 09 04:30:00.457715 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:00.457695    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a3ce90-29815470-w22mx"]
Sep 09 04:30:00.459455 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:30:00.459434    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3132b51a_1376_4bee_9c6b_71024b105699.slice/crio-543aa4e129b4448b98540674415fb9aa044e77de698365120470fa0374d1d182 WatchSource:0}: Error finding container 543aa4e129b4448b98540674415fb9aa044e77de698365120470fa0374d1d182: Status 404 returned error can't find the container with id 543aa4e129b4448b98540674415fb9aa044e77de698365120470fa0374d1d182
Sep 09 04:30:00.898785 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:00.898748    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a3ce90-29815470-w22mx" event={"ID":"3132b51a-1376-4bee-9c6b-71024b105699","Type":"ContainerStarted","Data":"4cc61412bc2c90d775be23f0da1fc1537926bc55725a671cd2126307dbb04867"}
Sep 09 04:30:00.898937 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:00.898805    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a3ce90-29815470-w22mx" event={"ID":"3132b51a-1376-4bee-9c6b-71024b105699","Type":"ContainerStarted","Data":"543aa4e129b4448b98540674415fb9aa044e77de698365120470fa0374d1d182"}
Sep 09 04:30:00.900275 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:00.900251    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-426183-29815470-hkc87" event={"ID":"17dcf909-82c7-4ac9-b9df-ef773a2f4681","Type":"ContainerStarted","Data":"e6d69aed58da59cebeb00c9b409b5de7cd5c7878cc042244d9008ab765106beb"}
Sep 09 04:30:00.900375 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:00.900288    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-426183-29815470-hkc87" event={"ID":"17dcf909-82c7-4ac9-b9df-ef773a2f4681","Type":"ContainerStarted","Data":"30b1ed60c92fa2fc920a301e0645d8b6d557636c14fafc929f26fe0c8618435b"}
Sep 09 04:30:00.914481 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:00.914443    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a3ce90-29815470-w22mx" podStartSLOduration=0.914429708 podStartE2EDuration="914.429708ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:30:00.913323696 +0000 UTC m=+1925.832494605" watchObservedRunningTime="2026-09-09 04:30:00.914429708 +0000 UTC m=+1925.833600617"
Sep 09 04:30:00.931385 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:00.931351    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-426183-29815470-hkc87" podStartSLOduration=0.931337316 podStartE2EDuration="931.337316ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:30:00.930433599 +0000 UTC m=+1925.849604507" watchObservedRunningTime="2026-09-09 04:30:00.931337316 +0000 UTC m=+1925.850508224"
Sep 09 04:30:01.907784 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:01.907752    2557 generic.go:356] "Generic (PLEG): container finished" podID="3132b51a-1376-4bee-9c6b-71024b105699" containerID="4cc61412bc2c90d775be23f0da1fc1537926bc55725a671cd2126307dbb04867" exitCode=0
Sep 09 04:30:01.908230 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:01.907842    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a3ce90-29815470-w22mx" event={"ID":"3132b51a-1376-4bee-9c6b-71024b105699","Type":"ContainerDied","Data":"4cc61412bc2c90d775be23f0da1fc1537926bc55725a671cd2126307dbb04867"}
Sep 09 04:30:01.909318 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:01.909296    2557 generic.go:356] "Generic (PLEG): container finished" podID="17dcf909-82c7-4ac9-b9df-ef773a2f4681" containerID="e6d69aed58da59cebeb00c9b409b5de7cd5c7878cc042244d9008ab765106beb" exitCode=0
Sep 09 04:30:01.909396 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:01.909358    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-426183-29815470-hkc87" event={"ID":"17dcf909-82c7-4ac9-b9df-ef773a2f4681","Type":"ContainerDied","Data":"e6d69aed58da59cebeb00c9b409b5de7cd5c7878cc042244d9008ab765106beb"}
Sep 09 04:30:03.039593 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:03.039568    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a3ce90-29815470-w22mx"
Sep 09 04:30:03.090034 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:03.090006    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-426183-29815470-hkc87"
Sep 09 04:30:03.927920 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:03.927841    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a3ce90-29815470-w22mx"
Sep 09 04:30:03.928077 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:03.927842    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a3ce90-29815470-w22mx" event={"ID":"3132b51a-1376-4bee-9c6b-71024b105699","Type":"ContainerDied","Data":"543aa4e129b4448b98540674415fb9aa044e77de698365120470fa0374d1d182"}
Sep 09 04:30:03.928077 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:03.927999    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="543aa4e129b4448b98540674415fb9aa044e77de698365120470fa0374d1d182"
Sep 09 04:30:03.929167 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:03.929148    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-426183-29815470-hkc87"
Sep 09 04:30:03.929167 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:03.929148    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-426183-29815470-hkc87" event={"ID":"17dcf909-82c7-4ac9-b9df-ef773a2f4681","Type":"ContainerDied","Data":"30b1ed60c92fa2fc920a301e0645d8b6d557636c14fafc929f26fe0c8618435b"}
Sep 09 04:30:03.929352 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:03.929195    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="30b1ed60c92fa2fc920a301e0645d8b6d557636c14fafc929f26fe0c8618435b"
Sep 09 04:30:06.064083 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:06.064051    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e787bf-57f697768b-z2bfs"]
Sep 09 04:30:06.064539 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:06.064322    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e787bf-57f697768b-z2bfs" podUID="9e053347-0cd2-4d99-b1f1-b057aaf20523" containerName="payload-pre-processing" containerID="cri-o://8dd331651076a7c4dc27167644b2b1c20627282f60319998ee89de9207dcb61a" gracePeriod=30
Sep 09 04:30:06.324963 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:06.324906    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e787bf-57f697768b-z2bfs"
Sep 09 04:30:06.406092 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:06.406022    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ml2wv\" (UniqueName: \"kubernetes.io/projected/9e053347-0cd2-4d99-b1f1-b057aaf20523-kube-api-access-ml2wv\") pod \"9e053347-0cd2-4d99-b1f1-b057aaf20523\" (UID: \"9e053347-0cd2-4d99-b1f1-b057aaf20523\") "
Sep 09 04:30:06.406354 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:06.406189    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e053347-0cd2-4d99-b1f1-b057aaf20523-plugins-config-volume\") pod \"9e053347-0cd2-4d99-b1f1-b057aaf20523\" (UID: \"9e053347-0cd2-4d99-b1f1-b057aaf20523\") "
Sep 09 04:30:06.406595 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:06.406563    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e053347-0cd2-4d99-b1f1-b057aaf20523-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9e053347-0cd2-4d99-b1f1-b057aaf20523" (UID: "9e053347-0cd2-4d99-b1f1-b057aaf20523"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 04:30:06.408383 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:06.408360    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e053347-0cd2-4d99-b1f1-b057aaf20523-kube-api-access-ml2wv" (OuterVolumeSpecName: "kube-api-access-ml2wv") pod "9e053347-0cd2-4d99-b1f1-b057aaf20523" (UID: "9e053347-0cd2-4d99-b1f1-b057aaf20523"). InnerVolumeSpecName "kube-api-access-ml2wv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 04:30:06.507883 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:06.507842    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ml2wv\" (UniqueName: \"kubernetes.io/projected/9e053347-0cd2-4d99-b1f1-b057aaf20523-kube-api-access-ml2wv\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:30:06.507883 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:06.507879    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e053347-0cd2-4d99-b1f1-b057aaf20523-plugins-config-volume\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:30:06.945999 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:06.945967    2557 generic.go:356] "Generic (PLEG): container finished" podID="9e053347-0cd2-4d99-b1f1-b057aaf20523" containerID="8dd331651076a7c4dc27167644b2b1c20627282f60319998ee89de9207dcb61a" exitCode=0
Sep 09 04:30:06.946199 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:06.946072    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e787bf-57f697768b-z2bfs"
Sep 09 04:30:06.946199 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:06.946102    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e787bf-57f697768b-z2bfs" event={"ID":"9e053347-0cd2-4d99-b1f1-b057aaf20523","Type":"ContainerDied","Data":"8dd331651076a7c4dc27167644b2b1c20627282f60319998ee89de9207dcb61a"}
Sep 09 04:30:06.946199 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:06.946164    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e787bf-57f697768b-z2bfs" event={"ID":"9e053347-0cd2-4d99-b1f1-b057aaf20523","Type":"ContainerDied","Data":"8b0d083be65a4a1f7e4cf8e8d46b04897cea65da8b086298f035721e6bbd2b45"}
Sep 09 04:30:06.946199 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:06.946188    2557 scope.go:117] "RemoveContainer" containerID="8dd331651076a7c4dc27167644b2b1c20627282f60319998ee89de9207dcb61a"
Sep 09 04:30:06.955713 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:06.955696    2557 scope.go:117] "RemoveContainer" containerID="8dd331651076a7c4dc27167644b2b1c20627282f60319998ee89de9207dcb61a"
Sep 09 04:30:06.955977 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:30:06.955955    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8dd331651076a7c4dc27167644b2b1c20627282f60319998ee89de9207dcb61a\": container with ID starting with 8dd331651076a7c4dc27167644b2b1c20627282f60319998ee89de9207dcb61a not found: ID does not exist" containerID="8dd331651076a7c4dc27167644b2b1c20627282f60319998ee89de9207dcb61a"
Sep 09 04:30:06.956040 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:06.955984    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dd331651076a7c4dc27167644b2b1c20627282f60319998ee89de9207dcb61a"} err="failed to get container status \"8dd331651076a7c4dc27167644b2b1c20627282f60319998ee89de9207dcb61a\": rpc error: code = NotFound desc = could not find container \"8dd331651076a7c4dc27167644b2b1c20627282f60319998ee89de9207dcb61a\": container with ID starting with 8dd331651076a7c4dc27167644b2b1c20627282f60319998ee89de9207dcb61a not found: ID does not exist"
Sep 09 04:30:06.968633 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:06.968607    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e787bf-57f697768b-z2bfs"]
Sep 09 04:30:06.972847 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:06.972825    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e787bf-57f697768b-z2bfs"]
Sep 09 04:30:07.618821 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:07.618790    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e053347-0cd2-4d99-b1f1-b057aaf20523" path="/var/lib/kubelet/pods/9e053347-0cd2-4d99-b1f1-b057aaf20523/volumes"
Sep 09 04:30:32.482350 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:32.482302    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw"]
Sep 09 04:30:32.483004 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:32.482576    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw" podUID="57960f27-5f23-417b-82be-883363e51719" containerName="maas-api" containerID="cri-o://82acc72b3fe6255ead8cd7139ec18aba7182982bd10519b34254097bdbcd2942" gracePeriod=30
Sep 09 04:30:32.514939 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:32.514907    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a3ce90-29815470-w22mx"]
Sep 09 04:30:32.516689 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:32.516663    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-a3ce90-29815470-w22mx"]
Sep 09 04:30:32.548423 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:32.548398    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a3ce90-66c9f5b478-p8z7j"]
Sep 09 04:30:32.548788 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:32.548655    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a3ce90-66c9f5b478-p8z7j" podUID="44fc4aac-f5ff-4c2d-af27-b2624fc70366" containerName="payload-pre-processing" containerID="cri-o://fa79ff5964794dfdb2aea1fabf68787be21e101633bc11ecb9539f3ae7800d44" gracePeriod=30
Sep 09 04:30:32.800307 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:32.800279    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw"
Sep 09 04:30:32.836236 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:32.836208    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57960f27-5f23-417b-82be-883363e51719-metrics-certs\") pod \"57960f27-5f23-417b-82be-883363e51719\" (UID: \"57960f27-5f23-417b-82be-883363e51719\") "
Sep 09 04:30:32.836386 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:32.836296    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hktsk\" (UniqueName: \"kubernetes.io/projected/57960f27-5f23-417b-82be-883363e51719-kube-api-access-hktsk\") pod \"57960f27-5f23-417b-82be-883363e51719\" (UID: \"57960f27-5f23-417b-82be-883363e51719\") "
Sep 09 04:30:32.836456 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:32.836393    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57960f27-5f23-417b-82be-883363e51719-openshift-service-ca\") pod \"57960f27-5f23-417b-82be-883363e51719\" (UID: \"57960f27-5f23-417b-82be-883363e51719\") "
Sep 09 04:30:32.836456 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:32.836432    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57960f27-5f23-417b-82be-883363e51719-maas-api-tls\") pod \"57960f27-5f23-417b-82be-883363e51719\" (UID: \"57960f27-5f23-417b-82be-883363e51719\") "
Sep 09 04:30:32.836840 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:32.836802    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57960f27-5f23-417b-82be-883363e51719-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "57960f27-5f23-417b-82be-883363e51719" (UID: "57960f27-5f23-417b-82be-883363e51719"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 04:30:32.838862 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:32.838831    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57960f27-5f23-417b-82be-883363e51719-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "57960f27-5f23-417b-82be-883363e51719" (UID: "57960f27-5f23-417b-82be-883363e51719"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 04:30:32.838962 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:32.838883    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57960f27-5f23-417b-82be-883363e51719-kube-api-access-hktsk" (OuterVolumeSpecName: "kube-api-access-hktsk") pod "57960f27-5f23-417b-82be-883363e51719" (UID: "57960f27-5f23-417b-82be-883363e51719"). InnerVolumeSpecName "kube-api-access-hktsk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 04:30:32.839736 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:32.839710    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57960f27-5f23-417b-82be-883363e51719-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "57960f27-5f23-417b-82be-883363e51719" (UID: "57960f27-5f23-417b-82be-883363e51719"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 04:30:32.847675 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:32.847653    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a3ce90-66c9f5b478-p8z7j"
Sep 09 04:30:32.937798 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:32.937767    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44fc4aac-f5ff-4c2d-af27-b2624fc70366-plugins-config-volume\") pod \"44fc4aac-f5ff-4c2d-af27-b2624fc70366\" (UID: \"44fc4aac-f5ff-4c2d-af27-b2624fc70366\") "
Sep 09 04:30:32.937954 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:32.937847    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9nj98\" (UniqueName: \"kubernetes.io/projected/44fc4aac-f5ff-4c2d-af27-b2624fc70366-kube-api-access-9nj98\") pod \"44fc4aac-f5ff-4c2d-af27-b2624fc70366\" (UID: \"44fc4aac-f5ff-4c2d-af27-b2624fc70366\") "
Sep 09 04:30:32.938057 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:32.938046    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57960f27-5f23-417b-82be-883363e51719-openshift-service-ca\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:30:32.938140 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:32.938059    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57960f27-5f23-417b-82be-883363e51719-maas-api-tls\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:30:32.938140 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:32.938067    2557 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57960f27-5f23-417b-82be-883363e51719-metrics-certs\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:30:32.938140 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:32.938075    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hktsk\" (UniqueName: \"kubernetes.io/projected/57960f27-5f23-417b-82be-883363e51719-kube-api-access-hktsk\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:30:32.938246 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:32.938139    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44fc4aac-f5ff-4c2d-af27-b2624fc70366-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "44fc4aac-f5ff-4c2d-af27-b2624fc70366" (UID: "44fc4aac-f5ff-4c2d-af27-b2624fc70366"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 04:30:32.939776 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:32.939752    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44fc4aac-f5ff-4c2d-af27-b2624fc70366-kube-api-access-9nj98" (OuterVolumeSpecName: "kube-api-access-9nj98") pod "44fc4aac-f5ff-4c2d-af27-b2624fc70366" (UID: "44fc4aac-f5ff-4c2d-af27-b2624fc70366"). InnerVolumeSpecName "kube-api-access-9nj98". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 04:30:33.039194 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:33.039097    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9nj98\" (UniqueName: \"kubernetes.io/projected/44fc4aac-f5ff-4c2d-af27-b2624fc70366-kube-api-access-9nj98\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:30:33.039194 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:33.039145    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44fc4aac-f5ff-4c2d-af27-b2624fc70366-plugins-config-volume\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:30:33.076551 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:33.076517    2557 generic.go:356] "Generic (PLEG): container finished" podID="44fc4aac-f5ff-4c2d-af27-b2624fc70366" containerID="fa79ff5964794dfdb2aea1fabf68787be21e101633bc11ecb9539f3ae7800d44" exitCode=0
Sep 09 04:30:33.076692 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:33.076612    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a3ce90-66c9f5b478-p8z7j"
Sep 09 04:30:33.076692 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:33.076667    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a3ce90-66c9f5b478-p8z7j" event={"ID":"44fc4aac-f5ff-4c2d-af27-b2624fc70366","Type":"ContainerDied","Data":"fa79ff5964794dfdb2aea1fabf68787be21e101633bc11ecb9539f3ae7800d44"}
Sep 09 04:30:33.076847 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:33.076725    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-a3ce90-66c9f5b478-p8z7j" event={"ID":"44fc4aac-f5ff-4c2d-af27-b2624fc70366","Type":"ContainerDied","Data":"4d07b285d7ab545f95a04e65806ab403b66c1aa7a6a7cf9473df4edf2b8b7ff3"}
Sep 09 04:30:33.076847 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:33.076758    2557 scope.go:117] "RemoveContainer" containerID="fa79ff5964794dfdb2aea1fabf68787be21e101633bc11ecb9539f3ae7800d44"
Sep 09 04:30:33.078270 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:33.078245    2557 generic.go:356] "Generic (PLEG): container finished" podID="57960f27-5f23-417b-82be-883363e51719" containerID="82acc72b3fe6255ead8cd7139ec18aba7182982bd10519b34254097bdbcd2942" exitCode=0
Sep 09 04:30:33.078424 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:33.078287    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw"
Sep 09 04:30:33.078424 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:33.078316    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw" event={"ID":"57960f27-5f23-417b-82be-883363e51719","Type":"ContainerDied","Data":"82acc72b3fe6255ead8cd7139ec18aba7182982bd10519b34254097bdbcd2942"}
Sep 09 04:30:33.078638 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:33.078431    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw" event={"ID":"57960f27-5f23-417b-82be-883363e51719","Type":"ContainerDied","Data":"6742d4ea765efebd5120c89047b086c4f95c7c565b49854c7f4d1cd886411cbb"}
Sep 09 04:30:33.087295 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:33.087280    2557 scope.go:117] "RemoveContainer" containerID="fa79ff5964794dfdb2aea1fabf68787be21e101633bc11ecb9539f3ae7800d44"
Sep 09 04:30:33.087566 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:30:33.087537    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa79ff5964794dfdb2aea1fabf68787be21e101633bc11ecb9539f3ae7800d44\": container with ID starting with fa79ff5964794dfdb2aea1fabf68787be21e101633bc11ecb9539f3ae7800d44 not found: ID does not exist" containerID="fa79ff5964794dfdb2aea1fabf68787be21e101633bc11ecb9539f3ae7800d44"
Sep 09 04:30:33.087658 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:33.087577    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa79ff5964794dfdb2aea1fabf68787be21e101633bc11ecb9539f3ae7800d44"} err="failed to get container status \"fa79ff5964794dfdb2aea1fabf68787be21e101633bc11ecb9539f3ae7800d44\": rpc error: code = NotFound desc = could not find container \"fa79ff5964794dfdb2aea1fabf68787be21e101633bc11ecb9539f3ae7800d44\": container with ID starting with fa79ff5964794dfdb2aea1fabf68787be21e101633bc11ecb9539f3ae7800d44 not found: ID does not exist"
Sep 09 04:30:33.087658 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:33.087596    2557 scope.go:117] "RemoveContainer" containerID="82acc72b3fe6255ead8cd7139ec18aba7182982bd10519b34254097bdbcd2942"
Sep 09 04:30:33.095639 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:33.095624    2557 scope.go:117] "RemoveContainer" containerID="82acc72b3fe6255ead8cd7139ec18aba7182982bd10519b34254097bdbcd2942"
Sep 09 04:30:33.095885 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:30:33.095865    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82acc72b3fe6255ead8cd7139ec18aba7182982bd10519b34254097bdbcd2942\": container with ID starting with 82acc72b3fe6255ead8cd7139ec18aba7182982bd10519b34254097bdbcd2942 not found: ID does not exist" containerID="82acc72b3fe6255ead8cd7139ec18aba7182982bd10519b34254097bdbcd2942"
Sep 09 04:30:33.095959 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:33.095903    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82acc72b3fe6255ead8cd7139ec18aba7182982bd10519b34254097bdbcd2942"} err="failed to get container status \"82acc72b3fe6255ead8cd7139ec18aba7182982bd10519b34254097bdbcd2942\": rpc error: code = NotFound desc = could not find container \"82acc72b3fe6255ead8cd7139ec18aba7182982bd10519b34254097bdbcd2942\": container with ID starting with 82acc72b3fe6255ead8cd7139ec18aba7182982bd10519b34254097bdbcd2942 not found: ID does not exist"
Sep 09 04:30:33.100083 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:33.100063    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a3ce90-66c9f5b478-p8z7j"]
Sep 09 04:30:33.104637 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:33.104618    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-a3ce90-66c9f5b478-p8z7j"]
Sep 09 04:30:33.115377 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:33.115353    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw"]
Sep 09 04:30:33.119370 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:33.119352    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-a3ce90-5fbddbdf6b-pkqhw"]
Sep 09 04:30:33.619672 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:33.619642    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3132b51a-1376-4bee-9c6b-71024b105699" path="/var/lib/kubelet/pods/3132b51a-1376-4bee-9c6b-71024b105699/volumes"
Sep 09 04:30:33.620009 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:33.619958    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44fc4aac-f5ff-4c2d-af27-b2624fc70366" path="/var/lib/kubelet/pods/44fc4aac-f5ff-4c2d-af27-b2624fc70366/volumes"
Sep 09 04:30:33.620327 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:30:33.620314    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57960f27-5f23-417b-82be-883363e51719" path="/var/lib/kubelet/pods/57960f27-5f23-417b-82be-883363e51719/volumes"
Sep 09 04:31:32.770452 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:32.770410    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0db87d-8c8cd5cc-9l756"]
Sep 09 04:31:32.770967 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:32.770703    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-0db87d-8c8cd5cc-9l756" podUID="c106ec1a-0d7e-43ce-b371-22ff3fda3baa" containerName="payload-processing" containerID="cri-o://cb2de555c7096abaeafb2ea5fa37ea636d718060a70906675616adeb4ba25220" gracePeriod=30
Sep 09 04:31:33.029891 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:33.029830    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-0db87d-8c8cd5cc-9l756"
Sep 09 04:31:33.142689 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:33.142657    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rptn7\" (UniqueName: \"kubernetes.io/projected/c106ec1a-0d7e-43ce-b371-22ff3fda3baa-kube-api-access-rptn7\") pod \"c106ec1a-0d7e-43ce-b371-22ff3fda3baa\" (UID: \"c106ec1a-0d7e-43ce-b371-22ff3fda3baa\") "
Sep 09 04:31:33.142868 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:33.142735    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c106ec1a-0d7e-43ce-b371-22ff3fda3baa-plugins-config-volume\") pod \"c106ec1a-0d7e-43ce-b371-22ff3fda3baa\" (UID: \"c106ec1a-0d7e-43ce-b371-22ff3fda3baa\") "
Sep 09 04:31:33.143236 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:33.143196    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c106ec1a-0d7e-43ce-b371-22ff3fda3baa-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c106ec1a-0d7e-43ce-b371-22ff3fda3baa" (UID: "c106ec1a-0d7e-43ce-b371-22ff3fda3baa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 04:31:33.145133 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:33.145074    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c106ec1a-0d7e-43ce-b371-22ff3fda3baa-kube-api-access-rptn7" (OuterVolumeSpecName: "kube-api-access-rptn7") pod "c106ec1a-0d7e-43ce-b371-22ff3fda3baa" (UID: "c106ec1a-0d7e-43ce-b371-22ff3fda3baa"). InnerVolumeSpecName "kube-api-access-rptn7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 04:31:33.244098 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:33.244071    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rptn7\" (UniqueName: \"kubernetes.io/projected/c106ec1a-0d7e-43ce-b371-22ff3fda3baa-kube-api-access-rptn7\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:31:33.244098 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:33.244095    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c106ec1a-0d7e-43ce-b371-22ff3fda3baa-plugins-config-volume\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:31:33.374337 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:33.374307    2557 generic.go:356] "Generic (PLEG): container finished" podID="c106ec1a-0d7e-43ce-b371-22ff3fda3baa" containerID="cb2de555c7096abaeafb2ea5fa37ea636d718060a70906675616adeb4ba25220" exitCode=0
Sep 09 04:31:33.374508 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:33.374357    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-0db87d-8c8cd5cc-9l756"
Sep 09 04:31:33.374508 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:33.374408    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-0db87d-8c8cd5cc-9l756" event={"ID":"c106ec1a-0d7e-43ce-b371-22ff3fda3baa","Type":"ContainerDied","Data":"cb2de555c7096abaeafb2ea5fa37ea636d718060a70906675616adeb4ba25220"}
Sep 09 04:31:33.374508 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:33.374472    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-0db87d-8c8cd5cc-9l756" event={"ID":"c106ec1a-0d7e-43ce-b371-22ff3fda3baa","Type":"ContainerDied","Data":"4f640bc1ad8f01a2d7d8db80da4b8abb46fcda37ecc561aa09ccd4bfde38d1bb"}
Sep 09 04:31:33.374508 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:33.374501    2557 scope.go:117] "RemoveContainer" containerID="cb2de555c7096abaeafb2ea5fa37ea636d718060a70906675616adeb4ba25220"
Sep 09 04:31:33.383265 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:33.383242    2557 scope.go:117] "RemoveContainer" containerID="cb2de555c7096abaeafb2ea5fa37ea636d718060a70906675616adeb4ba25220"
Sep 09 04:31:33.383513 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:31:33.383497    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb2de555c7096abaeafb2ea5fa37ea636d718060a70906675616adeb4ba25220\": container with ID starting with cb2de555c7096abaeafb2ea5fa37ea636d718060a70906675616adeb4ba25220 not found: ID does not exist" containerID="cb2de555c7096abaeafb2ea5fa37ea636d718060a70906675616adeb4ba25220"
Sep 09 04:31:33.383569 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:33.383521    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb2de555c7096abaeafb2ea5fa37ea636d718060a70906675616adeb4ba25220"} err="failed to get container status \"cb2de555c7096abaeafb2ea5fa37ea636d718060a70906675616adeb4ba25220\": rpc error: code = NotFound desc = could not find container \"cb2de555c7096abaeafb2ea5fa37ea636d718060a70906675616adeb4ba25220\": container with ID starting with cb2de555c7096abaeafb2ea5fa37ea636d718060a70906675616adeb4ba25220 not found: ID does not exist"
Sep 09 04:31:33.396401 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:33.396375    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0db87d-8c8cd5cc-9l756"]
Sep 09 04:31:33.403259 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:33.403237    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0db87d-8c8cd5cc-9l756"]
Sep 09 04:31:33.619539 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:33.619510    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c106ec1a-0d7e-43ce-b371-22ff3fda3baa" path="/var/lib/kubelet/pods/c106ec1a-0d7e-43ce-b371-22ff3fda3baa/volumes"
Sep 09 04:31:39.847976 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:39.847936    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"]
Sep 09 04:31:39.848572 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:39.848217    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f" podUID="835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf" containerName="istio-proxy" containerID="cri-o://97450b70be4c1608858870cadbe87bcdaddf23f1e217076ac78a298ebdf0c28a" gracePeriod=30
Sep 09 04:31:45.101819 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.101798    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:31:45.245645 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.245550    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-istio-data\") pod \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\" (UID: \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\") "
Sep 09 04:31:45.245645 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.245601    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-istio-token\") pod \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\" (UID: \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\") "
Sep 09 04:31:45.245645 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.245622    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-istio-envoy\") pod \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\" (UID: \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\") "
Sep 09 04:31:45.245645 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.245642    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-istiod-ca-cert\") pod \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\" (UID: \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\") "
Sep 09 04:31:45.245970 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.245667    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-workload-certs\") pod \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\" (UID: \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\") "
Sep 09 04:31:45.245970 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.245715    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-workload-socket\") pod \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\" (UID: \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\") "
Sep 09 04:31:45.245970 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.245749    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-istio-podinfo\") pod \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\" (UID: \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\") "
Sep 09 04:31:45.245970 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.245775    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69cvw\" (UniqueName: \"kubernetes.io/projected/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-kube-api-access-69cvw\") pod \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\" (UID: \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\") "
Sep 09 04:31:45.245970 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.245843    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-credential-socket\") pod \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\" (UID: \"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf\") "
Sep 09 04:31:45.245970 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.245855    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-istio-data" (OuterVolumeSpecName: "istio-data") pod "835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf" (UID: "835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 04:31:45.246313 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.246073    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf" (UID: "835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 04:31:45.246313 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.246141    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-istio-data\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:31:45.246313 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.246138    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf" (UID: "835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 04:31:45.247056 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.246963    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf" (UID: "835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 04:31:45.247207 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.247051    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf" (UID: "835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 04:31:45.248683 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.248242    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf" (UID: "835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 04:31:45.248815 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.248719    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf" (UID: "835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 04:31:45.255025 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.254950    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-istio-token" (OuterVolumeSpecName: "istio-token") pod "835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf" (UID: "835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 04:31:45.255182 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.254950    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-kube-api-access-69cvw" (OuterVolumeSpecName: "kube-api-access-69cvw") pod "835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf" (UID: "835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf"). InnerVolumeSpecName "kube-api-access-69cvw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 04:31:45.347404 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.347366    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-credential-socket\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:31:45.347404 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.347405    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-istio-token\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:31:45.347593 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.347419    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-istio-envoy\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:31:45.347593 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.347432    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-istiod-ca-cert\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:31:45.347593 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.347444    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-workload-certs\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:31:45.347593 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.347457    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-workload-socket\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:31:45.347593 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.347468    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-istio-podinfo\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:31:45.347593 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.347482    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69cvw\" (UniqueName: \"kubernetes.io/projected/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf-kube-api-access-69cvw\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:31:45.434495 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.434459    2557 generic.go:356] "Generic (PLEG): container finished" podID="835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf" containerID="97450b70be4c1608858870cadbe87bcdaddf23f1e217076ac78a298ebdf0c28a" exitCode=0
Sep 09 04:31:45.434645 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.434528    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"
Sep 09 04:31:45.434645 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.434542    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f" event={"ID":"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf","Type":"ContainerDied","Data":"97450b70be4c1608858870cadbe87bcdaddf23f1e217076ac78a298ebdf0c28a"}
Sep 09 04:31:45.434645 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.434637    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f" event={"ID":"835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf","Type":"ContainerDied","Data":"7739b300fbaa051791a57c8e39a6a4f2e9337f76781fb85fcb1e87de9eb66366"}
Sep 09 04:31:45.434756 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.434666    2557 scope.go:117] "RemoveContainer" containerID="97450b70be4c1608858870cadbe87bcdaddf23f1e217076ac78a298ebdf0c28a"
Sep 09 04:31:45.443868 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.443850    2557 scope.go:117] "RemoveContainer" containerID="d50498dc4b939d38325c94a2f0ab34e34c5e69e4cf0e831c54c00781b7c810c3"
Sep 09 04:31:45.451481 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.451466    2557 scope.go:117] "RemoveContainer" containerID="97450b70be4c1608858870cadbe87bcdaddf23f1e217076ac78a298ebdf0c28a"
Sep 09 04:31:45.451735 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:31:45.451717    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97450b70be4c1608858870cadbe87bcdaddf23f1e217076ac78a298ebdf0c28a\": container with ID starting with 97450b70be4c1608858870cadbe87bcdaddf23f1e217076ac78a298ebdf0c28a not found: ID does not exist" containerID="97450b70be4c1608858870cadbe87bcdaddf23f1e217076ac78a298ebdf0c28a"
Sep 09 04:31:45.451774 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.451745    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97450b70be4c1608858870cadbe87bcdaddf23f1e217076ac78a298ebdf0c28a"} err="failed to get container status \"97450b70be4c1608858870cadbe87bcdaddf23f1e217076ac78a298ebdf0c28a\": rpc error: code = NotFound desc = could not find container \"97450b70be4c1608858870cadbe87bcdaddf23f1e217076ac78a298ebdf0c28a\": container with ID starting with 97450b70be4c1608858870cadbe87bcdaddf23f1e217076ac78a298ebdf0c28a not found: ID does not exist"
Sep 09 04:31:45.451774 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.451761    2557 scope.go:117] "RemoveContainer" containerID="d50498dc4b939d38325c94a2f0ab34e34c5e69e4cf0e831c54c00781b7c810c3"
Sep 09 04:31:45.451996 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:31:45.451978    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d50498dc4b939d38325c94a2f0ab34e34c5e69e4cf0e831c54c00781b7c810c3\": container with ID starting with d50498dc4b939d38325c94a2f0ab34e34c5e69e4cf0e831c54c00781b7c810c3 not found: ID does not exist" containerID="d50498dc4b939d38325c94a2f0ab34e34c5e69e4cf0e831c54c00781b7c810c3"
Sep 09 04:31:45.452033 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.452004    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d50498dc4b939d38325c94a2f0ab34e34c5e69e4cf0e831c54c00781b7c810c3"} err="failed to get container status \"d50498dc4b939d38325c94a2f0ab34e34c5e69e4cf0e831c54c00781b7c810c3\": rpc error: code = NotFound desc = could not find container \"d50498dc4b939d38325c94a2f0ab34e34c5e69e4cf0e831c54c00781b7c810c3\": container with ID starting with d50498dc4b939d38325c94a2f0ab34e34c5e69e4cf0e831c54c00781b7c810c3 not found: ID does not exist"
Sep 09 04:31:45.457621 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.457599    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"]
Sep 09 04:31:45.461044 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.461022    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-0db87d-openshift-default-575f544d4b-ctt4f"]
Sep 09 04:31:45.619571 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:45.619539    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf" path="/var/lib/kubelet/pods/835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf/volumes"
Sep 09 04:31:50.857161 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:50.857124    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-426183-29815470-hkc87"]
Sep 09 04:31:50.859229 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:50.859201    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-426183-29815470-hkc87"]
Sep 09 04:31:50.883288 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:50.883259    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-426183-6b9f4c95f5-hgch5"]
Sep 09 04:31:50.883616 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:50.883555    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-426183-6b9f4c95f5-hgch5" podUID="2cdc0724-54a2-424f-839d-203b761acd08" containerName="payload-processing" containerID="cri-o://f436252fb58e6e424526c31c375c524ce2f040531da40a5cf93df8c30f566198" gracePeriod=30
Sep 09 04:31:51.145556 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:51.145529    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-426183-6b9f4c95f5-hgch5"
Sep 09 04:31:51.302435 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:51.302393    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2cdc0724-54a2-424f-839d-203b761acd08-plugins-config-volume\") pod \"2cdc0724-54a2-424f-839d-203b761acd08\" (UID: \"2cdc0724-54a2-424f-839d-203b761acd08\") "
Sep 09 04:31:51.302435 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:51.302440    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6lbcl\" (UniqueName: \"kubernetes.io/projected/2cdc0724-54a2-424f-839d-203b761acd08-kube-api-access-6lbcl\") pod \"2cdc0724-54a2-424f-839d-203b761acd08\" (UID: \"2cdc0724-54a2-424f-839d-203b761acd08\") "
Sep 09 04:31:51.302854 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:51.302829    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2cdc0724-54a2-424f-839d-203b761acd08-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2cdc0724-54a2-424f-839d-203b761acd08" (UID: "2cdc0724-54a2-424f-839d-203b761acd08"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 04:31:51.304515 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:51.304493    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cdc0724-54a2-424f-839d-203b761acd08-kube-api-access-6lbcl" (OuterVolumeSpecName: "kube-api-access-6lbcl") pod "2cdc0724-54a2-424f-839d-203b761acd08" (UID: "2cdc0724-54a2-424f-839d-203b761acd08"). InnerVolumeSpecName "kube-api-access-6lbcl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 04:31:51.403857 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:51.403789    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2cdc0724-54a2-424f-839d-203b761acd08-plugins-config-volume\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:31:51.403857 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:51.403814    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6lbcl\" (UniqueName: \"kubernetes.io/projected/2cdc0724-54a2-424f-839d-203b761acd08-kube-api-access-6lbcl\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:31:51.476648 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:51.476618    2557 generic.go:356] "Generic (PLEG): container finished" podID="2cdc0724-54a2-424f-839d-203b761acd08" containerID="f436252fb58e6e424526c31c375c524ce2f040531da40a5cf93df8c30f566198" exitCode=0
Sep 09 04:31:51.476829 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:51.476654    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-426183-6b9f4c95f5-hgch5" event={"ID":"2cdc0724-54a2-424f-839d-203b761acd08","Type":"ContainerDied","Data":"f436252fb58e6e424526c31c375c524ce2f040531da40a5cf93df8c30f566198"}
Sep 09 04:31:51.476829 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:51.476667    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-426183-6b9f4c95f5-hgch5"
Sep 09 04:31:51.476829 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:51.476691    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-426183-6b9f4c95f5-hgch5" event={"ID":"2cdc0724-54a2-424f-839d-203b761acd08","Type":"ContainerDied","Data":"68e12c107099ed0492248409f2f7ae1fd48f68fcac87ccc27f84545ccd6112f1"}
Sep 09 04:31:51.476829 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:51.476712    2557 scope.go:117] "RemoveContainer" containerID="f436252fb58e6e424526c31c375c524ce2f040531da40a5cf93df8c30f566198"
Sep 09 04:31:51.485301 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:51.485282    2557 scope.go:117] "RemoveContainer" containerID="f436252fb58e6e424526c31c375c524ce2f040531da40a5cf93df8c30f566198"
Sep 09 04:31:51.485584 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:31:51.485563    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f436252fb58e6e424526c31c375c524ce2f040531da40a5cf93df8c30f566198\": container with ID starting with f436252fb58e6e424526c31c375c524ce2f040531da40a5cf93df8c30f566198 not found: ID does not exist" containerID="f436252fb58e6e424526c31c375c524ce2f040531da40a5cf93df8c30f566198"
Sep 09 04:31:51.485657 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:51.485593    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f436252fb58e6e424526c31c375c524ce2f040531da40a5cf93df8c30f566198"} err="failed to get container status \"f436252fb58e6e424526c31c375c524ce2f040531da40a5cf93df8c30f566198\": rpc error: code = NotFound desc = could not find container \"f436252fb58e6e424526c31c375c524ce2f040531da40a5cf93df8c30f566198\": container with ID starting with f436252fb58e6e424526c31c375c524ce2f040531da40a5cf93df8c30f566198 not found: ID does not exist"
Sep 09 04:31:51.497773 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:51.497751    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-426183-6b9f4c95f5-hgch5"]
Sep 09 04:31:51.503565 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:51.503545    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-426183-6b9f4c95f5-hgch5"]
Sep 09 04:31:51.619759 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:51.619729    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17dcf909-82c7-4ac9-b9df-ef773a2f4681" path="/var/lib/kubelet/pods/17dcf909-82c7-4ac9-b9df-ef773a2f4681/volumes"
Sep 09 04:31:51.620072 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:31:51.620059    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2cdc0724-54a2-424f-839d-203b761acd08" path="/var/lib/kubelet/pods/2cdc0724-54a2-424f-839d-203b761acd08/volumes"
Sep 09 04:32:08.940209 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:32:08.940176    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq"]
Sep 09 04:32:08.940677 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:32:08.940383    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq" podUID="de42a0ec-7e12-41cb-b2a2-29c04d7758a8" containerName="maas-api" containerID="cri-o://764be7eaaecbaa32957a6711f5f3c1f0678907e807d447314eae27f922988098" gracePeriod=30
Sep 09 04:32:09.190938 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:32:09.190872    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq"
Sep 09 04:32:09.364058 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:32:09.364020    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de42a0ec-7e12-41cb-b2a2-29c04d7758a8-openshift-service-ca\") pod \"de42a0ec-7e12-41cb-b2a2-29c04d7758a8\" (UID: \"de42a0ec-7e12-41cb-b2a2-29c04d7758a8\") "
Sep 09 04:32:09.364248 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:32:09.364067    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de42a0ec-7e12-41cb-b2a2-29c04d7758a8-maas-api-tls\") pod \"de42a0ec-7e12-41cb-b2a2-29c04d7758a8\" (UID: \"de42a0ec-7e12-41cb-b2a2-29c04d7758a8\") "
Sep 09 04:32:09.364248 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:32:09.364144    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de42a0ec-7e12-41cb-b2a2-29c04d7758a8-metrics-certs\") pod \"de42a0ec-7e12-41cb-b2a2-29c04d7758a8\" (UID: \"de42a0ec-7e12-41cb-b2a2-29c04d7758a8\") "
Sep 09 04:32:09.364248 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:32:09.364179    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2qjv\" (UniqueName: \"kubernetes.io/projected/de42a0ec-7e12-41cb-b2a2-29c04d7758a8-kube-api-access-n2qjv\") pod \"de42a0ec-7e12-41cb-b2a2-29c04d7758a8\" (UID: \"de42a0ec-7e12-41cb-b2a2-29c04d7758a8\") "
Sep 09 04:32:09.364432 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:32:09.364409    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de42a0ec-7e12-41cb-b2a2-29c04d7758a8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "de42a0ec-7e12-41cb-b2a2-29c04d7758a8" (UID: "de42a0ec-7e12-41cb-b2a2-29c04d7758a8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 04:32:09.366317 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:32:09.366287    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de42a0ec-7e12-41cb-b2a2-29c04d7758a8-kube-api-access-n2qjv" (OuterVolumeSpecName: "kube-api-access-n2qjv") pod "de42a0ec-7e12-41cb-b2a2-29c04d7758a8" (UID: "de42a0ec-7e12-41cb-b2a2-29c04d7758a8"). InnerVolumeSpecName "kube-api-access-n2qjv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 04:32:09.366317 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:32:09.366311    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de42a0ec-7e12-41cb-b2a2-29c04d7758a8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "de42a0ec-7e12-41cb-b2a2-29c04d7758a8" (UID: "de42a0ec-7e12-41cb-b2a2-29c04d7758a8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 04:32:09.366477 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:32:09.366370    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de42a0ec-7e12-41cb-b2a2-29c04d7758a8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "de42a0ec-7e12-41cb-b2a2-29c04d7758a8" (UID: "de42a0ec-7e12-41cb-b2a2-29c04d7758a8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 04:32:09.465533 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:32:09.465455    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de42a0ec-7e12-41cb-b2a2-29c04d7758a8-openshift-service-ca\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:32:09.465533 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:32:09.465482    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de42a0ec-7e12-41cb-b2a2-29c04d7758a8-maas-api-tls\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:32:09.465533 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:32:09.465491    2557 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de42a0ec-7e12-41cb-b2a2-29c04d7758a8-metrics-certs\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:32:09.465533 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:32:09.465498    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2qjv\" (UniqueName: \"kubernetes.io/projected/de42a0ec-7e12-41cb-b2a2-29c04d7758a8-kube-api-access-n2qjv\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:32:09.565219 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:32:09.565182    2557 generic.go:356] "Generic (PLEG): container finished" podID="de42a0ec-7e12-41cb-b2a2-29c04d7758a8" containerID="764be7eaaecbaa32957a6711f5f3c1f0678907e807d447314eae27f922988098" exitCode=0
Sep 09 04:32:09.565414 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:32:09.565246    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq"
Sep 09 04:32:09.565414 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:32:09.565274    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq" event={"ID":"de42a0ec-7e12-41cb-b2a2-29c04d7758a8","Type":"ContainerDied","Data":"764be7eaaecbaa32957a6711f5f3c1f0678907e807d447314eae27f922988098"}
Sep 09 04:32:09.565414 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:32:09.565372    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq" event={"ID":"de42a0ec-7e12-41cb-b2a2-29c04d7758a8","Type":"ContainerDied","Data":"ffa23f34de2979d06fc48e4d5949795595da27b475600ed72dd7e60fb33a734f"}
Sep 09 04:32:09.565414 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:32:09.565403    2557 scope.go:117] "RemoveContainer" containerID="764be7eaaecbaa32957a6711f5f3c1f0678907e807d447314eae27f922988098"
Sep 09 04:32:09.574812 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:32:09.574796    2557 scope.go:117] "RemoveContainer" containerID="764be7eaaecbaa32957a6711f5f3c1f0678907e807d447314eae27f922988098"
Sep 09 04:32:09.575074 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:32:09.575057    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"764be7eaaecbaa32957a6711f5f3c1f0678907e807d447314eae27f922988098\": container with ID starting with 764be7eaaecbaa32957a6711f5f3c1f0678907e807d447314eae27f922988098 not found: ID does not exist" containerID="764be7eaaecbaa32957a6711f5f3c1f0678907e807d447314eae27f922988098"
Sep 09 04:32:09.575127 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:32:09.575083    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"764be7eaaecbaa32957a6711f5f3c1f0678907e807d447314eae27f922988098"} err="failed to get container status \"764be7eaaecbaa32957a6711f5f3c1f0678907e807d447314eae27f922988098\": rpc error: code = NotFound desc = could not find container \"764be7eaaecbaa32957a6711f5f3c1f0678907e807d447314eae27f922988098\": container with ID starting with 764be7eaaecbaa32957a6711f5f3c1f0678907e807d447314eae27f922988098 not found: ID does not exist"
Sep 09 04:32:09.586715 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:32:09.586692    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq"]
Sep 09 04:32:09.590657 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:32:09.590633    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9c44b5-74bf598976-h6qxq"]
Sep 09 04:32:09.619509 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:32:09.619486    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de42a0ec-7e12-41cb-b2a2-29c04d7758a8" path="/var/lib/kubelet/pods/de42a0ec-7e12-41cb-b2a2-29c04d7758a8/volumes"
Sep 09 04:32:55.676143 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:32:55.676099    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gj7h7_7816cd49-e9c2-4f7b-b8ec-4e5711b675a5/console-operator/2.log"
Sep 09 04:32:55.678764 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:32:55.678739    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gj7h7_7816cd49-e9c2-4f7b-b8ec-4e5711b675a5/console-operator/2.log"
Sep 09 04:32:55.682090 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:32:55.682068    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rfvrc_1828d4d8-0d24-4e4d-a502-aee013a3b116/ovn-acl-logging/0.log"
Sep 09 04:32:55.684334 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:32:55.684315    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rfvrc_1828d4d8-0d24-4e4d-a502-aee013a3b116/ovn-acl-logging/0.log"
Sep 09 04:36:55.887728 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:36:55.887652    2557 scope.go:117] "RemoveContainer" containerID="4cc61412bc2c90d775be23f0da1fc1537926bc55725a671cd2126307dbb04867"
Sep 09 04:36:55.895780 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:36:55.895752    2557 scope.go:117] "RemoveContainer" containerID="e6d69aed58da59cebeb00c9b409b5de7cd5c7878cc042244d9008ab765106beb"
Sep 09 04:37:55.703911 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:37:55.703879    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gj7h7_7816cd49-e9c2-4f7b-b8ec-4e5711b675a5/console-operator/2.log"
Sep 09 04:37:55.706490 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:37:55.706467    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gj7h7_7816cd49-e9c2-4f7b-b8ec-4e5711b675a5/console-operator/2.log"
Sep 09 04:37:55.709435 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:37:55.709415    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rfvrc_1828d4d8-0d24-4e4d-a502-aee013a3b116/ovn-acl-logging/0.log"
Sep 09 04:37:55.712001 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:37:55.711980    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rfvrc_1828d4d8-0d24-4e4d-a502-aee013a3b116/ovn-acl-logging/0.log"
Sep 09 04:38:03.157767 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.157734    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-69989b7c55-xxccc"]
Sep 09 04:38:03.158165 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.158058    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c106ec1a-0d7e-43ce-b371-22ff3fda3baa" containerName="payload-processing"
Sep 09 04:38:03.158165 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.158069    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c106ec1a-0d7e-43ce-b371-22ff3fda3baa" containerName="payload-processing"
Sep 09 04:38:03.158165 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.158082    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cdc0724-54a2-424f-839d-203b761acd08" containerName="payload-processing"
Sep 09 04:38:03.158165 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.158088    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cdc0724-54a2-424f-839d-203b761acd08" containerName="payload-processing"
Sep 09 04:38:03.158165 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.158097    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17dcf909-82c7-4ac9-b9df-ef773a2f4681" containerName="cleanup"
Sep 09 04:38:03.158165 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.158104    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="17dcf909-82c7-4ac9-b9df-ef773a2f4681" containerName="cleanup"
Sep 09 04:38:03.158165 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.158125    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57960f27-5f23-417b-82be-883363e51719" containerName="maas-api"
Sep 09 04:38:03.158165 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.158131    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="57960f27-5f23-417b-82be-883363e51719" containerName="maas-api"
Sep 09 04:38:03.158165 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.158137    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf" containerName="istio-proxy"
Sep 09 04:38:03.158165 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.158142    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf" containerName="istio-proxy"
Sep 09 04:38:03.158165 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.158148    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3132b51a-1376-4bee-9c6b-71024b105699" containerName="cleanup"
Sep 09 04:38:03.158165 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.158153    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="3132b51a-1376-4bee-9c6b-71024b105699" containerName="cleanup"
Sep 09 04:38:03.158165 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.158160    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf" containerName="istio-proxy"
Sep 09 04:38:03.158165 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.158165    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf" containerName="istio-proxy"
Sep 09 04:38:03.158165 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.158173    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e053347-0cd2-4d99-b1f1-b057aaf20523" containerName="payload-pre-processing"
Sep 09 04:38:03.158632 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.158178    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e053347-0cd2-4d99-b1f1-b057aaf20523" containerName="payload-pre-processing"
Sep 09 04:38:03.158632 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.158186    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44fc4aac-f5ff-4c2d-af27-b2624fc70366" containerName="payload-pre-processing"
Sep 09 04:38:03.158632 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.158191    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="44fc4aac-f5ff-4c2d-af27-b2624fc70366" containerName="payload-pre-processing"
Sep 09 04:38:03.158632 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.158198    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de42a0ec-7e12-41cb-b2a2-29c04d7758a8" containerName="maas-api"
Sep 09 04:38:03.158632 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.158203    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="de42a0ec-7e12-41cb-b2a2-29c04d7758a8" containerName="maas-api"
Sep 09 04:38:03.158632 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.158255    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="57960f27-5f23-417b-82be-883363e51719" containerName="maas-api"
Sep 09 04:38:03.158632 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.158264    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf" containerName="istio-proxy"
Sep 09 04:38:03.158632 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.158270    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="44fc4aac-f5ff-4c2d-af27-b2624fc70366" containerName="payload-pre-processing"
Sep 09 04:38:03.158632 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.158276    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="de42a0ec-7e12-41cb-b2a2-29c04d7758a8" containerName="maas-api"
Sep 09 04:38:03.158632 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.158282    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="3132b51a-1376-4bee-9c6b-71024b105699" containerName="cleanup"
Sep 09 04:38:03.158632 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.158287    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cdc0724-54a2-424f-839d-203b761acd08" containerName="payload-processing"
Sep 09 04:38:03.158632 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.158292    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c106ec1a-0d7e-43ce-b371-22ff3fda3baa" containerName="payload-processing"
Sep 09 04:38:03.158632 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.158298    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="17dcf909-82c7-4ac9-b9df-ef773a2f4681" containerName="cleanup"
Sep 09 04:38:03.158632 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.158303    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e053347-0cd2-4d99-b1f1-b057aaf20523" containerName="payload-pre-processing"
Sep 09 04:38:03.161123 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.161098    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-69989b7c55-xxccc"
Sep 09 04:38:03.163236 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.163214    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 09 04:38:03.172161 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.172140    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-69989b7c55-xxccc"]
Sep 09 04:38:03.260700 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.260666    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a6e36c4-181b-428b-be10-bd63eff7c1b1-openshift-service-ca\") pod \"maas-api-69989b7c55-xxccc\" (UID: \"8a6e36c4-181b-428b-be10-bd63eff7c1b1\") " pod="odh-ai-gateway-infra/maas-api-69989b7c55-xxccc"
Sep 09 04:38:03.260863 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.260710    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a6e36c4-181b-428b-be10-bd63eff7c1b1-maas-api-tls\") pod \"maas-api-69989b7c55-xxccc\" (UID: \"8a6e36c4-181b-428b-be10-bd63eff7c1b1\") " pod="odh-ai-gateway-infra/maas-api-69989b7c55-xxccc"
Sep 09 04:38:03.260863 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.260741    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a6e36c4-181b-428b-be10-bd63eff7c1b1-metrics-certs\") pod \"maas-api-69989b7c55-xxccc\" (UID: \"8a6e36c4-181b-428b-be10-bd63eff7c1b1\") " pod="odh-ai-gateway-infra/maas-api-69989b7c55-xxccc"
Sep 09 04:38:03.260863 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.260771    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64bdn\" (UniqueName: \"kubernetes.io/projected/8a6e36c4-181b-428b-be10-bd63eff7c1b1-kube-api-access-64bdn\") pod \"maas-api-69989b7c55-xxccc\" (UID: \"8a6e36c4-181b-428b-be10-bd63eff7c1b1\") " pod="odh-ai-gateway-infra/maas-api-69989b7c55-xxccc"
Sep 09 04:38:03.361403 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.361368    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a6e36c4-181b-428b-be10-bd63eff7c1b1-openshift-service-ca\") pod \"maas-api-69989b7c55-xxccc\" (UID: \"8a6e36c4-181b-428b-be10-bd63eff7c1b1\") " pod="odh-ai-gateway-infra/maas-api-69989b7c55-xxccc"
Sep 09 04:38:03.361569 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.361411    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a6e36c4-181b-428b-be10-bd63eff7c1b1-maas-api-tls\") pod \"maas-api-69989b7c55-xxccc\" (UID: \"8a6e36c4-181b-428b-be10-bd63eff7c1b1\") " pod="odh-ai-gateway-infra/maas-api-69989b7c55-xxccc"
Sep 09 04:38:03.361569 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.361430    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a6e36c4-181b-428b-be10-bd63eff7c1b1-metrics-certs\") pod \"maas-api-69989b7c55-xxccc\" (UID: \"8a6e36c4-181b-428b-be10-bd63eff7c1b1\") " pod="odh-ai-gateway-infra/maas-api-69989b7c55-xxccc"
Sep 09 04:38:03.361569 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.361454    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64bdn\" (UniqueName: \"kubernetes.io/projected/8a6e36c4-181b-428b-be10-bd63eff7c1b1-kube-api-access-64bdn\") pod \"maas-api-69989b7c55-xxccc\" (UID: \"8a6e36c4-181b-428b-be10-bd63eff7c1b1\") " pod="odh-ai-gateway-infra/maas-api-69989b7c55-xxccc"
Sep 09 04:38:03.362121 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.362092    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a6e36c4-181b-428b-be10-bd63eff7c1b1-openshift-service-ca\") pod \"maas-api-69989b7c55-xxccc\" (UID: \"8a6e36c4-181b-428b-be10-bd63eff7c1b1\") " pod="odh-ai-gateway-infra/maas-api-69989b7c55-xxccc"
Sep 09 04:38:03.363793 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.363771    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a6e36c4-181b-428b-be10-bd63eff7c1b1-metrics-certs\") pod \"maas-api-69989b7c55-xxccc\" (UID: \"8a6e36c4-181b-428b-be10-bd63eff7c1b1\") " pod="odh-ai-gateway-infra/maas-api-69989b7c55-xxccc"
Sep 09 04:38:03.363913 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.363838    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a6e36c4-181b-428b-be10-bd63eff7c1b1-maas-api-tls\") pod \"maas-api-69989b7c55-xxccc\" (UID: \"8a6e36c4-181b-428b-be10-bd63eff7c1b1\") " pod="odh-ai-gateway-infra/maas-api-69989b7c55-xxccc"
Sep 09 04:38:03.369969 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.369945    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64bdn\" (UniqueName: \"kubernetes.io/projected/8a6e36c4-181b-428b-be10-bd63eff7c1b1-kube-api-access-64bdn\") pod \"maas-api-69989b7c55-xxccc\" (UID: \"8a6e36c4-181b-428b-be10-bd63eff7c1b1\") " pod="odh-ai-gateway-infra/maas-api-69989b7c55-xxccc"
Sep 09 04:38:03.471173 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.471068    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-69989b7c55-xxccc"
Sep 09 04:38:03.742565 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.742538    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-69989b7c55-xxccc"]
Sep 09 04:38:03.744437 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:38:03.744397    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a6e36c4_181b_428b_be10_bd63eff7c1b1.slice/crio-530e106bfa0067704e59e81ed31c35f00e3703faf8e0606b73025dcf71e112ba WatchSource:0}: Error finding container 530e106bfa0067704e59e81ed31c35f00e3703faf8e0606b73025dcf71e112ba: Status 404 returned error can't find the container with id 530e106bfa0067704e59e81ed31c35f00e3703faf8e0606b73025dcf71e112ba
Sep 09 04:38:03.745804 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:03.745784    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 04:38:04.211311 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:04.211287    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-69989b7c55-xxccc" event={"ID":"8a6e36c4-181b-428b-be10-bd63eff7c1b1","Type":"ContainerStarted","Data":"530e106bfa0067704e59e81ed31c35f00e3703faf8e0606b73025dcf71e112ba"}
Sep 09 04:38:05.217298 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:05.217260    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-69989b7c55-xxccc" event={"ID":"8a6e36c4-181b-428b-be10-bd63eff7c1b1","Type":"ContainerStarted","Data":"bfd66d9ce2e3db7ad01e717982a59840cbaaa3dc59f07049baab43c4421cf07b"}
Sep 09 04:38:05.217676 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:05.217365    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-69989b7c55-xxccc"
Sep 09 04:38:05.237550 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:05.237499    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-69989b7c55-xxccc" podStartSLOduration=1.7735872289999999 podStartE2EDuration="2.237484553s" totalImagesPullingTime="463.897324ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:38:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:38:05.236081025 +0000 UTC m=+2410.155251936" watchObservedRunningTime="2026-09-09 04:38:05.237484553 +0000 UTC m=+2410.156655515"
Sep 09 04:38:11.228595 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:11.228568    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-69989b7c55-xxccc"
Sep 09 04:38:11.245890 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:11.245853    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-69989b7c55-xxccc"]
Sep 09 04:38:11.246048 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:11.246019    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-69989b7c55-xxccc" podUID="8a6e36c4-181b-428b-be10-bd63eff7c1b1" containerName="maas-api" containerID="cri-o://bfd66d9ce2e3db7ad01e717982a59840cbaaa3dc59f07049baab43c4421cf07b" gracePeriod=30
Sep 09 04:38:11.282430 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:11.282390    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-f6c58f77f-rqk6p"]
Sep 09 04:38:11.286506 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:11.283762    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="835c5d7a-9dd8-47d6-9ab9-f92a150a1ecf" containerName="istio-proxy"
Sep 09 04:38:11.286661 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:38:11.286625    2557 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a6e36c4_181b_428b_be10_bd63eff7c1b1.slice/crio-conmon-bfd66d9ce2e3db7ad01e717982a59840cbaaa3dc59f07049baab43c4421cf07b.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a6e36c4_181b_428b_be10_bd63eff7c1b1.slice/crio-bfd66d9ce2e3db7ad01e717982a59840cbaaa3dc59f07049baab43c4421cf07b.scope\": RecentStats: unable to find data in memory cache]"
Sep 09 04:38:11.290902 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:11.288751    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-f6c58f77f-rqk6p"
Sep 09 04:38:11.296266 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:11.296239    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-f6c58f77f-rqk6p"]
Sep 09 04:38:11.332395 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:11.332364    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c2a9f31-e84a-47a9-a7b2-4d33fdf9e6ac-metrics-certs\") pod \"maas-api-f6c58f77f-rqk6p\" (UID: \"6c2a9f31-e84a-47a9-a7b2-4d33fdf9e6ac\") " pod="odh-ai-gateway-infra/maas-api-f6c58f77f-rqk6p"
Sep 09 04:38:11.332515 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:11.332402    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fj62b\" (UniqueName: \"kubernetes.io/projected/6c2a9f31-e84a-47a9-a7b2-4d33fdf9e6ac-kube-api-access-fj62b\") pod \"maas-api-f6c58f77f-rqk6p\" (UID: \"6c2a9f31-e84a-47a9-a7b2-4d33fdf9e6ac\") " pod="odh-ai-gateway-infra/maas-api-f6c58f77f-rqk6p"
Sep 09 04:38:11.332515 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:11.332431    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c2a9f31-e84a-47a9-a7b2-4d33fdf9e6ac-maas-api-tls\") pod \"maas-api-f6c58f77f-rqk6p\" (UID: \"6c2a9f31-e84a-47a9-a7b2-4d33fdf9e6ac\") " pod="odh-ai-gateway-infra/maas-api-f6c58f77f-rqk6p"
Sep 09 04:38:11.332592 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:11.332562    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c2a9f31-e84a-47a9-a7b2-4d33fdf9e6ac-openshift-service-ca\") pod \"maas-api-f6c58f77f-rqk6p\" (UID: \"6c2a9f31-e84a-47a9-a7b2-4d33fdf9e6ac\") " pod="odh-ai-gateway-infra/maas-api-f6c58f77f-rqk6p"
Sep 09 04:38:11.433853 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:11.433821    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c2a9f31-e84a-47a9-a7b2-4d33fdf9e6ac-openshift-service-ca\") pod \"maas-api-f6c58f77f-rqk6p\" (UID: \"6c2a9f31-e84a-47a9-a7b2-4d33fdf9e6ac\") " pod="odh-ai-gateway-infra/maas-api-f6c58f77f-rqk6p"
Sep 09 04:38:11.434011 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:11.433882    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c2a9f31-e84a-47a9-a7b2-4d33fdf9e6ac-metrics-certs\") pod \"maas-api-f6c58f77f-rqk6p\" (UID: \"6c2a9f31-e84a-47a9-a7b2-4d33fdf9e6ac\") " pod="odh-ai-gateway-infra/maas-api-f6c58f77f-rqk6p"
Sep 09 04:38:11.434011 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:11.433908    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fj62b\" (UniqueName: \"kubernetes.io/projected/6c2a9f31-e84a-47a9-a7b2-4d33fdf9e6ac-kube-api-access-fj62b\") pod \"maas-api-f6c58f77f-rqk6p\" (UID: \"6c2a9f31-e84a-47a9-a7b2-4d33fdf9e6ac\") " pod="odh-ai-gateway-infra/maas-api-f6c58f77f-rqk6p"
Sep 09 04:38:11.434011 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:11.433934    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c2a9f31-e84a-47a9-a7b2-4d33fdf9e6ac-maas-api-tls\") pod \"maas-api-f6c58f77f-rqk6p\" (UID: \"6c2a9f31-e84a-47a9-a7b2-4d33fdf9e6ac\") " pod="odh-ai-gateway-infra/maas-api-f6c58f77f-rqk6p"
Sep 09 04:38:11.434642 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:11.434611    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c2a9f31-e84a-47a9-a7b2-4d33fdf9e6ac-openshift-service-ca\") pod \"maas-api-f6c58f77f-rqk6p\" (UID: \"6c2a9f31-e84a-47a9-a7b2-4d33fdf9e6ac\") " pod="odh-ai-gateway-infra/maas-api-f6c58f77f-rqk6p"
Sep 09 04:38:11.436571 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:11.436540    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c2a9f31-e84a-47a9-a7b2-4d33fdf9e6ac-metrics-certs\") pod \"maas-api-f6c58f77f-rqk6p\" (UID: \"6c2a9f31-e84a-47a9-a7b2-4d33fdf9e6ac\") " pod="odh-ai-gateway-infra/maas-api-f6c58f77f-rqk6p"
Sep 09 04:38:11.436932 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:11.436910    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c2a9f31-e84a-47a9-a7b2-4d33fdf9e6ac-maas-api-tls\") pod \"maas-api-f6c58f77f-rqk6p\" (UID: \"6c2a9f31-e84a-47a9-a7b2-4d33fdf9e6ac\") " pod="odh-ai-gateway-infra/maas-api-f6c58f77f-rqk6p"
Sep 09 04:38:11.443509 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:11.443481    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fj62b\" (UniqueName: \"kubernetes.io/projected/6c2a9f31-e84a-47a9-a7b2-4d33fdf9e6ac-kube-api-access-fj62b\") pod \"maas-api-f6c58f77f-rqk6p\" (UID: \"6c2a9f31-e84a-47a9-a7b2-4d33fdf9e6ac\") " pod="odh-ai-gateway-infra/maas-api-f6c58f77f-rqk6p"
Sep 09 04:38:11.493100 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:11.493044    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-69989b7c55-xxccc"
Sep 09 04:38:11.534229 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:11.534206    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a6e36c4-181b-428b-be10-bd63eff7c1b1-maas-api-tls\") pod \"8a6e36c4-181b-428b-be10-bd63eff7c1b1\" (UID: \"8a6e36c4-181b-428b-be10-bd63eff7c1b1\") "
Sep 09 04:38:11.534361 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:11.534250    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a6e36c4-181b-428b-be10-bd63eff7c1b1-openshift-service-ca\") pod \"8a6e36c4-181b-428b-be10-bd63eff7c1b1\" (UID: \"8a6e36c4-181b-428b-be10-bd63eff7c1b1\") "
Sep 09 04:38:11.534361 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:11.534280    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a6e36c4-181b-428b-be10-bd63eff7c1b1-metrics-certs\") pod \"8a6e36c4-181b-428b-be10-bd63eff7c1b1\" (UID: \"8a6e36c4-181b-428b-be10-bd63eff7c1b1\") "
Sep 09 04:38:11.534361 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:11.534309    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-64bdn\" (UniqueName: \"kubernetes.io/projected/8a6e36c4-181b-428b-be10-bd63eff7c1b1-kube-api-access-64bdn\") pod \"8a6e36c4-181b-428b-be10-bd63eff7c1b1\" (UID: \"8a6e36c4-181b-428b-be10-bd63eff7c1b1\") "
Sep 09 04:38:11.534800 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:11.534764    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a6e36c4-181b-428b-be10-bd63eff7c1b1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8a6e36c4-181b-428b-be10-bd63eff7c1b1" (UID: "8a6e36c4-181b-428b-be10-bd63eff7c1b1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 04:38:11.536256 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:11.536234    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a6e36c4-181b-428b-be10-bd63eff7c1b1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8a6e36c4-181b-428b-be10-bd63eff7c1b1" (UID: "8a6e36c4-181b-428b-be10-bd63eff7c1b1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 04:38:11.536357 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:11.536289    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a6e36c4-181b-428b-be10-bd63eff7c1b1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8a6e36c4-181b-428b-be10-bd63eff7c1b1" (UID: "8a6e36c4-181b-428b-be10-bd63eff7c1b1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 04:38:11.536357 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:11.536316    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a6e36c4-181b-428b-be10-bd63eff7c1b1-kube-api-access-64bdn" (OuterVolumeSpecName: "kube-api-access-64bdn") pod "8a6e36c4-181b-428b-be10-bd63eff7c1b1" (UID: "8a6e36c4-181b-428b-be10-bd63eff7c1b1"). InnerVolumeSpecName "kube-api-access-64bdn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 04:38:11.617811 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:11.617789    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-f6c58f77f-rqk6p"
Sep 09 04:38:11.635323 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:11.635301    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a6e36c4-181b-428b-be10-bd63eff7c1b1-maas-api-tls\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:38:11.635323 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:11.635325    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a6e36c4-181b-428b-be10-bd63eff7c1b1-openshift-service-ca\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:38:11.635440 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:11.635336    2557 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a6e36c4-181b-428b-be10-bd63eff7c1b1-metrics-certs\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:38:11.635440 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:11.635348    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64bdn\" (UniqueName: \"kubernetes.io/projected/8a6e36c4-181b-428b-be10-bd63eff7c1b1-kube-api-access-64bdn\") on node \"ip-10-0-141-71.ec2.internal\" DevicePath \"\""
Sep 09 04:38:11.890376 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:11.890271    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-f6c58f77f-rqk6p"]
Sep 09 04:38:11.893018 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:38:11.892987    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c2a9f31_e84a_47a9_a7b2_4d33fdf9e6ac.slice/crio-ced1d87dc34e076bc6c0ff034fabc58e5fb198137cec32a1ef98eccd8ae6184c WatchSource:0}: Error finding container ced1d87dc34e076bc6c0ff034fabc58e5fb198137cec32a1ef98eccd8ae6184c: Status 404 returned error can't find the container with id ced1d87dc34e076bc6c0ff034fabc58e5fb198137cec32a1ef98eccd8ae6184c
Sep 09 04:38:12.251642 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:12.251481    2557 generic.go:356] "Generic (PLEG): container finished" podID="8a6e36c4-181b-428b-be10-bd63eff7c1b1" containerID="bfd66d9ce2e3db7ad01e717982a59840cbaaa3dc59f07049baab43c4421cf07b" exitCode=0
Sep 09 04:38:12.251642 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:12.251525    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-69989b7c55-xxccc" event={"ID":"8a6e36c4-181b-428b-be10-bd63eff7c1b1","Type":"ContainerDied","Data":"bfd66d9ce2e3db7ad01e717982a59840cbaaa3dc59f07049baab43c4421cf07b"}
Sep 09 04:38:12.251642 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:12.251598    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-69989b7c55-xxccc" event={"ID":"8a6e36c4-181b-428b-be10-bd63eff7c1b1","Type":"ContainerDied","Data":"530e106bfa0067704e59e81ed31c35f00e3703faf8e0606b73025dcf71e112ba"}
Sep 09 04:38:12.251642 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:12.251628    2557 scope.go:117] "RemoveContainer" containerID="bfd66d9ce2e3db7ad01e717982a59840cbaaa3dc59f07049baab43c4421cf07b"
Sep 09 04:38:12.252358 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:12.251537    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-69989b7c55-xxccc"
Sep 09 04:38:12.254317 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:12.253806    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-f6c58f77f-rqk6p" event={"ID":"6c2a9f31-e84a-47a9-a7b2-4d33fdf9e6ac","Type":"ContainerStarted","Data":"ced1d87dc34e076bc6c0ff034fabc58e5fb198137cec32a1ef98eccd8ae6184c"}
Sep 09 04:38:12.271955 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:12.271934    2557 scope.go:117] "RemoveContainer" containerID="bfd66d9ce2e3db7ad01e717982a59840cbaaa3dc59f07049baab43c4421cf07b"
Sep 09 04:38:12.272422 ip-10-0-141-71 kubenswrapper[2557]: E0909 04:38:12.272354    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfd66d9ce2e3db7ad01e717982a59840cbaaa3dc59f07049baab43c4421cf07b\": container with ID starting with bfd66d9ce2e3db7ad01e717982a59840cbaaa3dc59f07049baab43c4421cf07b not found: ID does not exist" containerID="bfd66d9ce2e3db7ad01e717982a59840cbaaa3dc59f07049baab43c4421cf07b"
Sep 09 04:38:12.272422 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:12.272390    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfd66d9ce2e3db7ad01e717982a59840cbaaa3dc59f07049baab43c4421cf07b"} err="failed to get container status \"bfd66d9ce2e3db7ad01e717982a59840cbaaa3dc59f07049baab43c4421cf07b\": rpc error: code = NotFound desc = could not find container \"bfd66d9ce2e3db7ad01e717982a59840cbaaa3dc59f07049baab43c4421cf07b\": container with ID starting with bfd66d9ce2e3db7ad01e717982a59840cbaaa3dc59f07049baab43c4421cf07b not found: ID does not exist"
Sep 09 04:38:12.280695 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:12.280669    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-69989b7c55-xxccc"]
Sep 09 04:38:12.284839 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:12.284814    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-69989b7c55-xxccc"]
Sep 09 04:38:13.261012 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:13.260975    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-f6c58f77f-rqk6p" event={"ID":"6c2a9f31-e84a-47a9-a7b2-4d33fdf9e6ac","Type":"ContainerStarted","Data":"419bc4db1e77474a34df6ae309a0c1adf7a00454d6ed6bec29dea908ce355fdd"}
Sep 09 04:38:13.261443 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:13.261082    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-f6c58f77f-rqk6p"
Sep 09 04:38:13.279040 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:13.278993    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-f6c58f77f-rqk6p" podStartSLOduration=1.9187282030000001 podStartE2EDuration="2.278980206s" totalImagesPullingTime="360.252003ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 04:38:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:38:13.278176499 +0000 UTC m=+2418.197347411" watchObservedRunningTime="2026-09-09 04:38:13.278980206 +0000 UTC m=+2418.198151159"
Sep 09 04:38:13.619160 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:13.619131    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a6e36c4-181b-428b-be10-bd63eff7c1b1" path="/var/lib/kubelet/pods/8a6e36c4-181b-428b-be10-bd63eff7c1b1/volumes"
Sep 09 04:38:19.270159 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:19.270071    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-f6c58f77f-rqk6p"
Sep 09 04:38:44.168855 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:44.168818    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-fzkbx_cb6e639b-01bd-4884-ab79-76b09560d3c6/manager/0.log"
Sep 09 04:38:44.529648 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:44.529571    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5996d6cf95-w4tx6_9e4b9910-12de-4341-89be-3863b2274210/copy-manifests/0.log"
Sep 09 04:38:44.542909 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:44.542880    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5996d6cf95-w4tx6_9e4b9910-12de-4341-89be-3863b2274210/manager/0.log"
Sep 09 04:38:44.911697 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:44.911669    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-e2e-worker-w4-85732c-54589c45dd-ls9gc_a06fb867-32f0-4993-9bc1-bca2e03a953b/maas-api/0.log"
Sep 09 04:38:45.148334 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:45.148301    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-f6c58f77f-rqk6p_6c2a9f31-e84a-47a9-a7b2-4d33fdf9e6ac/maas-api/0.log"
Sep 09 04:38:47.766634 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:47.766597    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-k9m5x_55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5/discovery/0.log"
Sep 09 04:38:48.368837 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:48.368811    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-e2e-worker-w4-85732c-6d5cd7c4b6-ffzdb_01e315ac-1215-48b4-87fe-7a1108db4cc1/payload-pre-processing/0.log"
Sep 09 04:38:48.485933 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:48.485904    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-q4xpl_2f2c11af-63a3-41d0-8bab-12654e7b98be/payload-processing/0.log"
Sep 09 04:38:48.600170 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:48.600140    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-worker-w4-85732c-65749c6df9-c844k_20d1cfab-32f0-4f34-a789-b27e0c009583/payload-processing/0.log"
Sep 09 04:38:48.717754 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:48.717674    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6f7c46fdb6-89xrm_f831999b-7d59-4577-864d-c39b88896d64/router/0.log"
Sep 09 04:38:49.180524 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:49.180490    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-rjnh4_6902ae83-be2b-47fb-b30e-f3ba0788b5f5/main/0.log"
Sep 09 04:38:58.766060 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:58.766026    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-78hdg_85fe16e4-d362-49b1-b379-451d29551d1b/global-pull-secret-syncer/0.log"
Sep 09 04:38:58.961635 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:58.961601    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-scn7w_5452b40e-6ceb-4f8d-93d5-3dc83a424314/konnectivity-agent/0.log"
Sep 09 04:38:59.007386 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:38:59.007354    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-71.ec2.internal_2c06b6aa4345f7858e0b9cc81a5b603f/haproxy/0.log"
Sep 09 04:39:05.917255 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:05.917223    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-ln2j9_54f58d60-214f-4fb6-9ae8-02334c258100/cluster-monitoring-operator/0.log"
Sep 09 04:39:06.288703 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:06.288673    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wz2tp_d48875b3-ed83-4a71-818e-b56a6d54e394/node-exporter/0.log"
Sep 09 04:39:06.318177 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:06.318150    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wz2tp_d48875b3-ed83-4a71-818e-b56a6d54e394/kube-rbac-proxy/0.log"
Sep 09 04:39:06.346423 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:06.346402    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wz2tp_d48875b3-ed83-4a71-818e-b56a6d54e394/init-textfile/0.log"
Sep 09 04:39:08.960165 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:08.960133    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gj7h7_7816cd49-e9c2-4f7b-b8ec-4e5711b675a5/console-operator/2.log"
Sep 09 04:39:08.963488 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:08.963466    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gj7h7_7816cd49-e9c2-4f7b-b8ec-4e5711b675a5/console-operator/3.log"
Sep 09 04:39:09.292521 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:09.292444    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-smpj7/perf-node-gather-daemonset-xqvhd"]
Sep 09 04:39:09.292791 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:09.292779    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a6e36c4-181b-428b-be10-bd63eff7c1b1" containerName="maas-api"
Sep 09 04:39:09.292834 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:09.292793    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a6e36c4-181b-428b-be10-bd63eff7c1b1" containerName="maas-api"
Sep 09 04:39:09.292869 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:09.292850    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a6e36c4-181b-428b-be10-bd63eff7c1b1" containerName="maas-api"
Sep 09 04:39:09.295972 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:09.295953    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-smpj7/perf-node-gather-daemonset-xqvhd"
Sep 09 04:39:09.298320 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:09.298295    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-smpj7\"/\"openshift-service-ca.crt\""
Sep 09 04:39:09.298456 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:09.298297    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-smpj7\"/\"kube-root-ca.crt\""
Sep 09 04:39:09.299151 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:09.299134    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-smpj7\"/\"default-dockercfg-s57v2\""
Sep 09 04:39:09.303495 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:09.303477    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-smpj7/perf-node-gather-daemonset-xqvhd"]
Sep 09 04:39:09.433011 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:09.432979    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c9fb5d5b-cfe8-49ec-86bb-ccc28f2a4663-sys\") pod \"perf-node-gather-daemonset-xqvhd\" (UID: \"c9fb5d5b-cfe8-49ec-86bb-ccc28f2a4663\") " pod="openshift-must-gather-smpj7/perf-node-gather-daemonset-xqvhd"
Sep 09 04:39:09.433011 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:09.433012    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c9fb5d5b-cfe8-49ec-86bb-ccc28f2a4663-lib-modules\") pod \"perf-node-gather-daemonset-xqvhd\" (UID: \"c9fb5d5b-cfe8-49ec-86bb-ccc28f2a4663\") " pod="openshift-must-gather-smpj7/perf-node-gather-daemonset-xqvhd"
Sep 09 04:39:09.433244 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:09.433030    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c9fb5d5b-cfe8-49ec-86bb-ccc28f2a4663-podres\") pod \"perf-node-gather-daemonset-xqvhd\" (UID: \"c9fb5d5b-cfe8-49ec-86bb-ccc28f2a4663\") " pod="openshift-must-gather-smpj7/perf-node-gather-daemonset-xqvhd"
Sep 09 04:39:09.433244 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:09.433053    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qf8d\" (UniqueName: \"kubernetes.io/projected/c9fb5d5b-cfe8-49ec-86bb-ccc28f2a4663-kube-api-access-6qf8d\") pod \"perf-node-gather-daemonset-xqvhd\" (UID: \"c9fb5d5b-cfe8-49ec-86bb-ccc28f2a4663\") " pod="openshift-must-gather-smpj7/perf-node-gather-daemonset-xqvhd"
Sep 09 04:39:09.433244 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:09.433105    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c9fb5d5b-cfe8-49ec-86bb-ccc28f2a4663-proc\") pod \"perf-node-gather-daemonset-xqvhd\" (UID: \"c9fb5d5b-cfe8-49ec-86bb-ccc28f2a4663\") " pod="openshift-must-gather-smpj7/perf-node-gather-daemonset-xqvhd"
Sep 09 04:39:09.534320 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:09.534285    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c9fb5d5b-cfe8-49ec-86bb-ccc28f2a4663-sys\") pod \"perf-node-gather-daemonset-xqvhd\" (UID: \"c9fb5d5b-cfe8-49ec-86bb-ccc28f2a4663\") " pod="openshift-must-gather-smpj7/perf-node-gather-daemonset-xqvhd"
Sep 09 04:39:09.534320 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:09.534321    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c9fb5d5b-cfe8-49ec-86bb-ccc28f2a4663-lib-modules\") pod \"perf-node-gather-daemonset-xqvhd\" (UID: \"c9fb5d5b-cfe8-49ec-86bb-ccc28f2a4663\") " pod="openshift-must-gather-smpj7/perf-node-gather-daemonset-xqvhd"
Sep 09 04:39:09.534544 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:09.534338    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c9fb5d5b-cfe8-49ec-86bb-ccc28f2a4663-podres\") pod \"perf-node-gather-daemonset-xqvhd\" (UID: \"c9fb5d5b-cfe8-49ec-86bb-ccc28f2a4663\") " pod="openshift-must-gather-smpj7/perf-node-gather-daemonset-xqvhd"
Sep 09 04:39:09.534544 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:09.534374    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qf8d\" (UniqueName: \"kubernetes.io/projected/c9fb5d5b-cfe8-49ec-86bb-ccc28f2a4663-kube-api-access-6qf8d\") pod \"perf-node-gather-daemonset-xqvhd\" (UID: \"c9fb5d5b-cfe8-49ec-86bb-ccc28f2a4663\") " pod="openshift-must-gather-smpj7/perf-node-gather-daemonset-xqvhd"
Sep 09 04:39:09.534544 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:09.534406    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c9fb5d5b-cfe8-49ec-86bb-ccc28f2a4663-proc\") pod \"perf-node-gather-daemonset-xqvhd\" (UID: \"c9fb5d5b-cfe8-49ec-86bb-ccc28f2a4663\") " pod="openshift-must-gather-smpj7/perf-node-gather-daemonset-xqvhd"
Sep 09 04:39:09.534544 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:09.534419    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c9fb5d5b-cfe8-49ec-86bb-ccc28f2a4663-sys\") pod \"perf-node-gather-daemonset-xqvhd\" (UID: \"c9fb5d5b-cfe8-49ec-86bb-ccc28f2a4663\") " pod="openshift-must-gather-smpj7/perf-node-gather-daemonset-xqvhd"
Sep 09 04:39:09.534544 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:09.534480    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c9fb5d5b-cfe8-49ec-86bb-ccc28f2a4663-lib-modules\") pod \"perf-node-gather-daemonset-xqvhd\" (UID: \"c9fb5d5b-cfe8-49ec-86bb-ccc28f2a4663\") " pod="openshift-must-gather-smpj7/perf-node-gather-daemonset-xqvhd"
Sep 09 04:39:09.534544 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:09.534519    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c9fb5d5b-cfe8-49ec-86bb-ccc28f2a4663-podres\") pod \"perf-node-gather-daemonset-xqvhd\" (UID: \"c9fb5d5b-cfe8-49ec-86bb-ccc28f2a4663\") " pod="openshift-must-gather-smpj7/perf-node-gather-daemonset-xqvhd"
Sep 09 04:39:09.534544 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:09.534535    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c9fb5d5b-cfe8-49ec-86bb-ccc28f2a4663-proc\") pod \"perf-node-gather-daemonset-xqvhd\" (UID: \"c9fb5d5b-cfe8-49ec-86bb-ccc28f2a4663\") " pod="openshift-must-gather-smpj7/perf-node-gather-daemonset-xqvhd"
Sep 09 04:39:09.543081 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:09.543017    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qf8d\" (UniqueName: \"kubernetes.io/projected/c9fb5d5b-cfe8-49ec-86bb-ccc28f2a4663-kube-api-access-6qf8d\") pod \"perf-node-gather-daemonset-xqvhd\" (UID: \"c9fb5d5b-cfe8-49ec-86bb-ccc28f2a4663\") " pod="openshift-must-gather-smpj7/perf-node-gather-daemonset-xqvhd"
Sep 09 04:39:09.607165 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:09.607107    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-smpj7/perf-node-gather-daemonset-xqvhd"
Sep 09 04:39:09.881746 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:09.881722    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-smpj7/perf-node-gather-daemonset-xqvhd"]
Sep 09 04:39:09.883873 ip-10-0-141-71 kubenswrapper[2557]: W0909 04:39:09.883840    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc9fb5d5b_cfe8_49ec_86bb_ccc28f2a4663.slice/crio-87adf85b9be909ac1afd9336fe158187187553c4ca7627a3fec6e65e243e02aa WatchSource:0}: Error finding container 87adf85b9be909ac1afd9336fe158187187553c4ca7627a3fec6e65e243e02aa: Status 404 returned error can't find the container with id 87adf85b9be909ac1afd9336fe158187187553c4ca7627a3fec6e65e243e02aa
Sep 09 04:39:10.296978 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:10.296900    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-pnxkj_d55cca64-4230-4d1b-9e52-0d7379772d4b/volume-data-source-validator/0.log"
Sep 09 04:39:10.532605 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:10.532562    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-smpj7/perf-node-gather-daemonset-xqvhd" event={"ID":"c9fb5d5b-cfe8-49ec-86bb-ccc28f2a4663","Type":"ContainerStarted","Data":"aec72eaa28b0f351b6618814e57a7d98c6001d2eae66aa0dd09109f7b2fbb058"}
Sep 09 04:39:10.532807 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:10.532646    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-smpj7/perf-node-gather-daemonset-xqvhd" event={"ID":"c9fb5d5b-cfe8-49ec-86bb-ccc28f2a4663","Type":"ContainerStarted","Data":"87adf85b9be909ac1afd9336fe158187187553c4ca7627a3fec6e65e243e02aa"}
Sep 09 04:39:10.532807 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:10.532718    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-smpj7/perf-node-gather-daemonset-xqvhd"
Sep 09 04:39:10.550832 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:10.550668    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-smpj7/perf-node-gather-daemonset-xqvhd" podStartSLOduration=1.5506501799999999 podStartE2EDuration="1.55065018s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 04:39:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 04:39:10.550510462 +0000 UTC m=+2475.469681373" watchObservedRunningTime="2026-09-09 04:39:10.55065018 +0000 UTC m=+2475.469821091"
Sep 09 04:39:11.511064 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:11.511041    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9d9xs_b73c2563-ee66-481a-ba6e-122139fb3abc/dns/0.log"
Sep 09 04:39:11.534092 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:11.534066    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9d9xs_b73c2563-ee66-481a-ba6e-122139fb3abc/kube-rbac-proxy/0.log"
Sep 09 04:39:11.707581 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:11.707551    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hn6zn_34f28b0d-79cb-47df-9cc4-44f781c379fa/dns-node-resolver/0.log"
Sep 09 04:39:12.340017 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:12.339978    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kvnzq_6f01d502-4aa7-42a0-9617-093232b1f6a6/node-ca/0.log"
Sep 09 04:39:13.684663 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:13.684629    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-k9m5x_55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5/discovery/0.log"
Sep 09 04:39:13.838733 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:13.838698    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-e2e-worker-w4-85732c-6d5cd7c4b6-ffzdb_01e315ac-1215-48b4-87fe-7a1108db4cc1/payload-pre-processing/0.log"
Sep 09 04:39:13.877282 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:13.877249    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-q4xpl_2f2c11af-63a3-41d0-8bab-12654e7b98be/payload-processing/0.log"
Sep 09 04:39:13.899006 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:13.898976    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-worker-w4-85732c-65749c6df9-c844k_20d1cfab-32f0-4f34-a789-b27e0c009583/payload-processing/0.log"
Sep 09 04:39:13.925677 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:13.925642    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6f7c46fdb6-89xrm_f831999b-7d59-4577-864d-c39b88896d64/router/0.log"
Sep 09 04:39:14.905032 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:14.904999    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-pwxcn_aa51a428-9bbd-492a-98ae-e4bd03fab5ce/serve-healthcheck-canary/0.log"
Sep 09 04:39:15.436457 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:15.436426    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-57s54_55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7/insights-operator/0.log"
Sep 09 04:39:15.436709 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:15.436630    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-57s54_55a23ecc-3eeb-4c7f-859b-9dbccd0a7ff7/insights-operator/1.log"
Sep 09 04:39:15.457099 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:15.457081    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5dtmp_0c7e4f11-324d-4e2f-9d4b-97a2186beabc/kube-rbac-proxy/0.log"
Sep 09 04:39:15.479051 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:15.479028    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5dtmp_0c7e4f11-324d-4e2f-9d4b-97a2186beabc/exporter/0.log"
Sep 09 04:39:15.501984 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:15.501968    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5dtmp_0c7e4f11-324d-4e2f-9d4b-97a2186beabc/extractor/0.log"
Sep 09 04:39:16.547372 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:16.547345    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-smpj7/perf-node-gather-daemonset-xqvhd"
Sep 09 04:39:18.257692 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:18.257657    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-fzkbx_cb6e639b-01bd-4884-ab79-76b09560d3c6/manager/0.log"
Sep 09 04:39:18.426041 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:18.426015    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5996d6cf95-w4tx6_9e4b9910-12de-4341-89be-3863b2274210/manager/0.log"
Sep 09 04:39:18.443315 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:18.443292    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5996d6cf95-w4tx6_9e4b9910-12de-4341-89be-3863b2274210/copy-manifests/0.log"
Sep 09 04:39:19.959362 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:19.959334    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7486b797d6-x52dl_5f62770a-e2cc-40bf-922d-009ef53d32d8/manager/0.log"
Sep 09 04:39:20.060442 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:20.060412    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-k9m5x_55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5/discovery/0.log"
Sep 09 04:39:20.181561 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:20.181526    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-e2e-worker-w4-85732c-6d5cd7c4b6-ffzdb_01e315ac-1215-48b4-87fe-7a1108db4cc1/payload-pre-processing/0.log"
Sep 09 04:39:20.215590 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:20.215564    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-q4xpl_2f2c11af-63a3-41d0-8bab-12654e7b98be/payload-processing/0.log"
Sep 09 04:39:20.227566 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:20.227539    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-worker-w4-85732c-65749c6df9-c844k_20d1cfab-32f0-4f34-a789-b27e0c009583/payload-processing/0.log"
Sep 09 04:39:20.246970 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:20.246942    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6f7c46fdb6-89xrm_f831999b-7d59-4577-864d-c39b88896d64/router/0.log"
Sep 09 04:39:22.130190 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:22.130161    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-k9m5x_55e1dbd3-aeeb-4e90-bfa2-ad33f8d4c2c5/discovery/0.log"
Sep 09 04:39:22.253660 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:22.253633    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-e2e-worker-w4-85732c-6d5cd7c4b6-ffzdb_01e315ac-1215-48b4-87fe-7a1108db4cc1/payload-pre-processing/0.log"
Sep 09 04:39:22.286007 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:22.285984    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-q4xpl_2f2c11af-63a3-41d0-8bab-12654e7b98be/payload-processing/0.log"
Sep 09 04:39:22.299743 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:22.299713    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-worker-w4-85732c-65749c6df9-c844k_20d1cfab-32f0-4f34-a789-b27e0c009583/payload-processing/0.log"
Sep 09 04:39:22.318235 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:22.318211    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6f7c46fdb6-89xrm_f831999b-7d59-4577-864d-c39b88896d64/router/0.log"
Sep 09 04:39:25.861045 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:25.861012    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-4xg2m_6964ed13-fd43-48b3-aff1-8e37bcd05287/kube-storage-version-migrator-operator/1.log"
Sep 09 04:39:25.862135 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:25.862095    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-4xg2m_6964ed13-fd43-48b3-aff1-8e37bcd05287/kube-storage-version-migrator-operator/0.log"
Sep 09 04:39:26.950928 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:26.950852    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g6ggd_f26faf1a-61a1-4ff3-a8fb-5321f27480bc/kube-multus-additional-cni-plugins/0.log"
Sep 09 04:39:26.973079 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:26.973055    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g6ggd_f26faf1a-61a1-4ff3-a8fb-5321f27480bc/egress-router-binary-copy/0.log"
Sep 09 04:39:26.994570 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:26.994547    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g6ggd_f26faf1a-61a1-4ff3-a8fb-5321f27480bc/cni-plugins/0.log"
Sep 09 04:39:27.016791 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:27.016767    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g6ggd_f26faf1a-61a1-4ff3-a8fb-5321f27480bc/bond-cni-plugin/0.log"
Sep 09 04:39:27.038765 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:27.038746    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g6ggd_f26faf1a-61a1-4ff3-a8fb-5321f27480bc/routeoverride-cni/0.log"
Sep 09 04:39:27.061684 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:27.061663    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g6ggd_f26faf1a-61a1-4ff3-a8fb-5321f27480bc/whereabouts-cni-bincopy/0.log"
Sep 09 04:39:27.084390 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:27.084369    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-g6ggd_f26faf1a-61a1-4ff3-a8fb-5321f27480bc/whereabouts-cni/0.log"
Sep 09 04:39:27.449759 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:27.449729    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-gfd87_1c7a8c99-53ac-4578-aad2-25908242ff17/kube-multus/0.log"
Sep 09 04:39:27.615843 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:27.615810    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9lrvt_58280195-fa67-47e6-af03-e39d2bab379b/network-metrics-daemon/0.log"
Sep 09 04:39:27.646374 ip-10-0-141-71 kubenswrapper[2557]: I0909 04:39:27.646347    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9lrvt_58280195-fa67-47e6-af03-e39d2bab379b/kube-rbac-proxy/0.log"