Sep 14 02:50:50.922549 ip-10-0-141-180 systemd[1]: Starting Kubernetes Kubelet...
Sep 14 02:50:51.318493 ip-10-0-141-180 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 02:50:51.318493 ip-10-0-141-180 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 14 02:50:51.318493 ip-10-0-141-180 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 02:50:51.318493 ip-10-0-141-180 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 14 02:50:51.318493 ip-10-0-141-180 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 02:50:51.319525 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.319459    2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 14 02:50:51.322320 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322303    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 02:50:51.322320 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322318    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 02:50:51.322320 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322322    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 02:50:51.322500 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322328    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 02:50:51.322500 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322333    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 02:50:51.322500 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322336    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 02:50:51.322500 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322341    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 02:50:51.322500 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322344    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 02:50:51.322500 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322348    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 02:50:51.322500 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322352    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 02:50:51.322500 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322356    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 02:50:51.322500 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322360    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 02:50:51.322500 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322364    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 02:50:51.322500 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322367    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 02:50:51.322500 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322371    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 02:50:51.322500 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322375    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 02:50:51.322500 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322384    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 02:50:51.322500 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322388    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 02:50:51.322500 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322394    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 02:50:51.322500 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322400    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 02:50:51.322500 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322404    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 02:50:51.322500 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322416    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 02:50:51.323325 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322421    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 02:50:51.323325 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322426    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 02:50:51.323325 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322430    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 02:50:51.323325 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322434    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 02:50:51.323325 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322438    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 02:50:51.323325 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322442    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 02:50:51.323325 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322446    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 02:50:51.323325 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322450    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 02:50:51.323325 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322454    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 02:50:51.323325 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322458    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 02:50:51.323325 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322462    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 02:50:51.323325 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322467    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 02:50:51.323325 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322471    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 02:50:51.323325 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322475    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 02:50:51.323325 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322479    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 02:50:51.323325 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322485    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 02:50:51.323325 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322489    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 02:50:51.323325 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322493    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 02:50:51.323325 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322497    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 02:50:51.323325 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322501    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 02:50:51.324195 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322506    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 02:50:51.324195 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322510    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 02:50:51.324195 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322514    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 02:50:51.324195 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322525    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 02:50:51.324195 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322529    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 02:50:51.324195 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322534    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 02:50:51.324195 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322538    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 02:50:51.324195 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322543    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 02:50:51.324195 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322547    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 02:50:51.324195 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322551    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 02:50:51.324195 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322555    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 02:50:51.324195 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322562    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 02:50:51.324195 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322567    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 02:50:51.324195 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322572    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 02:50:51.324195 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322577    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 02:50:51.324195 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322582    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 02:50:51.324195 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322586    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 02:50:51.324195 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322591    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 02:50:51.324195 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322595    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 02:50:51.324195 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322599    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 02:50:51.325062 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322603    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 02:50:51.325062 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322607    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 02:50:51.325062 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322612    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 02:50:51.325062 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322616    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 02:50:51.325062 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322620    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 02:50:51.325062 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322624    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 02:50:51.325062 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322629    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 02:50:51.325062 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322633    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 02:50:51.325062 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322640    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 02:50:51.325062 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322644    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 02:50:51.325062 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322648    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 02:50:51.325062 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322653    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 02:50:51.325062 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322657    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 02:50:51.325062 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322661    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 02:50:51.325062 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322665    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 02:50:51.325062 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322670    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 02:50:51.325062 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322674    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 02:50:51.325062 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322678    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 02:50:51.325062 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322683    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 02:50:51.325062 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322687    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 02:50:51.325934 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322692    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 02:50:51.325934 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322696    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 02:50:51.325934 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322700    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 02:50:51.325934 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322704    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 02:50:51.325934 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322708    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 02:50:51.325934 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322713    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 02:50:51.325934 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322719    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 02:50:51.325934 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322723    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 02:50:51.325934 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322727    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 02:50:51.325934 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322731    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 02:50:51.325934 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322735    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 02:50:51.325934 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322742    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 02:50:51.325934 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322747    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 02:50:51.325934 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322752    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 02:50:51.325934 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322756    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 02:50:51.325934 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322761    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 02:50:51.325934 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.322766    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 02:50:51.325934 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.323873    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 02:50:51.325934 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.323884    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 02:50:51.325934 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.323890    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 02:50:51.326871 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.323894    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 02:50:51.326871 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.323900    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 02:50:51.326871 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.323904    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 02:50:51.326871 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.323908    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 02:50:51.326871 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.323913    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 02:50:51.326871 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.323917    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 02:50:51.326871 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.323921    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 02:50:51.326871 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.323926    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 02:50:51.326871 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.323930    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 02:50:51.326871 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.323934    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 02:50:51.326871 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.323938    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 02:50:51.326871 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.323943    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 02:50:51.326871 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.323948    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 02:50:51.326871 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.323952    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 02:50:51.326871 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.323956    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 02:50:51.326871 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.323963    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 02:50:51.326871 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.323969    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 02:50:51.326871 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.323974    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 02:50:51.326871 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.323979    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 02:50:51.326871 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.323983    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 02:50:51.327483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.323987    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 02:50:51.327483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.323992    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 02:50:51.327483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.323997    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 02:50:51.327483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324001    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 02:50:51.327483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324005    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 02:50:51.327483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324009    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 02:50:51.327483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324014    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 02:50:51.327483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324018    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 02:50:51.327483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324022    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 02:50:51.327483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324026    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 02:50:51.327483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324030    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 02:50:51.327483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324035    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 02:50:51.327483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324039    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 02:50:51.327483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324043    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 02:50:51.327483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324047    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 02:50:51.327483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324051    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 02:50:51.327483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324055    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 02:50:51.327483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324059    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 02:50:51.327483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324064    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 02:50:51.327483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324068    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 02:50:51.328051 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324074    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 02:50:51.328051 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324078    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 02:50:51.328051 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324085    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 02:50:51.328051 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324091    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 02:50:51.328051 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324096    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 02:50:51.328051 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324101    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 02:50:51.328051 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324105    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 02:50:51.328051 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324109    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 02:50:51.328051 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324113    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 02:50:51.328051 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324117    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 02:50:51.328051 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324121    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 02:50:51.328051 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324125    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 02:50:51.328051 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324130    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 02:50:51.328051 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324134    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 02:50:51.328051 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324139    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 02:50:51.328051 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324144    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 02:50:51.328051 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324150    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 02:50:51.328051 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324156    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 02:50:51.328051 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324161    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 02:50:51.328582 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324165    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 02:50:51.328582 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324170    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 02:50:51.328582 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324174    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 02:50:51.328582 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324179    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 02:50:51.328582 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324183    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 02:50:51.328582 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324187    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 02:50:51.328582 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324191    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 02:50:51.328582 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324195    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 02:50:51.328582 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324199    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 02:50:51.328582 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324205    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 02:50:51.328582 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324209    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 02:50:51.328582 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324213    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 02:50:51.328582 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324218    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 02:50:51.328582 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324222    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 02:50:51.328582 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324226    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 02:50:51.328582 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324230    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 02:50:51.328582 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324234    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 02:50:51.328582 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324239    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 02:50:51.328582 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324244    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 02:50:51.328582 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324248    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 02:50:51.329063 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324253    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 02:50:51.329063 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324257    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 02:50:51.329063 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324261    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 02:50:51.329063 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324284    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 02:50:51.329063 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324289    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 02:50:51.329063 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324294    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 02:50:51.329063 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324298    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 02:50:51.329063 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324303    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 02:50:51.329063 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324307    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 02:50:51.329063 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324311    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 02:50:51.329063 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324316    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 02:50:51.329063 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324320    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 02:50:51.329063 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324326    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 02:50:51.329063 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324329    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 02:50:51.329063 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324334    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 02:50:51.329063 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324338    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 02:50:51.329063 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324342    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 02:50:51.329063 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324924    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 02:50:51.329063 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324932    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 02:50:51.329063 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324936    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 02:50:51.329756 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324941    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 02:50:51.329756 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324946    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 02:50:51.329756 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324950    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 02:50:51.329756 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324954    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 02:50:51.329756 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324958    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 02:50:51.329756 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324963    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 02:50:51.329756 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324967    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 02:50:51.329756 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324971    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 02:50:51.329756 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324975    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 02:50:51.329756 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324980    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 02:50:51.329756 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324988    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 02:50:51.329756 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324993    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 02:50:51.329756 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.324998    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 02:50:51.329756 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325002    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 02:50:51.329756 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325006    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 02:50:51.329756 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325011    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 02:50:51.329756 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325015    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 02:50:51.329756 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325019    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 02:50:51.329756 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325028    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 02:50:51.329756 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325034    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 02:50:51.330242 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325038    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 02:50:51.330242 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325042    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 02:50:51.330242 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325047    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 02:50:51.330242 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325051    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 02:50:51.330242 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325055    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 02:50:51.330242 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325059    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 02:50:51.330242 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325063    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 02:50:51.330242 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325067    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 02:50:51.330242 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325071    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 02:50:51.330242 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325076    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 02:50:51.330242 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325080    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 02:50:51.330242 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325084    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 02:50:51.330242 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325088    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 02:50:51.330242 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325092    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 02:50:51.330242 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325096    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 02:50:51.330242 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325100    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 02:50:51.330242 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325104    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 02:50:51.330242 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325109    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 02:50:51.330242 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325113    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 02:50:51.330900 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325117    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 02:50:51.330900 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325122    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 02:50:51.330900 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325126    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 02:50:51.330900 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325129    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 02:50:51.330900 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325134    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 02:50:51.330900 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325139    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 02:50:51.330900 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325143    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 02:50:51.330900 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325147    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 02:50:51.330900 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325151    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 02:50:51.330900 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325155    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 02:50:51.330900 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325159    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 02:50:51.330900 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325165    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 02:50:51.330900 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325169    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 02:50:51.330900 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325174    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 02:50:51.330900 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325178    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 02:50:51.330900 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325182    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 02:50:51.330900 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325187    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 02:50:51.330900 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325194    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 02:50:51.330900 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325200    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 02:50:51.331404 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325204    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 02:50:51.331404 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325209    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 02:50:51.331404 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325213    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 02:50:51.331404 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325219    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 02:50:51.331404 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325225    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 02:50:51.331404 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325231    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 02:50:51.331404 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325236    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 02:50:51.331404 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325240    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 02:50:51.331404 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325244    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 02:50:51.331404 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325249    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 02:50:51.331404 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325254    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 02:50:51.331404 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325258    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 02:50:51.331404 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325262    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 02:50:51.331404 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325284    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 02:50:51.331404 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325288    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 02:50:51.331404 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325293    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 02:50:51.331404 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325300    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 02:50:51.331404 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325306    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 02:50:51.331404 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325312    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 02:50:51.331404 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325316    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 02:50:51.331913 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325321    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 02:50:51.331913 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325325    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 02:50:51.331913 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325329    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 02:50:51.331913 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325332    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 02:50:51.331913 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325339    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 02:50:51.331913 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325347    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 02:50:51.331913 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325351    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 02:50:51.331913 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325355    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 02:50:51.331913 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325359    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 02:50:51.331913 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325363    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 02:50:51.331913 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325367    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 02:50:51.331913 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325371    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 02:50:51.331913 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325376    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 02:50:51.331913 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325380    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 02:50:51.331913 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325384    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 02:50:51.331913 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325388    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 02:50:51.331913 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325392    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 02:50:51.331913 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.325396    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 02:50:51.331913 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325501    2577 flags.go:64] FLAG: --address="0.0.0.0"
Sep 14 02:50:51.331913 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325552    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 14 02:50:51.331913 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325561    2577 flags.go:64] FLAG: --anonymous-auth="true"
Sep 14 02:50:51.332435 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325567    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 14 02:50:51.332435 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325573    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 14 02:50:51.332435 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325578    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 14 02:50:51.332435 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325585    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 14 02:50:51.332435 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325591    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 14 02:50:51.332435 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325596    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 14 02:50:51.332435 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325601    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 14 02:50:51.332435 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325606    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 14 02:50:51.332435 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325611    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 14 02:50:51.332435 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325616    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 14 02:50:51.332435 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325622    2577 flags.go:64] FLAG: --cgroup-root=""
Sep 14 02:50:51.332435 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325628    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 14 02:50:51.332435 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325633    2577 flags.go:64] FLAG: --client-ca-file=""
Sep 14 02:50:51.332435 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325637    2577 flags.go:64] FLAG: --cloud-provider="external"
Sep 14 02:50:51.332435 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325642    2577 flags.go:64] FLAG: --cluster-dns="[]"
Sep 14 02:50:51.332435 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325650    2577 flags.go:64] FLAG: --cluster-domain=""
Sep 14 02:50:51.332435 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325656    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 14 02:50:51.332435 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325661    2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 14 02:50:51.332435 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325666    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 14 02:50:51.332435 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325672    2577 flags.go:64] FLAG: --container-log-max-files="5"
Sep 14 02:50:51.332435 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325678    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 14 02:50:51.332435 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325682    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 14 02:50:51.332435 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325687    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 14 02:50:51.332991 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325692    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 14 02:50:51.332991 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325697    2577 flags.go:64] FLAG: --contention-profiling="false"
Sep 14 02:50:51.332991 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325702    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 14 02:50:51.332991 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325706    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 14 02:50:51.332991 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325711    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 14 02:50:51.332991 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325716    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 14 02:50:51.332991 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325722    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 14 02:50:51.332991 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325727    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 14 02:50:51.332991 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325732    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 14 02:50:51.332991 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325737    2577 flags.go:64] FLAG: --enable-load-reader="false"
Sep 14 02:50:51.332991 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325742    2577 flags.go:64] FLAG: --enable-server="true"
Sep 14 02:50:51.332991 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325747    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 14 02:50:51.332991 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325753    2577 flags.go:64] FLAG: --event-burst="100"
Sep 14 02:50:51.332991 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325758    2577 flags.go:64] FLAG: --event-qps="50"
Sep 14 02:50:51.332991 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325763    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 14 02:50:51.332991 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325768    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 14 02:50:51.332991 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325773    2577 flags.go:64] FLAG: --eviction-hard=""
Sep 14 02:50:51.332991 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325779    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 14 02:50:51.332991 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325783    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 14 02:50:51.332991 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325789    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 14 02:50:51.332991 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325795    2577 flags.go:64] FLAG: --eviction-soft=""
Sep 14 02:50:51.332991 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325799    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 14 02:50:51.332991 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325804    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 14 02:50:51.332991 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325809    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 14 02:50:51.332991 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325815    2577 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 14 02:50:51.333709 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325822    2577 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 14 02:50:51.333709 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325827    2577 flags.go:64] FLAG: --fail-swap-on="true"
Sep 14 02:50:51.333709 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325831    2577 flags.go:64] FLAG: --feature-gates=""
Sep 14 02:50:51.333709 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325837    2577 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 14 02:50:51.333709 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325842    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 14 02:50:51.333709 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325847    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 14 02:50:51.333709 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325852    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 14 02:50:51.333709 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325857    2577 flags.go:64] FLAG: --healthz-port="10248"
Sep 14 02:50:51.333709 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325862    2577 flags.go:64] FLAG: --help="false"
Sep 14 02:50:51.333709 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325866    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-141-180.ec2.internal"
Sep 14 02:50:51.333709 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325871    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 14 02:50:51.333709 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325876    2577 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 14 02:50:51.333709 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325881    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 14 02:50:51.333709 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325886    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 14 02:50:51.333709 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325891    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 14 02:50:51.333709 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325896    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 14 02:50:51.333709 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325901    2577 flags.go:64] FLAG: --image-service-endpoint=""
Sep 14 02:50:51.333709 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325905    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 14 02:50:51.333709 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325910    2577 flags.go:64] FLAG: --kube-api-burst="100"
Sep 14 02:50:51.333709 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325914    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 14 02:50:51.333709 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325919    2577 flags.go:64] FLAG: --kube-api-qps="50"
Sep 14 02:50:51.333709 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325924    2577 flags.go:64] FLAG: --kube-reserved=""
Sep 14 02:50:51.333709 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325929    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 14 02:50:51.333709 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325933    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 14 02:50:51.334314 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325938    2577 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 14 02:50:51.334314 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325942    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 14 02:50:51.334314 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325946    2577 flags.go:64] FLAG: --lock-file=""
Sep 14 02:50:51.334314 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325951    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 14 02:50:51.334314 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325956    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 14 02:50:51.334314 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325961    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 14 02:50:51.334314 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325969    2577 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 14 02:50:51.334314 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325974    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 14 02:50:51.334314 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325979    2577 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 14 02:50:51.334314 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325985    2577 flags.go:64] FLAG: --logging-format="text"
Sep 14 02:50:51.334314 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325989    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 14 02:50:51.334314 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325994    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 14 02:50:51.334314 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.325998    2577 flags.go:64] FLAG: --manifest-url=""
Sep 14 02:50:51.334314 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326003    2577 flags.go:64] FLAG: --manifest-url-header=""
Sep 14 02:50:51.334314 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326010    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 14 02:50:51.334314 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326014    2577 flags.go:64] FLAG: --max-open-files="1000000"
Sep 14 02:50:51.334314 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326020    2577 flags.go:64] FLAG: --max-pods="110"
Sep 14 02:50:51.334314 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326025    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 14 02:50:51.334314 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326030    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 14 02:50:51.334314 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326034    2577 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 14 02:50:51.334314 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326038    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 14 02:50:51.334314 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326043    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 14 02:50:51.334314 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326048    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 14 02:50:51.334314 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326052    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 14 02:50:51.334314 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326062    2577 flags.go:64] FLAG: --node-status-max-images="50"
Sep 14 02:50:51.334941 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326067    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 14 02:50:51.334941 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326072    2577 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 14 02:50:51.334941 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326076    2577 flags.go:64] FLAG: --pod-cidr=""
Sep 14 02:50:51.334941 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326081    2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 14 02:50:51.334941 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326089    2577 flags.go:64] FLAG: --pod-manifest-path=""
Sep 14 02:50:51.334941 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326094    2577 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 14 02:50:51.334941 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326099    2577 flags.go:64] FLAG: --pods-per-core="0"
Sep 14 02:50:51.334941 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326103    2577 flags.go:64] FLAG: --port="10250"
Sep 14 02:50:51.334941 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326108    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 14 02:50:51.334941 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326113    2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-006ad7a97d85f319c"
Sep 14 02:50:51.334941 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326118    2577 flags.go:64] FLAG: --qos-reserved=""
Sep 14 02:50:51.334941 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326122    2577 flags.go:64] FLAG: --read-only-port="10255"
Sep 14 02:50:51.334941 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326128    2577 flags.go:64] FLAG: --register-node="true"
Sep 14 02:50:51.334941 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326132    2577 flags.go:64] FLAG: --register-with-taints=""
Sep 14 02:50:51.334941 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326139    2577 flags.go:64] FLAG: --registry-burst="10"
Sep 14 02:50:51.334941 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326144    2577 flags.go:64] FLAG: --registry-qps="5"
Sep 14 02:50:51.334941 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326148    2577 flags.go:64] FLAG: --reserved-cpus=""
Sep 14 02:50:51.334941 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326153    2577 flags.go:64] FLAG: --reserved-memory=""
Sep 14 02:50:51.334941 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326159    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 14 02:50:51.334941 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326164    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 14 02:50:51.334941 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326169    2577 flags.go:64] FLAG: --rotate-certificates="false"
Sep 14 02:50:51.334941 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326173    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 14 02:50:51.334941 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326178    2577 flags.go:64] FLAG: --runonce="false"
Sep 14 02:50:51.334941 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326183    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 14 02:50:51.334941 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326188    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 14 02:50:51.335571 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326193    2577 flags.go:64] FLAG: --seccomp-default="false"
Sep 14 02:50:51.335571 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326197    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 14 02:50:51.335571 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326202    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 14 02:50:51.335571 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326206    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 14 02:50:51.335571 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326211    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 14 02:50:51.335571 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326216    2577 flags.go:64] FLAG: --storage-driver-password="root"
Sep 14 02:50:51.335571 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326221    2577 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 14 02:50:51.335571 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326226    2577 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 14 02:50:51.335571 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326230    2577 flags.go:64] FLAG: --storage-driver-user="root"
Sep 14 02:50:51.335571 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326235    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 14 02:50:51.335571 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326240    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 14 02:50:51.335571 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326244    2577 flags.go:64] FLAG: --system-cgroups=""
Sep 14 02:50:51.335571 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326249    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 14 02:50:51.335571 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326257    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 14 02:50:51.335571 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326262    2577 flags.go:64] FLAG: --tls-cert-file=""
Sep 14 02:50:51.335571 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326283    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 14 02:50:51.335571 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326290    2577 flags.go:64] FLAG: --tls-min-version=""
Sep 14 02:50:51.335571 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326294    2577 flags.go:64] FLAG: --tls-private-key-file=""
Sep 14 02:50:51.335571 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326299    2577 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 14 02:50:51.335571 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326303    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 14 02:50:51.335571 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326309    2577 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 14 02:50:51.335571 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326314    2577 flags.go:64] FLAG: --v="2"
Sep 14 02:50:51.335571 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326320    2577 flags.go:64] FLAG: --version="false"
Sep 14 02:50:51.335571 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326327    2577 flags.go:64] FLAG: --vmodule=""
Sep 14 02:50:51.335571 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326332    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 14 02:50:51.336177 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.326338    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 14 02:50:51.336177 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326510    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 02:50:51.336177 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326525    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 02:50:51.336177 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326530    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 02:50:51.336177 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326534    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 02:50:51.336177 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326538    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 02:50:51.336177 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326543    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 02:50:51.336177 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326547    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 02:50:51.336177 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326551    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 02:50:51.336177 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326555    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 02:50:51.336177 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326560    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 02:50:51.336177 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326564    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 02:50:51.336177 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326569    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 02:50:51.336177 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326573    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 02:50:51.336177 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326577    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 02:50:51.336177 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326581    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 02:50:51.336177 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326585    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 02:50:51.336177 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326589    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 02:50:51.336177 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326593    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 02:50:51.336177 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326597    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 02:50:51.336674 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326601    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 02:50:51.336674 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326606    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 02:50:51.336674 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326610    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 02:50:51.336674 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326614    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 02:50:51.336674 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326618    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 02:50:51.336674 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326622    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 02:50:51.336674 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326626    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 02:50:51.336674 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326630    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 02:50:51.336674 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326636    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 02:50:51.336674 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326640    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 02:50:51.336674 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326645    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 02:50:51.336674 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326649    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 02:50:51.336674 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326653    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 02:50:51.336674 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326657    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 02:50:51.336674 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326662    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 02:50:51.336674 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326665    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 02:50:51.336674 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326670    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 02:50:51.336674 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326674    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 02:50:51.336674 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326678    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 02:50:51.336674 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326682    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 02:50:51.337176 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326686    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 02:50:51.337176 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326690    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 02:50:51.337176 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326695    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 02:50:51.337176 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326699    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 02:50:51.337176 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326703    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 02:50:51.337176 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326706    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 02:50:51.337176 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326711    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 02:50:51.337176 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326715    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 02:50:51.337176 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326719    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 02:50:51.337176 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326723    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 02:50:51.337176 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326727    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 02:50:51.337176 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326731    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 02:50:51.337176 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326735    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 02:50:51.337176 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326739    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 02:50:51.337176 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326743    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 02:50:51.337176 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326747    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 02:50:51.337176 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326752    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 02:50:51.337176 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326756    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 02:50:51.337176 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326760    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 02:50:51.337176 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326764    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 02:50:51.337759 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326770    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 02:50:51.337759 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326775    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 02:50:51.337759 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326782    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 02:50:51.337759 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326786    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 02:50:51.337759 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326790    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 02:50:51.337759 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326793    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 02:50:51.337759 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326798    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 02:50:51.337759 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326802    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 02:50:51.337759 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326806    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 02:50:51.337759 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326810    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 02:50:51.337759 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326814    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 02:50:51.337759 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326819    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 02:50:51.337759 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326823    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 02:50:51.337759 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326827    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 02:50:51.337759 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326831    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 02:50:51.337759 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326835    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 02:50:51.337759 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326839    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 02:50:51.337759 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326843    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 02:50:51.337759 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326847    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 02:50:51.337759 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326854    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 02:50:51.338280 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326859    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 02:50:51.338280 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326863    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 02:50:51.338280 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326867    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 02:50:51.338280 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326870    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 02:50:51.338280 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326874    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 02:50:51.338280 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326878    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 02:50:51.338280 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326882    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 02:50:51.338280 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326888    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 02:50:51.338280 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326894    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 02:50:51.338280 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326900    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 02:50:51.338280 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326906    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 02:50:51.338280 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326911    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 02:50:51.338280 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326918    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 02:50:51.338280 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326923    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 02:50:51.338280 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326930    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 02:50:51.338280 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326935    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 02:50:51.338280 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326939    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 02:50:51.338280 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326945    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 02:50:51.338280 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326949    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 02:50:51.338748 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.326954    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 02:50:51.338748 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.327814    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 02:50:51.338748 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.334613    2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 14 02:50:51.338748 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.334626    2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 14 02:50:51.338748 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.335395    2577 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 14 02:50:51.338748 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.335405    2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 14 02:50:51.338748 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335459    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 02:50:51.338748 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335463    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 02:50:51.338748 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335467    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 02:50:51.338748 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335471    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 02:50:51.338748 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335474    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 02:50:51.338748 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335477    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 02:50:51.338748 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335481    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 02:50:51.338748 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335484    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 02:50:51.338748 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335487    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 02:50:51.339114 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335489    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 02:50:51.339114 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335492    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 02:50:51.339114 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335495    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 02:50:51.339114 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335498    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 02:50:51.339114 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335505    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 02:50:51.339114 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335509    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 02:50:51.339114 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335513    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 02:50:51.339114 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335516    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 02:50:51.339114 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335519    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 02:50:51.339114 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335522    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 02:50:51.339114 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335526    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 02:50:51.339114 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335528    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 02:50:51.339114 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335533    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 02:50:51.339114 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335537    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 02:50:51.339114 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335541    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 02:50:51.339114 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335544    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 02:50:51.339114 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335546    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 02:50:51.339114 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335549    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 02:50:51.339114 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335552    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 02:50:51.339628 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335555    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 02:50:51.339628 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335557    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 02:50:51.339628 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335560    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 02:50:51.339628 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335563    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 02:50:51.339628 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335566    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 02:50:51.339628 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335570    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 02:50:51.339628 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335573    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 02:50:51.339628 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335576    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 02:50:51.339628 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335579    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 02:50:51.339628 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335582    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 02:50:51.339628 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335585    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 02:50:51.339628 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335588    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 02:50:51.339628 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335591    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 02:50:51.339628 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335594    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 02:50:51.339628 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335597    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 02:50:51.339628 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335600    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 02:50:51.339628 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335602    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 02:50:51.339628 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335605    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 02:50:51.340065 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335608    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 02:50:51.340065 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335611    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 02:50:51.340065 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335613    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 02:50:51.340065 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335616    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 02:50:51.340065 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335619    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 02:50:51.340065 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335622    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 02:50:51.340065 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335625    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 02:50:51.340065 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335627    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 02:50:51.340065 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335630    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 02:50:51.340065 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335632    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 02:50:51.340065 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335635    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 02:50:51.340065 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335637    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 02:50:51.340065 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335640    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 02:50:51.340065 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335642    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 02:50:51.340065 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335645    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 02:50:51.340065 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335647    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 02:50:51.340065 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335650    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 02:50:51.340065 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335653    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 02:50:51.340065 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335656    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 02:50:51.340065 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335658    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 02:50:51.340578 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335661    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 02:50:51.340578 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335663    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 02:50:51.340578 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335666    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 02:50:51.340578 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335668    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 02:50:51.340578 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335671    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 02:50:51.340578 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335673    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 02:50:51.340578 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335675    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 02:50:51.340578 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335678    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 02:50:51.340578 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335681    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 02:50:51.340578 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335683    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 02:50:51.340578 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335686    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 02:50:51.340578 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335688    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 02:50:51.340578 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335691    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 02:50:51.340578 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335693    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 02:50:51.340578 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335696    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 02:50:51.340578 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335698    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 02:50:51.340578 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335701    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 02:50:51.340578 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335704    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 02:50:51.340578 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335707    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 02:50:51.340578 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335709    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 02:50:51.341082 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335712    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 02:50:51.341082 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335714    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 02:50:51.341082 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335717    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 02:50:51.341082 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335719    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 02:50:51.341082 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335722    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 02:50:51.341082 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335724    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 02:50:51.341082 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335727    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 02:50:51.341082 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335729    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 02:50:51.341082 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335732    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 02:50:51.341082 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335734    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 02:50:51.341082 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335737    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 02:50:51.341082 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335739    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 02:50:51.341082 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335742    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 02:50:51.341082 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.335747    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 02:50:51.341082 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335847    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 02:50:51.341465 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335851    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 02:50:51.341465 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335854    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 02:50:51.341465 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335857    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 02:50:51.341465 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335860    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 02:50:51.341465 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335863    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 02:50:51.341465 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335865    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 02:50:51.341465 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335868    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 02:50:51.341465 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335871    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 02:50:51.341465 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335873    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 02:50:51.341465 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335876    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 02:50:51.341465 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335880    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 02:50:51.341465 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335883    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 02:50:51.341465 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335886    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 02:50:51.341465 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335889    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 02:50:51.341465 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335892    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 02:50:51.341465 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335894    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 02:50:51.341465 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335897    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 02:50:51.341465 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335900    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 02:50:51.341465 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335902    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 02:50:51.341465 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335905    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 02:50:51.341935 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335908    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 02:50:51.341935 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335911    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 02:50:51.341935 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335915    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 02:50:51.341935 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335918    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 02:50:51.341935 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335920    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 02:50:51.341935 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335923    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 02:50:51.341935 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335925    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 02:50:51.341935 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335928    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 02:50:51.341935 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335930    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 02:50:51.341935 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335933    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 02:50:51.341935 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335935    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 02:50:51.341935 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335938    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 02:50:51.341935 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335941    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 02:50:51.341935 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335943    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 02:50:51.341935 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335946    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 02:50:51.341935 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335948    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 02:50:51.341935 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335950    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 02:50:51.341935 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335953    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 02:50:51.341935 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335956    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 02:50:51.342483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335958    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 02:50:51.342483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335961    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 02:50:51.342483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335963    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 02:50:51.342483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335966    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 02:50:51.342483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335968    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 02:50:51.342483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335971    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 02:50:51.342483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335973    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 02:50:51.342483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335976    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 02:50:51.342483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335978    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 02:50:51.342483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335981    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 02:50:51.342483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335983    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 02:50:51.342483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335986    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 02:50:51.342483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335988    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 02:50:51.342483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335992    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 02:50:51.342483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335995    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 02:50:51.342483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.335998    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 02:50:51.342483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336000    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 02:50:51.342483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336003    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 02:50:51.342483 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336005    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 02:50:51.342941 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336007    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 02:50:51.342941 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336010    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 02:50:51.342941 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336012    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 02:50:51.342941 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336015    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 02:50:51.342941 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336017    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 02:50:51.342941 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336020    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 02:50:51.342941 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336022    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 02:50:51.342941 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336025    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 02:50:51.342941 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336027    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 02:50:51.342941 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336030    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 02:50:51.342941 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336033    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 02:50:51.342941 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336035    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 02:50:51.342941 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336038    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 02:50:51.342941 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336041    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 02:50:51.342941 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336043    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 02:50:51.342941 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336045    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 02:50:51.342941 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336048    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 02:50:51.342941 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336051    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 02:50:51.342941 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336053    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 02:50:51.343424 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336055    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 02:50:51.343424 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336058    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 02:50:51.343424 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336060    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 02:50:51.343424 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336063    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 02:50:51.343424 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336065    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 02:50:51.343424 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336068    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 02:50:51.343424 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336070    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 02:50:51.343424 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336073    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 02:50:51.343424 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336076    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 02:50:51.343424 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336078    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 02:50:51.343424 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336081    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 02:50:51.343424 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336083    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 02:50:51.343424 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336086    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 02:50:51.343424 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336088    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 02:50:51.343424 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336091    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 02:50:51.343424 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336093    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 02:50:51.343424 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336096    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 02:50:51.343424 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336098    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 02:50:51.343424 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336101    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 02:50:51.343424 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336103    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 02:50:51.343424 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:51.336105    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 02:50:51.343939 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.336110    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 02:50:51.343939 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.336196    2577 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 14 02:50:51.343939 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.340948    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 14 02:50:51.343939 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.341983    2577 server.go:1019] "Starting client certificate rotation"
Sep 14 02:50:51.343939 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.342087    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 14 02:50:51.343939 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.342125    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 14 02:50:51.344824 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.344810    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 14 02:50:51.348901 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.348880    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 14 02:50:51.359581 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.359563    2577 log.go:25] "Validated CRI v1 runtime API"
Sep 14 02:50:51.362587 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.362573    2577 log.go:25] "Validated CRI v1 image API"
Sep 14 02:50:51.364997 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.364983    2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 14 02:50:51.368312 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.368293    2577 fs.go:135] Filesystem UUIDs: map[5687e6c2-ce62-47a2-b2d6-76a6f743965c:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 8bdfc44e-6aa3-401d-ac77-4928a177cd83:/dev/nvme0n1p3]
Sep 14 02:50:51.368389 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.368310    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 14 02:50:51.373106 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.373009    2577 manager.go:216] Machine: {Timestamp:2026-09-14 02:50:51.371759131 +0000 UTC m=+0.349737150 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3089043 MemoryCapacity:33164464128 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b2d682a0c9b3ce755cbae1a528003 SystemUUID:ec2b2d68-2a0c-9b3c-e755-cbae1a528003 BootID:bb929033-5625-4561-a46b-037434079796 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582234112 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582230016 Type:vfs Inodes:4048396 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632894464 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:ce:0f:90:e3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:ce:0f:90:e3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a2:81:c6:c5:24:0d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164464128 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 14 02:50:51.373106 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.373098    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 14 02:50:51.373226 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.373184    2577 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 14 02:50:51.376727 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.376701    2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 14 02:50:51.376888 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.376732    2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-180.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 14 02:50:51.377679 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.377666    2577 topology_manager.go:138] "Creating topology manager with none policy"
Sep 14 02:50:51.377736 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.377685    2577 container_manager_linux.go:309] "Creating device plugin manager"
Sep 14 02:50:51.377736 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.377695    2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 14 02:50:51.378654 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.378641    2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 14 02:50:51.379802 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.379790    2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 14 02:50:51.383789 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.383773    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 14 02:50:51.384254 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.384241    2577 state_mem.go:36] "Initialized new in-memory state store"
Sep 14 02:50:51.384392 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.384381    2577 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 14 02:50:51.384446 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.384416    2577 kubelet.go:486] "Attempting to sync node with API server"
Sep 14 02:50:51.384446 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.384430    2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 14 02:50:51.384533 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.384449    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 14 02:50:51.384533 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.384462    2577 kubelet.go:398] "Adding apiserver pod source"
Sep 14 02:50:51.384533 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.384477    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 14 02:50:51.384799 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.384777    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 14 02:50:51.384868 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.384800    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 14 02:50:51.387060 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.387045    2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 14 02:50:51.389321 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.389306    2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 14 02:50:51.389383 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.389336    2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 14 02:50:51.391392 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.391381    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 14 02:50:51.391438 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.391396    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 14 02:50:51.391438 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.391402    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 14 02:50:51.391438 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.391407    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 14 02:50:51.391438 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.391415    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 14 02:50:51.391438 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.391420    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 14 02:50:51.391438 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.391433    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 14 02:50:51.391438 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.391438    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 14 02:50:51.391618 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.391445    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 14 02:50:51.391618 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.391450    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 14 02:50:51.391618 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.391458    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 14 02:50:51.391618 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.391463    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 14 02:50:51.392299 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.392286    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 14 02:50:51.392353 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.392302    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 14 02:50:51.394627 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.394605    2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-180.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 14 02:50:51.395490 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:51.395469    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-180.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 14 02:50:51.395554 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:51.395469    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 14 02:50:51.395740 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.395728    2577 server.go:1268] "Started kubelet"
Sep 14 02:50:51.395852 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.395781    2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 14 02:50:51.395910 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.395837    2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 14 02:50:51.395910 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.395898    2577 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 14 02:50:51.400340 ip-10-0-141-180 systemd[1]: Started Kubernetes Kubelet.
Sep 14 02:50:51.400740 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.400708    2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 14 02:50:51.402223 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.402204    2577 server.go:310] "Adding debug handlers to kubelet server"
Sep 14 02:50:51.404350 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:51.404329    2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 14 02:50:51.408792 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:51.407726    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-180.ec2.internal.18d5103613c6aafa  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-180.ec2.internal,UID:ip-10-0-141-180.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-180.ec2.internal,},FirstTimestamp:2026-09-14 02:50:51.395705594 +0000 UTC m=+0.373683615,LastTimestamp:2026-09-14 02:50:51.395705594 +0000 UTC m=+0.373683615,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-180.ec2.internal,}"
Sep 14 02:50:51.409573 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.409548    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 14 02:50:51.410223 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.410209    2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 14 02:50:51.410322 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:51.410309    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-180.ec2.internal\" not found"
Sep 14 02:50:51.410949 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.410934    2577 factory.go:55] Registering systemd factory
Sep 14 02:50:51.410949 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.410947    2577 factory.go:223] Registration of the systemd container factory successfully
Sep 14 02:50:51.411112 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.411083    2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 14 02:50:51.411170 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.411103    2577 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 14 02:50:51.411170 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.411121    2577 factory.go:153] Registering CRI-O factory
Sep 14 02:50:51.411170 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.411128    2577 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 14 02:50:51.411170 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.411136    2577 factory.go:223] Registration of the crio container factory successfully
Sep 14 02:50:51.411382 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.411185    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 14 02:50:51.411382 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.411197    2577 reconstruct.go:97] "Volume reconstruction finished"
Sep 14 02:50:51.411382 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.411202    2577 factory.go:103] Registering Raw factory
Sep 14 02:50:51.411382 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.411218    2577 manager.go:1196] Started watching for new ooms in manager
Sep 14 02:50:51.411382 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.411205    2577 reconciler.go:29] "Reconciler: start to sync state"
Sep 14 02:50:51.411669 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.411649    2577 manager.go:349] Starting recovery of all containers
Sep 14 02:50:51.421604 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.421564    2577 manager.go:354] Recovery completed
Sep 14 02:50:51.423412 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:51.423390    2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-180.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 14 02:50:51.425842 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.425829    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 02:50:51.427357 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.427340    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 02:50:51.427414 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.427367    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 02:50:51.427414 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.427377    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasSufficientPID"
Sep 14 02:50:51.427922 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.427905    2577 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 14 02:50:51.427922 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.427921    2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 14 02:50:51.428061 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.427918    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 02:50:51.428061 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.427941    2577 state_mem.go:36] "Initialized new in-memory state store"
Sep 14 02:50:51.428581 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.428564    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 02:50:51.428665 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.428595    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 02:50:51.428665 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.428609    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasSufficientPID"
Sep 14 02:50:51.429946 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.429933    2577 policy_none.go:49] "None policy: Start"
Sep 14 02:50:51.430012 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.429950    2577 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 14 02:50:51.430012 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.429960    2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 14 02:50:51.431894 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.431883    2577 policy_none.go:47] "Start"
Sep 14 02:50:51.439631 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.437829    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kqpq2"
Sep 14 02:50:51.439962 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:51.439860    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-180.ec2.internal.18d5103615a999c0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-180.ec2.internal,UID:ip-10-0-141-180.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-180.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-180.ec2.internal,},FirstTimestamp:2026-09-14 02:50:51.427355072 +0000 UTC m=+0.405333092,LastTimestamp:2026-09-14 02:50:51.427355072 +0000 UTC m=+0.405333092,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-180.ec2.internal,}"
Sep 14 02:50:51.440217 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:51.440111    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 14 02:50:51.443670 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:51.443579    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-180.ec2.internal.18d5103615a9dc03  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-180.ec2.internal,UID:ip-10-0-141-180.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-141-180.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-141-180.ec2.internal,},FirstTimestamp:2026-09-14 02:50:51.427372035 +0000 UTC m=+0.405350054,LastTimestamp:2026-09-14 02:50:51.427372035 +0000 UTC m=+0.405350054,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-180.ec2.internal,}"
Sep 14 02:50:51.448538 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:51.448478    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-180.ec2.internal.18d5103615a9fe98  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-180.ec2.internal,UID:ip-10-0-141-180.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-141-180.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-141-180.ec2.internal,},FirstTimestamp:2026-09-14 02:50:51.427380888 +0000 UTC m=+0.405358908,LastTimestamp:2026-09-14 02:50:51.427380888 +0000 UTC m=+0.405358908,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-180.ec2.internal,}"
Sep 14 02:50:51.454817 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.454803    2577 manager.go:337] "Starting Device Plugin manager"
Sep 14 02:50:51.454892 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:51.454849    2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 14 02:50:51.454892 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.454862    2577 server.go:87] "Starting device plugin registration server"
Sep 14 02:50:51.455092 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.455078    2577 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 14 02:50:51.455142 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.455090    2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 14 02:50:51.455424 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.455377    2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 14 02:50:51.455530 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.455461    2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 14 02:50:51.455530 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.455469    2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 14 02:50:51.455738 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:51.455710    2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 14 02:50:51.455823 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:51.455757    2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-180.ec2.internal\" not found"
Sep 14 02:50:51.462036 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:51.461969    2577 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-180.ec2.internal.18d5103615a999c0\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-180.ec2.internal.18d5103615a999c0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-180.ec2.internal,UID:ip-10-0-141-180.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-180.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-180.ec2.internal,},FirstTimestamp:2026-09-14 02:50:51.427355072 +0000 UTC m=+0.405333092,LastTimestamp:2026-09-14 02:50:51.428580407 +0000 UTC m=+0.406558434,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-180.ec2.internal,}"
Sep 14 02:50:51.581758 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.465011    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kqpq2"
Sep 14 02:50:51.581758 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:51.469311    2577 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-180.ec2.internal.18d5103615a9dc03\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-180.ec2.internal.18d5103615a9dc03  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-180.ec2.internal,UID:ip-10-0-141-180.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-141-180.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-141-180.ec2.internal,},FirstTimestamp:2026-09-14 02:50:51.427372035 +0000 UTC m=+0.405350054,LastTimestamp:2026-09-14 02:50:51.428601482 +0000 UTC m=+0.406579504,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-180.ec2.internal,}"
Sep 14 02:50:51.581758 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:51.481085    2577 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-180.ec2.internal.18d5103615a9fe98\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-180.ec2.internal.18d5103615a9fe98  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-180.ec2.internal,UID:ip-10-0-141-180.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-141-180.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-141-180.ec2.internal,},FirstTimestamp:2026-09-14 02:50:51.427380888 +0000 UTC m=+0.405358908,LastTimestamp:2026-09-14 02:50:51.428614841 +0000 UTC m=+0.406592863,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-180.ec2.internal,}"
Sep 14 02:50:51.581758 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:51.490446    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-180.ec2.internal.18d5103617f23922  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-180.ec2.internal,UID:ip-10-0-141-180.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-141-180.ec2.internal,},FirstTimestamp:2026-09-14 02:50:51.465668898 +0000 UTC m=+0.443646905,LastTimestamp:2026-09-14 02:50:51.465668898 +0000 UTC m=+0.443646905,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-180.ec2.internal,}"
Sep 14 02:50:51.581758 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.555818    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 02:50:51.581758 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.556544    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 02:50:51.581758 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.556566    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 02:50:51.581758 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.556576    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasSufficientPID"
Sep 14 02:50:51.581758 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.556602    2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-180.ec2.internal"
Sep 14 02:50:51.581758 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:51.570569    2577 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-180.ec2.internal.18d5103615a999c0\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-180.ec2.internal.18d5103615a999c0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-180.ec2.internal,UID:ip-10-0-141-180.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-180.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-180.ec2.internal,},FirstTimestamp:2026-09-14 02:50:51.427355072 +0000 UTC m=+0.405333092,LastTimestamp:2026-09-14 02:50:51.556554555 +0000 UTC m=+0.534532575,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-180.ec2.internal,}"
Sep 14 02:50:51.581758 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:51.576125    2577 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-180.ec2.internal.18d5103615a9dc03\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-180.ec2.internal.18d5103615a9dc03  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-180.ec2.internal,UID:ip-10-0-141-180.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-141-180.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-141-180.ec2.internal,},FirstTimestamp:2026-09-14 02:50:51.427372035 +0000 UTC m=+0.405350054,LastTimestamp:2026-09-14 02:50:51.556570912 +0000 UTC m=+0.534548932,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-180.ec2.internal,}"
Sep 14 02:50:51.582173 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:51.576183    2577 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-141-180.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-141-180.ec2.internal"
Sep 14 02:50:51.584964 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:51.584898    2577 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-141-180.ec2.internal.18d5103615a9fe98\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-180.ec2.internal.18d5103615a9fe98  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-180.ec2.internal,UID:ip-10-0-141-180.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-141-180.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-141-180.ec2.internal,},FirstTimestamp:2026-09-14 02:50:51.427380888 +0000 UTC m=+0.405358908,LastTimestamp:2026-09-14 02:50:51.556582148 +0000 UTC m=+0.534560169,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-180.ec2.internal,}"
Sep 14 02:50:51.609768 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.609733    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 14 02:50:51.610861 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.610842    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 14 02:50:51.610932 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.610885    2577 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 14 02:50:51.610932 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.610903    2577 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 14 02:50:51.611021 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:51.610937    2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 14 02:50:51.628282 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.628248    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 02:50:51.634483 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:51.634461    2577 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-141-180.ec2.internal\" not found" node="ip-10-0-141-180.ec2.internal"
Sep 14 02:50:51.711788 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.711765    2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-180.ec2.internal"]
Sep 14 02:50:51.711872 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.711822    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 02:50:51.712536 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.712522    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 02:50:51.712604 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.712547    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 02:50:51.712604 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.712559    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasSufficientPID"
Sep 14 02:50:51.714801 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.714787    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 02:50:51.714913 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.714899    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal"
Sep 14 02:50:51.714951 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.714926    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 02:50:51.715469 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.715451    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 02:50:51.715533 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.715480    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 02:50:51.715533 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.715491    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasSufficientPID"
Sep 14 02:50:51.716401 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.716384    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 02:50:51.716468 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.716416    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 02:50:51.716468 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.716433    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasSufficientPID"
Sep 14 02:50:51.718633 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.718614    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-180.ec2.internal"
Sep 14 02:50:51.718733 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.718642    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 02:50:51.719305 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.719291    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 02:50:51.719403 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.719319    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 02:50:51.719403 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.719333    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasSufficientPID"
Sep 14 02:50:51.742935 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:51.742914    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-180.ec2.internal\" not found" node="ip-10-0-141-180.ec2.internal"
Sep 14 02:50:51.747258 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:51.747245    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-180.ec2.internal\" not found" node="ip-10-0-141-180.ec2.internal"
Sep 14 02:50:51.776664 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.776645    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 02:50:51.777383 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.777370    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 02:50:51.777450 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.777392    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 02:50:51.777450 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.777401    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasSufficientPID"
Sep 14 02:50:51.777450 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.777419    2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-180.ec2.internal"
Sep 14 02:50:51.786881 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.786868    2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-180.ec2.internal"
Sep 14 02:50:51.786927 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:51.786892    2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-180.ec2.internal\": node \"ip-10-0-141-180.ec2.internal\" not found"
Sep 14 02:50:51.812465 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.812443    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/82fb3813d6a127b50e9babbd92b7fbcd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal\" (UID: \"82fb3813d6a127b50e9babbd92b7fbcd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal"
Sep 14 02:50:51.812548 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.812472    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/82fb3813d6a127b50e9babbd92b7fbcd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal\" (UID: \"82fb3813d6a127b50e9babbd92b7fbcd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal"
Sep 14 02:50:51.812548 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.812505    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2336dac84cecf30d916b12e72e4b2ecc-config\") pod \"kube-apiserver-proxy-ip-10-0-141-180.ec2.internal\" (UID: \"2336dac84cecf30d916b12e72e4b2ecc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-180.ec2.internal"
Sep 14 02:50:51.826890 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:51.826872    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-180.ec2.internal\" not found"
Sep 14 02:50:51.912821 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.912773    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/82fb3813d6a127b50e9babbd92b7fbcd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal\" (UID: \"82fb3813d6a127b50e9babbd92b7fbcd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal"
Sep 14 02:50:51.912821 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.912798    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/82fb3813d6a127b50e9babbd92b7fbcd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal\" (UID: \"82fb3813d6a127b50e9babbd92b7fbcd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal"
Sep 14 02:50:51.912933 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.912821    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2336dac84cecf30d916b12e72e4b2ecc-config\") pod \"kube-apiserver-proxy-ip-10-0-141-180.ec2.internal\" (UID: \"2336dac84cecf30d916b12e72e4b2ecc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-180.ec2.internal"
Sep 14 02:50:51.912933 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.912875    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2336dac84cecf30d916b12e72e4b2ecc-config\") pod \"kube-apiserver-proxy-ip-10-0-141-180.ec2.internal\" (UID: \"2336dac84cecf30d916b12e72e4b2ecc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-180.ec2.internal"
Sep 14 02:50:51.912933 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.912879    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/82fb3813d6a127b50e9babbd92b7fbcd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal\" (UID: \"82fb3813d6a127b50e9babbd92b7fbcd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal"
Sep 14 02:50:51.912933 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:51.912884    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/82fb3813d6a127b50e9babbd92b7fbcd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal\" (UID: \"82fb3813d6a127b50e9babbd92b7fbcd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal"
Sep 14 02:50:51.927527 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:51.927505    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-180.ec2.internal\" not found"
Sep 14 02:50:52.028616 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:52.028582    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-180.ec2.internal\" not found"
Sep 14 02:50:52.045742 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:52.045725    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal"
Sep 14 02:50:52.050744 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:52.050439    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-180.ec2.internal"
Sep 14 02:50:52.129359 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:52.129336    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-180.ec2.internal\" not found"
Sep 14 02:50:52.230571 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:52.230420    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-180.ec2.internal\" not found"
Sep 14 02:50:52.331603 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:52.331577    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-180.ec2.internal\" not found"
Sep 14 02:50:52.341461 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:52.341433    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 14 02:50:52.341624 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:52.341602    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 14 02:50:52.432013 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:52.431988    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-180.ec2.internal\" not found"
Sep 14 02:50:52.532686 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:52.532633    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-180.ec2.internal\" not found"
Sep 14 02:50:52.577921 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:52.577901    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 02:50:52.582413 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:52.582398    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 02:50:52.582497 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:52.582426    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 02:50:52.582497 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:52.582439    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeHasSufficientPID"
Sep 14 02:50:52.585572 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:52.585548    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-13 02:45:52 +0000 UTC" deadline="2028-06-27 02:02:42.54039468 +0000 UTC"
Sep 14 02:50:52.585572 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:52.585570    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15647h11m49.954826724s"
Sep 14 02:50:52.633463 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:52.633443    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-180.ec2.internal\" not found"
Sep 14 02:50:52.656714 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:52.656694    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 02:50:52.732027 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:52.731996    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2336dac84cecf30d916b12e72e4b2ecc.slice/crio-6aaa67b68bdfa57f425872ef3930f4f35c8eb5032de8d7098c1237e2d3c15844 WatchSource:0}: Error finding container 6aaa67b68bdfa57f425872ef3930f4f35c8eb5032de8d7098c1237e2d3c15844: Status 404 returned error can't find the container with id 6aaa67b68bdfa57f425872ef3930f4f35c8eb5032de8d7098c1237e2d3c15844
Sep 14 02:50:52.732865 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:52.732845    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82fb3813d6a127b50e9babbd92b7fbcd.slice/crio-525f072f48dde51f0c526a4cbf5fe0034df3ee764887411946196aaafd200648 WatchSource:0}: Error finding container 525f072f48dde51f0c526a4cbf5fe0034df3ee764887411946196aaafd200648: Status 404 returned error can't find the container with id 525f072f48dde51f0c526a4cbf5fe0034df3ee764887411946196aaafd200648
Sep 14 02:50:52.734003 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:52.733987    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-180.ec2.internal\" not found"
Sep 14 02:50:52.737869 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:52.737854    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 02:50:52.805750 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:52.805699    2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 02:50:52.822936 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:52.822916    2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 02:50:52.911874 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:52.911843    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal"
Sep 14 02:50:52.950201 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:52.950180    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 14 02:50:52.951381 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:52.951366    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-180.ec2.internal"
Sep 14 02:50:52.966117 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:52.966098    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 14 02:50:53.387526 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.387498    2577 apiserver.go:52] "Watching apiserver"
Sep 14 02:50:53.396777 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.396751    2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 14 02:50:53.399149 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.399120    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-141-180.ec2.internal","openshift-cluster-node-tuning-operator/tuned-q6xjk","openshift-dns/node-resolver-szm7r","openshift-image-registry/node-ca-w8fds","openshift-multus/network-metrics-daemon-n7hgz","openshift-network-operator/iptables-alerter-p4r72","kube-system/global-pull-secret-syncer-hckjv","kube-system/konnectivity-agent-r8rwc","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-846dt","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal","openshift-multus/multus-additional-cni-plugins-6bfjv","openshift-multus/multus-qkcvg","openshift-network-diagnostics/network-check-target-n4kdw","openshift-ovn-kubernetes/ovnkube-node-xd5pl"]
Sep 14 02:50:53.401857 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.401835    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hckjv"
Sep 14 02:50:53.401955 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:53.401921    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hckjv" podUID="003c37b0-6bdd-4b28-9a26-fa2a4ed813ec"
Sep 14 02:50:53.404276 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.404256    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4kdw"
Sep 14 02:50:53.404396 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:53.404323    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n4kdw" podUID="615f196a-4cb6-42a8-9e5f-fdea4a8672d6"
Sep 14 02:50:53.406468 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.406446    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-szm7r"
Sep 14 02:50:53.408710 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.408692    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-w8fds"
Sep 14 02:50:53.410775 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.410751    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 14 02:50:53.410990 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.410970    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 14 02:50:53.412536 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.412253    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 14 02:50:53.412536 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.412295    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 14 02:50:53.412536 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.412433    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 14 02:50:53.412741 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.412618    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7hgz"
Sep 14 02:50:53.412741 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:53.412693    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n7hgz" podUID="e746671e-19d5-48e7-8b3b-a11462954481"
Sep 14 02:50:53.413827 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.413787    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-p4r72"
Sep 14 02:50:53.415862 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.415842    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-846dt"
Sep 14 02:50:53.417304 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.417127    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 14 02:50:53.420395 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.419346    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ck67h\""
Sep 14 02:50:53.420395 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.419641    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.420395 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.420139    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 14 02:50:53.422535 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.420866    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 14 02:50:53.422535 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.420906    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 14 02:50:53.422535 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.421847    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rkg9\" (UniqueName: \"kubernetes.io/projected/e746671e-19d5-48e7-8b3b-a11462954481-kube-api-access-7rkg9\") pod \"network-metrics-daemon-n7hgz\" (UID: \"e746671e-19d5-48e7-8b3b-a11462954481\") " pod="openshift-multus/network-metrics-daemon-n7hgz"
Sep 14 02:50:53.422535 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.421879    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9d1ef897-6ce7-429d-871c-104e72bc90da-host-slash\") pod \"iptables-alerter-p4r72\" (UID: \"9d1ef897-6ce7-429d-871c-104e72bc90da\") " pod="openshift-network-operator/iptables-alerter-p4r72"
Sep 14 02:50:53.422535 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.421905    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gkkz\" (UniqueName: \"kubernetes.io/projected/9d1ef897-6ce7-429d-871c-104e72bc90da-kube-api-access-4gkkz\") pod \"iptables-alerter-p4r72\" (UID: \"9d1ef897-6ce7-429d-871c-104e72bc90da\") " pod="openshift-network-operator/iptables-alerter-p4r72"
Sep 14 02:50:53.422535 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.421931    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/79878e2e-9231-4688-b888-dc8db6184780-hosts-file\") pod \"node-resolver-szm7r\" (UID: \"79878e2e-9231-4688-b888-dc8db6184780\") " pod="openshift-dns/node-resolver-szm7r"
Sep 14 02:50:53.422535 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.421958    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9d1ef897-6ce7-429d-871c-104e72bc90da-iptables-alerter-script\") pod \"iptables-alerter-p4r72\" (UID: \"9d1ef897-6ce7-429d-871c-104e72bc90da\") " pod="openshift-network-operator/iptables-alerter-p4r72"
Sep 14 02:50:53.422535 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.421983    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/003c37b0-6bdd-4b28-9a26-fa2a4ed813ec-dbus\") pod \"global-pull-secret-syncer-hckjv\" (UID: \"003c37b0-6bdd-4b28-9a26-fa2a4ed813ec\") " pod="kube-system/global-pull-secret-syncer-hckjv"
Sep 14 02:50:53.422535 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.422009    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79878e2e-9231-4688-b888-dc8db6184780-tmp-dir\") pod \"node-resolver-szm7r\" (UID: \"79878e2e-9231-4688-b888-dc8db6184780\") " pod="openshift-dns/node-resolver-szm7r"
Sep 14 02:50:53.422535 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.422035    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/af1c701f-81cb-47aa-85a1-439d4f260cea-host\") pod \"node-ca-w8fds\" (UID: \"af1c701f-81cb-47aa-85a1-439d4f260cea\") " pod="openshift-image-registry/node-ca-w8fds"
Sep 14 02:50:53.422535 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.422066    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e746671e-19d5-48e7-8b3b-a11462954481-metrics-certs\") pod \"network-metrics-daemon-n7hgz\" (UID: \"e746671e-19d5-48e7-8b3b-a11462954481\") " pod="openshift-multus/network-metrics-daemon-n7hgz"
Sep 14 02:50:53.422535 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.422095    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/003c37b0-6bdd-4b28-9a26-fa2a4ed813ec-kubelet-config\") pod \"global-pull-secret-syncer-hckjv\" (UID: \"003c37b0-6bdd-4b28-9a26-fa2a4ed813ec\") " pod="kube-system/global-pull-secret-syncer-hckjv"
Sep 14 02:50:53.422535 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.422123    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/003c37b0-6bdd-4b28-9a26-fa2a4ed813ec-original-pull-secret\") pod \"global-pull-secret-syncer-hckjv\" (UID: \"003c37b0-6bdd-4b28-9a26-fa2a4ed813ec\") " pod="kube-system/global-pull-secret-syncer-hckjv"
Sep 14 02:50:53.422535 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.422152    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rkhd\" (UniqueName: \"kubernetes.io/projected/615f196a-4cb6-42a8-9e5f-fdea4a8672d6-kube-api-access-4rkhd\") pod \"network-check-target-n4kdw\" (UID: \"615f196a-4cb6-42a8-9e5f-fdea4a8672d6\") " pod="openshift-network-diagnostics/network-check-target-n4kdw"
Sep 14 02:50:53.422535 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.422181    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fldjz\" (UniqueName: \"kubernetes.io/projected/79878e2e-9231-4688-b888-dc8db6184780-kube-api-access-fldjz\") pod \"node-resolver-szm7r\" (UID: \"79878e2e-9231-4688-b888-dc8db6184780\") " pod="openshift-dns/node-resolver-szm7r"
Sep 14 02:50:53.423360 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.422528    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/af1c701f-81cb-47aa-85a1-439d4f260cea-serviceca\") pod \"node-ca-w8fds\" (UID: \"af1c701f-81cb-47aa-85a1-439d4f260cea\") " pod="openshift-image-registry/node-ca-w8fds"
Sep 14 02:50:53.423360 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.422809    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4p9xc\" (UniqueName: \"kubernetes.io/projected/af1c701f-81cb-47aa-85a1-439d4f260cea-kube-api-access-4p9xc\") pod \"node-ca-w8fds\" (UID: \"af1c701f-81cb-47aa-85a1-439d4f260cea\") " pod="openshift-image-registry/node-ca-w8fds"
Sep 14 02:50:53.427135 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.424335    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7lpxs\""
Sep 14 02:50:53.427135 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.424803    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 14 02:50:53.427135 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.424969    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-s99n5\""
Sep 14 02:50:53.427135 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.425073    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 14 02:50:53.427135 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.425298    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 14 02:50:53.427135 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.426120    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 14 02:50:53.427135 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.426418    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 14 02:50:53.427135 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.426551    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.427135 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.426684    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-r8rwc"
Sep 14 02:50:53.427996 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.427709    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 14 02:50:53.427996 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.427861    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 14 02:50:53.428873 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.428296    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6bfjv"
Sep 14 02:50:53.429889 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.429444    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vxp5l\""
Sep 14 02:50:53.429889 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.429515    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 14 02:50:53.429889 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.429571    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7tlp9\""
Sep 14 02:50:53.429889 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.429601    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 14 02:50:53.429889 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.429751    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 14 02:50:53.430207 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.429965    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 14 02:50:53.431137 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.425528    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-tphq9\""
Sep 14 02:50:53.431137 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.430722    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 14 02:50:53.435133 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.433005    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.435133 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.433337    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 14 02:50:53.435133 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.433509    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-72hdt\""
Sep 14 02:50:53.435133 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.433627    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 14 02:50:53.435133 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.433840    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 14 02:50:53.435133 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.433852    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 14 02:50:53.435133 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.433987    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 14 02:50:53.435133 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.434245    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fd9hg\""
Sep 14 02:50:53.435133 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.434408    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 14 02:50:53.439876 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.439856    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-pzw52\""
Sep 14 02:50:53.440631 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.440589    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 14 02:50:53.446913 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.446733    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 14 02:50:53.483592 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.483566    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rwzr8"
Sep 14 02:50:53.498569 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.498549    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rwzr8"
Sep 14 02:50:53.512263 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.512199    2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 14 02:50:53.523708 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.523683    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rkg9\" (UniqueName: \"kubernetes.io/projected/e746671e-19d5-48e7-8b3b-a11462954481-kube-api-access-7rkg9\") pod \"network-metrics-daemon-n7hgz\" (UID: \"e746671e-19d5-48e7-8b3b-a11462954481\") " pod="openshift-multus/network-metrics-daemon-n7hgz"
Sep 14 02:50:53.523820 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.523723    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9d1ef897-6ce7-429d-871c-104e72bc90da-host-slash\") pod \"iptables-alerter-p4r72\" (UID: \"9d1ef897-6ce7-429d-871c-104e72bc90da\") " pod="openshift-network-operator/iptables-alerter-p4r72"
Sep 14 02:50:53.523820 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.523750    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/79878e2e-9231-4688-b888-dc8db6184780-hosts-file\") pod \"node-resolver-szm7r\" (UID: \"79878e2e-9231-4688-b888-dc8db6184780\") " pod="openshift-dns/node-resolver-szm7r"
Sep 14 02:50:53.523820 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.523779    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-host-run-ovn-kubernetes\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.523820 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.523807    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-node-log\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.524040 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.523830    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5d26b1b-8e73-4ff3-b939-c12334ddb952-etc-kubernetes\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.524040 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.523854    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msbhk\" (UniqueName: \"kubernetes.io/projected/e448e444-53b3-4c07-8679-1c8c8960acff-kube-api-access-msbhk\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.524040 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.523880    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78m46\" (UniqueName: \"kubernetes.io/projected/13c3acff-3931-4204-96e2-0d7cf6559153-kube-api-access-78m46\") pod \"aws-ebs-csi-driver-node-846dt\" (UID: \"13c3acff-3931-4204-96e2-0d7cf6559153\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-846dt"
Sep 14 02:50:53.524040 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.523903    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d5d26b1b-8e73-4ff3-b939-c12334ddb952-host\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.524040 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.523929    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/38662066-a2a4-4e2e-8aef-f5478c1205a6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6bfjv\" (UID: \"38662066-a2a4-4e2e-8aef-f5478c1205a6\") " pod="openshift-multus/multus-additional-cni-plugins-6bfjv"
Sep 14 02:50:53.524040 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.523953    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-host-var-lib-kubelet\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.524040 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.523986    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9dcf46a1-8ecf-4333-8c37-13c9e717bde5-konnectivity-ca\") pod \"konnectivity-agent-r8rwc\" (UID: \"9dcf46a1-8ecf-4333-8c37-13c9e717bde5\") " pod="kube-system/konnectivity-agent-r8rwc"
Sep 14 02:50:53.524040 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.524013    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/003c37b0-6bdd-4b28-9a26-fa2a4ed813ec-kubelet-config\") pod \"global-pull-secret-syncer-hckjv\" (UID: \"003c37b0-6bdd-4b28-9a26-fa2a4ed813ec\") " pod="kube-system/global-pull-secret-syncer-hckjv"
Sep 14 02:50:53.524040 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.524040    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fldjz\" (UniqueName: \"kubernetes.io/projected/79878e2e-9231-4688-b888-dc8db6184780-kube-api-access-fldjz\") pod \"node-resolver-szm7r\" (UID: \"79878e2e-9231-4688-b888-dc8db6184780\") " pod="openshift-dns/node-resolver-szm7r"
Sep 14 02:50:53.524523 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.524082    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-var-lib-openvswitch\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.524523 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.524109    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-multus-socket-dir-parent\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.524523 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.524136    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.524523 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.524164    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/38662066-a2a4-4e2e-8aef-f5478c1205a6-cnibin\") pod \"multus-additional-cni-plugins-6bfjv\" (UID: \"38662066-a2a4-4e2e-8aef-f5478c1205a6\") " pod="openshift-multus/multus-additional-cni-plugins-6bfjv"
Sep 14 02:50:53.524523 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.524190    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2k4h\" (UniqueName: \"kubernetes.io/projected/38662066-a2a4-4e2e-8aef-f5478c1205a6-kube-api-access-w2k4h\") pod \"multus-additional-cni-plugins-6bfjv\" (UID: \"38662066-a2a4-4e2e-8aef-f5478c1205a6\") " pod="openshift-multus/multus-additional-cni-plugins-6bfjv"
Sep 14 02:50:53.524523 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.524215    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-etc-kubernetes\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.524523 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.524255    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79878e2e-9231-4688-b888-dc8db6184780-tmp-dir\") pod \"node-resolver-szm7r\" (UID: \"79878e2e-9231-4688-b888-dc8db6184780\") " pod="openshift-dns/node-resolver-szm7r"
Sep 14 02:50:53.524523 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.524292    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-etc-openvswitch\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.524523 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.524316    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-host-cni-netd\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.524523 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.524410    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-ovnkube-config\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.524523 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.524446    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxwt9\" (UniqueName: \"kubernetes.io/projected/d5d26b1b-8e73-4ff3-b939-c12334ddb952-kube-api-access-vxwt9\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.524523 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.524475    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-multus-cni-dir\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.524523 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.524502    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/38662066-a2a4-4e2e-8aef-f5478c1205a6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6bfjv\" (UID: \"38662066-a2a4-4e2e-8aef-f5478c1205a6\") " pod="openshift-multus/multus-additional-cni-plugins-6bfjv"
Sep 14 02:50:53.524523 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.524506    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/79878e2e-9231-4688-b888-dc8db6184780-hosts-file\") pod \"node-resolver-szm7r\" (UID: \"79878e2e-9231-4688-b888-dc8db6184780\") " pod="openshift-dns/node-resolver-szm7r"
Sep 14 02:50:53.525231 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.524531    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/af1c701f-81cb-47aa-85a1-439d4f260cea-host\") pod \"node-ca-w8fds\" (UID: \"af1c701f-81cb-47aa-85a1-439d4f260cea\") " pod="openshift-image-registry/node-ca-w8fds"
Sep 14 02:50:53.525231 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.524559    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-host-slash\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.525231 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.524416    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9d1ef897-6ce7-429d-871c-104e72bc90da-host-slash\") pod \"iptables-alerter-p4r72\" (UID: \"9d1ef897-6ce7-429d-871c-104e72bc90da\") " pod="openshift-network-operator/iptables-alerter-p4r72"
Sep 14 02:50:53.525231 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.524585    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d5d26b1b-8e73-4ff3-b939-c12334ddb952-etc-sysctl-d\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.525231 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.524609    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d5d26b1b-8e73-4ff3-b939-c12334ddb952-run\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.525231 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.524635    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d5d26b1b-8e73-4ff3-b939-c12334ddb952-etc-tuned\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.525231 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.524663    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-host-var-lib-cni-bin\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.525231 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.524689    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/13c3acff-3931-4204-96e2-0d7cf6559153-registration-dir\") pod \"aws-ebs-csi-driver-node-846dt\" (UID: \"13c3acff-3931-4204-96e2-0d7cf6559153\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-846dt"
Sep 14 02:50:53.525231 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.524703    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79878e2e-9231-4688-b888-dc8db6184780-tmp-dir\") pod \"node-resolver-szm7r\" (UID: \"79878e2e-9231-4688-b888-dc8db6184780\") " pod="openshift-dns/node-resolver-szm7r"
Sep 14 02:50:53.525231 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.524718    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rkhd\" (UniqueName: \"kubernetes.io/projected/615f196a-4cb6-42a8-9e5f-fdea4a8672d6-kube-api-access-4rkhd\") pod \"network-check-target-n4kdw\" (UID: \"615f196a-4cb6-42a8-9e5f-fdea4a8672d6\") " pod="openshift-network-diagnostics/network-check-target-n4kdw"
Sep 14 02:50:53.525231 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.524739    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/af1c701f-81cb-47aa-85a1-439d4f260cea-host\") pod \"node-ca-w8fds\" (UID: \"af1c701f-81cb-47aa-85a1-439d4f260cea\") " pod="openshift-image-registry/node-ca-w8fds"
Sep 14 02:50:53.525231 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.524767    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-systemd-units\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.525231 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.524561    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/003c37b0-6bdd-4b28-9a26-fa2a4ed813ec-kubelet-config\") pod \"global-pull-secret-syncer-hckjv\" (UID: \"003c37b0-6bdd-4b28-9a26-fa2a4ed813ec\") " pod="kube-system/global-pull-secret-syncer-hckjv"
Sep 14 02:50:53.525231 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.524796    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-ovn-node-metrics-cert\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.525231 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.524822    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e448e444-53b3-4c07-8679-1c8c8960acff-cni-binary-copy\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.525231 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.524848    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-host-run-netns\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.525231 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.524886    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-hostroot\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.526060 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.524926    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e448e444-53b3-4c07-8679-1c8c8960acff-multus-daemon-config\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.526060 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.524972    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-host-run-multus-certs\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.526060 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.525001    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/af1c701f-81cb-47aa-85a1-439d4f260cea-serviceca\") pod \"node-ca-w8fds\" (UID: \"af1c701f-81cb-47aa-85a1-439d4f260cea\") " pod="openshift-image-registry/node-ca-w8fds"
Sep 14 02:50:53.526060 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.525049    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7p6x\" (UniqueName: \"kubernetes.io/projected/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-kube-api-access-p7p6x\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.526060 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.525080    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-system-cni-dir\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.526060 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.525123    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-host-var-lib-cni-multus\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.526060 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.525171    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/003c37b0-6bdd-4b28-9a26-fa2a4ed813ec-dbus\") pod \"global-pull-secret-syncer-hckjv\" (UID: \"003c37b0-6bdd-4b28-9a26-fa2a4ed813ec\") " pod="kube-system/global-pull-secret-syncer-hckjv"
Sep 14 02:50:53.526060 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.525217    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-host-run-netns\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.526060 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.525293    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d5d26b1b-8e73-4ff3-b939-c12334ddb952-etc-systemd\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.526060 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.525321    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/38662066-a2a4-4e2e-8aef-f5478c1205a6-cni-binary-copy\") pod \"multus-additional-cni-plugins-6bfjv\" (UID: \"38662066-a2a4-4e2e-8aef-f5478c1205a6\") " pod="openshift-multus/multus-additional-cni-plugins-6bfjv"
Sep 14 02:50:53.526060 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.525361    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/13c3acff-3931-4204-96e2-0d7cf6559153-etc-selinux\") pod \"aws-ebs-csi-driver-node-846dt\" (UID: \"13c3acff-3931-4204-96e2-0d7cf6559153\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-846dt"
Sep 14 02:50:53.526060 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.525387    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/13c3acff-3931-4204-96e2-0d7cf6559153-kubelet-dir\") pod \"aws-ebs-csi-driver-node-846dt\" (UID: \"13c3acff-3931-4204-96e2-0d7cf6559153\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-846dt"
Sep 14 02:50:53.526060 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.525411    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9dcf46a1-8ecf-4333-8c37-13c9e717bde5-agent-certs\") pod \"konnectivity-agent-r8rwc\" (UID: \"9dcf46a1-8ecf-4333-8c37-13c9e717bde5\") " pod="kube-system/konnectivity-agent-r8rwc"
Sep 14 02:50:53.526060 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.525419    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/003c37b0-6bdd-4b28-9a26-fa2a4ed813ec-dbus\") pod \"global-pull-secret-syncer-hckjv\" (UID: \"003c37b0-6bdd-4b28-9a26-fa2a4ed813ec\") " pod="kube-system/global-pull-secret-syncer-hckjv"
Sep 14 02:50:53.526060 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.525439    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e746671e-19d5-48e7-8b3b-a11462954481-metrics-certs\") pod \"network-metrics-daemon-n7hgz\" (UID: \"e746671e-19d5-48e7-8b3b-a11462954481\") " pod="openshift-multus/network-metrics-daemon-n7hgz"
Sep 14 02:50:53.526060 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.525498    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/003c37b0-6bdd-4b28-9a26-fa2a4ed813ec-original-pull-secret\") pod \"global-pull-secret-syncer-hckjv\" (UID: \"003c37b0-6bdd-4b28-9a26-fa2a4ed813ec\") " pod="kube-system/global-pull-secret-syncer-hckjv"
Sep 14 02:50:53.526060 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:53.525533    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:50:53.526881 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.525530    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-cnibin\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.526881 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.525531    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/af1c701f-81cb-47aa-85a1-439d4f260cea-serviceca\") pod \"node-ca-w8fds\" (UID: \"af1c701f-81cb-47aa-85a1-439d4f260cea\") " pod="openshift-image-registry/node-ca-w8fds"
Sep 14 02:50:53.526881 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.525573    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4p9xc\" (UniqueName: \"kubernetes.io/projected/af1c701f-81cb-47aa-85a1-439d4f260cea-kube-api-access-4p9xc\") pod \"node-ca-w8fds\" (UID: \"af1c701f-81cb-47aa-85a1-439d4f260cea\") " pod="openshift-image-registry/node-ca-w8fds"
Sep 14 02:50:53.526881 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:53.525611    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 02:50:53.526881 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:53.525638    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e746671e-19d5-48e7-8b3b-a11462954481-metrics-certs podName:e746671e-19d5-48e7-8b3b-a11462954481 nodeName:}" failed. No retries permitted until 2026-09-14 02:50:54.025618366 +0000 UTC m=+3.003596414 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e746671e-19d5-48e7-8b3b-a11462954481-metrics-certs") pod "network-metrics-daemon-n7hgz" (UID: "e746671e-19d5-48e7-8b3b-a11462954481") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:50:53.526881 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.525659    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gkkz\" (UniqueName: \"kubernetes.io/projected/9d1ef897-6ce7-429d-871c-104e72bc90da-kube-api-access-4gkkz\") pod \"iptables-alerter-p4r72\" (UID: \"9d1ef897-6ce7-429d-871c-104e72bc90da\") " pod="openshift-network-operator/iptables-alerter-p4r72"
Sep 14 02:50:53.526881 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.525688    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d5d26b1b-8e73-4ff3-b939-c12334ddb952-etc-modprobe-d\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.526881 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.525790    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d5d26b1b-8e73-4ff3-b939-c12334ddb952-etc-sysctl-conf\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.526881 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.525825    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d5d26b1b-8e73-4ff3-b939-c12334ddb952-lib-modules\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.526881 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:53.525871    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/003c37b0-6bdd-4b28-9a26-fa2a4ed813ec-original-pull-secret podName:003c37b0-6bdd-4b28-9a26-fa2a4ed813ec nodeName:}" failed. No retries permitted until 2026-09-14 02:50:54.025856475 +0000 UTC m=+3.003834486 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/003c37b0-6bdd-4b28-9a26-fa2a4ed813ec-original-pull-secret") pod "global-pull-secret-syncer-hckjv" (UID: "003c37b0-6bdd-4b28-9a26-fa2a4ed813ec") : object "kube-system"/"original-pull-secret" not registered
Sep 14 02:50:53.526881 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.525925    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/38662066-a2a4-4e2e-8aef-f5478c1205a6-system-cni-dir\") pod \"multus-additional-cni-plugins-6bfjv\" (UID: \"38662066-a2a4-4e2e-8aef-f5478c1205a6\") " pod="openshift-multus/multus-additional-cni-plugins-6bfjv"
Sep 14 02:50:53.526881 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.525953    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-host-run-k8s-cni-cncf-io\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.526881 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.525991    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9d1ef897-6ce7-429d-871c-104e72bc90da-iptables-alerter-script\") pod \"iptables-alerter-p4r72\" (UID: \"9d1ef897-6ce7-429d-871c-104e72bc90da\") " pod="openshift-network-operator/iptables-alerter-p4r72"
Sep 14 02:50:53.526881 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.526035    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-ovnkube-script-lib\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.526881 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.526152    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5d26b1b-8e73-4ff3-b939-c12334ddb952-var-lib-kubelet\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.527672 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.526184    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/38662066-a2a4-4e2e-8aef-f5478c1205a6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6bfjv\" (UID: \"38662066-a2a4-4e2e-8aef-f5478c1205a6\") " pod="openshift-multus/multus-additional-cni-plugins-6bfjv"
Sep 14 02:50:53.527672 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.526230    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-multus-conf-dir\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.527672 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.526259    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/13c3acff-3931-4204-96e2-0d7cf6559153-socket-dir\") pod \"aws-ebs-csi-driver-node-846dt\" (UID: \"13c3acff-3931-4204-96e2-0d7cf6559153\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-846dt"
Sep 14 02:50:53.527672 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.526301    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-run-systemd\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.527672 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.526329    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-run-ovn\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.527672 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.526379    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-log-socket\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.527672 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.526424    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-host-cni-bin\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.527672 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.526450    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/13c3acff-3931-4204-96e2-0d7cf6559153-device-dir\") pod \"aws-ebs-csi-driver-node-846dt\" (UID: \"13c3acff-3931-4204-96e2-0d7cf6559153\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-846dt"
Sep 14 02:50:53.527672 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.526488    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/13c3acff-3931-4204-96e2-0d7cf6559153-sys-fs\") pod \"aws-ebs-csi-driver-node-846dt\" (UID: \"13c3acff-3931-4204-96e2-0d7cf6559153\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-846dt"
Sep 14 02:50:53.527672 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.526512    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-host-kubelet\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.527672 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.526560    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-run-openvswitch\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.527672 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.526599    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-env-overrides\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.527672 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.526657    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d5d26b1b-8e73-4ff3-b939-c12334ddb952-etc-sysconfig\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.527672 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.526693    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5d26b1b-8e73-4ff3-b939-c12334ddb952-sys\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.527672 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.526727    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d5d26b1b-8e73-4ff3-b939-c12334ddb952-tmp\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.527672 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.527210    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9d1ef897-6ce7-429d-871c-104e72bc90da-iptables-alerter-script\") pod \"iptables-alerter-p4r72\" (UID: \"9d1ef897-6ce7-429d-871c-104e72bc90da\") " pod="openshift-network-operator/iptables-alerter-p4r72"
Sep 14 02:50:53.545644 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.545620    2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 14 02:50:53.547639 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:53.547618    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 02:50:53.547728 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:53.547645    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 02:50:53.547728 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:53.547666    2577 projected.go:196] Error preparing data for projected volume kube-api-access-4rkhd for pod openshift-network-diagnostics/network-check-target-n4kdw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:50:53.547841 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:53.547728    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/615f196a-4cb6-42a8-9e5f-fdea4a8672d6-kube-api-access-4rkhd podName:615f196a-4cb6-42a8-9e5f-fdea4a8672d6 nodeName:}" failed. No retries permitted until 2026-09-14 02:50:54.047712573 +0000 UTC m=+3.025690588 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4rkhd" (UniqueName: "kubernetes.io/projected/615f196a-4cb6-42a8-9e5f-fdea4a8672d6-kube-api-access-4rkhd") pod "network-check-target-n4kdw" (UID: "615f196a-4cb6-42a8-9e5f-fdea4a8672d6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:50:53.549534 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.549510    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 02:50:53.550630 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.550579    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gkkz\" (UniqueName: \"kubernetes.io/projected/9d1ef897-6ce7-429d-871c-104e72bc90da-kube-api-access-4gkkz\") pod \"iptables-alerter-p4r72\" (UID: \"9d1ef897-6ce7-429d-871c-104e72bc90da\") " pod="openshift-network-operator/iptables-alerter-p4r72"
Sep 14 02:50:53.550726 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.550689    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rkg9\" (UniqueName: \"kubernetes.io/projected/e746671e-19d5-48e7-8b3b-a11462954481-kube-api-access-7rkg9\") pod \"network-metrics-daemon-n7hgz\" (UID: \"e746671e-19d5-48e7-8b3b-a11462954481\") " pod="openshift-multus/network-metrics-daemon-n7hgz"
Sep 14 02:50:53.555063 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.555037    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fldjz\" (UniqueName: \"kubernetes.io/projected/79878e2e-9231-4688-b888-dc8db6184780-kube-api-access-fldjz\") pod \"node-resolver-szm7r\" (UID: \"79878e2e-9231-4688-b888-dc8db6184780\") " pod="openshift-dns/node-resolver-szm7r"
Sep 14 02:50:53.566895 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.566874    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4p9xc\" (UniqueName: \"kubernetes.io/projected/af1c701f-81cb-47aa-85a1-439d4f260cea-kube-api-access-4p9xc\") pod \"node-ca-w8fds\" (UID: \"af1c701f-81cb-47aa-85a1-439d4f260cea\") " pod="openshift-image-registry/node-ca-w8fds"
Sep 14 02:50:53.616879 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.616838    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal" event={"ID":"82fb3813d6a127b50e9babbd92b7fbcd","Type":"ContainerStarted","Data":"525f072f48dde51f0c526a4cbf5fe0034df3ee764887411946196aaafd200648"}
Sep 14 02:50:53.618244 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.618219    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-180.ec2.internal" event={"ID":"2336dac84cecf30d916b12e72e4b2ecc","Type":"ContainerStarted","Data":"6aaa67b68bdfa57f425872ef3930f4f35c8eb5032de8d7098c1237e2d3c15844"}
Sep 14 02:50:53.627829 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.627802    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d5d26b1b-8e73-4ff3-b939-c12334ddb952-etc-modprobe-d\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.627937 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.627841    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d5d26b1b-8e73-4ff3-b939-c12334ddb952-etc-sysctl-conf\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.627937 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.627863    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d5d26b1b-8e73-4ff3-b939-c12334ddb952-lib-modules\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.627937 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.627884    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/38662066-a2a4-4e2e-8aef-f5478c1205a6-system-cni-dir\") pod \"multus-additional-cni-plugins-6bfjv\" (UID: \"38662066-a2a4-4e2e-8aef-f5478c1205a6\") " pod="openshift-multus/multus-additional-cni-plugins-6bfjv"
Sep 14 02:50:53.627937 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.627909    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-host-run-k8s-cni-cncf-io\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.628220 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.627951    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-ovnkube-script-lib\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.628220 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.627972    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5d26b1b-8e73-4ff3-b939-c12334ddb952-var-lib-kubelet\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.628220 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.627979    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d5d26b1b-8e73-4ff3-b939-c12334ddb952-etc-modprobe-d\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.628220 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.627982    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/38662066-a2a4-4e2e-8aef-f5478c1205a6-system-cni-dir\") pod \"multus-additional-cni-plugins-6bfjv\" (UID: \"38662066-a2a4-4e2e-8aef-f5478c1205a6\") " pod="openshift-multus/multus-additional-cni-plugins-6bfjv"
Sep 14 02:50:53.628220 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.627996    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-host-run-k8s-cni-cncf-io\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.628220 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.627996    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/38662066-a2a4-4e2e-8aef-f5478c1205a6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6bfjv\" (UID: \"38662066-a2a4-4e2e-8aef-f5478c1205a6\") " pod="openshift-multus/multus-additional-cni-plugins-6bfjv"
Sep 14 02:50:53.628220 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.627982    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d5d26b1b-8e73-4ff3-b939-c12334ddb952-lib-modules\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.628220 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.628014    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d5d26b1b-8e73-4ff3-b939-c12334ddb952-etc-sysctl-conf\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.628220 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.628048    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-multus-conf-dir\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.628220 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.628055    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5d26b1b-8e73-4ff3-b939-c12334ddb952-var-lib-kubelet\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.628220 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.628095    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-multus-conf-dir\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.628220 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.628110    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/13c3acff-3931-4204-96e2-0d7cf6559153-socket-dir\") pod \"aws-ebs-csi-driver-node-846dt\" (UID: \"13c3acff-3931-4204-96e2-0d7cf6559153\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-846dt"
Sep 14 02:50:53.628220 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.628140    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-run-systemd\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.628220 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.628166    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-run-ovn\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.628220 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.628192    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-log-socket\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.628220 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.628215    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-host-cni-bin\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.629064 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.628241    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-run-systemd\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.629064 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.628245    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/13c3acff-3931-4204-96e2-0d7cf6559153-device-dir\") pod \"aws-ebs-csi-driver-node-846dt\" (UID: \"13c3acff-3931-4204-96e2-0d7cf6559153\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-846dt"
Sep 14 02:50:53.629064 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.628308    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/13c3acff-3931-4204-96e2-0d7cf6559153-device-dir\") pod \"aws-ebs-csi-driver-node-846dt\" (UID: \"13c3acff-3931-4204-96e2-0d7cf6559153\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-846dt"
Sep 14 02:50:53.629064 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.628362    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/13c3acff-3931-4204-96e2-0d7cf6559153-sys-fs\") pod \"aws-ebs-csi-driver-node-846dt\" (UID: \"13c3acff-3931-4204-96e2-0d7cf6559153\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-846dt"
Sep 14 02:50:53.629064 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.628408    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-run-ovn\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.629064 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.628423    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/13c3acff-3931-4204-96e2-0d7cf6559153-socket-dir\") pod \"aws-ebs-csi-driver-node-846dt\" (UID: \"13c3acff-3931-4204-96e2-0d7cf6559153\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-846dt"
Sep 14 02:50:53.629064 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.628452    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-log-socket\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.629064 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.628309    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/13c3acff-3931-4204-96e2-0d7cf6559153-sys-fs\") pod \"aws-ebs-csi-driver-node-846dt\" (UID: \"13c3acff-3931-4204-96e2-0d7cf6559153\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-846dt"
Sep 14 02:50:53.629064 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.628479    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-host-cni-bin\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.629064 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.628489    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-host-kubelet\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.629064 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.628551    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-host-kubelet\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.629064 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.628583    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-run-openvswitch\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.629064 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.628590    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-ovnkube-script-lib\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.629064 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.628628    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-env-overrides\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.629064 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.628626    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-run-openvswitch\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.629064 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.628652    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d5d26b1b-8e73-4ff3-b939-c12334ddb952-etc-sysconfig\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.629064 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.628659    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/38662066-a2a4-4e2e-8aef-f5478c1205a6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6bfjv\" (UID: \"38662066-a2a4-4e2e-8aef-f5478c1205a6\") " pod="openshift-multus/multus-additional-cni-plugins-6bfjv"
Sep 14 02:50:53.629960 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.628684    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5d26b1b-8e73-4ff3-b939-c12334ddb952-sys\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.629960 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.628721    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d5d26b1b-8e73-4ff3-b939-c12334ddb952-etc-sysconfig\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.629960 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.628744    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5d26b1b-8e73-4ff3-b939-c12334ddb952-sys\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.629960 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.628782    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d5d26b1b-8e73-4ff3-b939-c12334ddb952-tmp\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.629960 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.628826    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-host-run-ovn-kubernetes\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.629960 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.628852    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-node-log\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.629960 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.628896    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-node-log\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.629960 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.628942    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-host-run-ovn-kubernetes\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.629960 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.628971    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5d26b1b-8e73-4ff3-b939-c12334ddb952-etc-kubernetes\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.629960 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.628996    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msbhk\" (UniqueName: \"kubernetes.io/projected/e448e444-53b3-4c07-8679-1c8c8960acff-kube-api-access-msbhk\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.629960 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.629024    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78m46\" (UniqueName: \"kubernetes.io/projected/13c3acff-3931-4204-96e2-0d7cf6559153-kube-api-access-78m46\") pod \"aws-ebs-csi-driver-node-846dt\" (UID: \"13c3acff-3931-4204-96e2-0d7cf6559153\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-846dt"
Sep 14 02:50:53.629960 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.629048    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d5d26b1b-8e73-4ff3-b939-c12334ddb952-host\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.629960 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.629075    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/38662066-a2a4-4e2e-8aef-f5478c1205a6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6bfjv\" (UID: \"38662066-a2a4-4e2e-8aef-f5478c1205a6\") " pod="openshift-multus/multus-additional-cni-plugins-6bfjv"
Sep 14 02:50:53.629960 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.629097    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-host-var-lib-kubelet\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.629960 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.629120    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9dcf46a1-8ecf-4333-8c37-13c9e717bde5-konnectivity-ca\") pod \"konnectivity-agent-r8rwc\" (UID: \"9dcf46a1-8ecf-4333-8c37-13c9e717bde5\") " pod="kube-system/konnectivity-agent-r8rwc"
Sep 14 02:50:53.629960 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.629137    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5d26b1b-8e73-4ff3-b939-c12334ddb952-etc-kubernetes\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.629960 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.629149    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-var-lib-openvswitch\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.630670 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.629173    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-multus-socket-dir-parent\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.630670 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.629197    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.630670 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.629221    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/38662066-a2a4-4e2e-8aef-f5478c1205a6-cnibin\") pod \"multus-additional-cni-plugins-6bfjv\" (UID: \"38662066-a2a4-4e2e-8aef-f5478c1205a6\") " pod="openshift-multus/multus-additional-cni-plugins-6bfjv"
Sep 14 02:50:53.630670 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.629245    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2k4h\" (UniqueName: \"kubernetes.io/projected/38662066-a2a4-4e2e-8aef-f5478c1205a6-kube-api-access-w2k4h\") pod \"multus-additional-cni-plugins-6bfjv\" (UID: \"38662066-a2a4-4e2e-8aef-f5478c1205a6\") " pod="openshift-multus/multus-additional-cni-plugins-6bfjv"
Sep 14 02:50:53.630670 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.629268    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-etc-kubernetes\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.630670 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.629049    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-env-overrides\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.630670 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.629295    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-etc-openvswitch\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.630670 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.629788    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-host-cni-netd\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.630670 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.629836    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-ovnkube-config\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.630670 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.629870    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxwt9\" (UniqueName: \"kubernetes.io/projected/d5d26b1b-8e73-4ff3-b939-c12334ddb952-kube-api-access-vxwt9\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.630670 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.629903    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-multus-cni-dir\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.630670 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.629936    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/38662066-a2a4-4e2e-8aef-f5478c1205a6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6bfjv\" (UID: \"38662066-a2a4-4e2e-8aef-f5478c1205a6\") " pod="openshift-multus/multus-additional-cni-plugins-6bfjv"
Sep 14 02:50:53.630670 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.629969    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-host-slash\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.630670 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.629994    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d5d26b1b-8e73-4ff3-b939-c12334ddb952-etc-sysctl-d\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.630670 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.630022    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d5d26b1b-8e73-4ff3-b939-c12334ddb952-run\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.630670 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.630056    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d5d26b1b-8e73-4ff3-b939-c12334ddb952-etc-tuned\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.630670 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.630590    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d5d26b1b-8e73-4ff3-b939-c12334ddb952-host\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.631183 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.630782    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-host-cni-netd\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.631260 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.631240    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d5d26b1b-8e73-4ff3-b939-c12334ddb952-tmp\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.631384 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.631365    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-ovnkube-config\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.631531 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.631514    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.631706 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.631687    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-multus-cni-dir\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.631864 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.631847    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/38662066-a2a4-4e2e-8aef-f5478c1205a6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6bfjv\" (UID: \"38662066-a2a4-4e2e-8aef-f5478c1205a6\") " pod="openshift-multus/multus-additional-cni-plugins-6bfjv"
Sep 14 02:50:53.631904 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.631860    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/38662066-a2a4-4e2e-8aef-f5478c1205a6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6bfjv\" (UID: \"38662066-a2a4-4e2e-8aef-f5478c1205a6\") " pod="openshift-multus/multus-additional-cni-plugins-6bfjv"
Sep 14 02:50:53.631949 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.631914    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-host-var-lib-kubelet\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.631985 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.631971    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d5d26b1b-8e73-4ff3-b939-c12334ddb952-etc-sysctl-d\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.632034 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.632016    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-host-slash\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.632092 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.632078    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d5d26b1b-8e73-4ff3-b939-c12334ddb952-run\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.632148 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.632133    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-etc-kubernetes\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.632196 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.632182    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/38662066-a2a4-4e2e-8aef-f5478c1205a6-cnibin\") pod \"multus-additional-cni-plugins-6bfjv\" (UID: \"38662066-a2a4-4e2e-8aef-f5478c1205a6\") " pod="openshift-multus/multus-additional-cni-plugins-6bfjv"
Sep 14 02:50:53.632387 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.632365    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d5d26b1b-8e73-4ff3-b939-c12334ddb952-etc-tuned\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.632476 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.632452    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9dcf46a1-8ecf-4333-8c37-13c9e717bde5-konnectivity-ca\") pod \"konnectivity-agent-r8rwc\" (UID: \"9dcf46a1-8ecf-4333-8c37-13c9e717bde5\") " pod="kube-system/konnectivity-agent-r8rwc"
Sep 14 02:50:53.632538 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.632473    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-etc-openvswitch\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.632538 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.632455    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-var-lib-openvswitch\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.632538 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.632530    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-multus-socket-dir-parent\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.632701 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.632609    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-host-var-lib-cni-bin\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.632701 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.632655    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/13c3acff-3931-4204-96e2-0d7cf6559153-registration-dir\") pod \"aws-ebs-csi-driver-node-846dt\" (UID: \"13c3acff-3931-4204-96e2-0d7cf6559153\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-846dt"
Sep 14 02:50:53.632805 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.632707    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-systemd-units\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.632805 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.632741    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-ovn-node-metrics-cert\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.632805 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.632775    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e448e444-53b3-4c07-8679-1c8c8960acff-cni-binary-copy\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.632805 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.632802    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-host-run-netns\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.633000 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.632834    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-hostroot\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.633000 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.632867    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e448e444-53b3-4c07-8679-1c8c8960acff-multus-daemon-config\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.633000 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.632899    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-host-run-multus-certs\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.633000 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.632934    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7p6x\" (UniqueName: \"kubernetes.io/projected/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-kube-api-access-p7p6x\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.633000 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.632967    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-system-cni-dir\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.633238 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.633000    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-host-var-lib-cni-multus\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.633238 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.633036    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-host-run-netns\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.633238 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.633060    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d5d26b1b-8e73-4ff3-b939-c12334ddb952-etc-systemd\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.633238 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.633078    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/38662066-a2a4-4e2e-8aef-f5478c1205a6-cni-binary-copy\") pod \"multus-additional-cni-plugins-6bfjv\" (UID: \"38662066-a2a4-4e2e-8aef-f5478c1205a6\") " pod="openshift-multus/multus-additional-cni-plugins-6bfjv"
Sep 14 02:50:53.633238 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.633097    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/13c3acff-3931-4204-96e2-0d7cf6559153-etc-selinux\") pod \"aws-ebs-csi-driver-node-846dt\" (UID: \"13c3acff-3931-4204-96e2-0d7cf6559153\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-846dt"
Sep 14 02:50:53.633238 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.633116    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/13c3acff-3931-4204-96e2-0d7cf6559153-kubelet-dir\") pod \"aws-ebs-csi-driver-node-846dt\" (UID: \"13c3acff-3931-4204-96e2-0d7cf6559153\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-846dt"
Sep 14 02:50:53.633238 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.633135    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9dcf46a1-8ecf-4333-8c37-13c9e717bde5-agent-certs\") pod \"konnectivity-agent-r8rwc\" (UID: \"9dcf46a1-8ecf-4333-8c37-13c9e717bde5\") " pod="kube-system/konnectivity-agent-r8rwc"
Sep 14 02:50:53.633238 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.633172    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-cnibin\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.633495 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.633244    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-cnibin\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.633495 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.633456    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/13c3acff-3931-4204-96e2-0d7cf6559153-registration-dir\") pod \"aws-ebs-csi-driver-node-846dt\" (UID: \"13c3acff-3931-4204-96e2-0d7cf6559153\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-846dt"
Sep 14 02:50:53.633559 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.633525    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-host-var-lib-cni-bin\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.633594 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.633578    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-host-var-lib-cni-multus\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.633649 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.633623    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-host-run-netns\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.633725 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.633672    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-hostroot\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.633888 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.633862    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e448e444-53b3-4c07-8679-1c8c8960acff-cni-binary-copy\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.633985 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.633963    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-host-run-netns\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.634070 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.634052    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d5d26b1b-8e73-4ff3-b939-c12334ddb952-etc-systemd\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.634158 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.634135    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e448e444-53b3-4c07-8679-1c8c8960acff-multus-daemon-config\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.634239 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.634210    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-host-run-multus-certs\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.634545 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.634526    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e448e444-53b3-4c07-8679-1c8c8960acff-system-cni-dir\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.634627 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.634567    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/38662066-a2a4-4e2e-8aef-f5478c1205a6-cni-binary-copy\") pod \"multus-additional-cni-plugins-6bfjv\" (UID: \"38662066-a2a4-4e2e-8aef-f5478c1205a6\") " pod="openshift-multus/multus-additional-cni-plugins-6bfjv"
Sep 14 02:50:53.634627 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.634610    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/13c3acff-3931-4204-96e2-0d7cf6559153-kubelet-dir\") pod \"aws-ebs-csi-driver-node-846dt\" (UID: \"13c3acff-3931-4204-96e2-0d7cf6559153\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-846dt"
Sep 14 02:50:53.634731 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.634709    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/13c3acff-3931-4204-96e2-0d7cf6559153-etc-selinux\") pod \"aws-ebs-csi-driver-node-846dt\" (UID: \"13c3acff-3931-4204-96e2-0d7cf6559153\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-846dt"
Sep 14 02:50:53.634794 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.634757    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-systemd-units\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.636042 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.636017    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-ovn-node-metrics-cert\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.637192 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.637170    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9dcf46a1-8ecf-4333-8c37-13c9e717bde5-agent-certs\") pod \"konnectivity-agent-r8rwc\" (UID: \"9dcf46a1-8ecf-4333-8c37-13c9e717bde5\") " pod="kube-system/konnectivity-agent-r8rwc"
Sep 14 02:50:53.645607 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.645502    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78m46\" (UniqueName: \"kubernetes.io/projected/13c3acff-3931-4204-96e2-0d7cf6559153-kube-api-access-78m46\") pod \"aws-ebs-csi-driver-node-846dt\" (UID: \"13c3acff-3931-4204-96e2-0d7cf6559153\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-846dt"
Sep 14 02:50:53.645685 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.645600    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxwt9\" (UniqueName: \"kubernetes.io/projected/d5d26b1b-8e73-4ff3-b939-c12334ddb952-kube-api-access-vxwt9\") pod \"tuned-q6xjk\" (UID: \"d5d26b1b-8e73-4ff3-b939-c12334ddb952\") " pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.646131 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.646109    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msbhk\" (UniqueName: \"kubernetes.io/projected/e448e444-53b3-4c07-8679-1c8c8960acff-kube-api-access-msbhk\") pod \"multus-qkcvg\" (UID: \"e448e444-53b3-4c07-8679-1c8c8960acff\") " pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:53.647891 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.647847    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2k4h\" (UniqueName: \"kubernetes.io/projected/38662066-a2a4-4e2e-8aef-f5478c1205a6-kube-api-access-w2k4h\") pod \"multus-additional-cni-plugins-6bfjv\" (UID: \"38662066-a2a4-4e2e-8aef-f5478c1205a6\") " pod="openshift-multus/multus-additional-cni-plugins-6bfjv"
Sep 14 02:50:53.649188 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.649169    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7p6x\" (UniqueName: \"kubernetes.io/projected/4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e-kube-api-access-p7p6x\") pod \"ovnkube-node-xd5pl\" (UID: \"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.725217 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.725181    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-szm7r"
Sep 14 02:50:53.741020 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.740993    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-w8fds"
Sep 14 02:50:53.755909 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.755890    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-p4r72"
Sep 14 02:50:53.760584 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.760565    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-846dt"
Sep 14 02:50:53.770453 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.770431    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:50:53.778388 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.778368    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-q6xjk"
Sep 14 02:50:53.786266 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.786249    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-r8rwc"
Sep 14 02:50:53.794502 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.794481    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6bfjv"
Sep 14 02:50:53.801152 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:53.801131    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qkcvg"
Sep 14 02:50:54.036918 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:54.036846    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e746671e-19d5-48e7-8b3b-a11462954481-metrics-certs\") pod \"network-metrics-daemon-n7hgz\" (UID: \"e746671e-19d5-48e7-8b3b-a11462954481\") " pod="openshift-multus/network-metrics-daemon-n7hgz"
Sep 14 02:50:54.036918 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:54.036877    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/003c37b0-6bdd-4b28-9a26-fa2a4ed813ec-original-pull-secret\") pod \"global-pull-secret-syncer-hckjv\" (UID: \"003c37b0-6bdd-4b28-9a26-fa2a4ed813ec\") " pod="kube-system/global-pull-secret-syncer-hckjv"
Sep 14 02:50:54.037128 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:54.036992    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 02:50:54.037128 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:54.037044    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/003c37b0-6bdd-4b28-9a26-fa2a4ed813ec-original-pull-secret podName:003c37b0-6bdd-4b28-9a26-fa2a4ed813ec nodeName:}" failed. No retries permitted until 2026-09-14 02:50:55.037030577 +0000 UTC m=+4.015008585 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/003c37b0-6bdd-4b28-9a26-fa2a4ed813ec-original-pull-secret") pod "global-pull-secret-syncer-hckjv" (UID: "003c37b0-6bdd-4b28-9a26-fa2a4ed813ec") : object "kube-system"/"original-pull-secret" not registered
Sep 14 02:50:54.037128 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:54.036996    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:50:54.037270 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:54.037137    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e746671e-19d5-48e7-8b3b-a11462954481-metrics-certs podName:e746671e-19d5-48e7-8b3b-a11462954481 nodeName:}" failed. No retries permitted until 2026-09-14 02:50:55.037117556 +0000 UTC m=+4.015095592 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e746671e-19d5-48e7-8b3b-a11462954481-metrics-certs") pod "network-metrics-daemon-n7hgz" (UID: "e746671e-19d5-48e7-8b3b-a11462954481") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:50:54.137734 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:54.137704    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rkhd\" (UniqueName: \"kubernetes.io/projected/615f196a-4cb6-42a8-9e5f-fdea4a8672d6-kube-api-access-4rkhd\") pod \"network-check-target-n4kdw\" (UID: \"615f196a-4cb6-42a8-9e5f-fdea4a8672d6\") " pod="openshift-network-diagnostics/network-check-target-n4kdw"
Sep 14 02:50:54.137880 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:54.137860    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 02:50:54.137880 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:54.137879    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 02:50:54.137954 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:54.137890    2577 projected.go:196] Error preparing data for projected volume kube-api-access-4rkhd for pod openshift-network-diagnostics/network-check-target-n4kdw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:50:54.137954 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:54.137940    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/615f196a-4cb6-42a8-9e5f-fdea4a8672d6-kube-api-access-4rkhd podName:615f196a-4cb6-42a8-9e5f-fdea4a8672d6 nodeName:}" failed. No retries permitted until 2026-09-14 02:50:55.13792318 +0000 UTC m=+4.115901192 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4rkhd" (UniqueName: "kubernetes.io/projected/615f196a-4cb6-42a8-9e5f-fdea4a8672d6-kube-api-access-4rkhd") pod "network-check-target-n4kdw" (UID: "615f196a-4cb6-42a8-9e5f-fdea4a8672d6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:50:54.502553 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:54.502478    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-13 02:45:54 +0000 UTC" deadline="2028-06-20 07:30:49.157590324 +0000 UTC"
Sep 14 02:50:54.502553 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:54.502506    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15484h39m54.655087788s"
Sep 14 02:50:54.611123 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:54.611090    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4kdw"
Sep 14 02:50:54.611248 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:54.611191    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n4kdw" podUID="615f196a-4cb6-42a8-9e5f-fdea4a8672d6"
Sep 14 02:50:54.824350 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:54.824310    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf1c701f_81cb_47aa_85a1_439d4f260cea.slice/crio-ed966b5a9724f0592b706435d47cddd175f6c9866989d094d70587c092675a5b WatchSource:0}: Error finding container ed966b5a9724f0592b706435d47cddd175f6c9866989d094d70587c092675a5b: Status 404 returned error can't find the container with id ed966b5a9724f0592b706435d47cddd175f6c9866989d094d70587c092675a5b
Sep 14 02:50:54.825440 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:54.825416    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38662066_a2a4_4e2e_8aef_f5478c1205a6.slice/crio-f1178c621caa7790c35b97e6fd2d72cd574f1515ba1c5844ccd3b25058e97b2f WatchSource:0}: Error finding container f1178c621caa7790c35b97e6fd2d72cd574f1515ba1c5844ccd3b25058e97b2f: Status 404 returned error can't find the container with id f1178c621caa7790c35b97e6fd2d72cd574f1515ba1c5844ccd3b25058e97b2f
Sep 14 02:50:54.827261 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:54.826929    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79878e2e_9231_4688_b888_dc8db6184780.slice/crio-52f631125bc5837aceaa2e244a92da5298076031ba21da8a339ba62b8ea4f6ae WatchSource:0}: Error finding container 52f631125bc5837aceaa2e244a92da5298076031ba21da8a339ba62b8ea4f6ae: Status 404 returned error can't find the container with id 52f631125bc5837aceaa2e244a92da5298076031ba21da8a339ba62b8ea4f6ae
Sep 14 02:50:54.829943 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:54.829901    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d1ef897_6ce7_429d_871c_104e72bc90da.slice/crio-216fd4afa579bae370aebfca51fcefb6565b8a053655a61c9e9e6a874b173977 WatchSource:0}: Error finding container 216fd4afa579bae370aebfca51fcefb6565b8a053655a61c9e9e6a874b173977: Status 404 returned error can't find the container with id 216fd4afa579bae370aebfca51fcefb6565b8a053655a61c9e9e6a874b173977
Sep 14 02:50:54.831337 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:54.831312    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4848f05c_1f6e_4c0a_8f3a_6eac6f5e814e.slice/crio-a002a2f4544664d4438560125ab781946a910d4742652661bb46225de94bcdc7 WatchSource:0}: Error finding container a002a2f4544664d4438560125ab781946a910d4742652661bb46225de94bcdc7: Status 404 returned error can't find the container with id a002a2f4544664d4438560125ab781946a910d4742652661bb46225de94bcdc7
Sep 14 02:50:54.833251 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:54.833089    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode448e444_53b3_4c07_8679_1c8c8960acff.slice/crio-440a33e094395502099b02315814e0ae9efb7eb011708f3e33756f3c6b63ca7f WatchSource:0}: Error finding container 440a33e094395502099b02315814e0ae9efb7eb011708f3e33756f3c6b63ca7f: Status 404 returned error can't find the container with id 440a33e094395502099b02315814e0ae9efb7eb011708f3e33756f3c6b63ca7f
Sep 14 02:50:54.834327 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:54.834080    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dcf46a1_8ecf_4333_8c37_13c9e717bde5.slice/crio-fe5c96da0f96fc7fbe674a24e8a057d99365bd6cc33fd882e00250ff2be785c0 WatchSource:0}: Error finding container fe5c96da0f96fc7fbe674a24e8a057d99365bd6cc33fd882e00250ff2be785c0: Status 404 returned error can't find the container with id fe5c96da0f96fc7fbe674a24e8a057d99365bd6cc33fd882e00250ff2be785c0
Sep 14 02:50:54.834803 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:54.834782    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5d26b1b_8e73_4ff3_b939_c12334ddb952.slice/crio-ffd5c042b1e608bd8e0d98b247dfab8be16188dc81ab131d139f1a3786529eef WatchSource:0}: Error finding container ffd5c042b1e608bd8e0d98b247dfab8be16188dc81ab131d139f1a3786529eef: Status 404 returned error can't find the container with id ffd5c042b1e608bd8e0d98b247dfab8be16188dc81ab131d139f1a3786529eef
Sep 14 02:50:54.836535 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:50:54.836507    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13c3acff_3931_4204_96e2_0d7cf6559153.slice/crio-84ef214f15a83eeab77a7103ee0118a950cfa4d9e33567d5fec5332e9d9137c5 WatchSource:0}: Error finding container 84ef214f15a83eeab77a7103ee0118a950cfa4d9e33567d5fec5332e9d9137c5: Status 404 returned error can't find the container with id 84ef214f15a83eeab77a7103ee0118a950cfa4d9e33567d5fec5332e9d9137c5
Sep 14 02:50:55.045134 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:55.045107    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e746671e-19d5-48e7-8b3b-a11462954481-metrics-certs\") pod \"network-metrics-daemon-n7hgz\" (UID: \"e746671e-19d5-48e7-8b3b-a11462954481\") " pod="openshift-multus/network-metrics-daemon-n7hgz"
Sep 14 02:50:55.045234 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:55.045141    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/003c37b0-6bdd-4b28-9a26-fa2a4ed813ec-original-pull-secret\") pod \"global-pull-secret-syncer-hckjv\" (UID: \"003c37b0-6bdd-4b28-9a26-fa2a4ed813ec\") " pod="kube-system/global-pull-secret-syncer-hckjv"
Sep 14 02:50:55.045294 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:55.045260    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:50:55.045337 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:55.045315    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 02:50:55.045402 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:55.045341    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e746671e-19d5-48e7-8b3b-a11462954481-metrics-certs podName:e746671e-19d5-48e7-8b3b-a11462954481 nodeName:}" failed. No retries permitted until 2026-09-14 02:50:57.045326867 +0000 UTC m=+6.023304900 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e746671e-19d5-48e7-8b3b-a11462954481-metrics-certs") pod "network-metrics-daemon-n7hgz" (UID: "e746671e-19d5-48e7-8b3b-a11462954481") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:50:55.045402 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:55.045361    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/003c37b0-6bdd-4b28-9a26-fa2a4ed813ec-original-pull-secret podName:003c37b0-6bdd-4b28-9a26-fa2a4ed813ec nodeName:}" failed. No retries permitted until 2026-09-14 02:50:57.045350428 +0000 UTC m=+6.023328437 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/003c37b0-6bdd-4b28-9a26-fa2a4ed813ec-original-pull-secret") pod "global-pull-secret-syncer-hckjv" (UID: "003c37b0-6bdd-4b28-9a26-fa2a4ed813ec") : object "kube-system"/"original-pull-secret" not registered
Sep 14 02:50:55.146350 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:55.146318    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rkhd\" (UniqueName: \"kubernetes.io/projected/615f196a-4cb6-42a8-9e5f-fdea4a8672d6-kube-api-access-4rkhd\") pod \"network-check-target-n4kdw\" (UID: \"615f196a-4cb6-42a8-9e5f-fdea4a8672d6\") " pod="openshift-network-diagnostics/network-check-target-n4kdw"
Sep 14 02:50:55.146501 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:55.146482    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 02:50:55.146545 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:55.146506    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 02:50:55.146545 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:55.146516    2577 projected.go:196] Error preparing data for projected volume kube-api-access-4rkhd for pod openshift-network-diagnostics/network-check-target-n4kdw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:50:55.146638 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:55.146574    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/615f196a-4cb6-42a8-9e5f-fdea4a8672d6-kube-api-access-4rkhd podName:615f196a-4cb6-42a8-9e5f-fdea4a8672d6 nodeName:}" failed. No retries permitted until 2026-09-14 02:50:57.146548597 +0000 UTC m=+6.124526623 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4rkhd" (UniqueName: "kubernetes.io/projected/615f196a-4cb6-42a8-9e5f-fdea4a8672d6-kube-api-access-4rkhd") pod "network-check-target-n4kdw" (UID: "615f196a-4cb6-42a8-9e5f-fdea4a8672d6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:50:55.503067 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:55.502958    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-13 02:45:54 +0000 UTC" deadline="2028-06-13 05:31:55.873391926 +0000 UTC"
Sep 14 02:50:55.503067 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:55.503027    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15314h41m0.370369016s"
Sep 14 02:50:55.611582 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:55.611340    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7hgz"
Sep 14 02:50:55.611755 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:55.611700    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n7hgz" podUID="e746671e-19d5-48e7-8b3b-a11462954481"
Sep 14 02:50:55.612135 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:55.612114    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hckjv"
Sep 14 02:50:55.612273 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:55.612212    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hckjv" podUID="003c37b0-6bdd-4b28-9a26-fa2a4ed813ec"
Sep 14 02:50:55.640753 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:55.640726    2577 generic.go:356] "Generic (PLEG): container finished" podID="82fb3813d6a127b50e9babbd92b7fbcd" containerID="8ea7aed67be38e64dfbf9326dfb3a5aced6670a6f98e409b3a73ac161e7bccdd" exitCode=0
Sep 14 02:50:55.640857 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:55.640809    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal" event={"ID":"82fb3813d6a127b50e9babbd92b7fbcd","Type":"ContainerDied","Data":"8ea7aed67be38e64dfbf9326dfb3a5aced6670a6f98e409b3a73ac161e7bccdd"}
Sep 14 02:50:55.651139 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:55.651109    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-180.ec2.internal" event={"ID":"2336dac84cecf30d916b12e72e4b2ecc","Type":"ContainerStarted","Data":"69a5b26f6c507913e1a09618bb8e99eecffd1c6ebd95cee188c303096db2a5fe"}
Sep 14 02:50:55.659281 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:55.659230    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-q6xjk" event={"ID":"d5d26b1b-8e73-4ff3-b939-c12334ddb952","Type":"ContainerStarted","Data":"ffd5c042b1e608bd8e0d98b247dfab8be16188dc81ab131d139f1a3786529eef"}
Sep 14 02:50:55.674547 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:55.674521    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6bfjv" event={"ID":"38662066-a2a4-4e2e-8aef-f5478c1205a6","Type":"ContainerStarted","Data":"f1178c621caa7790c35b97e6fd2d72cd574f1515ba1c5844ccd3b25058e97b2f"}
Sep 14 02:50:55.679936 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:55.679800    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-w8fds" event={"ID":"af1c701f-81cb-47aa-85a1-439d4f260cea","Type":"ContainerStarted","Data":"ed966b5a9724f0592b706435d47cddd175f6c9866989d094d70587c092675a5b"}
Sep 14 02:50:55.714690 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:55.714658    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-846dt" event={"ID":"13c3acff-3931-4204-96e2-0d7cf6559153","Type":"ContainerStarted","Data":"84ef214f15a83eeab77a7103ee0118a950cfa4d9e33567d5fec5332e9d9137c5"}
Sep 14 02:50:55.728461 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:55.728414    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-r8rwc" event={"ID":"9dcf46a1-8ecf-4333-8c37-13c9e717bde5","Type":"ContainerStarted","Data":"fe5c96da0f96fc7fbe674a24e8a057d99365bd6cc33fd882e00250ff2be785c0"}
Sep 14 02:50:55.736542 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:55.736511    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qkcvg" event={"ID":"e448e444-53b3-4c07-8679-1c8c8960acff","Type":"ContainerStarted","Data":"440a33e094395502099b02315814e0ae9efb7eb011708f3e33756f3c6b63ca7f"}
Sep 14 02:50:55.741993 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:55.741966    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl" event={"ID":"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e","Type":"ContainerStarted","Data":"a002a2f4544664d4438560125ab781946a910d4742652661bb46225de94bcdc7"}
Sep 14 02:50:55.761278 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:55.761164    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-p4r72" event={"ID":"9d1ef897-6ce7-429d-871c-104e72bc90da","Type":"ContainerStarted","Data":"216fd4afa579bae370aebfca51fcefb6565b8a053655a61c9e9e6a874b173977"}
Sep 14 02:50:55.770453 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:55.770390    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-szm7r" event={"ID":"79878e2e-9231-4688-b888-dc8db6184780","Type":"ContainerStarted","Data":"52f631125bc5837aceaa2e244a92da5298076031ba21da8a339ba62b8ea4f6ae"}
Sep 14 02:50:56.611383 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:56.611353    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4kdw"
Sep 14 02:50:56.611806 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:56.611488    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n4kdw" podUID="615f196a-4cb6-42a8-9e5f-fdea4a8672d6"
Sep 14 02:50:56.793453 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:56.793367    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal" event={"ID":"82fb3813d6a127b50e9babbd92b7fbcd","Type":"ContainerStarted","Data":"672384500b98587ad7ba23ec5fdd49dc77b3b2f1e0fbc89fbb5c0df8a4a22bc2"}
Sep 14 02:50:56.816664 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:56.816606    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-180.ec2.internal" podStartSLOduration=3.8165842 podStartE2EDuration="3.8165842s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 02:50:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:50:55.699970587 +0000 UTC m=+4.677948621" watchObservedRunningTime="2026-09-14 02:50:56.8165842 +0000 UTC m=+5.794562234"
Sep 14 02:50:57.062731 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:57.062652    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e746671e-19d5-48e7-8b3b-a11462954481-metrics-certs\") pod \"network-metrics-daemon-n7hgz\" (UID: \"e746671e-19d5-48e7-8b3b-a11462954481\") " pod="openshift-multus/network-metrics-daemon-n7hgz"
Sep 14 02:50:57.062731 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:57.062702    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/003c37b0-6bdd-4b28-9a26-fa2a4ed813ec-original-pull-secret\") pod \"global-pull-secret-syncer-hckjv\" (UID: \"003c37b0-6bdd-4b28-9a26-fa2a4ed813ec\") " pod="kube-system/global-pull-secret-syncer-hckjv"
Sep 14 02:50:57.062948 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:57.062847    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 02:50:57.062948 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:57.062907    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/003c37b0-6bdd-4b28-9a26-fa2a4ed813ec-original-pull-secret podName:003c37b0-6bdd-4b28-9a26-fa2a4ed813ec nodeName:}" failed. No retries permitted until 2026-09-14 02:51:01.062889289 +0000 UTC m=+10.040867304 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/003c37b0-6bdd-4b28-9a26-fa2a4ed813ec-original-pull-secret") pod "global-pull-secret-syncer-hckjv" (UID: "003c37b0-6bdd-4b28-9a26-fa2a4ed813ec") : object "kube-system"/"original-pull-secret" not registered
Sep 14 02:50:57.063338 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:57.063317    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:50:57.063462 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:57.063399    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e746671e-19d5-48e7-8b3b-a11462954481-metrics-certs podName:e746671e-19d5-48e7-8b3b-a11462954481 nodeName:}" failed. No retries permitted until 2026-09-14 02:51:01.063383031 +0000 UTC m=+10.041361046 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e746671e-19d5-48e7-8b3b-a11462954481-metrics-certs") pod "network-metrics-daemon-n7hgz" (UID: "e746671e-19d5-48e7-8b3b-a11462954481") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:50:57.163341 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:57.163066    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rkhd\" (UniqueName: \"kubernetes.io/projected/615f196a-4cb6-42a8-9e5f-fdea4a8672d6-kube-api-access-4rkhd\") pod \"network-check-target-n4kdw\" (UID: \"615f196a-4cb6-42a8-9e5f-fdea4a8672d6\") " pod="openshift-network-diagnostics/network-check-target-n4kdw"
Sep 14 02:50:57.163341 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:57.163256    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 02:50:57.163341 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:57.163276    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 02:50:57.163341 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:57.163289    2577 projected.go:196] Error preparing data for projected volume kube-api-access-4rkhd for pod openshift-network-diagnostics/network-check-target-n4kdw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:50:57.164137 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:57.163806    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/615f196a-4cb6-42a8-9e5f-fdea4a8672d6-kube-api-access-4rkhd podName:615f196a-4cb6-42a8-9e5f-fdea4a8672d6 nodeName:}" failed. No retries permitted until 2026-09-14 02:51:01.163783754 +0000 UTC m=+10.141761769 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4rkhd" (UniqueName: "kubernetes.io/projected/615f196a-4cb6-42a8-9e5f-fdea4a8672d6-kube-api-access-4rkhd") pod "network-check-target-n4kdw" (UID: "615f196a-4cb6-42a8-9e5f-fdea4a8672d6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:50:57.626325 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:57.626293    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hckjv"
Sep 14 02:50:57.626842 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:57.626640    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hckjv" podUID="003c37b0-6bdd-4b28-9a26-fa2a4ed813ec"
Sep 14 02:50:57.626919 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:57.626871    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7hgz"
Sep 14 02:50:57.627030 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:57.627009    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n7hgz" podUID="e746671e-19d5-48e7-8b3b-a11462954481"
Sep 14 02:50:58.612628 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:58.612123    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4kdw"
Sep 14 02:50:58.612628 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:58.612244    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n4kdw" podUID="615f196a-4cb6-42a8-9e5f-fdea4a8672d6"
Sep 14 02:50:59.611473 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:59.611438    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7hgz"
Sep 14 02:50:59.611929 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:59.611599    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n7hgz" podUID="e746671e-19d5-48e7-8b3b-a11462954481"
Sep 14 02:50:59.611929 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:50:59.611665    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hckjv"
Sep 14 02:50:59.611929 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:50:59.611739    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hckjv" podUID="003c37b0-6bdd-4b28-9a26-fa2a4ed813ec"
Sep 14 02:51:00.611504 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:00.611466    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4kdw"
Sep 14 02:51:00.611918 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:00.611595    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n4kdw" podUID="615f196a-4cb6-42a8-9e5f-fdea4a8672d6"
Sep 14 02:51:01.099185 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:01.099147    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e746671e-19d5-48e7-8b3b-a11462954481-metrics-certs\") pod \"network-metrics-daemon-n7hgz\" (UID: \"e746671e-19d5-48e7-8b3b-a11462954481\") " pod="openshift-multus/network-metrics-daemon-n7hgz"
Sep 14 02:51:01.099416 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:01.099198    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/003c37b0-6bdd-4b28-9a26-fa2a4ed813ec-original-pull-secret\") pod \"global-pull-secret-syncer-hckjv\" (UID: \"003c37b0-6bdd-4b28-9a26-fa2a4ed813ec\") " pod="kube-system/global-pull-secret-syncer-hckjv"
Sep 14 02:51:01.099416 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:01.099353    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 02:51:01.099545 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:01.099418    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/003c37b0-6bdd-4b28-9a26-fa2a4ed813ec-original-pull-secret podName:003c37b0-6bdd-4b28-9a26-fa2a4ed813ec nodeName:}" failed. No retries permitted until 2026-09-14 02:51:09.099398836 +0000 UTC m=+18.077376850 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/003c37b0-6bdd-4b28-9a26-fa2a4ed813ec-original-pull-secret") pod "global-pull-secret-syncer-hckjv" (UID: "003c37b0-6bdd-4b28-9a26-fa2a4ed813ec") : object "kube-system"/"original-pull-secret" not registered
Sep 14 02:51:01.099883 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:01.099807    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:51:01.099883 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:01.099856    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e746671e-19d5-48e7-8b3b-a11462954481-metrics-certs podName:e746671e-19d5-48e7-8b3b-a11462954481 nodeName:}" failed. No retries permitted until 2026-09-14 02:51:09.099840942 +0000 UTC m=+18.077818955 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e746671e-19d5-48e7-8b3b-a11462954481-metrics-certs") pod "network-metrics-daemon-n7hgz" (UID: "e746671e-19d5-48e7-8b3b-a11462954481") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:51:01.199877 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:01.199840    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rkhd\" (UniqueName: \"kubernetes.io/projected/615f196a-4cb6-42a8-9e5f-fdea4a8672d6-kube-api-access-4rkhd\") pod \"network-check-target-n4kdw\" (UID: \"615f196a-4cb6-42a8-9e5f-fdea4a8672d6\") " pod="openshift-network-diagnostics/network-check-target-n4kdw"
Sep 14 02:51:01.200049 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:01.200027    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 02:51:01.200100 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:01.200058    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 02:51:01.200100 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:01.200073    2577 projected.go:196] Error preparing data for projected volume kube-api-access-4rkhd for pod openshift-network-diagnostics/network-check-target-n4kdw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:51:01.200198 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:01.200141    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/615f196a-4cb6-42a8-9e5f-fdea4a8672d6-kube-api-access-4rkhd podName:615f196a-4cb6-42a8-9e5f-fdea4a8672d6 nodeName:}" failed. No retries permitted until 2026-09-14 02:51:09.200121562 +0000 UTC m=+18.178099574 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4rkhd" (UniqueName: "kubernetes.io/projected/615f196a-4cb6-42a8-9e5f-fdea4a8672d6-kube-api-access-4rkhd") pod "network-check-target-n4kdw" (UID: "615f196a-4cb6-42a8-9e5f-fdea4a8672d6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:51:01.617019 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:01.616892    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7hgz"
Sep 14 02:51:01.617503 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:01.617076    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hckjv"
Sep 14 02:51:01.617503 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:01.617114    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n7hgz" podUID="e746671e-19d5-48e7-8b3b-a11462954481"
Sep 14 02:51:01.617503 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:01.617225    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hckjv" podUID="003c37b0-6bdd-4b28-9a26-fa2a4ed813ec"
Sep 14 02:51:02.611419 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:02.611380    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4kdw"
Sep 14 02:51:02.611606 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:02.611509    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n4kdw" podUID="615f196a-4cb6-42a8-9e5f-fdea4a8672d6"
Sep 14 02:51:03.612012 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:03.611605    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7hgz"
Sep 14 02:51:03.612012 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:03.611758    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n7hgz" podUID="e746671e-19d5-48e7-8b3b-a11462954481"
Sep 14 02:51:03.612012 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:03.611837    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hckjv"
Sep 14 02:51:03.612012 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:03.611954    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hckjv" podUID="003c37b0-6bdd-4b28-9a26-fa2a4ed813ec"
Sep 14 02:51:04.611224 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:04.611196    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4kdw"
Sep 14 02:51:04.611398 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:04.611303    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n4kdw" podUID="615f196a-4cb6-42a8-9e5f-fdea4a8672d6"
Sep 14 02:51:05.611437 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:05.611405    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hckjv"
Sep 14 02:51:05.611781 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:05.611526    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hckjv" podUID="003c37b0-6bdd-4b28-9a26-fa2a4ed813ec"
Sep 14 02:51:05.611781 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:05.611563    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7hgz"
Sep 14 02:51:05.611781 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:05.611653    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n7hgz" podUID="e746671e-19d5-48e7-8b3b-a11462954481"
Sep 14 02:51:06.611908 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:06.611872    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4kdw"
Sep 14 02:51:06.612355 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:06.611997    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n4kdw" podUID="615f196a-4cb6-42a8-9e5f-fdea4a8672d6"
Sep 14 02:51:07.614731 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:07.614703    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hckjv"
Sep 14 02:51:07.615163 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:07.614819    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hckjv" podUID="003c37b0-6bdd-4b28-9a26-fa2a4ed813ec"
Sep 14 02:51:07.615163 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:07.614873    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7hgz"
Sep 14 02:51:07.615163 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:07.614989    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n7hgz" podUID="e746671e-19d5-48e7-8b3b-a11462954481"
Sep 14 02:51:08.611493 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:08.611462    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4kdw"
Sep 14 02:51:08.611708 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:08.611586    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n4kdw" podUID="615f196a-4cb6-42a8-9e5f-fdea4a8672d6"
Sep 14 02:51:09.164687 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:09.164630    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e746671e-19d5-48e7-8b3b-a11462954481-metrics-certs\") pod \"network-metrics-daemon-n7hgz\" (UID: \"e746671e-19d5-48e7-8b3b-a11462954481\") " pod="openshift-multus/network-metrics-daemon-n7hgz"
Sep 14 02:51:09.164687 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:09.164673    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/003c37b0-6bdd-4b28-9a26-fa2a4ed813ec-original-pull-secret\") pod \"global-pull-secret-syncer-hckjv\" (UID: \"003c37b0-6bdd-4b28-9a26-fa2a4ed813ec\") " pod="kube-system/global-pull-secret-syncer-hckjv"
Sep 14 02:51:09.165195 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:09.164783    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:51:09.165195 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:09.164839    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e746671e-19d5-48e7-8b3b-a11462954481-metrics-certs podName:e746671e-19d5-48e7-8b3b-a11462954481 nodeName:}" failed. No retries permitted until 2026-09-14 02:51:25.164821365 +0000 UTC m=+34.142799372 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e746671e-19d5-48e7-8b3b-a11462954481-metrics-certs") pod "network-metrics-daemon-n7hgz" (UID: "e746671e-19d5-48e7-8b3b-a11462954481") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:51:09.165195 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:09.164787    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 02:51:09.165195 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:09.164929    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/003c37b0-6bdd-4b28-9a26-fa2a4ed813ec-original-pull-secret podName:003c37b0-6bdd-4b28-9a26-fa2a4ed813ec nodeName:}" failed. No retries permitted until 2026-09-14 02:51:25.164908766 +0000 UTC m=+34.142886787 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/003c37b0-6bdd-4b28-9a26-fa2a4ed813ec-original-pull-secret") pod "global-pull-secret-syncer-hckjv" (UID: "003c37b0-6bdd-4b28-9a26-fa2a4ed813ec") : object "kube-system"/"original-pull-secret" not registered
Sep 14 02:51:09.265385 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:09.265354    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rkhd\" (UniqueName: \"kubernetes.io/projected/615f196a-4cb6-42a8-9e5f-fdea4a8672d6-kube-api-access-4rkhd\") pod \"network-check-target-n4kdw\" (UID: \"615f196a-4cb6-42a8-9e5f-fdea4a8672d6\") " pod="openshift-network-diagnostics/network-check-target-n4kdw"
Sep 14 02:51:09.265551 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:09.265487    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 02:51:09.265551 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:09.265507    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 02:51:09.265551 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:09.265519    2577 projected.go:196] Error preparing data for projected volume kube-api-access-4rkhd for pod openshift-network-diagnostics/network-check-target-n4kdw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:51:09.265664 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:09.265562    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/615f196a-4cb6-42a8-9e5f-fdea4a8672d6-kube-api-access-4rkhd podName:615f196a-4cb6-42a8-9e5f-fdea4a8672d6 nodeName:}" failed. No retries permitted until 2026-09-14 02:51:25.265548424 +0000 UTC m=+34.243526444 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4rkhd" (UniqueName: "kubernetes.io/projected/615f196a-4cb6-42a8-9e5f-fdea4a8672d6-kube-api-access-4rkhd") pod "network-check-target-n4kdw" (UID: "615f196a-4cb6-42a8-9e5f-fdea4a8672d6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:51:09.611537 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:09.611505    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hckjv"
Sep 14 02:51:09.611700 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:09.611632    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hckjv" podUID="003c37b0-6bdd-4b28-9a26-fa2a4ed813ec"
Sep 14 02:51:09.611766 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:09.611710    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7hgz"
Sep 14 02:51:09.611854 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:09.611831    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n7hgz" podUID="e746671e-19d5-48e7-8b3b-a11462954481"
Sep 14 02:51:10.611449 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:10.611417    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4kdw"
Sep 14 02:51:10.611848 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:10.611512    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n4kdw" podUID="615f196a-4cb6-42a8-9e5f-fdea4a8672d6"
Sep 14 02:51:11.611873 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:11.611842    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hckjv"
Sep 14 02:51:11.612282 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:11.611943    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hckjv" podUID="003c37b0-6bdd-4b28-9a26-fa2a4ed813ec"
Sep 14 02:51:11.612282 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:11.611999    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7hgz"
Sep 14 02:51:11.612282 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:11.612097    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n7hgz" podUID="e746671e-19d5-48e7-8b3b-a11462954481"
Sep 14 02:51:12.611716 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:12.611681    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4kdw"
Sep 14 02:51:12.611905 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:12.611804    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n4kdw" podUID="615f196a-4cb6-42a8-9e5f-fdea4a8672d6"
Sep 14 02:51:13.612096 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:13.611922    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hckjv"
Sep 14 02:51:13.612817 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:13.611928    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7hgz"
Sep 14 02:51:13.612817 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:13.612210    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hckjv" podUID="003c37b0-6bdd-4b28-9a26-fa2a4ed813ec"
Sep 14 02:51:13.612817 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:13.612320    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n7hgz" podUID="e746671e-19d5-48e7-8b3b-a11462954481"
Sep 14 02:51:13.847656 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:13.847458    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-846dt" event={"ID":"13c3acff-3931-4204-96e2-0d7cf6559153","Type":"ContainerStarted","Data":"ec720b90517356a4e5f6579db13751c9f257be28493f30e8928f3838b33f745b"}
Sep 14 02:51:13.849387 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:13.849358    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-r8rwc" event={"ID":"9dcf46a1-8ecf-4333-8c37-13c9e717bde5","Type":"ContainerStarted","Data":"5971be98028755c750cb2947be740bf1a61ac4ddbb212f6622e6ff31264eeec0"}
Sep 14 02:51:13.851015 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:13.850993    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qkcvg" event={"ID":"e448e444-53b3-4c07-8679-1c8c8960acff","Type":"ContainerStarted","Data":"74047328f3bd23222f2ff9c2ddf1c1da6006634bc264774efc8e6a8bdf891e9c"}
Sep 14 02:51:13.854261 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:13.854238    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl" event={"ID":"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e","Type":"ContainerStarted","Data":"26f9e06e3dc74a6b8fa271026b8d610f90b609b36669e35bc1d668f471c64ee8"}
Sep 14 02:51:13.854370 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:13.854315    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl" event={"ID":"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e","Type":"ContainerStarted","Data":"1c1c285d468a885f3e6c6c5ca1bca528f8d2807589c268ece1a5340a7c2b7856"}
Sep 14 02:51:13.854370 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:13.854349    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl" event={"ID":"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e","Type":"ContainerStarted","Data":"0f56e4161a141c95f47c36269ce84d726a120adb6eb2add131992e940b177848"}
Sep 14 02:51:13.854445 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:13.854383    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl" event={"ID":"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e","Type":"ContainerStarted","Data":"199b6f44cdb1e6ae46946ccd9d5a00b40fe478a28df82125ca34fd198ca791e0"}
Sep 14 02:51:13.854445 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:13.854412    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl" event={"ID":"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e","Type":"ContainerStarted","Data":"e514852d4fb28d8188f5c530b0c699e7e959902ad0a46e0fa3bacd389dd4d030"}
Sep 14 02:51:13.854526 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:13.854446    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl" event={"ID":"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e","Type":"ContainerStarted","Data":"0b4772ff9a91edcadf91cba56532e7c70968f32d523229a5bb7fe81639bf12ce"}
Sep 14 02:51:13.855642 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:13.855620    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-szm7r" event={"ID":"79878e2e-9231-4688-b888-dc8db6184780","Type":"ContainerStarted","Data":"b241442a93534e941b645073c3f9222d3e6aabe21609d419394d42e24615fa4c"}
Sep 14 02:51:13.856943 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:13.856923    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-q6xjk" event={"ID":"d5d26b1b-8e73-4ff3-b939-c12334ddb952","Type":"ContainerStarted","Data":"37dc062e724c22a4bd9dd045fe902fc0d8ecf66dc81324ba6963016c3759b7b4"}
Sep 14 02:51:13.858243 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:13.858224    2577 generic.go:356] "Generic (PLEG): container finished" podID="38662066-a2a4-4e2e-8aef-f5478c1205a6" containerID="d2baff5838f9e4b63ebca3fb6389201126ed4c94e456f962e24935a9ad08f93c" exitCode=0
Sep 14 02:51:13.858333 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:13.858291    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6bfjv" event={"ID":"38662066-a2a4-4e2e-8aef-f5478c1205a6","Type":"ContainerDied","Data":"d2baff5838f9e4b63ebca3fb6389201126ed4c94e456f962e24935a9ad08f93c"}
Sep 14 02:51:13.859683 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:13.859664    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-w8fds" event={"ID":"af1c701f-81cb-47aa-85a1-439d4f260cea","Type":"ContainerStarted","Data":"fdf52460a6cde5b89f131f8202684fc7967d9676107e8ede773b2a0a9c8329a6"}
Sep 14 02:51:13.871879 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:13.871837    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-180.ec2.internal" podStartSLOduration=20.87182592 podStartE2EDuration="20.87182592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 02:50:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:50:56.817499245 +0000 UTC m=+5.795477277" watchObservedRunningTime="2026-09-14 02:51:13.87182592 +0000 UTC m=+22.849803949"
Sep 14 02:51:13.901094 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:13.901057    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-q6xjk" podStartSLOduration=3.520668993 podStartE2EDuration="21.901044518s" totalImagesPullingTime="18.380375525s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 02:50:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:51:13.901008261 +0000 UTC m=+22.878986290" watchObservedRunningTime="2026-09-14 02:51:13.901044518 +0000 UTC m=+22.879022550"
Sep 14 02:51:13.904048 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:13.904010    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-r8rwc" podStartSLOduration=7.159668757 podStartE2EDuration="21.90399804s" totalImagesPullingTime="14.744329283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:50:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:51:13.872198309 +0000 UTC m=+22.850176338" watchObservedRunningTime="2026-09-14 02:51:13.90399804 +0000 UTC m=+22.881976069"
Sep 14 02:51:13.928862 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:13.928828    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-qkcvg" podStartSLOduration=3.504314871 podStartE2EDuration="21.928818292s" totalImagesPullingTime="18.424503421s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 02:50:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:51:13.928597277 +0000 UTC m=+22.906575306" watchObservedRunningTime="2026-09-14 02:51:13.928818292 +0000 UTC m=+22.906796320"
Sep 14 02:51:13.987631 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:13.987594    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-szm7r" podStartSLOduration=3.601037912 podStartE2EDuration="21.987583515s" totalImagesPullingTime="18.386545603s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 02:50:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:51:13.959069938 +0000 UTC m=+22.937047967" watchObservedRunningTime="2026-09-14 02:51:13.987583515 +0000 UTC m=+22.965561535"
Sep 14 02:51:14.023648 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:14.023607    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-w8fds" podStartSLOduration=3.634724087 podStartE2EDuration="22.023595686s" totalImagesPullingTime="18.388871599s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 02:50:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:51:13.987724602 +0000 UTC m=+22.965702635" watchObservedRunningTime="2026-09-14 02:51:14.023595686 +0000 UTC m=+23.001573715"
Sep 14 02:51:14.064563 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:14.064539    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-r8rwc"
Sep 14 02:51:14.065118 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:14.065103    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-r8rwc"
Sep 14 02:51:14.611606 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:14.611585    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4kdw"
Sep 14 02:51:14.611728 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:14.611686    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n4kdw" podUID="615f196a-4cb6-42a8-9e5f-fdea4a8672d6"
Sep 14 02:51:14.696152 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:14.696126    2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 14 02:51:14.864236 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:14.864167    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-846dt" event={"ID":"13c3acff-3931-4204-96e2-0d7cf6559153","Type":"ContainerStarted","Data":"0ca0482fc50895d879b124284c335ec4ddbd1be961bb3f0d21b41c7b98842ac5"}
Sep 14 02:51:14.865841 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:14.865812    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-p4r72" event={"ID":"9d1ef897-6ce7-429d-871c-104e72bc90da","Type":"ContainerStarted","Data":"bf08123613a75c7297450fdf4717a658004717a2fb0ec39796980d8dae4bab8c"}
Sep 14 02:51:14.866554 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:14.866504    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-r8rwc"
Sep 14 02:51:14.866741 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:14.866717    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-r8rwc"
Sep 14 02:51:14.887903 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:14.887866    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-p4r72" podStartSLOduration=4.541912688 podStartE2EDuration="22.887854469s" totalImagesPullingTime="18.345941781s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 02:50:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:51:14.887846527 +0000 UTC m=+23.865824569" watchObservedRunningTime="2026-09-14 02:51:14.887854469 +0000 UTC m=+23.865832497"
Sep 14 02:51:15.473778 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:15.473635    2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-14T02:51:14.696142075Z","UUID":"ef545cf4-a04c-4957-aa16-b95c376224a5","Handler":null,"Name":"","Endpoint":""}
Sep 14 02:51:15.475389 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:15.475363    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 14 02:51:15.475535 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:15.475400    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 14 02:51:15.611589 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:15.611558    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7hgz"
Sep 14 02:51:15.611752 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:15.611707    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n7hgz" podUID="e746671e-19d5-48e7-8b3b-a11462954481"
Sep 14 02:51:15.612014 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:15.611992    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hckjv"
Sep 14 02:51:15.612122 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:15.612101    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hckjv" podUID="003c37b0-6bdd-4b28-9a26-fa2a4ed813ec"
Sep 14 02:51:16.611603 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:16.611574    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4kdw"
Sep 14 02:51:16.611989 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:16.611674    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n4kdw" podUID="615f196a-4cb6-42a8-9e5f-fdea4a8672d6"
Sep 14 02:51:16.874301 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:16.874021    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl" event={"ID":"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e","Type":"ContainerStarted","Data":"7ff5240dfffd200e968f9364208c3aa352d1a65c3cac6f1b325265eab067c617"}
Sep 14 02:51:16.876068 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:16.876042    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-846dt" event={"ID":"13c3acff-3931-4204-96e2-0d7cf6559153","Type":"ContainerStarted","Data":"155a0f15f5a2b96bd63e9c172eacb7e56092d7fad70cd34d2c01992fc5962982"}
Sep 14 02:51:16.907656 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:16.907609    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-846dt" podStartSLOduration=4.126958742 podStartE2EDuration="24.907593904s" totalImagesPullingTime="20.780635162s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 02:50:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:51:16.907430842 +0000 UTC m=+25.885408870" watchObservedRunningTime="2026-09-14 02:51:16.907593904 +0000 UTC m=+25.885571936"
Sep 14 02:51:17.614288 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:17.614250    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hckjv"
Sep 14 02:51:17.614741 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:17.614303    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7hgz"
Sep 14 02:51:17.614741 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:17.614380    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hckjv" podUID="003c37b0-6bdd-4b28-9a26-fa2a4ed813ec"
Sep 14 02:51:17.614741 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:17.614515    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n7hgz" podUID="e746671e-19d5-48e7-8b3b-a11462954481"
Sep 14 02:51:18.611746 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:18.611720    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4kdw"
Sep 14 02:51:18.611856 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:18.611823    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n4kdw" podUID="615f196a-4cb6-42a8-9e5f-fdea4a8672d6"
Sep 14 02:51:18.882198 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:18.882020    2577 generic.go:356] "Generic (PLEG): container finished" podID="38662066-a2a4-4e2e-8aef-f5478c1205a6" containerID="eba8f819926e513a0127695657127ee37f27691d821818f642da0021455e12c5" exitCode=0
Sep 14 02:51:18.882909 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:18.882097    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6bfjv" event={"ID":"38662066-a2a4-4e2e-8aef-f5478c1205a6","Type":"ContainerDied","Data":"eba8f819926e513a0127695657127ee37f27691d821818f642da0021455e12c5"}
Sep 14 02:51:18.888959 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:18.888935    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl" event={"ID":"4848f05c-1f6e-4c0a-8f3a-6eac6f5e814e","Type":"ContainerStarted","Data":"4e80d6694b3bb718ba928463b975523a6dba29dcec020d2e3de63f5a7fbe9ab5"}
Sep 14 02:51:18.889239 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:18.889225    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:51:18.902800 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:18.902756    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:51:18.952916 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:18.952877    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl" podStartSLOduration=8.518487695 podStartE2EDuration="26.952864475s" totalImagesPullingTime="18.43437678s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 02:50:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:51:18.950765517 +0000 UTC m=+27.928743545" watchObservedRunningTime="2026-09-14 02:51:18.952864475 +0000 UTC m=+27.930842530"
Sep 14 02:51:19.611070 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:19.611037    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hckjv"
Sep 14 02:51:19.611257 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:19.611146    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hckjv" podUID="003c37b0-6bdd-4b28-9a26-fa2a4ed813ec"
Sep 14 02:51:19.611257 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:19.611193    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7hgz"
Sep 14 02:51:19.611365 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:19.611325    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n7hgz" podUID="e746671e-19d5-48e7-8b3b-a11462954481"
Sep 14 02:51:19.910408 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:19.910381    2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 14 02:51:19.910916 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:19.910875    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:51:19.927874 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:19.927840    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:51:20.364333 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:20.364299    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hckjv"]
Sep 14 02:51:20.364486 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:20.364389    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hckjv"
Sep 14 02:51:20.364534 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:20.364475    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hckjv" podUID="003c37b0-6bdd-4b28-9a26-fa2a4ed813ec"
Sep 14 02:51:20.367578 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:20.367552    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-n4kdw"]
Sep 14 02:51:20.367709 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:20.367665    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4kdw"
Sep 14 02:51:20.367804 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:20.367783    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n4kdw" podUID="615f196a-4cb6-42a8-9e5f-fdea4a8672d6"
Sep 14 02:51:20.368217 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:20.368199    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-n7hgz"]
Sep 14 02:51:20.368310 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:20.368296    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7hgz"
Sep 14 02:51:20.368420 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:20.368399    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n7hgz" podUID="e746671e-19d5-48e7-8b3b-a11462954481"
Sep 14 02:51:20.896162 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:20.896129    2577 generic.go:356] "Generic (PLEG): container finished" podID="38662066-a2a4-4e2e-8aef-f5478c1205a6" containerID="d69fe09d4df32d1ec458976b482d0dc8afd1570e1c4b72ba8cc755b222138eb2" exitCode=0
Sep 14 02:51:20.896345 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:20.896217    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6bfjv" event={"ID":"38662066-a2a4-4e2e-8aef-f5478c1205a6","Type":"ContainerDied","Data":"d69fe09d4df32d1ec458976b482d0dc8afd1570e1c4b72ba8cc755b222138eb2"}
Sep 14 02:51:20.896345 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:20.896314    2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 14 02:51:21.612093 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:21.612069    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hckjv"
Sep 14 02:51:21.612385 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:21.612149    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hckjv" podUID="003c37b0-6bdd-4b28-9a26-fa2a4ed813ec"
Sep 14 02:51:21.901083 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:21.900866    2577 generic.go:356] "Generic (PLEG): container finished" podID="38662066-a2a4-4e2e-8aef-f5478c1205a6" containerID="5ea148d1c94a58c3f463dafaf3a5ff6912e28d6df06131fd616cd51f67d2c50d" exitCode=0
Sep 14 02:51:21.901194 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:21.900939    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6bfjv" event={"ID":"38662066-a2a4-4e2e-8aef-f5478c1205a6","Type":"ContainerDied","Data":"5ea148d1c94a58c3f463dafaf3a5ff6912e28d6df06131fd616cd51f67d2c50d"}
Sep 14 02:51:21.901236 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:21.901224    2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 14 02:51:21.961288 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:21.961257    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:51:22.611629 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:22.611596    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4kdw"
Sep 14 02:51:22.611791 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:22.611643    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7hgz"
Sep 14 02:51:22.611791 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:22.611725    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n4kdw" podUID="615f196a-4cb6-42a8-9e5f-fdea4a8672d6"
Sep 14 02:51:22.611898 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:22.611862    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n7hgz" podUID="e746671e-19d5-48e7-8b3b-a11462954481"
Sep 14 02:51:23.611812 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:23.611779    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hckjv"
Sep 14 02:51:23.612258 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:23.611936    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hckjv" podUID="003c37b0-6bdd-4b28-9a26-fa2a4ed813ec"
Sep 14 02:51:24.611451 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:24.611377    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4kdw"
Sep 14 02:51:24.611621 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:24.611381    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7hgz"
Sep 14 02:51:24.611621 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:24.611496    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n4kdw" podUID="615f196a-4cb6-42a8-9e5f-fdea4a8672d6"
Sep 14 02:51:24.611621 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:24.611587    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n7hgz" podUID="e746671e-19d5-48e7-8b3b-a11462954481"
Sep 14 02:51:25.181923 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:25.181879    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e746671e-19d5-48e7-8b3b-a11462954481-metrics-certs\") pod \"network-metrics-daemon-n7hgz\" (UID: \"e746671e-19d5-48e7-8b3b-a11462954481\") " pod="openshift-multus/network-metrics-daemon-n7hgz"
Sep 14 02:51:25.181923 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:25.181926    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/003c37b0-6bdd-4b28-9a26-fa2a4ed813ec-original-pull-secret\") pod \"global-pull-secret-syncer-hckjv\" (UID: \"003c37b0-6bdd-4b28-9a26-fa2a4ed813ec\") " pod="kube-system/global-pull-secret-syncer-hckjv"
Sep 14 02:51:25.182457 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:25.182036    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 02:51:25.182457 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:25.182041    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:51:25.182457 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:25.182088    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/003c37b0-6bdd-4b28-9a26-fa2a4ed813ec-original-pull-secret podName:003c37b0-6bdd-4b28-9a26-fa2a4ed813ec nodeName:}" failed. No retries permitted until 2026-09-14 02:51:57.18207473 +0000 UTC m=+66.160052738 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/003c37b0-6bdd-4b28-9a26-fa2a4ed813ec-original-pull-secret") pod "global-pull-secret-syncer-hckjv" (UID: "003c37b0-6bdd-4b28-9a26-fa2a4ed813ec") : object "kube-system"/"original-pull-secret" not registered
Sep 14 02:51:25.182457 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:25.182135    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e746671e-19d5-48e7-8b3b-a11462954481-metrics-certs podName:e746671e-19d5-48e7-8b3b-a11462954481 nodeName:}" failed. No retries permitted until 2026-09-14 02:51:57.182119965 +0000 UTC m=+66.160097992 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e746671e-19d5-48e7-8b3b-a11462954481-metrics-certs") pod "network-metrics-daemon-n7hgz" (UID: "e746671e-19d5-48e7-8b3b-a11462954481") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:51:25.282834 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:25.282791    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rkhd\" (UniqueName: \"kubernetes.io/projected/615f196a-4cb6-42a8-9e5f-fdea4a8672d6-kube-api-access-4rkhd\") pod \"network-check-target-n4kdw\" (UID: \"615f196a-4cb6-42a8-9e5f-fdea4a8672d6\") " pod="openshift-network-diagnostics/network-check-target-n4kdw"
Sep 14 02:51:25.283005 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:25.282935    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 02:51:25.283005 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:25.282954    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 02:51:25.283005 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:25.282964    2577 projected.go:196] Error preparing data for projected volume kube-api-access-4rkhd for pod openshift-network-diagnostics/network-check-target-n4kdw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:51:25.283170 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:25.283012    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/615f196a-4cb6-42a8-9e5f-fdea4a8672d6-kube-api-access-4rkhd podName:615f196a-4cb6-42a8-9e5f-fdea4a8672d6 nodeName:}" failed. No retries permitted until 2026-09-14 02:51:57.282999425 +0000 UTC m=+66.260977432 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4rkhd" (UniqueName: "kubernetes.io/projected/615f196a-4cb6-42a8-9e5f-fdea4a8672d6-kube-api-access-4rkhd") pod "network-check-target-n4kdw" (UID: "615f196a-4cb6-42a8-9e5f-fdea4a8672d6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:51:25.611923 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:25.611891    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hckjv"
Sep 14 02:51:25.612107 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:25.612035    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hckjv" podUID="003c37b0-6bdd-4b28-9a26-fa2a4ed813ec"
Sep 14 02:51:25.802876 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:25.802846    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-180.ec2.internal" event="NodeReady"
Sep 14 02:51:25.803035 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:25.802984    2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 14 02:51:25.873040 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:25.872933    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-v5tc4"]
Sep 14 02:51:25.927081 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:25.927055    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-lvxg5"]
Sep 14 02:51:25.927216 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:25.927180    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-v5tc4"
Sep 14 02:51:25.933837 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:25.933818    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-nzbvq\""
Sep 14 02:51:25.933936 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:25.933835    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 14 02:51:25.934064 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:25.934052    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 14 02:51:25.947805 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:25.947777    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lvxg5"
Sep 14 02:51:25.947805 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:25.947777    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-v5tc4"]
Sep 14 02:51:25.947944 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:25.947848    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lvxg5"]
Sep 14 02:51:25.953239 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:25.953221    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-hnrbf\""
Sep 14 02:51:25.953378 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:25.953294    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 14 02:51:25.953443 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:25.953405    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 14 02:51:25.953669 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:25.953655    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 14 02:51:26.088938 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:26.088919    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/df25d2a1-9825-4117-a68b-bdb22001d3d6-config-volume\") pod \"dns-default-v5tc4\" (UID: \"df25d2a1-9825-4117-a68b-bdb22001d3d6\") " pod="openshift-dns/dns-default-v5tc4"
Sep 14 02:51:26.089028 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:26.088948    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kf7bm\" (UniqueName: \"kubernetes.io/projected/b0761e95-5448-4b3d-a607-5eebfc1afc7d-kube-api-access-kf7bm\") pod \"ingress-canary-lvxg5\" (UID: \"b0761e95-5448-4b3d-a607-5eebfc1afc7d\") " pod="openshift-ingress-canary/ingress-canary-lvxg5"
Sep 14 02:51:26.089028 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:26.088980    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/df25d2a1-9825-4117-a68b-bdb22001d3d6-metrics-tls\") pod \"dns-default-v5tc4\" (UID: \"df25d2a1-9825-4117-a68b-bdb22001d3d6\") " pod="openshift-dns/dns-default-v5tc4"
Sep 14 02:51:26.089125 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:26.089066    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b0761e95-5448-4b3d-a607-5eebfc1afc7d-cert\") pod \"ingress-canary-lvxg5\" (UID: \"b0761e95-5448-4b3d-a607-5eebfc1afc7d\") " pod="openshift-ingress-canary/ingress-canary-lvxg5"
Sep 14 02:51:26.089125 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:26.089115    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df25d2a1-9825-4117-a68b-bdb22001d3d6-tmp-dir\") pod \"dns-default-v5tc4\" (UID: \"df25d2a1-9825-4117-a68b-bdb22001d3d6\") " pod="openshift-dns/dns-default-v5tc4"
Sep 14 02:51:26.089204 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:26.089134    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc6rs\" (UniqueName: \"kubernetes.io/projected/df25d2a1-9825-4117-a68b-bdb22001d3d6-kube-api-access-wc6rs\") pod \"dns-default-v5tc4\" (UID: \"df25d2a1-9825-4117-a68b-bdb22001d3d6\") " pod="openshift-dns/dns-default-v5tc4"
Sep 14 02:51:26.190279 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:26.190250    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b0761e95-5448-4b3d-a607-5eebfc1afc7d-cert\") pod \"ingress-canary-lvxg5\" (UID: \"b0761e95-5448-4b3d-a607-5eebfc1afc7d\") " pod="openshift-ingress-canary/ingress-canary-lvxg5"
Sep 14 02:51:26.190660 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:26.190300    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df25d2a1-9825-4117-a68b-bdb22001d3d6-tmp-dir\") pod \"dns-default-v5tc4\" (UID: \"df25d2a1-9825-4117-a68b-bdb22001d3d6\") " pod="openshift-dns/dns-default-v5tc4"
Sep 14 02:51:26.190660 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:26.190319    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc6rs\" (UniqueName: \"kubernetes.io/projected/df25d2a1-9825-4117-a68b-bdb22001d3d6-kube-api-access-wc6rs\") pod \"dns-default-v5tc4\" (UID: \"df25d2a1-9825-4117-a68b-bdb22001d3d6\") " pod="openshift-dns/dns-default-v5tc4"
Sep 14 02:51:26.190660 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:26.190354    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/df25d2a1-9825-4117-a68b-bdb22001d3d6-config-volume\") pod \"dns-default-v5tc4\" (UID: \"df25d2a1-9825-4117-a68b-bdb22001d3d6\") " pod="openshift-dns/dns-default-v5tc4"
Sep 14 02:51:26.190660 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:26.190374    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kf7bm\" (UniqueName: \"kubernetes.io/projected/b0761e95-5448-4b3d-a607-5eebfc1afc7d-kube-api-access-kf7bm\") pod \"ingress-canary-lvxg5\" (UID: \"b0761e95-5448-4b3d-a607-5eebfc1afc7d\") " pod="openshift-ingress-canary/ingress-canary-lvxg5"
Sep 14 02:51:26.190660 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:26.190391    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/df25d2a1-9825-4117-a68b-bdb22001d3d6-metrics-tls\") pod \"dns-default-v5tc4\" (UID: \"df25d2a1-9825-4117-a68b-bdb22001d3d6\") " pod="openshift-dns/dns-default-v5tc4"
Sep 14 02:51:26.190660 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:26.190479    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 02:51:26.190939 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:26.190525    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df25d2a1-9825-4117-a68b-bdb22001d3d6-metrics-tls podName:df25d2a1-9825-4117-a68b-bdb22001d3d6 nodeName:}" failed. No retries permitted until 2026-09-14 02:51:26.690510864 +0000 UTC m=+35.668488871 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/df25d2a1-9825-4117-a68b-bdb22001d3d6-metrics-tls") pod "dns-default-v5tc4" (UID: "df25d2a1-9825-4117-a68b-bdb22001d3d6") : secret "dns-default-metrics-tls" not found
Sep 14 02:51:26.190939 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:26.190883    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df25d2a1-9825-4117-a68b-bdb22001d3d6-tmp-dir\") pod \"dns-default-v5tc4\" (UID: \"df25d2a1-9825-4117-a68b-bdb22001d3d6\") " pod="openshift-dns/dns-default-v5tc4"
Sep 14 02:51:26.191211 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:26.191190    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/df25d2a1-9825-4117-a68b-bdb22001d3d6-config-volume\") pod \"dns-default-v5tc4\" (UID: \"df25d2a1-9825-4117-a68b-bdb22001d3d6\") " pod="openshift-dns/dns-default-v5tc4"
Sep 14 02:51:26.195391 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:26.195372    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b0761e95-5448-4b3d-a607-5eebfc1afc7d-cert\") pod \"ingress-canary-lvxg5\" (UID: \"b0761e95-5448-4b3d-a607-5eebfc1afc7d\") " pod="openshift-ingress-canary/ingress-canary-lvxg5"
Sep 14 02:51:26.203427 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:26.203406    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kf7bm\" (UniqueName: \"kubernetes.io/projected/b0761e95-5448-4b3d-a607-5eebfc1afc7d-kube-api-access-kf7bm\") pod \"ingress-canary-lvxg5\" (UID: \"b0761e95-5448-4b3d-a607-5eebfc1afc7d\") " pod="openshift-ingress-canary/ingress-canary-lvxg5"
Sep 14 02:51:26.203800 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:26.203775    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc6rs\" (UniqueName: \"kubernetes.io/projected/df25d2a1-9825-4117-a68b-bdb22001d3d6-kube-api-access-wc6rs\") pod \"dns-default-v5tc4\" (UID: \"df25d2a1-9825-4117-a68b-bdb22001d3d6\") " pod="openshift-dns/dns-default-v5tc4"
Sep 14 02:51:26.255254 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:26.255235    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lvxg5"
Sep 14 02:51:26.611397 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:26.611370    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4kdw"
Sep 14 02:51:26.611508 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:26.611370    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7hgz"
Sep 14 02:51:26.614622 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:26.614601    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 14 02:51:26.614622 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:26.614614    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 14 02:51:26.614795 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:26.614614    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mk9cr\""
Sep 14 02:51:26.614795 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:26.614614    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-b88wl\""
Sep 14 02:51:26.614795 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:26.614649    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 14 02:51:26.640186 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:26.640164    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lvxg5"]
Sep 14 02:51:26.643048 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:51:26.643022    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0761e95_5448_4b3d_a607_5eebfc1afc7d.slice/crio-e213565765e9c1a5c7e21d69dc99abb6852cba5a479cfe9a173c7e2f82d1aade WatchSource:0}: Error finding container e213565765e9c1a5c7e21d69dc99abb6852cba5a479cfe9a173c7e2f82d1aade: Status 404 returned error can't find the container with id e213565765e9c1a5c7e21d69dc99abb6852cba5a479cfe9a173c7e2f82d1aade
Sep 14 02:51:26.694013 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:26.693993    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/df25d2a1-9825-4117-a68b-bdb22001d3d6-metrics-tls\") pod \"dns-default-v5tc4\" (UID: \"df25d2a1-9825-4117-a68b-bdb22001d3d6\") " pod="openshift-dns/dns-default-v5tc4"
Sep 14 02:51:26.694116 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:26.694102    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 02:51:26.694165 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:26.694156    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df25d2a1-9825-4117-a68b-bdb22001d3d6-metrics-tls podName:df25d2a1-9825-4117-a68b-bdb22001d3d6 nodeName:}" failed. No retries permitted until 2026-09-14 02:51:27.694144337 +0000 UTC m=+36.672122358 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/df25d2a1-9825-4117-a68b-bdb22001d3d6-metrics-tls") pod "dns-default-v5tc4" (UID: "df25d2a1-9825-4117-a68b-bdb22001d3d6") : secret "dns-default-metrics-tls" not found
Sep 14 02:51:26.915190 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:26.915160    2577 generic.go:356] "Generic (PLEG): container finished" podID="38662066-a2a4-4e2e-8aef-f5478c1205a6" containerID="021d606bb04365f297409d59739f4f619f74220a6bd7c1bb9cc721dde8107a14" exitCode=0
Sep 14 02:51:26.915347 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:26.915238    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6bfjv" event={"ID":"38662066-a2a4-4e2e-8aef-f5478c1205a6","Type":"ContainerDied","Data":"021d606bb04365f297409d59739f4f619f74220a6bd7c1bb9cc721dde8107a14"}
Sep 14 02:51:26.916387 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:26.916365    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lvxg5" event={"ID":"b0761e95-5448-4b3d-a607-5eebfc1afc7d","Type":"ContainerStarted","Data":"e213565765e9c1a5c7e21d69dc99abb6852cba5a479cfe9a173c7e2f82d1aade"}
Sep 14 02:51:27.611219 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:27.611182    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hckjv"
Sep 14 02:51:27.614879 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:27.614856    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 14 02:51:27.701500 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:27.701468    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/df25d2a1-9825-4117-a68b-bdb22001d3d6-metrics-tls\") pod \"dns-default-v5tc4\" (UID: \"df25d2a1-9825-4117-a68b-bdb22001d3d6\") " pod="openshift-dns/dns-default-v5tc4"
Sep 14 02:51:27.701671 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:27.701632    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 02:51:27.701740 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:27.701707    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df25d2a1-9825-4117-a68b-bdb22001d3d6-metrics-tls podName:df25d2a1-9825-4117-a68b-bdb22001d3d6 nodeName:}" failed. No retries permitted until 2026-09-14 02:51:29.701686629 +0000 UTC m=+38.679664641 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/df25d2a1-9825-4117-a68b-bdb22001d3d6-metrics-tls") pod "dns-default-v5tc4" (UID: "df25d2a1-9825-4117-a68b-bdb22001d3d6") : secret "dns-default-metrics-tls" not found
Sep 14 02:51:27.923636 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:27.923554    2577 generic.go:356] "Generic (PLEG): container finished" podID="38662066-a2a4-4e2e-8aef-f5478c1205a6" containerID="203518532caa0d66edf9e6cc527f71e473b73731ac99dc521394efffe05beffd" exitCode=0
Sep 14 02:51:27.923636 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:27.923610    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6bfjv" event={"ID":"38662066-a2a4-4e2e-8aef-f5478c1205a6","Type":"ContainerDied","Data":"203518532caa0d66edf9e6cc527f71e473b73731ac99dc521394efffe05beffd"}
Sep 14 02:51:28.929203 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:28.929165    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6bfjv" event={"ID":"38662066-a2a4-4e2e-8aef-f5478c1205a6","Type":"ContainerStarted","Data":"62b9fbc4010128079005e76847165745fbbd5dde152012c604e50ffd70942351"}
Sep 14 02:51:28.971966 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:28.971923    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6bfjv" podStartSLOduration=7.782665442 podStartE2EDuration="36.971910013s" totalImagesPullingTime="29.189244571s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 02:50:52 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:51:28.971633483 +0000 UTC m=+37.949611512" watchObservedRunningTime="2026-09-14 02:51:28.971910013 +0000 UTC m=+37.949888042"
Sep 14 02:51:29.717369 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:29.717333    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/df25d2a1-9825-4117-a68b-bdb22001d3d6-metrics-tls\") pod \"dns-default-v5tc4\" (UID: \"df25d2a1-9825-4117-a68b-bdb22001d3d6\") " pod="openshift-dns/dns-default-v5tc4"
Sep 14 02:51:29.717546 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:29.717489    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 02:51:29.717590 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:29.717553    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df25d2a1-9825-4117-a68b-bdb22001d3d6-metrics-tls podName:df25d2a1-9825-4117-a68b-bdb22001d3d6 nodeName:}" failed. No retries permitted until 2026-09-14 02:51:33.717538731 +0000 UTC m=+42.695516768 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/df25d2a1-9825-4117-a68b-bdb22001d3d6-metrics-tls") pod "dns-default-v5tc4" (UID: "df25d2a1-9825-4117-a68b-bdb22001d3d6") : secret "dns-default-metrics-tls" not found
Sep 14 02:51:29.933208 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:29.933039    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lvxg5" event={"ID":"b0761e95-5448-4b3d-a607-5eebfc1afc7d","Type":"ContainerStarted","Data":"87979b893f9d2337387999a4689e3ab590b157ea25bd7183e12776fd7c3ff7da"}
Sep 14 02:51:29.950559 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:29.950480    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-lvxg5" podStartSLOduration=2.5499450120000002 podStartE2EDuration="4.95046262s" totalImagesPullingTime="2.400517608s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:51:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:51:29.950418043 +0000 UTC m=+38.928396072" watchObservedRunningTime="2026-09-14 02:51:29.95046262 +0000 UTC m=+38.928440650"
Sep 14 02:51:33.745057 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:33.745016    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/df25d2a1-9825-4117-a68b-bdb22001d3d6-metrics-tls\") pod \"dns-default-v5tc4\" (UID: \"df25d2a1-9825-4117-a68b-bdb22001d3d6\") " pod="openshift-dns/dns-default-v5tc4"
Sep 14 02:51:33.745495 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:33.745191    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 02:51:33.745495 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:33.745263    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df25d2a1-9825-4117-a68b-bdb22001d3d6-metrics-tls podName:df25d2a1-9825-4117-a68b-bdb22001d3d6 nodeName:}" failed. No retries permitted until 2026-09-14 02:51:41.745246368 +0000 UTC m=+50.723224376 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/df25d2a1-9825-4117-a68b-bdb22001d3d6-metrics-tls") pod "dns-default-v5tc4" (UID: "df25d2a1-9825-4117-a68b-bdb22001d3d6") : secret "dns-default-metrics-tls" not found
Sep 14 02:51:41.802621 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:41.802587    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/df25d2a1-9825-4117-a68b-bdb22001d3d6-metrics-tls\") pod \"dns-default-v5tc4\" (UID: \"df25d2a1-9825-4117-a68b-bdb22001d3d6\") " pod="openshift-dns/dns-default-v5tc4"
Sep 14 02:51:41.803104 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:41.802707    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 02:51:41.803104 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:41.802761    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df25d2a1-9825-4117-a68b-bdb22001d3d6-metrics-tls podName:df25d2a1-9825-4117-a68b-bdb22001d3d6 nodeName:}" failed. No retries permitted until 2026-09-14 02:51:57.802746799 +0000 UTC m=+66.780724811 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/df25d2a1-9825-4117-a68b-bdb22001d3d6-metrics-tls") pod "dns-default-v5tc4" (UID: "df25d2a1-9825-4117-a68b-bdb22001d3d6") : secret "dns-default-metrics-tls" not found
Sep 14 02:51:52.913849 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:52.913812    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xd5pl"
Sep 14 02:51:57.196156 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:57.196124    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e746671e-19d5-48e7-8b3b-a11462954481-metrics-certs\") pod \"network-metrics-daemon-n7hgz\" (UID: \"e746671e-19d5-48e7-8b3b-a11462954481\") " pod="openshift-multus/network-metrics-daemon-n7hgz"
Sep 14 02:51:57.196672 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:57.196162    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/003c37b0-6bdd-4b28-9a26-fa2a4ed813ec-original-pull-secret\") pod \"global-pull-secret-syncer-hckjv\" (UID: \"003c37b0-6bdd-4b28-9a26-fa2a4ed813ec\") " pod="kube-system/global-pull-secret-syncer-hckjv"
Sep 14 02:51:57.198641 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:57.198613    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 14 02:51:57.198716 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:57.198698    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 14 02:51:57.206824 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:57.206801    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 14 02:51:57.206928 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:57.206878    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e746671e-19d5-48e7-8b3b-a11462954481-metrics-certs podName:e746671e-19d5-48e7-8b3b-a11462954481 nodeName:}" failed. No retries permitted until 2026-09-14 02:53:01.206862963 +0000 UTC m=+130.184840970 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e746671e-19d5-48e7-8b3b-a11462954481-metrics-certs") pod "network-metrics-daemon-n7hgz" (UID: "e746671e-19d5-48e7-8b3b-a11462954481") : secret "metrics-daemon-secret" not found
Sep 14 02:51:57.210130 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:57.210107    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/003c37b0-6bdd-4b28-9a26-fa2a4ed813ec-original-pull-secret\") pod \"global-pull-secret-syncer-hckjv\" (UID: \"003c37b0-6bdd-4b28-9a26-fa2a4ed813ec\") " pod="kube-system/global-pull-secret-syncer-hckjv"
Sep 14 02:51:57.296547 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:57.296526    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rkhd\" (UniqueName: \"kubernetes.io/projected/615f196a-4cb6-42a8-9e5f-fdea4a8672d6-kube-api-access-4rkhd\") pod \"network-check-target-n4kdw\" (UID: \"615f196a-4cb6-42a8-9e5f-fdea4a8672d6\") " pod="openshift-network-diagnostics/network-check-target-n4kdw"
Sep 14 02:51:57.299491 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:57.299475    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 14 02:51:57.309643 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:57.309629    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 14 02:51:57.319609 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:57.319589    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rkhd\" (UniqueName: \"kubernetes.io/projected/615f196a-4cb6-42a8-9e5f-fdea4a8672d6-kube-api-access-4rkhd\") pod \"network-check-target-n4kdw\" (UID: \"615f196a-4cb6-42a8-9e5f-fdea4a8672d6\") " pod="openshift-network-diagnostics/network-check-target-n4kdw"
Sep 14 02:51:57.322409 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:57.322390    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hckjv"
Sep 14 02:51:57.544776 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:57.544755    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-b88wl\""
Sep 14 02:51:57.552942 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:57.552924    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4kdw"
Sep 14 02:51:57.601159 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:57.601036    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hckjv"]
Sep 14 02:51:57.604152 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:51:57.604123    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod003c37b0_6bdd_4b28_9a26_fa2a4ed813ec.slice/crio-40a570c777fa968f6cd166787639ca0ee1cd2c87d41b8b1228b44569bdf39fad WatchSource:0}: Error finding container 40a570c777fa968f6cd166787639ca0ee1cd2c87d41b8b1228b44569bdf39fad: Status 404 returned error can't find the container with id 40a570c777fa968f6cd166787639ca0ee1cd2c87d41b8b1228b44569bdf39fad
Sep 14 02:51:57.819762 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:57.819675    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-n4kdw"]
Sep 14 02:51:57.823040 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:51:57.823013    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod615f196a_4cb6_42a8_9e5f_fdea4a8672d6.slice/crio-dd08293e2f5549b3fef7af9869b45fc9ac15dc13f2e88308739c5edd73d9b0a9 WatchSource:0}: Error finding container dd08293e2f5549b3fef7af9869b45fc9ac15dc13f2e88308739c5edd73d9b0a9: Status 404 returned error can't find the container with id dd08293e2f5549b3fef7af9869b45fc9ac15dc13f2e88308739c5edd73d9b0a9
Sep 14 02:51:57.903440 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:57.903412    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/df25d2a1-9825-4117-a68b-bdb22001d3d6-metrics-tls\") pod \"dns-default-v5tc4\" (UID: \"df25d2a1-9825-4117-a68b-bdb22001d3d6\") " pod="openshift-dns/dns-default-v5tc4"
Sep 14 02:51:57.903584 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:57.903533    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 02:51:57.903584 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:57.903584    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df25d2a1-9825-4117-a68b-bdb22001d3d6-metrics-tls podName:df25d2a1-9825-4117-a68b-bdb22001d3d6 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:29.903569869 +0000 UTC m=+98.881547875 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/df25d2a1-9825-4117-a68b-bdb22001d3d6-metrics-tls") pod "dns-default-v5tc4" (UID: "df25d2a1-9825-4117-a68b-bdb22001d3d6") : secret "dns-default-metrics-tls" not found
Sep 14 02:51:57.995811 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:57.995774    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hckjv" event={"ID":"003c37b0-6bdd-4b28-9a26-fa2a4ed813ec","Type":"ContainerStarted","Data":"40a570c777fa968f6cd166787639ca0ee1cd2c87d41b8b1228b44569bdf39fad"}
Sep 14 02:51:57.997161 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:57.997129    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-n4kdw" event={"ID":"615f196a-4cb6-42a8-9e5f-fdea4a8672d6","Type":"ContainerStarted","Data":"dd08293e2f5549b3fef7af9869b45fc9ac15dc13f2e88308739c5edd73d9b0a9"}
Sep 14 02:51:58.737867 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.737833    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-s8847"]
Sep 14 02:51:58.739980 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.739718    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-s8847"
Sep 14 02:51:58.742932 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.742845    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 14 02:51:58.743073 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.742976    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 14 02:51:58.743405 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.743382    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 14 02:51:58.743855 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.743834    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 14 02:51:58.743934 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.743839    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-5mb77\""
Sep 14 02:51:58.751805 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.751756    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-s8847"]
Sep 14 02:51:58.810477 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.810287    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7wzq\" (UniqueName: \"kubernetes.io/projected/c10dbb6e-709f-46d7-af54-39fe56f4720e-kube-api-access-f7wzq\") pod \"service-ca-operator-cf7c776b7-s8847\" (UID: \"c10dbb6e-709f-46d7-af54-39fe56f4720e\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-s8847"
Sep 14 02:51:58.810477 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.810343    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c10dbb6e-709f-46d7-af54-39fe56f4720e-config\") pod \"service-ca-operator-cf7c776b7-s8847\" (UID: \"c10dbb6e-709f-46d7-af54-39fe56f4720e\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-s8847"
Sep 14 02:51:58.810477 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.810416    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c10dbb6e-709f-46d7-af54-39fe56f4720e-serving-cert\") pod \"service-ca-operator-cf7c776b7-s8847\" (UID: \"c10dbb6e-709f-46d7-af54-39fe56f4720e\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-s8847"
Sep 14 02:51:58.867252 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.867189    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-6vkfm"]
Sep 14 02:51:58.869093 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.869074    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6d9b48664b-648rd"]
Sep 14 02:51:58.869235 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.869215    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6vkfm"
Sep 14 02:51:58.870945 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.870904    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6d9b48664b-648rd"
Sep 14 02:51:58.872334 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:58.871905    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"networking-console-plugin-5dcbc7df6b-6vkfm\" is forbidden: User \"system:node:ip-10-0-141-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-network-console\": no relationship found between node 'ip-10-0-141-180.ec2.internal' and this object" podUID="1b15bf18-f4ef-4d01-9916-780ac2d581ab" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6vkfm"
Sep 14 02:51:58.872334 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:58.872058    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"default-dockercfg-4f29d\" is forbidden: User \"system:node:ip-10-0-141-180.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-network-console\": no relationship found between node 'ip-10-0-141-180.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-network-console\"/\"default-dockercfg-4f29d\"" type="*v1.Secret"
Sep 14 02:51:58.872334 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:58.872245    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"networking-console-plugin-cert\" is forbidden: User \"system:node:ip-10-0-141-180.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-network-console\": no relationship found between node 'ip-10-0-141-180.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" type="*v1.Secret"
Sep 14 02:51:58.872994 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:58.872965    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"networking-console-plugin\" is forbidden: User \"system:node:ip-10-0-141-180.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-network-console\": no relationship found between node 'ip-10-0-141-180.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" type="*v1.ConfigMap"
Sep 14 02:51:58.873299 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:58.873245    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"router-stats-default\" is forbidden: User \"system:node:ip-10-0-141-180.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-141-180.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" type="*v1.Secret"
Sep 14 02:51:58.873406 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:58.873258    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-141-180.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-141-180.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap"
Sep 14 02:51:58.873725 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:58.873698    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"service-ca-bundle\" is forbidden: User \"system:node:ip-10-0-141-180.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-141-180.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" type="*v1.ConfigMap"
Sep 14 02:51:58.873812 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:58.873770    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-141-180.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-141-180.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap"
Sep 14 02:51:58.874686 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.874666    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 14 02:51:58.874881 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.874860    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 14 02:51:58.875103 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.875041    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-q2jsk\""
Sep 14 02:51:58.885026 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.885000    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-6vkfm"]
Sep 14 02:51:58.887887 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.887839    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6d9b48664b-648rd"]
Sep 14 02:51:58.910833 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.910798    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/713ae3f6-786b-4911-92ae-854cf6c825b1-default-certificate\") pod \"router-default-6d9b48664b-648rd\" (UID: \"713ae3f6-786b-4911-92ae-854cf6c825b1\") " pod="openshift-ingress/router-default-6d9b48664b-648rd"
Sep 14 02:51:58.910940 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.910842    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/713ae3f6-786b-4911-92ae-854cf6c825b1-stats-auth\") pod \"router-default-6d9b48664b-648rd\" (UID: \"713ae3f6-786b-4911-92ae-854cf6c825b1\") " pod="openshift-ingress/router-default-6d9b48664b-648rd"
Sep 14 02:51:58.910940 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.910870    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1b15bf18-f4ef-4d01-9916-780ac2d581ab-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-6vkfm\" (UID: \"1b15bf18-f4ef-4d01-9916-780ac2d581ab\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6vkfm"
Sep 14 02:51:58.910940 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.910895    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1b15bf18-f4ef-4d01-9916-780ac2d581ab-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-6vkfm\" (UID: \"1b15bf18-f4ef-4d01-9916-780ac2d581ab\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6vkfm"
Sep 14 02:51:58.911108 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.910969    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/713ae3f6-786b-4911-92ae-854cf6c825b1-metrics-certs\") pod \"router-default-6d9b48664b-648rd\" (UID: \"713ae3f6-786b-4911-92ae-854cf6c825b1\") " pod="openshift-ingress/router-default-6d9b48664b-648rd"
Sep 14 02:51:58.911108 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.911019    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c10dbb6e-709f-46d7-af54-39fe56f4720e-serving-cert\") pod \"service-ca-operator-cf7c776b7-s8847\" (UID: \"c10dbb6e-709f-46d7-af54-39fe56f4720e\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-s8847"
Sep 14 02:51:58.911108 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.911048    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/713ae3f6-786b-4911-92ae-854cf6c825b1-service-ca-bundle\") pod \"router-default-6d9b48664b-648rd\" (UID: \"713ae3f6-786b-4911-92ae-854cf6c825b1\") " pod="openshift-ingress/router-default-6d9b48664b-648rd"
Sep 14 02:51:58.911108 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.911082    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7wzq\" (UniqueName: \"kubernetes.io/projected/c10dbb6e-709f-46d7-af54-39fe56f4720e-kube-api-access-f7wzq\") pod \"service-ca-operator-cf7c776b7-s8847\" (UID: \"c10dbb6e-709f-46d7-af54-39fe56f4720e\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-s8847"
Sep 14 02:51:58.911311 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.911133    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c10dbb6e-709f-46d7-af54-39fe56f4720e-config\") pod \"service-ca-operator-cf7c776b7-s8847\" (UID: \"c10dbb6e-709f-46d7-af54-39fe56f4720e\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-s8847"
Sep 14 02:51:58.911311 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.911174    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rq949\" (UniqueName: \"kubernetes.io/projected/713ae3f6-786b-4911-92ae-854cf6c825b1-kube-api-access-rq949\") pod \"router-default-6d9b48664b-648rd\" (UID: \"713ae3f6-786b-4911-92ae-854cf6c825b1\") " pod="openshift-ingress/router-default-6d9b48664b-648rd"
Sep 14 02:51:58.911789 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.911754    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c10dbb6e-709f-46d7-af54-39fe56f4720e-config\") pod \"service-ca-operator-cf7c776b7-s8847\" (UID: \"c10dbb6e-709f-46d7-af54-39fe56f4720e\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-s8847"
Sep 14 02:51:58.913459 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.913437    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c10dbb6e-709f-46d7-af54-39fe56f4720e-serving-cert\") pod \"service-ca-operator-cf7c776b7-s8847\" (UID: \"c10dbb6e-709f-46d7-af54-39fe56f4720e\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-s8847"
Sep 14 02:51:58.920211 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.920192    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7wzq\" (UniqueName: \"kubernetes.io/projected/c10dbb6e-709f-46d7-af54-39fe56f4720e-kube-api-access-f7wzq\") pod \"service-ca-operator-cf7c776b7-s8847\" (UID: \"c10dbb6e-709f-46d7-af54-39fe56f4720e\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-s8847"
Sep 14 02:51:58.970805 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.970780    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7666cf9458-rqpks"]
Sep 14 02:51:58.972898 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.972875    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7666cf9458-rqpks"
Sep 14 02:51:58.975961 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.975791    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 14 02:51:58.975961 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.975803    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 14 02:51:58.975961 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.975869    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 14 02:51:58.975961 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.975960    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tz6jr\""
Sep 14 02:51:58.985279 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.985238    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7666cf9458-rqpks"]
Sep 14 02:51:58.989374 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:58.989139    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 14 02:51:59.012387 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:59.012336    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/713ae3f6-786b-4911-92ae-854cf6c825b1-service-ca-bundle\") pod \"router-default-6d9b48664b-648rd\" (UID: \"713ae3f6-786b-4911-92ae-854cf6c825b1\") " pod="openshift-ingress/router-default-6d9b48664b-648rd"
Sep 14 02:51:59.012477 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:59.012403    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-image-registry-private-configuration\") pod \"image-registry-7666cf9458-rqpks\" (UID: \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\") " pod="openshift-image-registry/image-registry-7666cf9458-rqpks"
Sep 14 02:51:59.012477 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:59.012435    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-installation-pull-secrets\") pod \"image-registry-7666cf9458-rqpks\" (UID: \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\") " pod="openshift-image-registry/image-registry-7666cf9458-rqpks"
Sep 14 02:51:59.012601 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:59.012523    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rq949\" (UniqueName: \"kubernetes.io/projected/713ae3f6-786b-4911-92ae-854cf6c825b1-kube-api-access-rq949\") pod \"router-default-6d9b48664b-648rd\" (UID: \"713ae3f6-786b-4911-92ae-854cf6c825b1\") " pod="openshift-ingress/router-default-6d9b48664b-648rd"
Sep 14 02:51:59.012601 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:59.012569    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-ca-trust-extracted\") pod \"image-registry-7666cf9458-rqpks\" (UID: \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\") " pod="openshift-image-registry/image-registry-7666cf9458-rqpks"
Sep 14 02:51:59.012601 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:59.012596    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-registry-certificates\") pod \"image-registry-7666cf9458-rqpks\" (UID: \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\") " pod="openshift-image-registry/image-registry-7666cf9458-rqpks"
Sep 14 02:51:59.012752 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:59.012645    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-trusted-ca\") pod \"image-registry-7666cf9458-rqpks\" (UID: \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\") " pod="openshift-image-registry/image-registry-7666cf9458-rqpks"
Sep 14 02:51:59.012752 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:59.012680    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcv7f\" (UniqueName: \"kubernetes.io/projected/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-kube-api-access-qcv7f\") pod \"image-registry-7666cf9458-rqpks\" (UID: \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\") " pod="openshift-image-registry/image-registry-7666cf9458-rqpks"
Sep 14 02:51:59.012856 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:59.012778    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/713ae3f6-786b-4911-92ae-854cf6c825b1-default-certificate\") pod \"router-default-6d9b48664b-648rd\" (UID: \"713ae3f6-786b-4911-92ae-854cf6c825b1\") " pod="openshift-ingress/router-default-6d9b48664b-648rd"
Sep 14 02:51:59.012856 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:59.012824    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/713ae3f6-786b-4911-92ae-854cf6c825b1-stats-auth\") pod \"router-default-6d9b48664b-648rd\" (UID: \"713ae3f6-786b-4911-92ae-854cf6c825b1\") " pod="openshift-ingress/router-default-6d9b48664b-648rd"
Sep 14 02:51:59.012856 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:59.012846    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1b15bf18-f4ef-4d01-9916-780ac2d581ab-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-6vkfm\" (UID: \"1b15bf18-f4ef-4d01-9916-780ac2d581ab\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6vkfm"
Sep 14 02:51:59.012989 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:59.012862    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1b15bf18-f4ef-4d01-9916-780ac2d581ab-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-6vkfm\" (UID: \"1b15bf18-f4ef-4d01-9916-780ac2d581ab\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6vkfm"
Sep 14 02:51:59.012989 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:59.012889    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/713ae3f6-786b-4911-92ae-854cf6c825b1-metrics-certs\") pod \"router-default-6d9b48664b-648rd\" (UID: \"713ae3f6-786b-4911-92ae-854cf6c825b1\") " pod="openshift-ingress/router-default-6d9b48664b-648rd"
Sep 14 02:51:59.012989 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:59.012917    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-bound-sa-token\") pod \"image-registry-7666cf9458-rqpks\" (UID: \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\") " pod="openshift-image-registry/image-registry-7666cf9458-rqpks"
Sep 14 02:51:59.012989 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:59.012944    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-registry-tls\") pod \"image-registry-7666cf9458-rqpks\" (UID: \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\") " pod="openshift-image-registry/image-registry-7666cf9458-rqpks"
Sep 14 02:51:59.013179 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:59.013002    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 14 02:51:59.013179 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:59.013063    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/713ae3f6-786b-4911-92ae-854cf6c825b1-metrics-certs podName:713ae3f6-786b-4911-92ae-854cf6c825b1 nodeName:}" failed. No retries permitted until 2026-09-14 02:51:59.513044211 +0000 UTC m=+68.491022229 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/713ae3f6-786b-4911-92ae-854cf6c825b1-metrics-certs") pod "router-default-6d9b48664b-648rd" (UID: "713ae3f6-786b-4911-92ae-854cf6c825b1") : secret "router-metrics-certs-default" not found
Sep 14 02:51:59.015626 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:59.015598    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/713ae3f6-786b-4911-92ae-854cf6c825b1-default-certificate\") pod \"router-default-6d9b48664b-648rd\" (UID: \"713ae3f6-786b-4911-92ae-854cf6c825b1\") " pod="openshift-ingress/router-default-6d9b48664b-648rd"
Sep 14 02:51:59.051171 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:59.051149    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-s8847"
Sep 14 02:51:59.114247 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:59.113759    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-trusted-ca\") pod \"image-registry-7666cf9458-rqpks\" (UID: \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\") " pod="openshift-image-registry/image-registry-7666cf9458-rqpks"
Sep 14 02:51:59.114247 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:59.113802    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcv7f\" (UniqueName: \"kubernetes.io/projected/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-kube-api-access-qcv7f\") pod \"image-registry-7666cf9458-rqpks\" (UID: \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\") " pod="openshift-image-registry/image-registry-7666cf9458-rqpks"
Sep 14 02:51:59.114247 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:59.113876    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-bound-sa-token\") pod \"image-registry-7666cf9458-rqpks\" (UID: \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\") " pod="openshift-image-registry/image-registry-7666cf9458-rqpks"
Sep 14 02:51:59.114247 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:59.113902    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-registry-tls\") pod \"image-registry-7666cf9458-rqpks\" (UID: \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\") " pod="openshift-image-registry/image-registry-7666cf9458-rqpks"
Sep 14 02:51:59.114247 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:59.113929    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-image-registry-private-configuration\") pod \"image-registry-7666cf9458-rqpks\" (UID: \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\") " pod="openshift-image-registry/image-registry-7666cf9458-rqpks"
Sep 14 02:51:59.114247 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:59.113968    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-installation-pull-secrets\") pod \"image-registry-7666cf9458-rqpks\" (UID: \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\") " pod="openshift-image-registry/image-registry-7666cf9458-rqpks"
Sep 14 02:51:59.114247 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:59.114036    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-ca-trust-extracted\") pod \"image-registry-7666cf9458-rqpks\" (UID: \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\") " pod="openshift-image-registry/image-registry-7666cf9458-rqpks"
Sep 14 02:51:59.114247 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:59.114058    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-registry-certificates\") pod \"image-registry-7666cf9458-rqpks\" (UID: \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\") " pod="openshift-image-registry/image-registry-7666cf9458-rqpks"
Sep 14 02:51:59.115136 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:59.114893    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-ca-trust-extracted\") pod \"image-registry-7666cf9458-rqpks\" (UID: \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\") " pod="openshift-image-registry/image-registry-7666cf9458-rqpks"
Sep 14 02:51:59.115136 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:59.115084    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-registry-certificates\") pod \"image-registry-7666cf9458-rqpks\" (UID: \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\") " pod="openshift-image-registry/image-registry-7666cf9458-rqpks"
Sep 14 02:51:59.115294 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:59.115169    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 02:51:59.115294 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:59.115189    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7666cf9458-rqpks: secret "image-registry-tls" not found
Sep 14 02:51:59.115294 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:59.115247    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-registry-tls podName:321a267f-4dd7-4cfb-b28f-7b2f57caf02b nodeName:}" failed. No retries permitted until 2026-09-14 02:51:59.615227003 +0000 UTC m=+68.593205016 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-registry-tls") pod "image-registry-7666cf9458-rqpks" (UID: "321a267f-4dd7-4cfb-b28f-7b2f57caf02b") : secret "image-registry-tls" not found
Sep 14 02:51:59.116231 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:59.116207    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-trusted-ca\") pod \"image-registry-7666cf9458-rqpks\" (UID: \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\") " pod="openshift-image-registry/image-registry-7666cf9458-rqpks"
Sep 14 02:51:59.117358 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:59.117317    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-installation-pull-secrets\") pod \"image-registry-7666cf9458-rqpks\" (UID: \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\") " pod="openshift-image-registry/image-registry-7666cf9458-rqpks"
Sep 14 02:51:59.118731 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:59.118681    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-image-registry-private-configuration\") pod \"image-registry-7666cf9458-rqpks\" (UID: \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\") " pod="openshift-image-registry/image-registry-7666cf9458-rqpks"
Sep 14 02:51:59.128456 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:59.128423    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcv7f\" (UniqueName: \"kubernetes.io/projected/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-kube-api-access-qcv7f\") pod \"image-registry-7666cf9458-rqpks\" (UID: \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\") " pod="openshift-image-registry/image-registry-7666cf9458-rqpks"
Sep 14 02:51:59.130387 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:59.130192    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-bound-sa-token\") pod \"image-registry-7666cf9458-rqpks\" (UID: \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\") " pod="openshift-image-registry/image-registry-7666cf9458-rqpks"
Sep 14 02:51:59.333453 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:59.333386    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-s8847"]
Sep 14 02:51:59.337454 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:51:59.337424    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc10dbb6e_709f_46d7_af54_39fe56f4720e.slice/crio-3405bed78497a89479cfc3cfc13f91104d4a36cf1316f8f026184d8da2cc24b7 WatchSource:0}: Error finding container 3405bed78497a89479cfc3cfc13f91104d4a36cf1316f8f026184d8da2cc24b7: Status 404 returned error can't find the container with id 3405bed78497a89479cfc3cfc13f91104d4a36cf1316f8f026184d8da2cc24b7
Sep 14 02:51:59.517173 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:59.517136    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/713ae3f6-786b-4911-92ae-854cf6c825b1-metrics-certs\") pod \"router-default-6d9b48664b-648rd\" (UID: \"713ae3f6-786b-4911-92ae-854cf6c825b1\") " pod="openshift-ingress/router-default-6d9b48664b-648rd"
Sep 14 02:51:59.517326 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:59.517313    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 14 02:51:59.517386 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:59.517373    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/713ae3f6-786b-4911-92ae-854cf6c825b1-metrics-certs podName:713ae3f6-786b-4911-92ae-854cf6c825b1 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:00.517354982 +0000 UTC m=+69.495332990 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/713ae3f6-786b-4911-92ae-854cf6c825b1-metrics-certs") pod "router-default-6d9b48664b-648rd" (UID: "713ae3f6-786b-4911-92ae-854cf6c825b1") : secret "router-metrics-certs-default" not found
Sep 14 02:51:59.617677 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:59.617613    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-registry-tls\") pod \"image-registry-7666cf9458-rqpks\" (UID: \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\") " pod="openshift-image-registry/image-registry-7666cf9458-rqpks"
Sep 14 02:51:59.617813 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:59.617790    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 02:51:59.617813 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:59.617813    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7666cf9458-rqpks: secret "image-registry-tls" not found
Sep 14 02:51:59.617949 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:51:59.617890    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-registry-tls podName:321a267f-4dd7-4cfb-b28f-7b2f57caf02b nodeName:}" failed. No retries permitted until 2026-09-14 02:52:00.617870976 +0000 UTC m=+69.595848998 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-registry-tls") pod "image-registry-7666cf9458-rqpks" (UID: "321a267f-4dd7-4cfb-b28f-7b2f57caf02b") : secret "image-registry-tls" not found
Sep 14 02:51:59.868075 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:51:59.868004    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 14 02:52:00.005204 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:00.005138    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-s8847" event={"ID":"c10dbb6e-709f-46d7-af54-39fe56f4720e","Type":"ContainerStarted","Data":"3405bed78497a89479cfc3cfc13f91104d4a36cf1316f8f026184d8da2cc24b7"}
Sep 14 02:52:00.013059 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:00.012756    2577 configmap.go:193] Couldn't get configMap openshift-ingress/service-ca-bundle: failed to sync configmap cache: timed out waiting for the condition
Sep 14 02:52:00.013059 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:00.012845    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/713ae3f6-786b-4911-92ae-854cf6c825b1-service-ca-bundle podName:713ae3f6-786b-4911-92ae-854cf6c825b1 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:00.512824209 +0000 UTC m=+69.490802235 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/713ae3f6-786b-4911-92ae-854cf6c825b1-service-ca-bundle") pod "router-default-6d9b48664b-648rd" (UID: "713ae3f6-786b-4911-92ae-854cf6c825b1") : failed to sync configmap cache: timed out waiting for the condition
Sep 14 02:52:00.013253 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:00.013208    2577 secret.go:189] Couldn't get secret openshift-ingress/router-stats-default: failed to sync secret cache: timed out waiting for the condition
Sep 14 02:52:00.013339 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:00.013298    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/713ae3f6-786b-4911-92ae-854cf6c825b1-stats-auth podName:713ae3f6-786b-4911-92ae-854cf6c825b1 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:00.513263364 +0000 UTC m=+69.491241387 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "stats-auth" (UniqueName: "kubernetes.io/secret/713ae3f6-786b-4911-92ae-854cf6c825b1-stats-auth") pod "router-default-6d9b48664b-648rd" (UID: "713ae3f6-786b-4911-92ae-854cf6c825b1") : failed to sync secret cache: timed out waiting for the condition
Sep 14 02:52:00.013339 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:00.013325    2577 configmap.go:193] Couldn't get configMap openshift-network-console/networking-console-plugin: failed to sync configmap cache: timed out waiting for the condition
Sep 14 02:52:00.013455 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:00.013330    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: failed to sync secret cache: timed out waiting for the condition
Sep 14 02:52:00.013455 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:00.013360    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1b15bf18-f4ef-4d01-9916-780ac2d581ab-nginx-conf podName:1b15bf18-f4ef-4d01-9916-780ac2d581ab nodeName:}" failed. No retries permitted until 2026-09-14 02:52:00.513350922 +0000 UTC m=+69.491328929 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "nginx-conf" (UniqueName: "kubernetes.io/configmap/1b15bf18-f4ef-4d01-9916-780ac2d581ab-nginx-conf") pod "networking-console-plugin-5dcbc7df6b-6vkfm" (UID: "1b15bf18-f4ef-4d01-9916-780ac2d581ab") : failed to sync configmap cache: timed out waiting for the condition
Sep 14 02:52:00.013455 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:00.013413    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b15bf18-f4ef-4d01-9916-780ac2d581ab-networking-console-plugin-cert podName:1b15bf18-f4ef-4d01-9916-780ac2d581ab nodeName:}" failed. No retries permitted until 2026-09-14 02:52:00.513396013 +0000 UTC m=+69.491374041 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1b15bf18-f4ef-4d01-9916-780ac2d581ab-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-6vkfm" (UID: "1b15bf18-f4ef-4d01-9916-780ac2d581ab") : failed to sync secret cache: timed out waiting for the condition
Sep 14 02:52:00.021304 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:00.021282    2577 projected.go:291] Couldn't get configMap openshift-ingress/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition
Sep 14 02:52:00.021420 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:00.021319    2577 projected.go:196] Error preparing data for projected volume kube-api-access-rq949 for pod openshift-ingress/router-default-6d9b48664b-648rd: failed to sync configmap cache: timed out waiting for the condition
Sep 14 02:52:00.021420 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:00.021367    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/713ae3f6-786b-4911-92ae-854cf6c825b1-kube-api-access-rq949 podName:713ae3f6-786b-4911-92ae-854cf6c825b1 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:00.521352911 +0000 UTC m=+69.499330941 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rq949" (UniqueName: "kubernetes.io/projected/713ae3f6-786b-4911-92ae-854cf6c825b1-kube-api-access-rq949") pod "router-default-6d9b48664b-648rd" (UID: "713ae3f6-786b-4911-92ae-854cf6c825b1") : failed to sync configmap cache: timed out waiting for the condition
Sep 14 02:52:00.174494 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:00.174428    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 14 02:52:00.244584 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:00.244555    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 14 02:52:00.337916 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:00.337829    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-4f29d\""
Sep 14 02:52:00.359344 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:00.359318    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 14 02:52:00.379236 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:00.379211    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 14 02:52:00.471991 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:00.471923    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 14 02:52:00.526721 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:00.526691    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rq949\" (UniqueName: \"kubernetes.io/projected/713ae3f6-786b-4911-92ae-854cf6c825b1-kube-api-access-rq949\") pod \"router-default-6d9b48664b-648rd\" (UID: \"713ae3f6-786b-4911-92ae-854cf6c825b1\") " pod="openshift-ingress/router-default-6d9b48664b-648rd"
Sep 14 02:52:00.526845 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:00.526756    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/713ae3f6-786b-4911-92ae-854cf6c825b1-stats-auth\") pod \"router-default-6d9b48664b-648rd\" (UID: \"713ae3f6-786b-4911-92ae-854cf6c825b1\") " pod="openshift-ingress/router-default-6d9b48664b-648rd"
Sep 14 02:52:00.526845 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:00.526776    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1b15bf18-f4ef-4d01-9916-780ac2d581ab-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-6vkfm\" (UID: \"1b15bf18-f4ef-4d01-9916-780ac2d581ab\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6vkfm"
Sep 14 02:52:00.526845 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:00.526801    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1b15bf18-f4ef-4d01-9916-780ac2d581ab-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-6vkfm\" (UID: \"1b15bf18-f4ef-4d01-9916-780ac2d581ab\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6vkfm"
Sep 14 02:52:00.526845 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:00.526820    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/713ae3f6-786b-4911-92ae-854cf6c825b1-metrics-certs\") pod \"router-default-6d9b48664b-648rd\" (UID: \"713ae3f6-786b-4911-92ae-854cf6c825b1\") " pod="openshift-ingress/router-default-6d9b48664b-648rd"
Sep 14 02:52:00.527055 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:00.526847    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/713ae3f6-786b-4911-92ae-854cf6c825b1-service-ca-bundle\") pod \"router-default-6d9b48664b-648rd\" (UID: \"713ae3f6-786b-4911-92ae-854cf6c825b1\") " pod="openshift-ingress/router-default-6d9b48664b-648rd"
Sep 14 02:52:00.527055 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:00.526973    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 14 02:52:00.527055 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:00.527033    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b15bf18-f4ef-4d01-9916-780ac2d581ab-networking-console-plugin-cert podName:1b15bf18-f4ef-4d01-9916-780ac2d581ab nodeName:}" failed. No retries permitted until 2026-09-14 02:52:01.527015289 +0000 UTC m=+70.504993315 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1b15bf18-f4ef-4d01-9916-780ac2d581ab-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-6vkfm" (UID: "1b15bf18-f4ef-4d01-9916-780ac2d581ab") : secret "networking-console-plugin-cert" not found
Sep 14 02:52:00.527055 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:00.527031    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 14 02:52:00.527295 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:00.527073    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/713ae3f6-786b-4911-92ae-854cf6c825b1-metrics-certs podName:713ae3f6-786b-4911-92ae-854cf6c825b1 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:02.52706149 +0000 UTC m=+71.505039510 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/713ae3f6-786b-4911-92ae-854cf6c825b1-metrics-certs") pod "router-default-6d9b48664b-648rd" (UID: "713ae3f6-786b-4911-92ae-854cf6c825b1") : secret "router-metrics-certs-default" not found
Sep 14 02:52:00.527295 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:00.527169    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/713ae3f6-786b-4911-92ae-854cf6c825b1-service-ca-bundle podName:713ae3f6-786b-4911-92ae-854cf6c825b1 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:01.527151101 +0000 UTC m=+70.505129131 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/713ae3f6-786b-4911-92ae-854cf6c825b1-service-ca-bundle") pod "router-default-6d9b48664b-648rd" (UID: "713ae3f6-786b-4911-92ae-854cf6c825b1") : configmap references non-existent config key: service-ca.crt
Sep 14 02:52:00.527591 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:00.527570    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1b15bf18-f4ef-4d01-9916-780ac2d581ab-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-6vkfm\" (UID: \"1b15bf18-f4ef-4d01-9916-780ac2d581ab\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6vkfm"
Sep 14 02:52:00.529406 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:00.529378    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/713ae3f6-786b-4911-92ae-854cf6c825b1-stats-auth\") pod \"router-default-6d9b48664b-648rd\" (UID: \"713ae3f6-786b-4911-92ae-854cf6c825b1\") " pod="openshift-ingress/router-default-6d9b48664b-648rd"
Sep 14 02:52:00.529691 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:00.529673    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rq949\" (UniqueName: \"kubernetes.io/projected/713ae3f6-786b-4911-92ae-854cf6c825b1-kube-api-access-rq949\") pod \"router-default-6d9b48664b-648rd\" (UID: \"713ae3f6-786b-4911-92ae-854cf6c825b1\") " pod="openshift-ingress/router-default-6d9b48664b-648rd"
Sep 14 02:52:00.627926 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:00.627896    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-registry-tls\") pod \"image-registry-7666cf9458-rqpks\" (UID: \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\") " pod="openshift-image-registry/image-registry-7666cf9458-rqpks"
Sep 14 02:52:00.628062 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:00.628014    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 02:52:00.628062 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:00.628031    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7666cf9458-rqpks: secret "image-registry-tls" not found
Sep 14 02:52:00.628167 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:00.628086    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-registry-tls podName:321a267f-4dd7-4cfb-b28f-7b2f57caf02b nodeName:}" failed. No retries permitted until 2026-09-14 02:52:02.628070529 +0000 UTC m=+71.606048540 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-registry-tls") pod "image-registry-7666cf9458-rqpks" (UID: "321a267f-4dd7-4cfb-b28f-7b2f57caf02b") : secret "image-registry-tls" not found
Sep 14 02:52:01.534964 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:01.534925    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1b15bf18-f4ef-4d01-9916-780ac2d581ab-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-6vkfm\" (UID: \"1b15bf18-f4ef-4d01-9916-780ac2d581ab\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6vkfm"
Sep 14 02:52:01.535434 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:01.534995    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/713ae3f6-786b-4911-92ae-854cf6c825b1-service-ca-bundle\") pod \"router-default-6d9b48664b-648rd\" (UID: \"713ae3f6-786b-4911-92ae-854cf6c825b1\") " pod="openshift-ingress/router-default-6d9b48664b-648rd"
Sep 14 02:52:01.535434 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:01.535064    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 14 02:52:01.535434 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:01.535146    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b15bf18-f4ef-4d01-9916-780ac2d581ab-networking-console-plugin-cert podName:1b15bf18-f4ef-4d01-9916-780ac2d581ab nodeName:}" failed. No retries permitted until 2026-09-14 02:52:03.535124936 +0000 UTC m=+72.513102944 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1b15bf18-f4ef-4d01-9916-780ac2d581ab-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-6vkfm" (UID: "1b15bf18-f4ef-4d01-9916-780ac2d581ab") : secret "networking-console-plugin-cert" not found
Sep 14 02:52:01.535434 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:01.535165    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/713ae3f6-786b-4911-92ae-854cf6c825b1-service-ca-bundle podName:713ae3f6-786b-4911-92ae-854cf6c825b1 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:03.535157466 +0000 UTC m=+72.513135472 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/713ae3f6-786b-4911-92ae-854cf6c825b1-service-ca-bundle") pod "router-default-6d9b48664b-648rd" (UID: "713ae3f6-786b-4911-92ae-854cf6c825b1") : configmap references non-existent config key: service-ca.crt
Sep 14 02:52:02.543505 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:02.543464    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/713ae3f6-786b-4911-92ae-854cf6c825b1-metrics-certs\") pod \"router-default-6d9b48664b-648rd\" (UID: \"713ae3f6-786b-4911-92ae-854cf6c825b1\") " pod="openshift-ingress/router-default-6d9b48664b-648rd"
Sep 14 02:52:02.543937 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:02.543590    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 14 02:52:02.543937 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:02.543657    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/713ae3f6-786b-4911-92ae-854cf6c825b1-metrics-certs podName:713ae3f6-786b-4911-92ae-854cf6c825b1 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:06.543641814 +0000 UTC m=+75.521619827 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/713ae3f6-786b-4911-92ae-854cf6c825b1-metrics-certs") pod "router-default-6d9b48664b-648rd" (UID: "713ae3f6-786b-4911-92ae-854cf6c825b1") : secret "router-metrics-certs-default" not found
Sep 14 02:52:02.644430 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:02.644396    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-registry-tls\") pod \"image-registry-7666cf9458-rqpks\" (UID: \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\") " pod="openshift-image-registry/image-registry-7666cf9458-rqpks"
Sep 14 02:52:02.644592 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:02.644541    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 02:52:02.644592 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:02.644555    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7666cf9458-rqpks: secret "image-registry-tls" not found
Sep 14 02:52:02.644690 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:02.644608    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-registry-tls podName:321a267f-4dd7-4cfb-b28f-7b2f57caf02b nodeName:}" failed. No retries permitted until 2026-09-14 02:52:06.644591893 +0000 UTC m=+75.622569919 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-registry-tls") pod "image-registry-7666cf9458-rqpks" (UID: "321a267f-4dd7-4cfb-b28f-7b2f57caf02b") : secret "image-registry-tls" not found
Sep 14 02:52:03.016019 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:03.015980    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hckjv" event={"ID":"003c37b0-6bdd-4b28-9a26-fa2a4ed813ec","Type":"ContainerStarted","Data":"0a06f29ab656573c2b289f8801bf2fa71c9e7fb1151547bf019c1afb57431a5f"}
Sep 14 02:52:03.017657 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:03.017629    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-n4kdw" event={"ID":"615f196a-4cb6-42a8-9e5f-fdea4a8672d6","Type":"ContainerStarted","Data":"217ee94610992dfd264d6b8d2bbbf049aaf937816fa011a48e0e2020de5b4e42"}
Sep 14 02:52:03.017745 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:03.017691    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-n4kdw"
Sep 14 02:52:03.019150 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:03.019132    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-s8847" event={"ID":"c10dbb6e-709f-46d7-af54-39fe56f4720e","Type":"ContainerStarted","Data":"c87803b32a8e80cc3632bb3f008b807586b32b4154e53d8b53338ca6308c474d"}
Sep 14 02:52:03.067070 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:03.067025    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-hckjv" podStartSLOduration=65.785183492 podStartE2EDuration="1m11.067011116s" totalImagesPullingTime="5.281827624s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 02:50:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:03.037711081 +0000 UTC m=+72.015689111" watchObservedRunningTime="2026-09-14 02:52:03.067011116 +0000 UTC m=+72.044989145"
Sep 14 02:52:03.067440 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:03.067418    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-s8847" podStartSLOduration=1.5136593299999999 podStartE2EDuration="5.067408901s" totalImagesPullingTime="3.553749571s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:51:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:03.066079779 +0000 UTC m=+72.044057807" watchObservedRunningTime="2026-09-14 02:52:03.067408901 +0000 UTC m=+72.045386929"
Sep 14 02:52:03.083502 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:03.083463    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-n4kdw" podStartSLOduration=66.01961968 podStartE2EDuration="1m11.083450193s" totalImagesPullingTime="5.063830513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:50:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:03.082341831 +0000 UTC m=+72.060319860" watchObservedRunningTime="2026-09-14 02:52:03.083450193 +0000 UTC m=+72.061428221"
Sep 14 02:52:03.551088 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:03.551037    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1b15bf18-f4ef-4d01-9916-780ac2d581ab-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-6vkfm\" (UID: \"1b15bf18-f4ef-4d01-9916-780ac2d581ab\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6vkfm"
Sep 14 02:52:03.551605 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:03.551110    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/713ae3f6-786b-4911-92ae-854cf6c825b1-service-ca-bundle\") pod \"router-default-6d9b48664b-648rd\" (UID: \"713ae3f6-786b-4911-92ae-854cf6c825b1\") " pod="openshift-ingress/router-default-6d9b48664b-648rd"
Sep 14 02:52:03.551605 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:03.551196    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 14 02:52:03.551605 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:03.551263    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b15bf18-f4ef-4d01-9916-780ac2d581ab-networking-console-plugin-cert podName:1b15bf18-f4ef-4d01-9916-780ac2d581ab nodeName:}" failed. No retries permitted until 2026-09-14 02:52:07.551247311 +0000 UTC m=+76.529225332 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1b15bf18-f4ef-4d01-9916-780ac2d581ab-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-6vkfm" (UID: "1b15bf18-f4ef-4d01-9916-780ac2d581ab") : secret "networking-console-plugin-cert" not found
Sep 14 02:52:03.551605 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:03.551302    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/713ae3f6-786b-4911-92ae-854cf6c825b1-service-ca-bundle podName:713ae3f6-786b-4911-92ae-854cf6c825b1 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:07.551293982 +0000 UTC m=+76.529271992 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/713ae3f6-786b-4911-92ae-854cf6c825b1-service-ca-bundle") pod "router-default-6d9b48664b-648rd" (UID: "713ae3f6-786b-4911-92ae-854cf6c825b1") : configmap references non-existent config key: service-ca.crt
Sep 14 02:52:06.573493 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:06.573457    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/713ae3f6-786b-4911-92ae-854cf6c825b1-metrics-certs\") pod \"router-default-6d9b48664b-648rd\" (UID: \"713ae3f6-786b-4911-92ae-854cf6c825b1\") " pod="openshift-ingress/router-default-6d9b48664b-648rd"
Sep 14 02:52:06.573874 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:06.573602    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 14 02:52:06.573874 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:06.573662    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/713ae3f6-786b-4911-92ae-854cf6c825b1-metrics-certs podName:713ae3f6-786b-4911-92ae-854cf6c825b1 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:14.573649399 +0000 UTC m=+83.551627407 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/713ae3f6-786b-4911-92ae-854cf6c825b1-metrics-certs") pod "router-default-6d9b48664b-648rd" (UID: "713ae3f6-786b-4911-92ae-854cf6c825b1") : secret "router-metrics-certs-default" not found
Sep 14 02:52:06.674203 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:06.674174    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-registry-tls\") pod \"image-registry-7666cf9458-rqpks\" (UID: \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\") " pod="openshift-image-registry/image-registry-7666cf9458-rqpks"
Sep 14 02:52:06.674342 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:06.674283    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 02:52:06.674342 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:06.674296    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7666cf9458-rqpks: secret "image-registry-tls" not found
Sep 14 02:52:06.674342 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:06.674334    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-registry-tls podName:321a267f-4dd7-4cfb-b28f-7b2f57caf02b nodeName:}" failed. No retries permitted until 2026-09-14 02:52:14.674322522 +0000 UTC m=+83.652300528 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-registry-tls") pod "image-registry-7666cf9458-rqpks" (UID: "321a267f-4dd7-4cfb-b28f-7b2f57caf02b") : secret "image-registry-tls" not found
Sep 14 02:52:07.580156 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:07.580116    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/713ae3f6-786b-4911-92ae-854cf6c825b1-service-ca-bundle\") pod \"router-default-6d9b48664b-648rd\" (UID: \"713ae3f6-786b-4911-92ae-854cf6c825b1\") " pod="openshift-ingress/router-default-6d9b48664b-648rd"
Sep 14 02:52:07.580556 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:07.580199    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1b15bf18-f4ef-4d01-9916-780ac2d581ab-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-6vkfm\" (UID: \"1b15bf18-f4ef-4d01-9916-780ac2d581ab\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6vkfm"
Sep 14 02:52:07.580556 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:07.580309    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 14 02:52:07.580556 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:07.580311    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/713ae3f6-786b-4911-92ae-854cf6c825b1-service-ca-bundle podName:713ae3f6-786b-4911-92ae-854cf6c825b1 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:15.580292919 +0000 UTC m=+84.558270943 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/713ae3f6-786b-4911-92ae-854cf6c825b1-service-ca-bundle") pod "router-default-6d9b48664b-648rd" (UID: "713ae3f6-786b-4911-92ae-854cf6c825b1") : configmap references non-existent config key: service-ca.crt
Sep 14 02:52:07.580556 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:07.580357    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b15bf18-f4ef-4d01-9916-780ac2d581ab-networking-console-plugin-cert podName:1b15bf18-f4ef-4d01-9916-780ac2d581ab nodeName:}" failed. No retries permitted until 2026-09-14 02:52:15.580346033 +0000 UTC m=+84.558324040 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1b15bf18-f4ef-4d01-9916-780ac2d581ab-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-6vkfm" (UID: "1b15bf18-f4ef-4d01-9916-780ac2d581ab") : secret "networking-console-plugin-cert" not found
Sep 14 02:52:10.183233 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:10.183206    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-szm7r_79878e2e-9231-4688-b888-dc8db6184780/dns-node-resolver/0.log"
Sep 14 02:52:11.583143 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:11.583113    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-w8fds_af1c701f-81cb-47aa-85a1-439d4f260cea/node-ca/0.log"
Sep 14 02:52:12.382775 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:12.382753    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lvxg5_b0761e95-5448-4b3d-a607-5eebfc1afc7d/serve-healthcheck-canary/0.log"
Sep 14 02:52:14.634875 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:14.634759    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/713ae3f6-786b-4911-92ae-854cf6c825b1-metrics-certs\") pod \"router-default-6d9b48664b-648rd\" (UID: \"713ae3f6-786b-4911-92ae-854cf6c825b1\") " pod="openshift-ingress/router-default-6d9b48664b-648rd"
Sep 14 02:52:14.635223 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:14.634892    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 14 02:52:14.635223 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:14.634947    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/713ae3f6-786b-4911-92ae-854cf6c825b1-metrics-certs podName:713ae3f6-786b-4911-92ae-854cf6c825b1 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:30.634932887 +0000 UTC m=+99.612910899 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/713ae3f6-786b-4911-92ae-854cf6c825b1-metrics-certs") pod "router-default-6d9b48664b-648rd" (UID: "713ae3f6-786b-4911-92ae-854cf6c825b1") : secret "router-metrics-certs-default" not found
Sep 14 02:52:14.735748 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:14.735715    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-registry-tls\") pod \"image-registry-7666cf9458-rqpks\" (UID: \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\") " pod="openshift-image-registry/image-registry-7666cf9458-rqpks"
Sep 14 02:52:14.735864 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:14.735812    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 02:52:14.735864 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:14.735822    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7666cf9458-rqpks: secret "image-registry-tls" not found
Sep 14 02:52:14.735864 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:14.735860    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-registry-tls podName:321a267f-4dd7-4cfb-b28f-7b2f57caf02b nodeName:}" failed. No retries permitted until 2026-09-14 02:52:30.735849753 +0000 UTC m=+99.713827760 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-registry-tls") pod "image-registry-7666cf9458-rqpks" (UID: "321a267f-4dd7-4cfb-b28f-7b2f57caf02b") : secret "image-registry-tls" not found
Sep 14 02:52:15.642306 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:15.642283    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1b15bf18-f4ef-4d01-9916-780ac2d581ab-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-6vkfm\" (UID: \"1b15bf18-f4ef-4d01-9916-780ac2d581ab\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6vkfm"
Sep 14 02:52:15.642625 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:15.642329    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/713ae3f6-786b-4911-92ae-854cf6c825b1-service-ca-bundle\") pod \"router-default-6d9b48664b-648rd\" (UID: \"713ae3f6-786b-4911-92ae-854cf6c825b1\") " pod="openshift-ingress/router-default-6d9b48664b-648rd"
Sep 14 02:52:15.642625 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:15.642420    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 14 02:52:15.642625 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:15.642446    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/713ae3f6-786b-4911-92ae-854cf6c825b1-service-ca-bundle podName:713ae3f6-786b-4911-92ae-854cf6c825b1 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:31.642431763 +0000 UTC m=+100.620409773 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/713ae3f6-786b-4911-92ae-854cf6c825b1-service-ca-bundle") pod "router-default-6d9b48664b-648rd" (UID: "713ae3f6-786b-4911-92ae-854cf6c825b1") : configmap references non-existent config key: service-ca.crt
Sep 14 02:52:15.642625 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:15.642475    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b15bf18-f4ef-4d01-9916-780ac2d581ab-networking-console-plugin-cert podName:1b15bf18-f4ef-4d01-9916-780ac2d581ab nodeName:}" failed. No retries permitted until 2026-09-14 02:52:31.642463919 +0000 UTC m=+100.620441926 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1b15bf18-f4ef-4d01-9916-780ac2d581ab-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-6vkfm" (UID: "1b15bf18-f4ef-4d01-9916-780ac2d581ab") : secret "networking-console-plugin-cert" not found
Sep 14 02:52:26.078680 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.078633    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-bqblc"]
Sep 14 02:52:26.082876 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.082859    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-79f577896c-vjgd2"]
Sep 14 02:52:26.082996 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.082977    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bqblc"
Sep 14 02:52:26.085593 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.085570    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79f577896c-vjgd2"
Sep 14 02:52:26.086337 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.086321    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pmfgr\""
Sep 14 02:52:26.086814 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.086799    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 14 02:52:26.086978 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.086961    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 14 02:52:26.087097 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.087081    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 14 02:52:26.088505 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.088485    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 14 02:52:26.088635 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.088611    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 14 02:52:26.088867 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.088846    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 14 02:52:26.088955 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.088944    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 14 02:52:26.090738 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.090722    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 14 02:52:26.102011 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.101988    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bqblc"]
Sep 14 02:52:26.129539 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.129515    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-79f577896c-vjgd2"]
Sep 14 02:52:26.146520 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.146499    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7666cf9458-rqpks"]
Sep 14 02:52:26.146679 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:26.146662    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-7666cf9458-rqpks" podUID="321a267f-4dd7-4cfb-b28f-7b2f57caf02b"
Sep 14 02:52:26.216306 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.216284    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8a44bfd9-c93d-4366-a958-bd8ba5d2d97e-klusterlet-config\") pod \"klusterlet-addon-workmgr-79f577896c-vjgd2\" (UID: \"8a44bfd9-c93d-4366-a958-bd8ba5d2d97e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79f577896c-vjgd2"
Sep 14 02:52:26.216403 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.216313    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55mrn\" (UniqueName: \"kubernetes.io/projected/8a44bfd9-c93d-4366-a958-bd8ba5d2d97e-kube-api-access-55mrn\") pod \"klusterlet-addon-workmgr-79f577896c-vjgd2\" (UID: \"8a44bfd9-c93d-4366-a958-bd8ba5d2d97e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79f577896c-vjgd2"
Sep 14 02:52:26.216403 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.216336    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a44bfd9-c93d-4366-a958-bd8ba5d2d97e-tmp\") pod \"klusterlet-addon-workmgr-79f577896c-vjgd2\" (UID: \"8a44bfd9-c93d-4366-a958-bd8ba5d2d97e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79f577896c-vjgd2"
Sep 14 02:52:26.216403 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.216354    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rl7ll\" (UniqueName: \"kubernetes.io/projected/7484daa2-029b-4488-b2aa-1123a49b3fd8-kube-api-access-rl7ll\") pod \"insights-runtime-extractor-bqblc\" (UID: \"7484daa2-029b-4488-b2aa-1123a49b3fd8\") " pod="openshift-insights/insights-runtime-extractor-bqblc"
Sep 14 02:52:26.216520 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.216414    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7484daa2-029b-4488-b2aa-1123a49b3fd8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bqblc\" (UID: \"7484daa2-029b-4488-b2aa-1123a49b3fd8\") " pod="openshift-insights/insights-runtime-extractor-bqblc"
Sep 14 02:52:26.216520 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.216509    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7484daa2-029b-4488-b2aa-1123a49b3fd8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bqblc\" (UID: \"7484daa2-029b-4488-b2aa-1123a49b3fd8\") " pod="openshift-insights/insights-runtime-extractor-bqblc"
Sep 14 02:52:26.216591 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.216574    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7484daa2-029b-4488-b2aa-1123a49b3fd8-crio-socket\") pod \"insights-runtime-extractor-bqblc\" (UID: \"7484daa2-029b-4488-b2aa-1123a49b3fd8\") " pod="openshift-insights/insights-runtime-extractor-bqblc"
Sep 14 02:52:26.216635 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.216602    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7484daa2-029b-4488-b2aa-1123a49b3fd8-data-volume\") pod \"insights-runtime-extractor-bqblc\" (UID: \"7484daa2-029b-4488-b2aa-1123a49b3fd8\") " pod="openshift-insights/insights-runtime-extractor-bqblc"
Sep 14 02:52:26.253350 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.253327    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-57d69d9c4c-rvt8r"]
Sep 14 02:52:26.256060 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.256044    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57d69d9c4c-rvt8r"
Sep 14 02:52:26.288366 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.288348    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-57d69d9c4c-rvt8r"]
Sep 14 02:52:26.317606 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.317582    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7484daa2-029b-4488-b2aa-1123a49b3fd8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bqblc\" (UID: \"7484daa2-029b-4488-b2aa-1123a49b3fd8\") " pod="openshift-insights/insights-runtime-extractor-bqblc"
Sep 14 02:52:26.317701 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.317628    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/673eb2ed-da11-47d3-b059-228075d65381-ca-trust-extracted\") pod \"image-registry-57d69d9c4c-rvt8r\" (UID: \"673eb2ed-da11-47d3-b059-228075d65381\") " pod="openshift-image-registry/image-registry-57d69d9c4c-rvt8r"
Sep 14 02:52:26.317701 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.317645    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/673eb2ed-da11-47d3-b059-228075d65381-registry-tls\") pod \"image-registry-57d69d9c4c-rvt8r\" (UID: \"673eb2ed-da11-47d3-b059-228075d65381\") " pod="openshift-image-registry/image-registry-57d69d9c4c-rvt8r"
Sep 14 02:52:26.317795 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.317698    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7484daa2-029b-4488-b2aa-1123a49b3fd8-crio-socket\") pod \"insights-runtime-extractor-bqblc\" (UID: \"7484daa2-029b-4488-b2aa-1123a49b3fd8\") " pod="openshift-insights/insights-runtime-extractor-bqblc"
Sep 14 02:52:26.317795 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.317788    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7484daa2-029b-4488-b2aa-1123a49b3fd8-data-volume\") pod \"insights-runtime-extractor-bqblc\" (UID: \"7484daa2-029b-4488-b2aa-1123a49b3fd8\") " pod="openshift-insights/insights-runtime-extractor-bqblc"
Sep 14 02:52:26.317870 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.317795    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7484daa2-029b-4488-b2aa-1123a49b3fd8-crio-socket\") pod \"insights-runtime-extractor-bqblc\" (UID: \"7484daa2-029b-4488-b2aa-1123a49b3fd8\") " pod="openshift-insights/insights-runtime-extractor-bqblc"
Sep 14 02:52:26.317870 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.317853    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/673eb2ed-da11-47d3-b059-228075d65381-image-registry-private-configuration\") pod \"image-registry-57d69d9c4c-rvt8r\" (UID: \"673eb2ed-da11-47d3-b059-228075d65381\") " pod="openshift-image-registry/image-registry-57d69d9c4c-rvt8r"
Sep 14 02:52:26.317944 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.317889    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/673eb2ed-da11-47d3-b059-228075d65381-registry-certificates\") pod \"image-registry-57d69d9c4c-rvt8r\" (UID: \"673eb2ed-da11-47d3-b059-228075d65381\") " pod="openshift-image-registry/image-registry-57d69d9c4c-rvt8r"
Sep 14 02:52:26.317944 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.317907    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8a44bfd9-c93d-4366-a958-bd8ba5d2d97e-klusterlet-config\") pod \"klusterlet-addon-workmgr-79f577896c-vjgd2\" (UID: \"8a44bfd9-c93d-4366-a958-bd8ba5d2d97e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79f577896c-vjgd2"
Sep 14 02:52:26.317944 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.317928    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55mrn\" (UniqueName: \"kubernetes.io/projected/8a44bfd9-c93d-4366-a958-bd8ba5d2d97e-kube-api-access-55mrn\") pod \"klusterlet-addon-workmgr-79f577896c-vjgd2\" (UID: \"8a44bfd9-c93d-4366-a958-bd8ba5d2d97e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79f577896c-vjgd2"
Sep 14 02:52:26.318066 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.317977    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7484daa2-029b-4488-b2aa-1123a49b3fd8-data-volume\") pod \"insights-runtime-extractor-bqblc\" (UID: \"7484daa2-029b-4488-b2aa-1123a49b3fd8\") " pod="openshift-insights/insights-runtime-extractor-bqblc"
Sep 14 02:52:26.318066 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.317975    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/673eb2ed-da11-47d3-b059-228075d65381-installation-pull-secrets\") pod \"image-registry-57d69d9c4c-rvt8r\" (UID: \"673eb2ed-da11-47d3-b059-228075d65381\") " pod="openshift-image-registry/image-registry-57d69d9c4c-rvt8r"
Sep 14 02:52:26.318066 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.318018    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a44bfd9-c93d-4366-a958-bd8ba5d2d97e-tmp\") pod \"klusterlet-addon-workmgr-79f577896c-vjgd2\" (UID: \"8a44bfd9-c93d-4366-a958-bd8ba5d2d97e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79f577896c-vjgd2"
Sep 14 02:52:26.318066 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.318042    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl7ll\" (UniqueName: \"kubernetes.io/projected/7484daa2-029b-4488-b2aa-1123a49b3fd8-kube-api-access-rl7ll\") pod \"insights-runtime-extractor-bqblc\" (UID: \"7484daa2-029b-4488-b2aa-1123a49b3fd8\") " pod="openshift-insights/insights-runtime-extractor-bqblc"
Sep 14 02:52:26.318066 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.318063    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7484daa2-029b-4488-b2aa-1123a49b3fd8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bqblc\" (UID: \"7484daa2-029b-4488-b2aa-1123a49b3fd8\") " pod="openshift-insights/insights-runtime-extractor-bqblc"
Sep 14 02:52:26.318314 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.318081    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqkzf\" (UniqueName: \"kubernetes.io/projected/673eb2ed-da11-47d3-b059-228075d65381-kube-api-access-cqkzf\") pod \"image-registry-57d69d9c4c-rvt8r\" (UID: \"673eb2ed-da11-47d3-b059-228075d65381\") " pod="openshift-image-registry/image-registry-57d69d9c4c-rvt8r"
Sep 14 02:52:26.318314 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.318101    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/673eb2ed-da11-47d3-b059-228075d65381-bound-sa-token\") pod \"image-registry-57d69d9c4c-rvt8r\" (UID: \"673eb2ed-da11-47d3-b059-228075d65381\") " pod="openshift-image-registry/image-registry-57d69d9c4c-rvt8r"
Sep 14 02:52:26.318314 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.318124    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/673eb2ed-da11-47d3-b059-228075d65381-trusted-ca\") pod \"image-registry-57d69d9c4c-rvt8r\" (UID: \"673eb2ed-da11-47d3-b059-228075d65381\") " pod="openshift-image-registry/image-registry-57d69d9c4c-rvt8r"
Sep 14 02:52:26.318314 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.318042    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7484daa2-029b-4488-b2aa-1123a49b3fd8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bqblc\" (UID: \"7484daa2-029b-4488-b2aa-1123a49b3fd8\") " pod="openshift-insights/insights-runtime-extractor-bqblc"
Sep 14 02:52:26.318501 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.318404    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a44bfd9-c93d-4366-a958-bd8ba5d2d97e-tmp\") pod \"klusterlet-addon-workmgr-79f577896c-vjgd2\" (UID: \"8a44bfd9-c93d-4366-a958-bd8ba5d2d97e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79f577896c-vjgd2"
Sep 14 02:52:26.320321 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.320301    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7484daa2-029b-4488-b2aa-1123a49b3fd8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bqblc\" (UID: \"7484daa2-029b-4488-b2aa-1123a49b3fd8\") " pod="openshift-insights/insights-runtime-extractor-bqblc"
Sep 14 02:52:26.320588 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.320568    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8a44bfd9-c93d-4366-a958-bd8ba5d2d97e-klusterlet-config\") pod \"klusterlet-addon-workmgr-79f577896c-vjgd2\" (UID: \"8a44bfd9-c93d-4366-a958-bd8ba5d2d97e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79f577896c-vjgd2"
Sep 14 02:52:26.327615 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.327591    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55mrn\" (UniqueName: \"kubernetes.io/projected/8a44bfd9-c93d-4366-a958-bd8ba5d2d97e-kube-api-access-55mrn\") pod \"klusterlet-addon-workmgr-79f577896c-vjgd2\" (UID: \"8a44bfd9-c93d-4366-a958-bd8ba5d2d97e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79f577896c-vjgd2"
Sep 14 02:52:26.327755 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.327736    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rl7ll\" (UniqueName: \"kubernetes.io/projected/7484daa2-029b-4488-b2aa-1123a49b3fd8-kube-api-access-rl7ll\") pod \"insights-runtime-extractor-bqblc\" (UID: \"7484daa2-029b-4488-b2aa-1123a49b3fd8\") " pod="openshift-insights/insights-runtime-extractor-bqblc"
Sep 14 02:52:26.392595 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.392547    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bqblc"
Sep 14 02:52:26.398315 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.398292    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79f577896c-vjgd2"
Sep 14 02:52:26.418868 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.418841    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/673eb2ed-da11-47d3-b059-228075d65381-image-registry-private-configuration\") pod \"image-registry-57d69d9c4c-rvt8r\" (UID: \"673eb2ed-da11-47d3-b059-228075d65381\") " pod="openshift-image-registry/image-registry-57d69d9c4c-rvt8r"
Sep 14 02:52:26.418971 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.418890    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/673eb2ed-da11-47d3-b059-228075d65381-registry-certificates\") pod \"image-registry-57d69d9c4c-rvt8r\" (UID: \"673eb2ed-da11-47d3-b059-228075d65381\") " pod="openshift-image-registry/image-registry-57d69d9c4c-rvt8r"
Sep 14 02:52:26.418971 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.418921    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/673eb2ed-da11-47d3-b059-228075d65381-installation-pull-secrets\") pod \"image-registry-57d69d9c4c-rvt8r\" (UID: \"673eb2ed-da11-47d3-b059-228075d65381\") " pod="openshift-image-registry/image-registry-57d69d9c4c-rvt8r"
Sep 14 02:52:26.418971 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.418958    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqkzf\" (UniqueName: \"kubernetes.io/projected/673eb2ed-da11-47d3-b059-228075d65381-kube-api-access-cqkzf\") pod \"image-registry-57d69d9c4c-rvt8r\" (UID: \"673eb2ed-da11-47d3-b059-228075d65381\") " pod="openshift-image-registry/image-registry-57d69d9c4c-rvt8r"
Sep 14 02:52:26.419122 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.418985    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/673eb2ed-da11-47d3-b059-228075d65381-bound-sa-token\") pod \"image-registry-57d69d9c4c-rvt8r\" (UID: \"673eb2ed-da11-47d3-b059-228075d65381\") " pod="openshift-image-registry/image-registry-57d69d9c4c-rvt8r"
Sep 14 02:52:26.419122 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.419011    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/673eb2ed-da11-47d3-b059-228075d65381-trusted-ca\") pod \"image-registry-57d69d9c4c-rvt8r\" (UID: \"673eb2ed-da11-47d3-b059-228075d65381\") " pod="openshift-image-registry/image-registry-57d69d9c4c-rvt8r"
Sep 14 02:52:26.419604 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.419421    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/673eb2ed-da11-47d3-b059-228075d65381-ca-trust-extracted\") pod \"image-registry-57d69d9c4c-rvt8r\" (UID: \"673eb2ed-da11-47d3-b059-228075d65381\") " pod="openshift-image-registry/image-registry-57d69d9c4c-rvt8r"
Sep 14 02:52:26.419604 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.419462    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/673eb2ed-da11-47d3-b059-228075d65381-registry-tls\") pod \"image-registry-57d69d9c4c-rvt8r\" (UID: \"673eb2ed-da11-47d3-b059-228075d65381\") " pod="openshift-image-registry/image-registry-57d69d9c4c-rvt8r"
Sep 14 02:52:26.419752 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.419650    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/673eb2ed-da11-47d3-b059-228075d65381-registry-certificates\") pod \"image-registry-57d69d9c4c-rvt8r\" (UID: \"673eb2ed-da11-47d3-b059-228075d65381\") " pod="openshift-image-registry/image-registry-57d69d9c4c-rvt8r"
Sep 14 02:52:26.419883 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.419859    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/673eb2ed-da11-47d3-b059-228075d65381-ca-trust-extracted\") pod \"image-registry-57d69d9c4c-rvt8r\" (UID: \"673eb2ed-da11-47d3-b059-228075d65381\") " pod="openshift-image-registry/image-registry-57d69d9c4c-rvt8r"
Sep 14 02:52:26.420618 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.420586    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/673eb2ed-da11-47d3-b059-228075d65381-trusted-ca\") pod \"image-registry-57d69d9c4c-rvt8r\" (UID: \"673eb2ed-da11-47d3-b059-228075d65381\") " pod="openshift-image-registry/image-registry-57d69d9c4c-rvt8r"
Sep 14 02:52:26.421921 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.421882    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/673eb2ed-da11-47d3-b059-228075d65381-registry-tls\") pod \"image-registry-57d69d9c4c-rvt8r\" (UID: \"673eb2ed-da11-47d3-b059-228075d65381\") " pod="openshift-image-registry/image-registry-57d69d9c4c-rvt8r"
Sep 14 02:52:26.422187 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.422167    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/673eb2ed-da11-47d3-b059-228075d65381-image-registry-private-configuration\") pod \"image-registry-57d69d9c4c-rvt8r\" (UID: \"673eb2ed-da11-47d3-b059-228075d65381\") " pod="openshift-image-registry/image-registry-57d69d9c4c-rvt8r"
Sep 14 02:52:26.422716 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.422696    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/673eb2ed-da11-47d3-b059-228075d65381-installation-pull-secrets\") pod \"image-registry-57d69d9c4c-rvt8r\" (UID: \"673eb2ed-da11-47d3-b059-228075d65381\") " pod="openshift-image-registry/image-registry-57d69d9c4c-rvt8r"
Sep 14 02:52:26.429905 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.429871    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/673eb2ed-da11-47d3-b059-228075d65381-bound-sa-token\") pod \"image-registry-57d69d9c4c-rvt8r\" (UID: \"673eb2ed-da11-47d3-b059-228075d65381\") " pod="openshift-image-registry/image-registry-57d69d9c4c-rvt8r"
Sep 14 02:52:26.430079 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.430060    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqkzf\" (UniqueName: \"kubernetes.io/projected/673eb2ed-da11-47d3-b059-228075d65381-kube-api-access-cqkzf\") pod \"image-registry-57d69d9c4c-rvt8r\" (UID: \"673eb2ed-da11-47d3-b059-228075d65381\") " pod="openshift-image-registry/image-registry-57d69d9c4c-rvt8r"
Sep 14 02:52:26.563968 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.563945    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57d69d9c4c-rvt8r"
Sep 14 02:52:26.695024 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.694968    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bqblc"]
Sep 14 02:52:26.703411 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:52:26.703382    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7484daa2_029b_4488_b2aa_1123a49b3fd8.slice/crio-65a363065ed54a055d35ec1a5d1016900b5aaa1dea09f35c059249bd0b30eadf WatchSource:0}: Error finding container 65a363065ed54a055d35ec1a5d1016900b5aaa1dea09f35c059249bd0b30eadf: Status 404 returned error can't find the container with id 65a363065ed54a055d35ec1a5d1016900b5aaa1dea09f35c059249bd0b30eadf
Sep 14 02:52:26.703575 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.703557    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-79f577896c-vjgd2"]
Sep 14 02:52:26.706062 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:52:26.705996    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a44bfd9_c93d_4366_a958_bd8ba5d2d97e.slice/crio-86b1a9ce4ac62cf33f05ac95847ff3e7221bd99a38d6db7362625a51a33d4852 WatchSource:0}: Error finding container 86b1a9ce4ac62cf33f05ac95847ff3e7221bd99a38d6db7362625a51a33d4852: Status 404 returned error can't find the container with id 86b1a9ce4ac62cf33f05ac95847ff3e7221bd99a38d6db7362625a51a33d4852
Sep 14 02:52:26.836852 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:26.836822    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-57d69d9c4c-rvt8r"]
Sep 14 02:52:26.840799 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:52:26.840773    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod673eb2ed_da11_47d3_b059_228075d65381.slice/crio-06b5f230eba9f9a2b98ba4b3a2712edf508cab491b85984924eff2f61a4266aa WatchSource:0}: Error finding container 06b5f230eba9f9a2b98ba4b3a2712edf508cab491b85984924eff2f61a4266aa: Status 404 returned error can't find the container with id 06b5f230eba9f9a2b98ba4b3a2712edf508cab491b85984924eff2f61a4266aa
Sep 14 02:52:27.079714 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:27.079684    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bqblc" event={"ID":"7484daa2-029b-4488-b2aa-1123a49b3fd8","Type":"ContainerStarted","Data":"e5d90f662e3bdf2e28801a49ac9b3c4dba52e555cc9e33ecddee097b3723e70c"}
Sep 14 02:52:27.080129 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:27.079744    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bqblc" event={"ID":"7484daa2-029b-4488-b2aa-1123a49b3fd8","Type":"ContainerStarted","Data":"65a363065ed54a055d35ec1a5d1016900b5aaa1dea09f35c059249bd0b30eadf"}
Sep 14 02:52:27.081425 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:27.081401    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-57d69d9c4c-rvt8r" event={"ID":"673eb2ed-da11-47d3-b059-228075d65381","Type":"ContainerStarted","Data":"809a7830158b050984ce6bf3c57fa8d9c4dc00919e629b698d9b91f66dd118c7"}
Sep 14 02:52:27.081555 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:27.081439    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-57d69d9c4c-rvt8r" event={"ID":"673eb2ed-da11-47d3-b059-228075d65381","Type":"ContainerStarted","Data":"06b5f230eba9f9a2b98ba4b3a2712edf508cab491b85984924eff2f61a4266aa"}
Sep 14 02:52:27.081555 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:27.081484    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-57d69d9c4c-rvt8r"
Sep 14 02:52:27.082625 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:27.082605    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79f577896c-vjgd2" event={"ID":"8a44bfd9-c93d-4366-a958-bd8ba5d2d97e","Type":"ContainerStarted","Data":"86b1a9ce4ac62cf33f05ac95847ff3e7221bd99a38d6db7362625a51a33d4852"}
Sep 14 02:52:27.082737 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:27.082632    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7666cf9458-rqpks"
Sep 14 02:52:27.088761 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:27.088745    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7666cf9458-rqpks"
Sep 14 02:52:27.101427 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:27.101371    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-57d69d9c4c-rvt8r" podStartSLOduration=1.10135775 podStartE2EDuration="1.10135775s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:52:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:27.100195587 +0000 UTC m=+96.078173610" watchObservedRunningTime="2026-09-14 02:52:27.10135775 +0000 UTC m=+96.079335794"
Sep 14 02:52:27.225545 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:27.225506    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-registry-certificates\") pod \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\" (UID: \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\") "
Sep 14 02:52:27.225545 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:27.225544    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qcv7f\" (UniqueName: \"kubernetes.io/projected/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-kube-api-access-qcv7f\") pod \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\" (UID: \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\") "
Sep 14 02:52:27.225767 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:27.225567    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-trusted-ca\") pod \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\" (UID: \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\") "
Sep 14 02:52:27.225767 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:27.225678    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-image-registry-private-configuration\") pod \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\" (UID: \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\") "
Sep 14 02:52:27.225767 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:27.225715    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-ca-trust-extracted\") pod \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\" (UID: \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\") "
Sep 14 02:52:27.225767 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:27.225747    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-installation-pull-secrets\") pod \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\" (UID: \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\") "
Sep 14 02:52:27.225971 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:27.225808    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-bound-sa-token\") pod \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\" (UID: \"321a267f-4dd7-4cfb-b28f-7b2f57caf02b\") "
Sep 14 02:52:27.225971 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:27.225856    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "321a267f-4dd7-4cfb-b28f-7b2f57caf02b" (UID: "321a267f-4dd7-4cfb-b28f-7b2f57caf02b"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 02:52:27.225971 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:27.225922    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "321a267f-4dd7-4cfb-b28f-7b2f57caf02b" (UID: "321a267f-4dd7-4cfb-b28f-7b2f57caf02b"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:52:27.226122 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:27.226111    2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-ca-trust-extracted\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:52:27.226175 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:27.226120    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "321a267f-4dd7-4cfb-b28f-7b2f57caf02b" (UID: "321a267f-4dd7-4cfb-b28f-7b2f57caf02b"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 02:52:27.226175 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:27.226130    2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-registry-certificates\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:52:27.227951 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:27.227927    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "321a267f-4dd7-4cfb-b28f-7b2f57caf02b" (UID: "321a267f-4dd7-4cfb-b28f-7b2f57caf02b"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 02:52:27.228171 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:27.228147    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "321a267f-4dd7-4cfb-b28f-7b2f57caf02b" (UID: "321a267f-4dd7-4cfb-b28f-7b2f57caf02b"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 02:52:27.228171 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:27.228144    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "321a267f-4dd7-4cfb-b28f-7b2f57caf02b" (UID: "321a267f-4dd7-4cfb-b28f-7b2f57caf02b"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:52:27.228327 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:27.228198    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-kube-api-access-qcv7f" (OuterVolumeSpecName: "kube-api-access-qcv7f") pod "321a267f-4dd7-4cfb-b28f-7b2f57caf02b" (UID: "321a267f-4dd7-4cfb-b28f-7b2f57caf02b"). InnerVolumeSpecName "kube-api-access-qcv7f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:52:27.327279 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:27.327245    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qcv7f\" (UniqueName: \"kubernetes.io/projected/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-kube-api-access-qcv7f\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:52:27.327412 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:27.327287    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-trusted-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:52:27.327412 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:27.327299    2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-image-registry-private-configuration\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:52:27.327412 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:27.327309    2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-installation-pull-secrets\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:52:27.327412 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:27.327319    2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-bound-sa-token\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:52:28.089092 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:28.089056    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bqblc" event={"ID":"7484daa2-029b-4488-b2aa-1123a49b3fd8","Type":"ContainerStarted","Data":"a789f1f6363807dd5d2180801b1ff5dee8fd39dbbd200bf1e9fab2337805e2b5"}
Sep 14 02:52:28.089577 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:28.089477    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7666cf9458-rqpks"
Sep 14 02:52:28.123901 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:28.123876    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7666cf9458-rqpks"]
Sep 14 02:52:28.127614 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:28.127590    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7666cf9458-rqpks"]
Sep 14 02:52:28.233143 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:28.233112    2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/321a267f-4dd7-4cfb-b28f-7b2f57caf02b-registry-tls\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:52:29.616503 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:29.616465    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="321a267f-4dd7-4cfb-b28f-7b2f57caf02b" path="/var/lib/kubelet/pods/321a267f-4dd7-4cfb-b28f-7b2f57caf02b/volumes"
Sep 14 02:52:29.948012 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:29.947989    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/df25d2a1-9825-4117-a68b-bdb22001d3d6-metrics-tls\") pod \"dns-default-v5tc4\" (UID: \"df25d2a1-9825-4117-a68b-bdb22001d3d6\") " pod="openshift-dns/dns-default-v5tc4"
Sep 14 02:52:29.950411 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:29.950384    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/df25d2a1-9825-4117-a68b-bdb22001d3d6-metrics-tls\") pod \"dns-default-v5tc4\" (UID: \"df25d2a1-9825-4117-a68b-bdb22001d3d6\") " pod="openshift-dns/dns-default-v5tc4"
Sep 14 02:52:30.098544 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:30.098509    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bqblc" event={"ID":"7484daa2-029b-4488-b2aa-1123a49b3fd8","Type":"ContainerStarted","Data":"0090324e04ab0d2f8bcb555606075cdf96485215100891338609fcca02a61396"}
Sep 14 02:52:30.115624 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:30.115571    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-bqblc" podStartSLOduration=1.082023079 podStartE2EDuration="4.115553438s" totalImagesPullingTime="3.033530359s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 02:52:26 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:30.114852559 +0000 UTC m=+99.092830590" watchObservedRunningTime="2026-09-14 02:52:30.115553438 +0000 UTC m=+99.093531469"
Sep 14 02:52:30.138218 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:30.138161    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-nzbvq\""
Sep 14 02:52:30.146287 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:30.146251    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-v5tc4"
Sep 14 02:52:30.654284 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:30.654241    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/713ae3f6-786b-4911-92ae-854cf6c825b1-metrics-certs\") pod \"router-default-6d9b48664b-648rd\" (UID: \"713ae3f6-786b-4911-92ae-854cf6c825b1\") " pod="openshift-ingress/router-default-6d9b48664b-648rd"
Sep 14 02:52:30.656976 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:30.656953    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/713ae3f6-786b-4911-92ae-854cf6c825b1-metrics-certs\") pod \"router-default-6d9b48664b-648rd\" (UID: \"713ae3f6-786b-4911-92ae-854cf6c825b1\") " pod="openshift-ingress/router-default-6d9b48664b-648rd"
Sep 14 02:52:31.388067 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:31.388009    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-v5tc4"]
Sep 14 02:52:31.390645 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:52:31.390604    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf25d2a1_9825_4117_a68b_bdb22001d3d6.slice/crio-c386931482a5c3c98a0931caad02e54a57c9621a7ff607f922fb60832614de70 WatchSource:0}: Error finding container c386931482a5c3c98a0931caad02e54a57c9621a7ff607f922fb60832614de70: Status 404 returned error can't find the container with id c386931482a5c3c98a0931caad02e54a57c9621a7ff607f922fb60832614de70
Sep 14 02:52:31.661508 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:31.661454    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1b15bf18-f4ef-4d01-9916-780ac2d581ab-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-6vkfm\" (UID: \"1b15bf18-f4ef-4d01-9916-780ac2d581ab\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6vkfm"
Sep 14 02:52:31.661508 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:31.661488    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/713ae3f6-786b-4911-92ae-854cf6c825b1-service-ca-bundle\") pod \"router-default-6d9b48664b-648rd\" (UID: \"713ae3f6-786b-4911-92ae-854cf6c825b1\") " pod="openshift-ingress/router-default-6d9b48664b-648rd"
Sep 14 02:52:31.662046 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:31.662031    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/713ae3f6-786b-4911-92ae-854cf6c825b1-service-ca-bundle\") pod \"router-default-6d9b48664b-648rd\" (UID: \"713ae3f6-786b-4911-92ae-854cf6c825b1\") " pod="openshift-ingress/router-default-6d9b48664b-648rd"
Sep 14 02:52:31.663965 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:31.663947    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1b15bf18-f4ef-4d01-9916-780ac2d581ab-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-6vkfm\" (UID: \"1b15bf18-f4ef-4d01-9916-780ac2d581ab\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6vkfm"
Sep 14 02:52:31.882563 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:31.882535    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6vkfm"
Sep 14 02:52:31.890464 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:31.890430    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6d9b48664b-648rd"
Sep 14 02:52:32.106985 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:32.106774    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79f577896c-vjgd2" event={"ID":"8a44bfd9-c93d-4366-a958-bd8ba5d2d97e","Type":"ContainerStarted","Data":"0d80c776c399035ccd905ddbabbb67efeaa7648f015667ec28aa4a781490a110"}
Sep 14 02:52:32.106985 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:32.106949    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79f577896c-vjgd2"
Sep 14 02:52:32.108188 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:32.108158    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v5tc4" event={"ID":"df25d2a1-9825-4117-a68b-bdb22001d3d6","Type":"ContainerStarted","Data":"c386931482a5c3c98a0931caad02e54a57c9621a7ff607f922fb60832614de70"}
Sep 14 02:52:32.108582 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:32.108561    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79f577896c-vjgd2"
Sep 14 02:52:32.129286 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:32.129227    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79f577896c-vjgd2" podStartSLOduration=1.628064768 podStartE2EDuration="6.129208692s" totalImagesPullingTime="4.501143924s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:52:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:32.128871166 +0000 UTC m=+101.106849195" watchObservedRunningTime="2026-09-14 02:52:32.129208692 +0000 UTC m=+101.107186722"
Sep 14 02:52:32.172506 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:32.172467    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-6vkfm"]
Sep 14 02:52:32.174684 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:52:32.174658    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b15bf18_f4ef_4d01_9916_780ac2d581ab.slice/crio-0a2fc395cd64aa58c8b5c5319f68dbccde5e348b1bef542a2eb1cf78cc712e8c WatchSource:0}: Error finding container 0a2fc395cd64aa58c8b5c5319f68dbccde5e348b1bef542a2eb1cf78cc712e8c: Status 404 returned error can't find the container with id 0a2fc395cd64aa58c8b5c5319f68dbccde5e348b1bef542a2eb1cf78cc712e8c
Sep 14 02:52:32.181228 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:32.178897    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6d9b48664b-648rd"]
Sep 14 02:52:32.182852 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:52:32.182750    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod713ae3f6_786b_4911_92ae_854cf6c825b1.slice/crio-e06045594dc93c5e9a0cd545b2b982a35ea080d745c592ab3a0ba0a5031a0fa9 WatchSource:0}: Error finding container e06045594dc93c5e9a0cd545b2b982a35ea080d745c592ab3a0ba0a5031a0fa9: Status 404 returned error can't find the container with id e06045594dc93c5e9a0cd545b2b982a35ea080d745c592ab3a0ba0a5031a0fa9
Sep 14 02:52:33.112075 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:33.112038    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6d9b48664b-648rd" event={"ID":"713ae3f6-786b-4911-92ae-854cf6c825b1","Type":"ContainerStarted","Data":"6324b94976eec66f86da08611ed97d0f58ff681194818c392bf894044babed2c"}
Sep 14 02:52:33.112494 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:33.112096    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6d9b48664b-648rd" event={"ID":"713ae3f6-786b-4911-92ae-854cf6c825b1","Type":"ContainerStarted","Data":"e06045594dc93c5e9a0cd545b2b982a35ea080d745c592ab3a0ba0a5031a0fa9"}
Sep 14 02:52:33.113205 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:33.113178    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6vkfm" event={"ID":"1b15bf18-f4ef-4d01-9916-780ac2d581ab","Type":"ContainerStarted","Data":"0a2fc395cd64aa58c8b5c5319f68dbccde5e348b1bef542a2eb1cf78cc712e8c"}
Sep 14 02:52:33.134801 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:33.134752    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6d9b48664b-648rd" podStartSLOduration=35.134736617 podStartE2EDuration="35.134736617s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:51:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:33.133770312 +0000 UTC m=+102.111748343" watchObservedRunningTime="2026-09-14 02:52:33.134736617 +0000 UTC m=+102.112714645"
Sep 14 02:52:33.891395 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:33.891359    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6d9b48664b-648rd"
Sep 14 02:52:33.894314 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:33.894287    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6d9b48664b-648rd"
Sep 14 02:52:34.024576 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:34.024553    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-n4kdw"
Sep 14 02:52:34.120545 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:34.120516    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v5tc4" event={"ID":"df25d2a1-9825-4117-a68b-bdb22001d3d6","Type":"ContainerStarted","Data":"7756cbc3a1c352b4860bd226b7e69520a7a879fc4132b064186385d240d6334b"}
Sep 14 02:52:34.120928 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:34.120573    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v5tc4" event={"ID":"df25d2a1-9825-4117-a68b-bdb22001d3d6","Type":"ContainerStarted","Data":"872164a607b1e61c245d8d62f58657bd12cf821b217b6875d94895d303f3ff91"}
Sep 14 02:52:34.120928 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:34.120605    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-v5tc4"
Sep 14 02:52:34.121942 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:34.121920    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6vkfm" event={"ID":"1b15bf18-f4ef-4d01-9916-780ac2d581ab","Type":"ContainerStarted","Data":"3a897925d8806170373c80a4a6c1704abadff4e33156b1b1c75ad0231083ba7d"}
Sep 14 02:52:34.122206 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:34.122186    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6d9b48664b-648rd"
Sep 14 02:52:34.123328 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:34.123311    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6d9b48664b-648rd"
Sep 14 02:52:34.146234 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:34.146166    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-v5tc4" podStartSLOduration=67.392068656 podStartE2EDuration="1m9.146148638s" totalImagesPullingTime="1.754079982s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:51:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:34.14603132 +0000 UTC m=+103.124009349" watchObservedRunningTime="2026-09-14 02:52:34.146148638 +0000 UTC m=+103.124126662"
Sep 14 02:52:34.196366 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:34.196321    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-6vkfm" podStartSLOduration=34.388546973 podStartE2EDuration="36.196307782s" totalImagesPullingTime="1.807760809s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:51:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:34.195531072 +0000 UTC m=+103.173509100" watchObservedRunningTime="2026-09-14 02:52:34.196307782 +0000 UTC m=+103.174285814"
Sep 14 02:52:34.371046 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:34.371019    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-t2l4g"]
Sep 14 02:52:34.392338 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:34.392316    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-t2l4g"]
Sep 14 02:52:34.392441 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:34.392406    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-t2l4g"
Sep 14 02:52:34.395162 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:34.395139    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 14 02:52:34.395301 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:34.395221    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-96zbb\""
Sep 14 02:52:34.484289 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:34.484255    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c6505647-069b-4004-bab3-64af301005f1-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-t2l4g\" (UID: \"c6505647-069b-4004-bab3-64af301005f1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-t2l4g"
Sep 14 02:52:34.584795 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:34.584774    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c6505647-069b-4004-bab3-64af301005f1-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-t2l4g\" (UID: \"c6505647-069b-4004-bab3-64af301005f1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-t2l4g"
Sep 14 02:52:34.584908 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:34.584894    2577 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 14 02:52:34.584970 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:34.584960    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6505647-069b-4004-bab3-64af301005f1-tls-certificates podName:c6505647-069b-4004-bab3-64af301005f1 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:35.084942577 +0000 UTC m=+104.062920585 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/c6505647-069b-4004-bab3-64af301005f1-tls-certificates") pod "prometheus-operator-admission-webhook-5864cd99d7-t2l4g" (UID: "c6505647-069b-4004-bab3-64af301005f1") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 14 02:52:35.087429 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:35.087397    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c6505647-069b-4004-bab3-64af301005f1-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-t2l4g\" (UID: \"c6505647-069b-4004-bab3-64af301005f1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-t2l4g"
Sep 14 02:52:35.089906 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:35.089882    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c6505647-069b-4004-bab3-64af301005f1-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-t2l4g\" (UID: \"c6505647-069b-4004-bab3-64af301005f1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-t2l4g"
Sep 14 02:52:35.300922 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:35.300893    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-t2l4g"
Sep 14 02:52:35.595845 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:35.595817    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-t2l4g"]
Sep 14 02:52:35.599595 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:52:35.599572    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6505647_069b_4004_bab3_64af301005f1.slice/crio-ecb7511c1f9b735c2aeb4ed9f1b9aa1df5c5aaaebf88bb5d2cb3eae05d3cdc35 WatchSource:0}: Error finding container ecb7511c1f9b735c2aeb4ed9f1b9aa1df5c5aaaebf88bb5d2cb3eae05d3cdc35: Status 404 returned error can't find the container with id ecb7511c1f9b735c2aeb4ed9f1b9aa1df5c5aaaebf88bb5d2cb3eae05d3cdc35
Sep 14 02:52:35.823493 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:35.823467    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-557bbd7fcd-sb9qd"]
Sep 14 02:52:35.856806 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:35.856781    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-557bbd7fcd-sb9qd"
Sep 14 02:52:35.856905 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:35.856803    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-557bbd7fcd-sb9qd"]
Sep 14 02:52:35.859377 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:35.859351    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 14 02:52:35.859377 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:35.859367    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 14 02:52:35.859552 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:35.859438    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 14 02:52:35.859552 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:35.859477    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 14 02:52:35.859552 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:35.859483    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 14 02:52:35.860453 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:35.860435    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 14 02:52:35.860559 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:35.860443    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-vw64g\""
Sep 14 02:52:35.863711 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:35.863691    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 14 02:52:35.865703 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:35.865686    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 14 02:52:35.994697 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:35.994673    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1553f21c-eabf-4881-89b0-8957cc55d52b-service-ca\") pod \"console-557bbd7fcd-sb9qd\" (UID: \"1553f21c-eabf-4881-89b0-8957cc55d52b\") " pod="openshift-console/console-557bbd7fcd-sb9qd"
Sep 14 02:52:35.994794 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:35.994707    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1553f21c-eabf-4881-89b0-8957cc55d52b-console-config\") pod \"console-557bbd7fcd-sb9qd\" (UID: \"1553f21c-eabf-4881-89b0-8957cc55d52b\") " pod="openshift-console/console-557bbd7fcd-sb9qd"
Sep 14 02:52:35.994794 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:35.994726    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1553f21c-eabf-4881-89b0-8957cc55d52b-console-oauth-config\") pod \"console-557bbd7fcd-sb9qd\" (UID: \"1553f21c-eabf-4881-89b0-8957cc55d52b\") " pod="openshift-console/console-557bbd7fcd-sb9qd"
Sep 14 02:52:35.994794 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:35.994761    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1553f21c-eabf-4881-89b0-8957cc55d52b-oauth-serving-cert\") pod \"console-557bbd7fcd-sb9qd\" (UID: \"1553f21c-eabf-4881-89b0-8957cc55d52b\") " pod="openshift-console/console-557bbd7fcd-sb9qd"
Sep 14 02:52:35.994913 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:35.994805    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1553f21c-eabf-4881-89b0-8957cc55d52b-trusted-ca-bundle\") pod \"console-557bbd7fcd-sb9qd\" (UID: \"1553f21c-eabf-4881-89b0-8957cc55d52b\") " pod="openshift-console/console-557bbd7fcd-sb9qd"
Sep 14 02:52:35.994913 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:35.994836    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m65j\" (UniqueName: \"kubernetes.io/projected/1553f21c-eabf-4881-89b0-8957cc55d52b-kube-api-access-5m65j\") pod \"console-557bbd7fcd-sb9qd\" (UID: \"1553f21c-eabf-4881-89b0-8957cc55d52b\") " pod="openshift-console/console-557bbd7fcd-sb9qd"
Sep 14 02:52:35.994913 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:35.994861    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1553f21c-eabf-4881-89b0-8957cc55d52b-console-serving-cert\") pod \"console-557bbd7fcd-sb9qd\" (UID: \"1553f21c-eabf-4881-89b0-8957cc55d52b\") " pod="openshift-console/console-557bbd7fcd-sb9qd"
Sep 14 02:52:36.095883 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:36.095831    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1553f21c-eabf-4881-89b0-8957cc55d52b-console-serving-cert\") pod \"console-557bbd7fcd-sb9qd\" (UID: \"1553f21c-eabf-4881-89b0-8957cc55d52b\") " pod="openshift-console/console-557bbd7fcd-sb9qd"
Sep 14 02:52:36.095883 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:36.095875    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1553f21c-eabf-4881-89b0-8957cc55d52b-service-ca\") pod \"console-557bbd7fcd-sb9qd\" (UID: \"1553f21c-eabf-4881-89b0-8957cc55d52b\") " pod="openshift-console/console-557bbd7fcd-sb9qd"
Sep 14 02:52:36.096001 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:36.095894    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1553f21c-eabf-4881-89b0-8957cc55d52b-console-config\") pod \"console-557bbd7fcd-sb9qd\" (UID: \"1553f21c-eabf-4881-89b0-8957cc55d52b\") " pod="openshift-console/console-557bbd7fcd-sb9qd"
Sep 14 02:52:36.096001 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:36.095912    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1553f21c-eabf-4881-89b0-8957cc55d52b-console-oauth-config\") pod \"console-557bbd7fcd-sb9qd\" (UID: \"1553f21c-eabf-4881-89b0-8957cc55d52b\") " pod="openshift-console/console-557bbd7fcd-sb9qd"
Sep 14 02:52:36.096001 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:36.095936    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1553f21c-eabf-4881-89b0-8957cc55d52b-oauth-serving-cert\") pod \"console-557bbd7fcd-sb9qd\" (UID: \"1553f21c-eabf-4881-89b0-8957cc55d52b\") " pod="openshift-console/console-557bbd7fcd-sb9qd"
Sep 14 02:52:36.096001 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:36.095952    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1553f21c-eabf-4881-89b0-8957cc55d52b-trusted-ca-bundle\") pod \"console-557bbd7fcd-sb9qd\" (UID: \"1553f21c-eabf-4881-89b0-8957cc55d52b\") " pod="openshift-console/console-557bbd7fcd-sb9qd"
Sep 14 02:52:36.096001 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:36.095972    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m65j\" (UniqueName: \"kubernetes.io/projected/1553f21c-eabf-4881-89b0-8957cc55d52b-kube-api-access-5m65j\") pod \"console-557bbd7fcd-sb9qd\" (UID: \"1553f21c-eabf-4881-89b0-8957cc55d52b\") " pod="openshift-console/console-557bbd7fcd-sb9qd"
Sep 14 02:52:36.096683 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:36.096657    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1553f21c-eabf-4881-89b0-8957cc55d52b-service-ca\") pod \"console-557bbd7fcd-sb9qd\" (UID: \"1553f21c-eabf-4881-89b0-8957cc55d52b\") " pod="openshift-console/console-557bbd7fcd-sb9qd"
Sep 14 02:52:36.096777 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:36.096657    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1553f21c-eabf-4881-89b0-8957cc55d52b-console-config\") pod \"console-557bbd7fcd-sb9qd\" (UID: \"1553f21c-eabf-4881-89b0-8957cc55d52b\") " pod="openshift-console/console-557bbd7fcd-sb9qd"
Sep 14 02:52:36.096777 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:36.096669    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1553f21c-eabf-4881-89b0-8957cc55d52b-oauth-serving-cert\") pod \"console-557bbd7fcd-sb9qd\" (UID: \"1553f21c-eabf-4881-89b0-8957cc55d52b\") " pod="openshift-console/console-557bbd7fcd-sb9qd"
Sep 14 02:52:36.096889 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:36.096850    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1553f21c-eabf-4881-89b0-8957cc55d52b-trusted-ca-bundle\") pod \"console-557bbd7fcd-sb9qd\" (UID: \"1553f21c-eabf-4881-89b0-8957cc55d52b\") " pod="openshift-console/console-557bbd7fcd-sb9qd"
Sep 14 02:52:36.098446 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:36.098427    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1553f21c-eabf-4881-89b0-8957cc55d52b-console-oauth-config\") pod \"console-557bbd7fcd-sb9qd\" (UID: \"1553f21c-eabf-4881-89b0-8957cc55d52b\") " pod="openshift-console/console-557bbd7fcd-sb9qd"
Sep 14 02:52:36.098682 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:36.098664    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1553f21c-eabf-4881-89b0-8957cc55d52b-console-serving-cert\") pod \"console-557bbd7fcd-sb9qd\" (UID: \"1553f21c-eabf-4881-89b0-8957cc55d52b\") " pod="openshift-console/console-557bbd7fcd-sb9qd"
Sep 14 02:52:36.112957 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:36.112928    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m65j\" (UniqueName: \"kubernetes.io/projected/1553f21c-eabf-4881-89b0-8957cc55d52b-kube-api-access-5m65j\") pod \"console-557bbd7fcd-sb9qd\" (UID: \"1553f21c-eabf-4881-89b0-8957cc55d52b\") " pod="openshift-console/console-557bbd7fcd-sb9qd"
Sep 14 02:52:36.130565 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:36.130541    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-t2l4g" event={"ID":"c6505647-069b-4004-bab3-64af301005f1","Type":"ContainerStarted","Data":"ecb7511c1f9b735c2aeb4ed9f1b9aa1df5c5aaaebf88bb5d2cb3eae05d3cdc35"}
Sep 14 02:52:36.166472 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:36.166457    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-557bbd7fcd-sb9qd"
Sep 14 02:52:36.454621 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:36.454558    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-557bbd7fcd-sb9qd"]
Sep 14 02:52:36.457690 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:52:36.457664    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1553f21c_eabf_4881_89b0_8957cc55d52b.slice/crio-314a903306e9937b47f0d274312935fcff78012e16ece982c0f44fd3006372ea WatchSource:0}: Error finding container 314a903306e9937b47f0d274312935fcff78012e16ece982c0f44fd3006372ea: Status 404 returned error can't find the container with id 314a903306e9937b47f0d274312935fcff78012e16ece982c0f44fd3006372ea
Sep 14 02:52:37.135881 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:37.135846    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-557bbd7fcd-sb9qd" event={"ID":"1553f21c-eabf-4881-89b0-8957cc55d52b","Type":"ContainerStarted","Data":"314a903306e9937b47f0d274312935fcff78012e16ece982c0f44fd3006372ea"}
Sep 14 02:52:38.141352 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:38.141315    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-t2l4g" event={"ID":"c6505647-069b-4004-bab3-64af301005f1","Type":"ContainerStarted","Data":"a9ae15756857a10cfedcdb8d0c9e6fb8ba55a2f2d7adb3fd9d73a6cb135a66af"}
Sep 14 02:52:38.141744 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:38.141435    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-t2l4g"
Sep 14 02:52:38.147384 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:38.147359    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-t2l4g"
Sep 14 02:52:38.161405 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:38.161364    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-t2l4g" podStartSLOduration=2.578726389 podStartE2EDuration="4.161349914s" totalImagesPullingTime="1.582623525s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:52:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:38.160525826 +0000 UTC m=+107.138503854" watchObservedRunningTime="2026-09-14 02:52:38.161349914 +0000 UTC m=+107.139327942"
Sep 14 02:52:38.437772 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:38.437686    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-nkzdx"]
Sep 14 02:52:38.464110 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:38.464073    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-nkzdx"]
Sep 14 02:52:38.464248 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:38.464198    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-nkzdx"
Sep 14 02:52:38.467036 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:38.467002    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 14 02:52:38.467153 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:38.467052    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 14 02:52:38.467153 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:38.467087    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 14 02:52:38.467153 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:38.467007    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-ff6fq\""
Sep 14 02:52:38.468070 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:38.468046    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 14 02:52:38.468070 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:38.468065    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 14 02:52:38.614228 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:38.614193    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/139bb2e4-4c56-4bdb-ba66-995245cc7a2c-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-nkzdx\" (UID: \"139bb2e4-4c56-4bdb-ba66-995245cc7a2c\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-nkzdx"
Sep 14 02:52:38.614381 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:38.614257    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/139bb2e4-4c56-4bdb-ba66-995245cc7a2c-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-nkzdx\" (UID: \"139bb2e4-4c56-4bdb-ba66-995245cc7a2c\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-nkzdx"
Sep 14 02:52:38.614381 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:38.614330    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/139bb2e4-4c56-4bdb-ba66-995245cc7a2c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-nkzdx\" (UID: \"139bb2e4-4c56-4bdb-ba66-995245cc7a2c\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-nkzdx"
Sep 14 02:52:38.614381 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:38.614378    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-945r5\" (UniqueName: \"kubernetes.io/projected/139bb2e4-4c56-4bdb-ba66-995245cc7a2c-kube-api-access-945r5\") pod \"prometheus-operator-68bcd7f987-nkzdx\" (UID: \"139bb2e4-4c56-4bdb-ba66-995245cc7a2c\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-nkzdx"
Sep 14 02:52:38.715703 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:38.715637    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/139bb2e4-4c56-4bdb-ba66-995245cc7a2c-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-nkzdx\" (UID: \"139bb2e4-4c56-4bdb-ba66-995245cc7a2c\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-nkzdx"
Sep 14 02:52:38.715703 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:38.715688    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/139bb2e4-4c56-4bdb-ba66-995245cc7a2c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-nkzdx\" (UID: \"139bb2e4-4c56-4bdb-ba66-995245cc7a2c\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-nkzdx"
Sep 14 02:52:38.715893 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:38.715738    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-945r5\" (UniqueName: \"kubernetes.io/projected/139bb2e4-4c56-4bdb-ba66-995245cc7a2c-kube-api-access-945r5\") pod \"prometheus-operator-68bcd7f987-nkzdx\" (UID: \"139bb2e4-4c56-4bdb-ba66-995245cc7a2c\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-nkzdx"
Sep 14 02:52:38.715893 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:38.715775    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/139bb2e4-4c56-4bdb-ba66-995245cc7a2c-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-nkzdx\" (UID: \"139bb2e4-4c56-4bdb-ba66-995245cc7a2c\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-nkzdx"
Sep 14 02:52:38.715893 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:38.715883    2577 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 14 02:52:38.716037 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:38.715955    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/139bb2e4-4c56-4bdb-ba66-995245cc7a2c-prometheus-operator-tls podName:139bb2e4-4c56-4bdb-ba66-995245cc7a2c nodeName:}" failed. No retries permitted until 2026-09-14 02:52:39.215934822 +0000 UTC m=+108.193912832 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/139bb2e4-4c56-4bdb-ba66-995245cc7a2c-prometheus-operator-tls") pod "prometheus-operator-68bcd7f987-nkzdx" (UID: "139bb2e4-4c56-4bdb-ba66-995245cc7a2c") : secret "prometheus-operator-tls" not found
Sep 14 02:52:38.716446 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:38.716415    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/139bb2e4-4c56-4bdb-ba66-995245cc7a2c-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-nkzdx\" (UID: \"139bb2e4-4c56-4bdb-ba66-995245cc7a2c\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-nkzdx"
Sep 14 02:52:38.718499 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:38.718474    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/139bb2e4-4c56-4bdb-ba66-995245cc7a2c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-nkzdx\" (UID: \"139bb2e4-4c56-4bdb-ba66-995245cc7a2c\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-nkzdx"
Sep 14 02:52:38.724700 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:38.724670    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-945r5\" (UniqueName: \"kubernetes.io/projected/139bb2e4-4c56-4bdb-ba66-995245cc7a2c-kube-api-access-945r5\") pod \"prometheus-operator-68bcd7f987-nkzdx\" (UID: \"139bb2e4-4c56-4bdb-ba66-995245cc7a2c\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-nkzdx"
Sep 14 02:52:39.220216 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:39.220177    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/139bb2e4-4c56-4bdb-ba66-995245cc7a2c-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-nkzdx\" (UID: \"139bb2e4-4c56-4bdb-ba66-995245cc7a2c\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-nkzdx"
Sep 14 02:52:39.222707 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:39.222675    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/139bb2e4-4c56-4bdb-ba66-995245cc7a2c-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-nkzdx\" (UID: \"139bb2e4-4c56-4bdb-ba66-995245cc7a2c\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-nkzdx"
Sep 14 02:52:39.374857 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:39.374833    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-nkzdx"
Sep 14 02:52:39.651719 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:39.651610    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-nkzdx"]
Sep 14 02:52:39.654177 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:52:39.654145    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod139bb2e4_4c56_4bdb_ba66_995245cc7a2c.slice/crio-7835ae98c58c526e6dc4589184912c92aefcf9939edf9f2bec642482f705fd63 WatchSource:0}: Error finding container 7835ae98c58c526e6dc4589184912c92aefcf9939edf9f2bec642482f705fd63: Status 404 returned error can't find the container with id 7835ae98c58c526e6dc4589184912c92aefcf9939edf9f2bec642482f705fd63
Sep 14 02:52:40.150418 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:40.150375    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-557bbd7fcd-sb9qd" event={"ID":"1553f21c-eabf-4881-89b0-8957cc55d52b","Type":"ContainerStarted","Data":"05a182cbcd7715294eef8efae32c5d970d6d5b8ac676bca223c61a5fc709d8e8"}
Sep 14 02:52:40.151842 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:40.151819    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-nkzdx" event={"ID":"139bb2e4-4c56-4bdb-ba66-995245cc7a2c","Type":"ContainerStarted","Data":"7835ae98c58c526e6dc4589184912c92aefcf9939edf9f2bec642482f705fd63"}
Sep 14 02:52:40.170159 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:40.170110    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-557bbd7fcd-sb9qd" podStartSLOduration=2.348846929 podStartE2EDuration="5.170095257s" totalImagesPullingTime="2.821248328s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:52:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:40.169085346 +0000 UTC m=+109.147063376" watchObservedRunningTime="2026-09-14 02:52:40.170095257 +0000 UTC m=+109.148073285"
Sep 14 02:52:41.158002 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:41.157923    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-nkzdx" event={"ID":"139bb2e4-4c56-4bdb-ba66-995245cc7a2c","Type":"ContainerStarted","Data":"0a7721dd3e82e63d8d406eaf9faba6700359f3a0dd279e6e31e1c19d49ae1488"}
Sep 14 02:52:41.158002 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:41.157977    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-nkzdx" event={"ID":"139bb2e4-4c56-4bdb-ba66-995245cc7a2c","Type":"ContainerStarted","Data":"891901dd6375175e6f840edd7904a0f63e9465d394163fa84dac52d7e7b1679d"}
Sep 14 02:52:41.186002 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:41.185957    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-nkzdx" podStartSLOduration=1.940920158 podStartE2EDuration="3.185943592s" totalImagesPullingTime="1.245023434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:52:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:41.184459903 +0000 UTC m=+110.162437942" watchObservedRunningTime="2026-09-14 02:52:41.185943592 +0000 UTC m=+110.163921621"
Sep 14 02:52:42.936126 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:42.936088    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-4zqc5"]
Sep 14 02:52:42.939278 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:42.939250    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-4zqc5"
Sep 14 02:52:42.943917 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:42.943893    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 14 02:52:42.944323 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:42.944292    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 14 02:52:42.944594 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:42.944575    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-vjvwv\""
Sep 14 02:52:42.965454 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:42.965426    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qvfz4"]
Sep 14 02:52:42.968901 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:42.968880    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-4zqc5"]
Sep 14 02:52:42.968997 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:42.968971    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qvfz4"
Sep 14 02:52:42.975508 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:42.975489    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 14 02:52:42.975607 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:42.975581    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 14 02:52:42.975667 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:42.975652    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 14 02:52:42.976382 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:42.976363    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-qnmp8\""
Sep 14 02:52:43.049814 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.049795    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cf1cfe36-99ab-461f-8845-2c4c8f3383ac-sys\") pod \"node-exporter-qvfz4\" (UID: \"cf1cfe36-99ab-461f-8845-2c4c8f3383ac\") " pod="openshift-monitoring/node-exporter-qvfz4"
Sep 14 02:52:43.049910 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.049821    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cf1cfe36-99ab-461f-8845-2c4c8f3383ac-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qvfz4\" (UID: \"cf1cfe36-99ab-461f-8845-2c4c8f3383ac\") " pod="openshift-monitoring/node-exporter-qvfz4"
Sep 14 02:52:43.049910 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.049843    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cf1cfe36-99ab-461f-8845-2c4c8f3383ac-node-exporter-tls\") pod \"node-exporter-qvfz4\" (UID: \"cf1cfe36-99ab-461f-8845-2c4c8f3383ac\") " pod="openshift-monitoring/node-exporter-qvfz4"
Sep 14 02:52:43.049910 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.049859    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cf1cfe36-99ab-461f-8845-2c4c8f3383ac-node-exporter-accelerators-collector-config\") pod \"node-exporter-qvfz4\" (UID: \"cf1cfe36-99ab-461f-8845-2c4c8f3383ac\") " pod="openshift-monitoring/node-exporter-qvfz4"
Sep 14 02:52:43.049910 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.049882    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/973e81ff-b6f7-4d26-bae3-c9f91ca1aaed-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-4zqc5\" (UID: \"973e81ff-b6f7-4d26-bae3-c9f91ca1aaed\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-4zqc5"
Sep 14 02:52:43.050065 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.049966    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/973e81ff-b6f7-4d26-bae3-c9f91ca1aaed-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-4zqc5\" (UID: \"973e81ff-b6f7-4d26-bae3-c9f91ca1aaed\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-4zqc5"
Sep 14 02:52:43.050065 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.050004    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/973e81ff-b6f7-4d26-bae3-c9f91ca1aaed-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-4zqc5\" (UID: \"973e81ff-b6f7-4d26-bae3-c9f91ca1aaed\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-4zqc5"
Sep 14 02:52:43.050065 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.050023    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cf1cfe36-99ab-461f-8845-2c4c8f3383ac-metrics-client-ca\") pod \"node-exporter-qvfz4\" (UID: \"cf1cfe36-99ab-461f-8845-2c4c8f3383ac\") " pod="openshift-monitoring/node-exporter-qvfz4"
Sep 14 02:52:43.050166 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.050066    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7mxx\" (UniqueName: \"kubernetes.io/projected/973e81ff-b6f7-4d26-bae3-c9f91ca1aaed-kube-api-access-z7mxx\") pod \"openshift-state-metrics-67dcf8c979-4zqc5\" (UID: \"973e81ff-b6f7-4d26-bae3-c9f91ca1aaed\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-4zqc5"
Sep 14 02:52:43.050166 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.050096    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cf1cfe36-99ab-461f-8845-2c4c8f3383ac-node-exporter-textfile\") pod \"node-exporter-qvfz4\" (UID: \"cf1cfe36-99ab-461f-8845-2c4c8f3383ac\") " pod="openshift-monitoring/node-exporter-qvfz4"
Sep 14 02:52:43.050166 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.050130    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qsdw\" (UniqueName: \"kubernetes.io/projected/cf1cfe36-99ab-461f-8845-2c4c8f3383ac-kube-api-access-9qsdw\") pod \"node-exporter-qvfz4\" (UID: \"cf1cfe36-99ab-461f-8845-2c4c8f3383ac\") " pod="openshift-monitoring/node-exporter-qvfz4"
Sep 14 02:52:43.050166 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.050156    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cf1cfe36-99ab-461f-8845-2c4c8f3383ac-root\") pod \"node-exporter-qvfz4\" (UID: \"cf1cfe36-99ab-461f-8845-2c4c8f3383ac\") " pod="openshift-monitoring/node-exporter-qvfz4"
Sep 14 02:52:43.050329 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.050182    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cf1cfe36-99ab-461f-8845-2c4c8f3383ac-node-exporter-wtmp\") pod \"node-exporter-qvfz4\" (UID: \"cf1cfe36-99ab-461f-8845-2c4c8f3383ac\") " pod="openshift-monitoring/node-exporter-qvfz4"
Sep 14 02:52:43.150665 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.150640    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cf1cfe36-99ab-461f-8845-2c4c8f3383ac-sys\") pod \"node-exporter-qvfz4\" (UID: \"cf1cfe36-99ab-461f-8845-2c4c8f3383ac\") " pod="openshift-monitoring/node-exporter-qvfz4"
Sep 14 02:52:43.150756 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.150669    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cf1cfe36-99ab-461f-8845-2c4c8f3383ac-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qvfz4\" (UID: \"cf1cfe36-99ab-461f-8845-2c4c8f3383ac\") " pod="openshift-monitoring/node-exporter-qvfz4"
Sep 14 02:52:43.150756 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.150692    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cf1cfe36-99ab-461f-8845-2c4c8f3383ac-node-exporter-tls\") pod \"node-exporter-qvfz4\" (UID: \"cf1cfe36-99ab-461f-8845-2c4c8f3383ac\") " pod="openshift-monitoring/node-exporter-qvfz4"
Sep 14 02:52:43.150756 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.150708    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cf1cfe36-99ab-461f-8845-2c4c8f3383ac-node-exporter-accelerators-collector-config\") pod \"node-exporter-qvfz4\" (UID: \"cf1cfe36-99ab-461f-8845-2c4c8f3383ac\") " pod="openshift-monitoring/node-exporter-qvfz4"
Sep 14 02:52:43.150756 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.150733    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/973e81ff-b6f7-4d26-bae3-c9f91ca1aaed-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-4zqc5\" (UID: \"973e81ff-b6f7-4d26-bae3-c9f91ca1aaed\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-4zqc5"
Sep 14 02:52:43.150928 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.150754    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cf1cfe36-99ab-461f-8845-2c4c8f3383ac-sys\") pod \"node-exporter-qvfz4\" (UID: \"cf1cfe36-99ab-461f-8845-2c4c8f3383ac\") " pod="openshift-monitoring/node-exporter-qvfz4"
Sep 14 02:52:43.150928 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.150762    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/973e81ff-b6f7-4d26-bae3-c9f91ca1aaed-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-4zqc5\" (UID: \"973e81ff-b6f7-4d26-bae3-c9f91ca1aaed\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-4zqc5"
Sep 14 02:52:43.150928 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:43.150812    2577 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 14 02:52:43.150928 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.150827    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/973e81ff-b6f7-4d26-bae3-c9f91ca1aaed-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-4zqc5\" (UID: \"973e81ff-b6f7-4d26-bae3-c9f91ca1aaed\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-4zqc5"
Sep 14 02:52:43.150928 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:43.150837    2577 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 14 02:52:43.150928 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.150856    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cf1cfe36-99ab-461f-8845-2c4c8f3383ac-metrics-client-ca\") pod \"node-exporter-qvfz4\" (UID: \"cf1cfe36-99ab-461f-8845-2c4c8f3383ac\") " pod="openshift-monitoring/node-exporter-qvfz4"
Sep 14 02:52:43.150928 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:43.150895    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf1cfe36-99ab-461f-8845-2c4c8f3383ac-node-exporter-tls podName:cf1cfe36-99ab-461f-8845-2c4c8f3383ac nodeName:}" failed. No retries permitted until 2026-09-14 02:52:43.650877057 +0000 UTC m=+112.628855067 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/cf1cfe36-99ab-461f-8845-2c4c8f3383ac-node-exporter-tls") pod "node-exporter-qvfz4" (UID: "cf1cfe36-99ab-461f-8845-2c4c8f3383ac") : secret "node-exporter-tls" not found
Sep 14 02:52:43.151292 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:43.150953    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/973e81ff-b6f7-4d26-bae3-c9f91ca1aaed-openshift-state-metrics-tls podName:973e81ff-b6f7-4d26-bae3-c9f91ca1aaed nodeName:}" failed. No retries permitted until 2026-09-14 02:52:43.650917705 +0000 UTC m=+112.628895730 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/973e81ff-b6f7-4d26-bae3-c9f91ca1aaed-openshift-state-metrics-tls") pod "openshift-state-metrics-67dcf8c979-4zqc5" (UID: "973e81ff-b6f7-4d26-bae3-c9f91ca1aaed") : secret "openshift-state-metrics-tls" not found
Sep 14 02:52:43.151292 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.151004    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7mxx\" (UniqueName: \"kubernetes.io/projected/973e81ff-b6f7-4d26-bae3-c9f91ca1aaed-kube-api-access-z7mxx\") pod \"openshift-state-metrics-67dcf8c979-4zqc5\" (UID: \"973e81ff-b6f7-4d26-bae3-c9f91ca1aaed\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-4zqc5"
Sep 14 02:52:43.151292 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.151046    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cf1cfe36-99ab-461f-8845-2c4c8f3383ac-node-exporter-textfile\") pod \"node-exporter-qvfz4\" (UID: \"cf1cfe36-99ab-461f-8845-2c4c8f3383ac\") " pod="openshift-monitoring/node-exporter-qvfz4"
Sep 14 02:52:43.151292 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.151085    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qsdw\" (UniqueName: \"kubernetes.io/projected/cf1cfe36-99ab-461f-8845-2c4c8f3383ac-kube-api-access-9qsdw\") pod \"node-exporter-qvfz4\" (UID: \"cf1cfe36-99ab-461f-8845-2c4c8f3383ac\") " pod="openshift-monitoring/node-exporter-qvfz4"
Sep 14 02:52:43.151292 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.151114    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cf1cfe36-99ab-461f-8845-2c4c8f3383ac-root\") pod \"node-exporter-qvfz4\" (UID: \"cf1cfe36-99ab-461f-8845-2c4c8f3383ac\") " pod="openshift-monitoring/node-exporter-qvfz4"
Sep 14 02:52:43.151292 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.151151    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cf1cfe36-99ab-461f-8845-2c4c8f3383ac-node-exporter-wtmp\") pod \"node-exporter-qvfz4\" (UID: \"cf1cfe36-99ab-461f-8845-2c4c8f3383ac\") " pod="openshift-monitoring/node-exporter-qvfz4"
Sep 14 02:52:43.151593 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.151345    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cf1cfe36-99ab-461f-8845-2c4c8f3383ac-node-exporter-wtmp\") pod \"node-exporter-qvfz4\" (UID: \"cf1cfe36-99ab-461f-8845-2c4c8f3383ac\") " pod="openshift-monitoring/node-exporter-qvfz4"
Sep 14 02:52:43.151593 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.151414    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cf1cfe36-99ab-461f-8845-2c4c8f3383ac-node-exporter-textfile\") pod \"node-exporter-qvfz4\" (UID: \"cf1cfe36-99ab-461f-8845-2c4c8f3383ac\") " pod="openshift-monitoring/node-exporter-qvfz4"
Sep 14 02:52:43.151593 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.151423    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cf1cfe36-99ab-461f-8845-2c4c8f3383ac-root\") pod \"node-exporter-qvfz4\" (UID: \"cf1cfe36-99ab-461f-8845-2c4c8f3383ac\") " pod="openshift-monitoring/node-exporter-qvfz4"
Sep 14 02:52:43.151593 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.151443    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cf1cfe36-99ab-461f-8845-2c4c8f3383ac-node-exporter-accelerators-collector-config\") pod \"node-exporter-qvfz4\" (UID: \"cf1cfe36-99ab-461f-8845-2c4c8f3383ac\") " pod="openshift-monitoring/node-exporter-qvfz4"
Sep 14 02:52:43.151593 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.151520    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cf1cfe36-99ab-461f-8845-2c4c8f3383ac-metrics-client-ca\") pod \"node-exporter-qvfz4\" (UID: \"cf1cfe36-99ab-461f-8845-2c4c8f3383ac\") " pod="openshift-monitoring/node-exporter-qvfz4"
Sep 14 02:52:43.151765 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.151619    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/973e81ff-b6f7-4d26-bae3-c9f91ca1aaed-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-4zqc5\" (UID: \"973e81ff-b6f7-4d26-bae3-c9f91ca1aaed\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-4zqc5"
Sep 14 02:52:43.153362 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.153343    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/973e81ff-b6f7-4d26-bae3-c9f91ca1aaed-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-4zqc5\" (UID: \"973e81ff-b6f7-4d26-bae3-c9f91ca1aaed\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-4zqc5"
Sep 14 02:52:43.153490 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.153473    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cf1cfe36-99ab-461f-8845-2c4c8f3383ac-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qvfz4\" (UID: \"cf1cfe36-99ab-461f-8845-2c4c8f3383ac\") " pod="openshift-monitoring/node-exporter-qvfz4"
Sep 14 02:52:43.161765 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.161743    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qsdw\" (UniqueName: \"kubernetes.io/projected/cf1cfe36-99ab-461f-8845-2c4c8f3383ac-kube-api-access-9qsdw\") pod \"node-exporter-qvfz4\" (UID: \"cf1cfe36-99ab-461f-8845-2c4c8f3383ac\") " pod="openshift-monitoring/node-exporter-qvfz4"
Sep 14 02:52:43.162829 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.162810    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7mxx\" (UniqueName: \"kubernetes.io/projected/973e81ff-b6f7-4d26-bae3-c9f91ca1aaed-kube-api-access-z7mxx\") pod \"openshift-state-metrics-67dcf8c979-4zqc5\" (UID: \"973e81ff-b6f7-4d26-bae3-c9f91ca1aaed\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-4zqc5"
Sep 14 02:52:43.655388 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.655346    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cf1cfe36-99ab-461f-8845-2c4c8f3383ac-node-exporter-tls\") pod \"node-exporter-qvfz4\" (UID: \"cf1cfe36-99ab-461f-8845-2c4c8f3383ac\") " pod="openshift-monitoring/node-exporter-qvfz4"
Sep 14 02:52:43.655576 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.655398    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/973e81ff-b6f7-4d26-bae3-c9f91ca1aaed-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-4zqc5\" (UID: \"973e81ff-b6f7-4d26-bae3-c9f91ca1aaed\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-4zqc5"
Sep 14 02:52:43.655576 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:43.655499    2577 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 14 02:52:43.655576 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:52:43.655550    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/973e81ff-b6f7-4d26-bae3-c9f91ca1aaed-openshift-state-metrics-tls podName:973e81ff-b6f7-4d26-bae3-c9f91ca1aaed nodeName:}" failed. No retries permitted until 2026-09-14 02:52:44.655537622 +0000 UTC m=+113.633515642 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/973e81ff-b6f7-4d26-bae3-c9f91ca1aaed-openshift-state-metrics-tls") pod "openshift-state-metrics-67dcf8c979-4zqc5" (UID: "973e81ff-b6f7-4d26-bae3-c9f91ca1aaed") : secret "openshift-state-metrics-tls" not found
Sep 14 02:52:43.657711 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.657686    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cf1cfe36-99ab-461f-8845-2c4c8f3383ac-node-exporter-tls\") pod \"node-exporter-qvfz4\" (UID: \"cf1cfe36-99ab-461f-8845-2c4c8f3383ac\") " pod="openshift-monitoring/node-exporter-qvfz4"
Sep 14 02:52:43.877231 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:43.877198    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qvfz4"
Sep 14 02:52:43.891008 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:52:43.890973    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf1cfe36_99ab_461f_8845_2c4c8f3383ac.slice/crio-9f35539c5a7bd923088ebd99faadf0add157b420b5867761858ddac1b6c6ac51 WatchSource:0}: Error finding container 9f35539c5a7bd923088ebd99faadf0add157b420b5867761858ddac1b6c6ac51: Status 404 returned error can't find the container with id 9f35539c5a7bd923088ebd99faadf0add157b420b5867761858ddac1b6c6ac51
Sep 14 02:52:44.128089 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:44.128064    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-v5tc4"
Sep 14 02:52:44.170669 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:44.170641    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qvfz4" event={"ID":"cf1cfe36-99ab-461f-8845-2c4c8f3383ac","Type":"ContainerStarted","Data":"9f35539c5a7bd923088ebd99faadf0add157b420b5867761858ddac1b6c6ac51"}
Sep 14 02:52:44.664549 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:44.664511    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/973e81ff-b6f7-4d26-bae3-c9f91ca1aaed-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-4zqc5\" (UID: \"973e81ff-b6f7-4d26-bae3-c9f91ca1aaed\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-4zqc5"
Sep 14 02:52:44.666970 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:44.666939    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/973e81ff-b6f7-4d26-bae3-c9f91ca1aaed-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-4zqc5\" (UID: \"973e81ff-b6f7-4d26-bae3-c9f91ca1aaed\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-4zqc5"
Sep 14 02:52:44.748973 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:44.748928    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-4zqc5"
Sep 14 02:52:45.097424 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:45.097404    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-4zqc5"]
Sep 14 02:52:45.098997 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:52:45.098971    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod973e81ff_b6f7_4d26_bae3_c9f91ca1aaed.slice/crio-47b81a8639992c589078e998033bede0a8b43a9a03977b55574465a686fe21e2 WatchSource:0}: Error finding container 47b81a8639992c589078e998033bede0a8b43a9a03977b55574465a686fe21e2: Status 404 returned error can't find the container with id 47b81a8639992c589078e998033bede0a8b43a9a03977b55574465a686fe21e2
Sep 14 02:52:45.178562 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:45.178113    2577 generic.go:356] "Generic (PLEG): container finished" podID="cf1cfe36-99ab-461f-8845-2c4c8f3383ac" containerID="bf22b85f4f0f1643f8f24b809b9306cc8bcc74dba15edd42ced24e0b1cb44e87" exitCode=0
Sep 14 02:52:45.178926 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:45.178617    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qvfz4" event={"ID":"cf1cfe36-99ab-461f-8845-2c4c8f3383ac","Type":"ContainerDied","Data":"bf22b85f4f0f1643f8f24b809b9306cc8bcc74dba15edd42ced24e0b1cb44e87"}
Sep 14 02:52:45.180354 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:45.180331    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-4zqc5" event={"ID":"973e81ff-b6f7-4d26-bae3-c9f91ca1aaed","Type":"ContainerStarted","Data":"1eda0d6f93def689c8583965bbc7f235c89a2daa55fccc4edf7c803c7ddf7cf9"}
Sep 14 02:52:45.180442 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:45.180382    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-4zqc5" event={"ID":"973e81ff-b6f7-4d26-bae3-c9f91ca1aaed","Type":"ContainerStarted","Data":"47b81a8639992c589078e998033bede0a8b43a9a03977b55574465a686fe21e2"}
Sep 14 02:52:45.911611 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:45.911578    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5798cfc64f-fd7cc"]
Sep 14 02:52:45.915361 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:45.915337    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5798cfc64f-fd7cc"
Sep 14 02:52:45.918813 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:45.918793    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 14 02:52:45.919063 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:45.919042    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 14 02:52:45.919195 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:45.919046    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 14 02:52:45.919258 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:45.919234    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-7oukj80vein4r\""
Sep 14 02:52:45.919942 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:45.919920    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 14 02:52:45.920047 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:45.919922    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 14 02:52:45.920047 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:45.919922    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-84v7s\""
Sep 14 02:52:45.932507 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:45.932486    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5798cfc64f-fd7cc"]
Sep 14 02:52:46.076684 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:46.076633    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/59a4ce0f-f4fe-4c46-a137-55f612b3240b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5798cfc64f-fd7cc\" (UID: \"59a4ce0f-f4fe-4c46-a137-55f612b3240b\") " pod="openshift-monitoring/thanos-querier-5798cfc64f-fd7cc"
Sep 14 02:52:46.076684 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:46.076689    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/59a4ce0f-f4fe-4c46-a137-55f612b3240b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5798cfc64f-fd7cc\" (UID: \"59a4ce0f-f4fe-4c46-a137-55f612b3240b\") " pod="openshift-monitoring/thanos-querier-5798cfc64f-fd7cc"
Sep 14 02:52:46.076908 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:46.076722    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/59a4ce0f-f4fe-4c46-a137-55f612b3240b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5798cfc64f-fd7cc\" (UID: \"59a4ce0f-f4fe-4c46-a137-55f612b3240b\") " pod="openshift-monitoring/thanos-querier-5798cfc64f-fd7cc"
Sep 14 02:52:46.076908 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:46.076756    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/59a4ce0f-f4fe-4c46-a137-55f612b3240b-metrics-client-ca\") pod \"thanos-querier-5798cfc64f-fd7cc\" (UID: \"59a4ce0f-f4fe-4c46-a137-55f612b3240b\") " pod="openshift-monitoring/thanos-querier-5798cfc64f-fd7cc"
Sep 14 02:52:46.076908 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:46.076799    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/59a4ce0f-f4fe-4c46-a137-55f612b3240b-secret-grpc-tls\") pod \"thanos-querier-5798cfc64f-fd7cc\" (UID: \"59a4ce0f-f4fe-4c46-a137-55f612b3240b\") " pod="openshift-monitoring/thanos-querier-5798cfc64f-fd7cc"
Sep 14 02:52:46.076908 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:46.076872    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/59a4ce0f-f4fe-4c46-a137-55f612b3240b-secret-thanos-querier-tls\") pod \"thanos-querier-5798cfc64f-fd7cc\" (UID: \"59a4ce0f-f4fe-4c46-a137-55f612b3240b\") " pod="openshift-monitoring/thanos-querier-5798cfc64f-fd7cc"
Sep 14 02:52:46.077084 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:46.076921    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gb5d4\" (UniqueName: \"kubernetes.io/projected/59a4ce0f-f4fe-4c46-a137-55f612b3240b-kube-api-access-gb5d4\") pod \"thanos-querier-5798cfc64f-fd7cc\" (UID: \"59a4ce0f-f4fe-4c46-a137-55f612b3240b\") " pod="openshift-monitoring/thanos-querier-5798cfc64f-fd7cc"
Sep 14 02:52:46.077084 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:46.077046    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/59a4ce0f-f4fe-4c46-a137-55f612b3240b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5798cfc64f-fd7cc\" (UID: \"59a4ce0f-f4fe-4c46-a137-55f612b3240b\") " pod="openshift-monitoring/thanos-querier-5798cfc64f-fd7cc"
Sep 14 02:52:46.166886 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:46.166827    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-557bbd7fcd-sb9qd"
Sep 14 02:52:46.166886 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:46.166878    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-557bbd7fcd-sb9qd"
Sep 14 02:52:46.171481 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:46.171460    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-557bbd7fcd-sb9qd"
Sep 14 02:52:46.177508 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:46.177489    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/59a4ce0f-f4fe-4c46-a137-55f612b3240b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5798cfc64f-fd7cc\" (UID: \"59a4ce0f-f4fe-4c46-a137-55f612b3240b\") " pod="openshift-monitoring/thanos-querier-5798cfc64f-fd7cc"
Sep 14 02:52:46.177586 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:46.177515    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/59a4ce0f-f4fe-4c46-a137-55f612b3240b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5798cfc64f-fd7cc\" (UID: \"59a4ce0f-f4fe-4c46-a137-55f612b3240b\") " pod="openshift-monitoring/thanos-querier-5798cfc64f-fd7cc"
Sep 14 02:52:46.177586 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:46.177534    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/59a4ce0f-f4fe-4c46-a137-55f612b3240b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5798cfc64f-fd7cc\" (UID: \"59a4ce0f-f4fe-4c46-a137-55f612b3240b\") " pod="openshift-monitoring/thanos-querier-5798cfc64f-fd7cc"
Sep 14 02:52:46.177586 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:46.177555    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/59a4ce0f-f4fe-4c46-a137-55f612b3240b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5798cfc64f-fd7cc\" (UID: \"59a4ce0f-f4fe-4c46-a137-55f612b3240b\") " pod="openshift-monitoring/thanos-querier-5798cfc64f-fd7cc"
Sep 14 02:52:46.177586 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:46.177574    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/59a4ce0f-f4fe-4c46-a137-55f612b3240b-metrics-client-ca\") pod \"thanos-querier-5798cfc64f-fd7cc\" (UID: \"59a4ce0f-f4fe-4c46-a137-55f612b3240b\") " pod="openshift-monitoring/thanos-querier-5798cfc64f-fd7cc"
Sep 14 02:52:46.177768 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:46.177714    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/59a4ce0f-f4fe-4c46-a137-55f612b3240b-secret-grpc-tls\") pod \"thanos-querier-5798cfc64f-fd7cc\" (UID: \"59a4ce0f-f4fe-4c46-a137-55f612b3240b\") " pod="openshift-monitoring/thanos-querier-5798cfc64f-fd7cc"
Sep 14 02:52:46.177867 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:46.177777    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/59a4ce0f-f4fe-4c46-a137-55f612b3240b-secret-thanos-querier-tls\") pod \"thanos-querier-5798cfc64f-fd7cc\" (UID: \"59a4ce0f-f4fe-4c46-a137-55f612b3240b\") " pod="openshift-monitoring/thanos-querier-5798cfc64f-fd7cc"
Sep 14 02:52:46.177867 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:46.177823    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gb5d4\" (UniqueName: \"kubernetes.io/projected/59a4ce0f-f4fe-4c46-a137-55f612b3240b-kube-api-access-gb5d4\") pod \"thanos-querier-5798cfc64f-fd7cc\" (UID: \"59a4ce0f-f4fe-4c46-a137-55f612b3240b\") " pod="openshift-monitoring/thanos-querier-5798cfc64f-fd7cc"
Sep 14 02:52:46.178309 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:46.178284    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/59a4ce0f-f4fe-4c46-a137-55f612b3240b-metrics-client-ca\") pod \"thanos-querier-5798cfc64f-fd7cc\" (UID: \"59a4ce0f-f4fe-4c46-a137-55f612b3240b\") " pod="openshift-monitoring/thanos-querier-5798cfc64f-fd7cc"
Sep 14 02:52:46.180353 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:46.180325    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/59a4ce0f-f4fe-4c46-a137-55f612b3240b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5798cfc64f-fd7cc\" (UID: \"59a4ce0f-f4fe-4c46-a137-55f612b3240b\") " pod="openshift-monitoring/thanos-querier-5798cfc64f-fd7cc"
Sep 14 02:52:46.180704 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:46.180525    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/59a4ce0f-f4fe-4c46-a137-55f612b3240b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5798cfc64f-fd7cc\" (UID: \"59a4ce0f-f4fe-4c46-a137-55f612b3240b\") " pod="openshift-monitoring/thanos-querier-5798cfc64f-fd7cc"
Sep 14 02:52:46.180704 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:46.180560    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/59a4ce0f-f4fe-4c46-a137-55f612b3240b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5798cfc64f-fd7cc\" (UID: \"59a4ce0f-f4fe-4c46-a137-55f612b3240b\") " pod="openshift-monitoring/thanos-querier-5798cfc64f-fd7cc"
Sep 14 02:52:46.180704 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:46.180694    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/59a4ce0f-f4fe-4c46-a137-55f612b3240b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5798cfc64f-fd7cc\" (UID: \"59a4ce0f-f4fe-4c46-a137-55f612b3240b\") " pod="openshift-monitoring/thanos-querier-5798cfc64f-fd7cc"
Sep 14 02:52:46.180857 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:46.180757    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/59a4ce0f-f4fe-4c46-a137-55f612b3240b-secret-thanos-querier-tls\") pod \"thanos-querier-5798cfc64f-fd7cc\" (UID: \"59a4ce0f-f4fe-4c46-a137-55f612b3240b\") " pod="openshift-monitoring/thanos-querier-5798cfc64f-fd7cc"
Sep 14 02:52:46.181073 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:46.181056    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/59a4ce0f-f4fe-4c46-a137-55f612b3240b-secret-grpc-tls\") pod \"thanos-querier-5798cfc64f-fd7cc\" (UID: \"59a4ce0f-f4fe-4c46-a137-55f612b3240b\") " pod="openshift-monitoring/thanos-querier-5798cfc64f-fd7cc"
Sep 14 02:52:46.185644 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:46.185620    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gb5d4\" (UniqueName: \"kubernetes.io/projected/59a4ce0f-f4fe-4c46-a137-55f612b3240b-kube-api-access-gb5d4\") pod \"thanos-querier-5798cfc64f-fd7cc\" (UID: \"59a4ce0f-f4fe-4c46-a137-55f612b3240b\") " pod="openshift-monitoring/thanos-querier-5798cfc64f-fd7cc"
Sep 14 02:52:46.186481 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:46.186454    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qvfz4" event={"ID":"cf1cfe36-99ab-461f-8845-2c4c8f3383ac","Type":"ContainerStarted","Data":"8c28f2be3edb51e7c7d471453d708ca529a5dd35e275acd9f3a72db7433fb5ed"}
Sep 14 02:52:46.186580 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:46.186502    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qvfz4" event={"ID":"cf1cfe36-99ab-461f-8845-2c4c8f3383ac","Type":"ContainerStarted","Data":"de3928022ce4fc73e249d7194a300930cfe883002c7cb0679ea3a0f30966609f"}
Sep 14 02:52:46.188364 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:46.188343    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-4zqc5" event={"ID":"973e81ff-b6f7-4d26-bae3-c9f91ca1aaed","Type":"ContainerStarted","Data":"d7da487036729c4fbfee43a9c3d9ee8f62e02676ecb9d3a2f5061184f66a95c2"}
Sep 14 02:52:46.194594 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:46.193651    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-557bbd7fcd-sb9qd"
Sep 14 02:52:46.210347 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:46.210294    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qvfz4" podStartSLOduration=3.170231527 podStartE2EDuration="4.210281056s" totalImagesPullingTime="981.862976ms" totalInitContainerRuntime="58.186553ms" isStatefulPod=true podCreationTimestamp="2026-09-14 02:52:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:46.208934631 +0000 UTC m=+115.186912696" watchObservedRunningTime="2026-09-14 02:52:46.210281056 +0000 UTC m=+115.188259076"
Sep 14 02:52:46.225680 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:46.225628    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5798cfc64f-fd7cc"
Sep 14 02:52:46.542276 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:46.542234    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5798cfc64f-fd7cc"]
Sep 14 02:52:46.546613 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:52:46.546585    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59a4ce0f_f4fe_4c46_a137_55f612b3240b.slice/crio-2746a0bd36b4b89c8e055d8df4ba6f063f41fd277da3c509d07caee677bc21d1 WatchSource:0}: Error finding container 2746a0bd36b4b89c8e055d8df4ba6f063f41fd277da3c509d07caee677bc21d1: Status 404 returned error can't find the container with id 2746a0bd36b4b89c8e055d8df4ba6f063f41fd277da3c509d07caee677bc21d1
Sep 14 02:52:46.568520 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:46.568498    2577 patch_prober.go:28] interesting pod/image-registry-57d69d9c4c-rvt8r container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 14 02:52:46.568626 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:46.568539    2577 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-57d69d9c4c-rvt8r" podUID="673eb2ed-da11-47d3-b059-228075d65381" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 02:52:47.194477 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:47.194445    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-4zqc5" event={"ID":"973e81ff-b6f7-4d26-bae3-c9f91ca1aaed","Type":"ContainerStarted","Data":"c9e304f6edd283da708cd2798f3fb3b89552cd87f38fc9abf135922c3518ef19"}
Sep 14 02:52:47.195951 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:47.195922    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5798cfc64f-fd7cc" event={"ID":"59a4ce0f-f4fe-4c46-a137-55f612b3240b","Type":"ContainerStarted","Data":"2746a0bd36b4b89c8e055d8df4ba6f063f41fd277da3c509d07caee677bc21d1"}
Sep 14 02:52:47.215848 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:47.215794    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-4zqc5" podStartSLOduration=4.256971581 podStartE2EDuration="5.215775668s" totalImagesPullingTime="958.804087ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:52:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:47.213951042 +0000 UTC m=+116.191929072" watchObservedRunningTime="2026-09-14 02:52:47.215775668 +0000 UTC m=+116.193753700"
Sep 14 02:52:47.584566 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:47.584535    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-d8jr5"]
Sep 14 02:52:47.588019 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:47.587994    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-d8jr5"
Sep 14 02:52:47.591236 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:47.590815    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-qq8st\""
Sep 14 02:52:47.591236 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:47.591102    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 14 02:52:47.597145 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:47.597122    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-d8jr5"]
Sep 14 02:52:47.692793 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:47.692763    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e5dc70e7-0fd4-457b-8e8b-8ceb4149c3d3-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-d8jr5\" (UID: \"e5dc70e7-0fd4-457b-8e8b-8ceb4149c3d3\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-d8jr5"
Sep 14 02:52:47.692918 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:47.692835    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x689\" (UniqueName: \"kubernetes.io/projected/e5dc70e7-0fd4-457b-8e8b-8ceb4149c3d3-kube-api-access-4x689\") pod \"monitoring-plugin-5d85778b9d-d8jr5\" (UID: \"e5dc70e7-0fd4-457b-8e8b-8ceb4149c3d3\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-d8jr5"
Sep 14 02:52:47.793623 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:47.793599    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e5dc70e7-0fd4-457b-8e8b-8ceb4149c3d3-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-d8jr5\" (UID: \"e5dc70e7-0fd4-457b-8e8b-8ceb4149c3d3\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-d8jr5"
Sep 14 02:52:47.793725 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:47.793645    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x689\" (UniqueName: \"kubernetes.io/projected/e5dc70e7-0fd4-457b-8e8b-8ceb4149c3d3-kube-api-access-4x689\") pod \"monitoring-plugin-5d85778b9d-d8jr5\" (UID: \"e5dc70e7-0fd4-457b-8e8b-8ceb4149c3d3\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-d8jr5"
Sep 14 02:52:47.795990 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:47.795972    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e5dc70e7-0fd4-457b-8e8b-8ceb4149c3d3-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-d8jr5\" (UID: \"e5dc70e7-0fd4-457b-8e8b-8ceb4149c3d3\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-d8jr5"
Sep 14 02:52:47.803365 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:47.803344    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x689\" (UniqueName: \"kubernetes.io/projected/e5dc70e7-0fd4-457b-8e8b-8ceb4149c3d3-kube-api-access-4x689\") pod \"monitoring-plugin-5d85778b9d-d8jr5\" (UID: \"e5dc70e7-0fd4-457b-8e8b-8ceb4149c3d3\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-d8jr5"
Sep 14 02:52:47.901844 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:47.901787    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-d8jr5"
Sep 14 02:52:48.094779 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:48.094752    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-57d69d9c4c-rvt8r"
Sep 14 02:52:48.179924 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:48.179802    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-d8jr5"]
Sep 14 02:52:48.519585 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:52:48.519552    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5dc70e7_0fd4_457b_8e8b_8ceb4149c3d3.slice/crio-013b43d90d5068e973434ad31f01889ebe9abb6464bd1a6b82dfaf9526ed425c WatchSource:0}: Error finding container 013b43d90d5068e973434ad31f01889ebe9abb6464bd1a6b82dfaf9526ed425c: Status 404 returned error can't find the container with id 013b43d90d5068e973434ad31f01889ebe9abb6464bd1a6b82dfaf9526ed425c
Sep 14 02:52:49.207875 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:49.207840    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5798cfc64f-fd7cc" event={"ID":"59a4ce0f-f4fe-4c46-a137-55f612b3240b","Type":"ContainerStarted","Data":"8c74a6ac3690974136af5cdc07feb6319f0e79c4e41c05458d1c063496c0f572"}
Sep 14 02:52:49.208063 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:49.207931    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5798cfc64f-fd7cc" event={"ID":"59a4ce0f-f4fe-4c46-a137-55f612b3240b","Type":"ContainerStarted","Data":"0328286b1741f4fbd5d58df927e3a1b8d75f5a4d34b69cd27bbce32efe187a41"}
Sep 14 02:52:49.208063 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:49.207991    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5798cfc64f-fd7cc" event={"ID":"59a4ce0f-f4fe-4c46-a137-55f612b3240b","Type":"ContainerStarted","Data":"81d3fe35115f06d5bf7c830e98a3875abbb8efa5936adf8353a4e2da7279d8d7"}
Sep 14 02:52:49.209217 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:49.209188    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-d8jr5" event={"ID":"e5dc70e7-0fd4-457b-8e8b-8ceb4149c3d3","Type":"ContainerStarted","Data":"013b43d90d5068e973434ad31f01889ebe9abb6464bd1a6b82dfaf9526ed425c"}
Sep 14 02:52:50.218162 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:50.217893    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5798cfc64f-fd7cc" event={"ID":"59a4ce0f-f4fe-4c46-a137-55f612b3240b","Type":"ContainerStarted","Data":"1f3b9f2d434973b3c594d87ffd4ea5df6b58b0d2d13b7ccf1c5b2a7c829d7002"}
Sep 14 02:52:50.218162 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:50.217974    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5798cfc64f-fd7cc" event={"ID":"59a4ce0f-f4fe-4c46-a137-55f612b3240b","Type":"ContainerStarted","Data":"bc6ee83a3f2b49fcd5c851311e08a61bf7bde18d3e16573b0081e8a163b85734"}
Sep 14 02:52:50.218162 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:50.218018    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5798cfc64f-fd7cc" event={"ID":"59a4ce0f-f4fe-4c46-a137-55f612b3240b","Type":"ContainerStarted","Data":"03c3e52a103d69930674f24ed0d0970924d92ac35b8e3023526c22faa8f244dc"}
Sep 14 02:52:50.218162 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:50.218065    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5798cfc64f-fd7cc"
Sep 14 02:52:50.219738 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:50.219707    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-d8jr5" event={"ID":"e5dc70e7-0fd4-457b-8e8b-8ceb4149c3d3","Type":"ContainerStarted","Data":"9d147a4da67e3f147826cd33aef09b77b8b01ccb8761c5d565ad8d5e59bb04c9"}
Sep 14 02:52:50.219850 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:50.219750    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-d8jr5"
Sep 14 02:52:50.225449 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:50.225423    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-d8jr5"
Sep 14 02:52:50.242090 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:50.242044    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5798cfc64f-fd7cc" podStartSLOduration=1.994458034 podStartE2EDuration="5.242032478s" totalImagesPullingTime="3.247574444s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:52:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:50.240714571 +0000 UTC m=+119.218692610" watchObservedRunningTime="2026-09-14 02:52:50.242032478 +0000 UTC m=+119.220010506"
Sep 14 02:52:50.256530 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:50.256491    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-d8jr5" podStartSLOduration=1.817629621 podStartE2EDuration="3.256477554s" totalImagesPullingTime="1.438847933s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:52:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:50.254800526 +0000 UTC m=+119.232778548" watchObservedRunningTime="2026-09-14 02:52:50.256477554 +0000 UTC m=+119.234455608"
Sep 14 02:52:51.222715 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:51.222680    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-ddbbff75f-ddr76"]
Sep 14 02:52:51.226244 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:51.226218    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-ddbbff75f-ddr76"
Sep 14 02:52:51.236650 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:51.236622    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-ddbbff75f-ddr76"]
Sep 14 02:52:51.324517 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:51.324479    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9f2aa931-4930-419f-8078-13ff29d6a8bc-service-ca\") pod \"console-ddbbff75f-ddr76\" (UID: \"9f2aa931-4930-419f-8078-13ff29d6a8bc\") " pod="openshift-console/console-ddbbff75f-ddr76"
Sep 14 02:52:51.324645 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:51.324526    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tbnl\" (UniqueName: \"kubernetes.io/projected/9f2aa931-4930-419f-8078-13ff29d6a8bc-kube-api-access-6tbnl\") pod \"console-ddbbff75f-ddr76\" (UID: \"9f2aa931-4930-419f-8078-13ff29d6a8bc\") " pod="openshift-console/console-ddbbff75f-ddr76"
Sep 14 02:52:51.324756 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:51.324712    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9f2aa931-4930-419f-8078-13ff29d6a8bc-console-serving-cert\") pod \"console-ddbbff75f-ddr76\" (UID: \"9f2aa931-4930-419f-8078-13ff29d6a8bc\") " pod="openshift-console/console-ddbbff75f-ddr76"
Sep 14 02:52:51.324819 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:51.324800    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9f2aa931-4930-419f-8078-13ff29d6a8bc-console-config\") pod \"console-ddbbff75f-ddr76\" (UID: \"9f2aa931-4930-419f-8078-13ff29d6a8bc\") " pod="openshift-console/console-ddbbff75f-ddr76"
Sep 14 02:52:51.324995 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:51.324966    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9f2aa931-4930-419f-8078-13ff29d6a8bc-oauth-serving-cert\") pod \"console-ddbbff75f-ddr76\" (UID: \"9f2aa931-4930-419f-8078-13ff29d6a8bc\") " pod="openshift-console/console-ddbbff75f-ddr76"
Sep 14 02:52:51.325135 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:51.325060    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9f2aa931-4930-419f-8078-13ff29d6a8bc-console-oauth-config\") pod \"console-ddbbff75f-ddr76\" (UID: \"9f2aa931-4930-419f-8078-13ff29d6a8bc\") " pod="openshift-console/console-ddbbff75f-ddr76"
Sep 14 02:52:51.325135 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:51.325087    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f2aa931-4930-419f-8078-13ff29d6a8bc-trusted-ca-bundle\") pod \"console-ddbbff75f-ddr76\" (UID: \"9f2aa931-4930-419f-8078-13ff29d6a8bc\") " pod="openshift-console/console-ddbbff75f-ddr76"
Sep 14 02:52:51.425607 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:51.425578    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9f2aa931-4930-419f-8078-13ff29d6a8bc-service-ca\") pod \"console-ddbbff75f-ddr76\" (UID: \"9f2aa931-4930-419f-8078-13ff29d6a8bc\") " pod="openshift-console/console-ddbbff75f-ddr76"
Sep 14 02:52:51.425607 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:51.425610    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tbnl\" (UniqueName: \"kubernetes.io/projected/9f2aa931-4930-419f-8078-13ff29d6a8bc-kube-api-access-6tbnl\") pod \"console-ddbbff75f-ddr76\" (UID: \"9f2aa931-4930-419f-8078-13ff29d6a8bc\") " pod="openshift-console/console-ddbbff75f-ddr76"
Sep 14 02:52:51.425783 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:51.425631    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9f2aa931-4930-419f-8078-13ff29d6a8bc-console-serving-cert\") pod \"console-ddbbff75f-ddr76\" (UID: \"9f2aa931-4930-419f-8078-13ff29d6a8bc\") " pod="openshift-console/console-ddbbff75f-ddr76"
Sep 14 02:52:51.425783 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:51.425666    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9f2aa931-4930-419f-8078-13ff29d6a8bc-console-config\") pod \"console-ddbbff75f-ddr76\" (UID: \"9f2aa931-4930-419f-8078-13ff29d6a8bc\") " pod="openshift-console/console-ddbbff75f-ddr76"
Sep 14 02:52:51.425783 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:51.425709    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9f2aa931-4930-419f-8078-13ff29d6a8bc-oauth-serving-cert\") pod \"console-ddbbff75f-ddr76\" (UID: \"9f2aa931-4930-419f-8078-13ff29d6a8bc\") " pod="openshift-console/console-ddbbff75f-ddr76"
Sep 14 02:52:51.425783 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:51.425755    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9f2aa931-4930-419f-8078-13ff29d6a8bc-console-oauth-config\") pod \"console-ddbbff75f-ddr76\" (UID: \"9f2aa931-4930-419f-8078-13ff29d6a8bc\") " pod="openshift-console/console-ddbbff75f-ddr76"
Sep 14 02:52:51.425783 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:51.425777    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f2aa931-4930-419f-8078-13ff29d6a8bc-trusted-ca-bundle\") pod \"console-ddbbff75f-ddr76\" (UID: \"9f2aa931-4930-419f-8078-13ff29d6a8bc\") " pod="openshift-console/console-ddbbff75f-ddr76"
Sep 14 02:52:51.426471 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:51.426439    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9f2aa931-4930-419f-8078-13ff29d6a8bc-service-ca\") pod \"console-ddbbff75f-ddr76\" (UID: \"9f2aa931-4930-419f-8078-13ff29d6a8bc\") " pod="openshift-console/console-ddbbff75f-ddr76"
Sep 14 02:52:51.426603 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:51.426578    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9f2aa931-4930-419f-8078-13ff29d6a8bc-console-config\") pod \"console-ddbbff75f-ddr76\" (UID: \"9f2aa931-4930-419f-8078-13ff29d6a8bc\") " pod="openshift-console/console-ddbbff75f-ddr76"
Sep 14 02:52:51.426682 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:51.426655    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9f2aa931-4930-419f-8078-13ff29d6a8bc-oauth-serving-cert\") pod \"console-ddbbff75f-ddr76\" (UID: \"9f2aa931-4930-419f-8078-13ff29d6a8bc\") " pod="openshift-console/console-ddbbff75f-ddr76"
Sep 14 02:52:51.426750 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:51.426730    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f2aa931-4930-419f-8078-13ff29d6a8bc-trusted-ca-bundle\") pod \"console-ddbbff75f-ddr76\" (UID: \"9f2aa931-4930-419f-8078-13ff29d6a8bc\") " pod="openshift-console/console-ddbbff75f-ddr76"
Sep 14 02:52:51.428420 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:51.428389    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9f2aa931-4930-419f-8078-13ff29d6a8bc-console-serving-cert\") pod \"console-ddbbff75f-ddr76\" (UID: \"9f2aa931-4930-419f-8078-13ff29d6a8bc\") " pod="openshift-console/console-ddbbff75f-ddr76"
Sep 14 02:52:51.428520 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:51.428430    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9f2aa931-4930-419f-8078-13ff29d6a8bc-console-oauth-config\") pod \"console-ddbbff75f-ddr76\" (UID: \"9f2aa931-4930-419f-8078-13ff29d6a8bc\") " pod="openshift-console/console-ddbbff75f-ddr76"
Sep 14 02:52:51.435282 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:51.435239    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tbnl\" (UniqueName: \"kubernetes.io/projected/9f2aa931-4930-419f-8078-13ff29d6a8bc-kube-api-access-6tbnl\") pod \"console-ddbbff75f-ddr76\" (UID: \"9f2aa931-4930-419f-8078-13ff29d6a8bc\") " pod="openshift-console/console-ddbbff75f-ddr76"
Sep 14 02:52:51.536410 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:51.536388    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-ddbbff75f-ddr76"
Sep 14 02:52:51.814038 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:51.813999    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-ddbbff75f-ddr76"]
Sep 14 02:52:51.816191 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:52:51.816164    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f2aa931_4930_419f_8078_13ff29d6a8bc.slice/crio-0825c900b41efc518d35ac819db174fee1154591ac3cd297eaaf573a54e9f0dd WatchSource:0}: Error finding container 0825c900b41efc518d35ac819db174fee1154591ac3cd297eaaf573a54e9f0dd: Status 404 returned error can't find the container with id 0825c900b41efc518d35ac819db174fee1154591ac3cd297eaaf573a54e9f0dd
Sep 14 02:52:52.229978 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:52.229884    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-ddbbff75f-ddr76" event={"ID":"9f2aa931-4930-419f-8078-13ff29d6a8bc","Type":"ContainerStarted","Data":"76301e4af0178a1b0e21f0a5668a7b7bf09097fe5bb055a33ce3d01d8a3ab8c0"}
Sep 14 02:52:52.229978 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:52.229936    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-ddbbff75f-ddr76" event={"ID":"9f2aa931-4930-419f-8078-13ff29d6a8bc","Type":"ContainerStarted","Data":"0825c900b41efc518d35ac819db174fee1154591ac3cd297eaaf573a54e9f0dd"}
Sep 14 02:52:52.246887 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:52.246838    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-ddbbff75f-ddr76" podStartSLOduration=1.246823996 podStartE2EDuration="1.246823996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:52:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:52.246674778 +0000 UTC m=+121.224652809" watchObservedRunningTime="2026-09-14 02:52:52.246823996 +0000 UTC m=+121.224802025"
Sep 14 02:52:54.245755 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:54.245719    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-ddbbff75f-ddr76"]
Sep 14 02:52:54.287557 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:54.287522    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5587bfd74c-2jrfx"]
Sep 14 02:52:54.291309 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:54.291288    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5587bfd74c-2jrfx"
Sep 14 02:52:54.327773 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:54.327741    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5587bfd74c-2jrfx"]
Sep 14 02:52:54.451884 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:54.451845    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-service-ca\") pod \"console-5587bfd74c-2jrfx\" (UID: \"8bd31446-bd3b-4eee-93ef-e31d2d1210f7\") " pod="openshift-console/console-5587bfd74c-2jrfx"
Sep 14 02:52:54.452026 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:54.451902    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-console-serving-cert\") pod \"console-5587bfd74c-2jrfx\" (UID: \"8bd31446-bd3b-4eee-93ef-e31d2d1210f7\") " pod="openshift-console/console-5587bfd74c-2jrfx"
Sep 14 02:52:54.452026 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:54.451936    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29f59\" (UniqueName: \"kubernetes.io/projected/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-kube-api-access-29f59\") pod \"console-5587bfd74c-2jrfx\" (UID: \"8bd31446-bd3b-4eee-93ef-e31d2d1210f7\") " pod="openshift-console/console-5587bfd74c-2jrfx"
Sep 14 02:52:54.452026 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:54.451960    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-trusted-ca-bundle\") pod \"console-5587bfd74c-2jrfx\" (UID: \"8bd31446-bd3b-4eee-93ef-e31d2d1210f7\") " pod="openshift-console/console-5587bfd74c-2jrfx"
Sep 14 02:52:54.452195 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:54.452086    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-console-oauth-config\") pod \"console-5587bfd74c-2jrfx\" (UID: \"8bd31446-bd3b-4eee-93ef-e31d2d1210f7\") " pod="openshift-console/console-5587bfd74c-2jrfx"
Sep 14 02:52:54.452195 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:54.452145    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-oauth-serving-cert\") pod \"console-5587bfd74c-2jrfx\" (UID: \"8bd31446-bd3b-4eee-93ef-e31d2d1210f7\") " pod="openshift-console/console-5587bfd74c-2jrfx"
Sep 14 02:52:54.452263 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:54.452204    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-console-config\") pod \"console-5587bfd74c-2jrfx\" (UID: \"8bd31446-bd3b-4eee-93ef-e31d2d1210f7\") " pod="openshift-console/console-5587bfd74c-2jrfx"
Sep 14 02:52:54.553192 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:54.553112    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29f59\" (UniqueName: \"kubernetes.io/projected/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-kube-api-access-29f59\") pod \"console-5587bfd74c-2jrfx\" (UID: \"8bd31446-bd3b-4eee-93ef-e31d2d1210f7\") " pod="openshift-console/console-5587bfd74c-2jrfx"
Sep 14 02:52:54.553192 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:54.553152    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-trusted-ca-bundle\") pod \"console-5587bfd74c-2jrfx\" (UID: \"8bd31446-bd3b-4eee-93ef-e31d2d1210f7\") " pod="openshift-console/console-5587bfd74c-2jrfx"
Sep 14 02:52:54.553192 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:54.553174    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-console-oauth-config\") pod \"console-5587bfd74c-2jrfx\" (UID: \"8bd31446-bd3b-4eee-93ef-e31d2d1210f7\") " pod="openshift-console/console-5587bfd74c-2jrfx"
Sep 14 02:52:54.553192 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:54.553195    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-oauth-serving-cert\") pod \"console-5587bfd74c-2jrfx\" (UID: \"8bd31446-bd3b-4eee-93ef-e31d2d1210f7\") " pod="openshift-console/console-5587bfd74c-2jrfx"
Sep 14 02:52:54.553545 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:54.553211    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-console-config\") pod \"console-5587bfd74c-2jrfx\" (UID: \"8bd31446-bd3b-4eee-93ef-e31d2d1210f7\") " pod="openshift-console/console-5587bfd74c-2jrfx"
Sep 14 02:52:54.553545 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:54.553238    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-service-ca\") pod \"console-5587bfd74c-2jrfx\" (UID: \"8bd31446-bd3b-4eee-93ef-e31d2d1210f7\") " pod="openshift-console/console-5587bfd74c-2jrfx"
Sep 14 02:52:54.553545 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:54.553295    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-console-serving-cert\") pod \"console-5587bfd74c-2jrfx\" (UID: \"8bd31446-bd3b-4eee-93ef-e31d2d1210f7\") " pod="openshift-console/console-5587bfd74c-2jrfx"
Sep 14 02:52:54.554074 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:54.554049    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-service-ca\") pod \"console-5587bfd74c-2jrfx\" (UID: \"8bd31446-bd3b-4eee-93ef-e31d2d1210f7\") " pod="openshift-console/console-5587bfd74c-2jrfx"
Sep 14 02:52:54.554189 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:54.554051    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-console-config\") pod \"console-5587bfd74c-2jrfx\" (UID: \"8bd31446-bd3b-4eee-93ef-e31d2d1210f7\") " pod="openshift-console/console-5587bfd74c-2jrfx"
Sep 14 02:52:54.554189 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:54.554052    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-oauth-serving-cert\") pod \"console-5587bfd74c-2jrfx\" (UID: \"8bd31446-bd3b-4eee-93ef-e31d2d1210f7\") " pod="openshift-console/console-5587bfd74c-2jrfx"
Sep 14 02:52:54.554189 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:54.554110    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-trusted-ca-bundle\") pod \"console-5587bfd74c-2jrfx\" (UID: \"8bd31446-bd3b-4eee-93ef-e31d2d1210f7\") " pod="openshift-console/console-5587bfd74c-2jrfx"
Sep 14 02:52:54.555718 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:54.555695    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-console-oauth-config\") pod \"console-5587bfd74c-2jrfx\" (UID: \"8bd31446-bd3b-4eee-93ef-e31d2d1210f7\") " pod="openshift-console/console-5587bfd74c-2jrfx"
Sep 14 02:52:54.555908 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:54.555891    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-console-serving-cert\") pod \"console-5587bfd74c-2jrfx\" (UID: \"8bd31446-bd3b-4eee-93ef-e31d2d1210f7\") " pod="openshift-console/console-5587bfd74c-2jrfx"
Sep 14 02:52:54.565990 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:54.565965    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29f59\" (UniqueName: \"kubernetes.io/projected/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-kube-api-access-29f59\") pod \"console-5587bfd74c-2jrfx\" (UID: \"8bd31446-bd3b-4eee-93ef-e31d2d1210f7\") " pod="openshift-console/console-5587bfd74c-2jrfx"
Sep 14 02:52:54.601788 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:54.601766    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5587bfd74c-2jrfx"
Sep 14 02:52:54.887113 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:54.887040    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5587bfd74c-2jrfx"]
Sep 14 02:52:54.890312 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:52:54.890289    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bd31446_bd3b_4eee_93ef_e31d2d1210f7.slice/crio-01f4373aa51d8b9603377d297163721925c33a14170ab0ed2656f207e54f1204 WatchSource:0}: Error finding container 01f4373aa51d8b9603377d297163721925c33a14170ab0ed2656f207e54f1204: Status 404 returned error can't find the container with id 01f4373aa51d8b9603377d297163721925c33a14170ab0ed2656f207e54f1204
Sep 14 02:52:55.243492 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:55.243455    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5587bfd74c-2jrfx" event={"ID":"8bd31446-bd3b-4eee-93ef-e31d2d1210f7","Type":"ContainerStarted","Data":"92398199369a334762a2aa9480968722294bcc5c8687beb4812f4922251df6a2"}
Sep 14 02:52:55.243651 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:55.243515    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5587bfd74c-2jrfx" event={"ID":"8bd31446-bd3b-4eee-93ef-e31d2d1210f7","Type":"ContainerStarted","Data":"01f4373aa51d8b9603377d297163721925c33a14170ab0ed2656f207e54f1204"}
Sep 14 02:52:55.262460 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:55.262393    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5587bfd74c-2jrfx" podStartSLOduration=1.262376145 podStartE2EDuration="1.262376145s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:52:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:55.260002955 +0000 UTC m=+124.237981003" watchObservedRunningTime="2026-09-14 02:52:55.262376145 +0000 UTC m=+124.240354248"
Sep 14 02:52:56.230858 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:52:56.230831    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5798cfc64f-fd7cc"
Sep 14 02:53:01.303111 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:01.303070    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e746671e-19d5-48e7-8b3b-a11462954481-metrics-certs\") pod \"network-metrics-daemon-n7hgz\" (UID: \"e746671e-19d5-48e7-8b3b-a11462954481\") " pod="openshift-multus/network-metrics-daemon-n7hgz"
Sep 14 02:53:01.305351 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:01.305321    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e746671e-19d5-48e7-8b3b-a11462954481-metrics-certs\") pod \"network-metrics-daemon-n7hgz\" (UID: \"e746671e-19d5-48e7-8b3b-a11462954481\") " pod="openshift-multus/network-metrics-daemon-n7hgz"
Sep 14 02:53:01.450835 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:01.450799    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mk9cr\""
Sep 14 02:53:01.458226 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:01.458197    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n7hgz"
Sep 14 02:53:01.536606 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:01.536579    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-ddbbff75f-ddr76"
Sep 14 02:53:01.744705 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:01.744675    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-n7hgz"]
Sep 14 02:53:01.748595 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:53:01.748563    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode746671e_19d5_48e7_8b3b_a11462954481.slice/crio-627fce1b2985956bd1fed6eccb033dd1d2294c399ed8292885369ed54bb1d20a WatchSource:0}: Error finding container 627fce1b2985956bd1fed6eccb033dd1d2294c399ed8292885369ed54bb1d20a: Status 404 returned error can't find the container with id 627fce1b2985956bd1fed6eccb033dd1d2294c399ed8292885369ed54bb1d20a
Sep 14 02:53:02.275018 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:02.274985    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n7hgz" event={"ID":"e746671e-19d5-48e7-8b3b-a11462954481","Type":"ContainerStarted","Data":"627fce1b2985956bd1fed6eccb033dd1d2294c399ed8292885369ed54bb1d20a"}
Sep 14 02:53:03.280675 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:03.280637    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n7hgz" event={"ID":"e746671e-19d5-48e7-8b3b-a11462954481","Type":"ContainerStarted","Data":"71814e6ce85c3c2f06a407b558bcb8db6e5bb5a5649dfeebd7b8873d3a3cdde9"}
Sep 14 02:53:03.281027 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:03.280698    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n7hgz" event={"ID":"e746671e-19d5-48e7-8b3b-a11462954481","Type":"ContainerStarted","Data":"0966d5aa3e3692d3b3a1721cec895ce69ae001e431515842259f20dd3ef5dd2c"}
Sep 14 02:53:03.300117 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:03.300072    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-n7hgz" podStartSLOduration=130.173466503 podStartE2EDuration="2m11.300035419s" totalImagesPullingTime="1.126568916s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:50:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:53:03.298955613 +0000 UTC m=+132.276933641" watchObservedRunningTime="2026-09-14 02:53:03.300035419 +0000 UTC m=+132.278013448"
Sep 14 02:53:04.602612 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:04.602574    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5587bfd74c-2jrfx"
Sep 14 02:53:04.603014 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:04.602680    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5587bfd74c-2jrfx"
Sep 14 02:53:04.607892 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:04.607864    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5587bfd74c-2jrfx"
Sep 14 02:53:05.293717 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:05.293687    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5587bfd74c-2jrfx"
Sep 14 02:53:05.365169 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:05.365135    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-557bbd7fcd-sb9qd"]
Sep 14 02:53:19.269638 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:19.269578    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-ddbbff75f-ddr76" podUID="9f2aa931-4930-419f-8078-13ff29d6a8bc" containerName="console" containerID="cri-o://76301e4af0178a1b0e21f0a5668a7b7bf09097fe5bb055a33ce3d01d8a3ab8c0" gracePeriod=15
Sep 14 02:53:19.511607 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:19.511585    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-ddbbff75f-ddr76"
Sep 14 02:53:19.637328 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:19.637256    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f2aa931-4930-419f-8078-13ff29d6a8bc-trusted-ca-bundle\") pod \"9f2aa931-4930-419f-8078-13ff29d6a8bc\" (UID: \"9f2aa931-4930-419f-8078-13ff29d6a8bc\") "
Sep 14 02:53:19.637328 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:19.637326    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6tbnl\" (UniqueName: \"kubernetes.io/projected/9f2aa931-4930-419f-8078-13ff29d6a8bc-kube-api-access-6tbnl\") pod \"9f2aa931-4930-419f-8078-13ff29d6a8bc\" (UID: \"9f2aa931-4930-419f-8078-13ff29d6a8bc\") "
Sep 14 02:53:19.637507 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:19.637348    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9f2aa931-4930-419f-8078-13ff29d6a8bc-console-config\") pod \"9f2aa931-4930-419f-8078-13ff29d6a8bc\" (UID: \"9f2aa931-4930-419f-8078-13ff29d6a8bc\") "
Sep 14 02:53:19.637507 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:19.637365    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9f2aa931-4930-419f-8078-13ff29d6a8bc-oauth-serving-cert\") pod \"9f2aa931-4930-419f-8078-13ff29d6a8bc\" (UID: \"9f2aa931-4930-419f-8078-13ff29d6a8bc\") "
Sep 14 02:53:19.637507 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:19.637388    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9f2aa931-4930-419f-8078-13ff29d6a8bc-service-ca\") pod \"9f2aa931-4930-419f-8078-13ff29d6a8bc\" (UID: \"9f2aa931-4930-419f-8078-13ff29d6a8bc\") "
Sep 14 02:53:19.637507 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:19.637410    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9f2aa931-4930-419f-8078-13ff29d6a8bc-console-oauth-config\") pod \"9f2aa931-4930-419f-8078-13ff29d6a8bc\" (UID: \"9f2aa931-4930-419f-8078-13ff29d6a8bc\") "
Sep 14 02:53:19.637507 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:19.637460    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9f2aa931-4930-419f-8078-13ff29d6a8bc-console-serving-cert\") pod \"9f2aa931-4930-419f-8078-13ff29d6a8bc\" (UID: \"9f2aa931-4930-419f-8078-13ff29d6a8bc\") "
Sep 14 02:53:19.637801 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:19.637691    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f2aa931-4930-419f-8078-13ff29d6a8bc-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "9f2aa931-4930-419f-8078-13ff29d6a8bc" (UID: "9f2aa931-4930-419f-8078-13ff29d6a8bc"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 02:53:19.637856 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:19.637816    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f2aa931-4930-419f-8078-13ff29d6a8bc-service-ca" (OuterVolumeSpecName: "service-ca") pod "9f2aa931-4930-419f-8078-13ff29d6a8bc" (UID: "9f2aa931-4930-419f-8078-13ff29d6a8bc"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 02:53:19.637856 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:19.637823    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f2aa931-4930-419f-8078-13ff29d6a8bc-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9f2aa931-4930-419f-8078-13ff29d6a8bc" (UID: "9f2aa931-4930-419f-8078-13ff29d6a8bc"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 02:53:19.637856 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:19.637836    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f2aa931-4930-419f-8078-13ff29d6a8bc-console-config" (OuterVolumeSpecName: "console-config") pod "9f2aa931-4930-419f-8078-13ff29d6a8bc" (UID: "9f2aa931-4930-419f-8078-13ff29d6a8bc"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 02:53:19.639633 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:19.639609    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f2aa931-4930-419f-8078-13ff29d6a8bc-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9f2aa931-4930-419f-8078-13ff29d6a8bc" (UID: "9f2aa931-4930-419f-8078-13ff29d6a8bc"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 02:53:19.639723 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:19.639664    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f2aa931-4930-419f-8078-13ff29d6a8bc-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9f2aa931-4930-419f-8078-13ff29d6a8bc" (UID: "9f2aa931-4930-419f-8078-13ff29d6a8bc"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 02:53:19.639723 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:19.639692    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f2aa931-4930-419f-8078-13ff29d6a8bc-kube-api-access-6tbnl" (OuterVolumeSpecName: "kube-api-access-6tbnl") pod "9f2aa931-4930-419f-8078-13ff29d6a8bc" (UID: "9f2aa931-4930-419f-8078-13ff29d6a8bc"). InnerVolumeSpecName "kube-api-access-6tbnl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:53:19.738779 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:19.738759    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f2aa931-4930-419f-8078-13ff29d6a8bc-trusted-ca-bundle\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:53:19.738779 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:19.738777    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tbnl\" (UniqueName: \"kubernetes.io/projected/9f2aa931-4930-419f-8078-13ff29d6a8bc-kube-api-access-6tbnl\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:53:19.738910 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:19.738787    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9f2aa931-4930-419f-8078-13ff29d6a8bc-console-config\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:53:19.738910 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:19.738796    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9f2aa931-4930-419f-8078-13ff29d6a8bc-oauth-serving-cert\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:53:19.738910 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:19.738804    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9f2aa931-4930-419f-8078-13ff29d6a8bc-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:53:19.738910 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:19.738811    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9f2aa931-4930-419f-8078-13ff29d6a8bc-console-oauth-config\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:53:19.738910 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:19.738820    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9f2aa931-4930-419f-8078-13ff29d6a8bc-console-serving-cert\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:53:20.348988 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:20.348960    2577 generic.go:356] "Generic (PLEG): container finished" podID="9f2aa931-4930-419f-8078-13ff29d6a8bc" containerID="76301e4af0178a1b0e21f0a5668a7b7bf09097fe5bb055a33ce3d01d8a3ab8c0" exitCode=0
Sep 14 02:53:20.349370 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:20.348995    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-ddbbff75f-ddr76"
Sep 14 02:53:20.349370 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:20.349014    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-ddbbff75f-ddr76" event={"ID":"9f2aa931-4930-419f-8078-13ff29d6a8bc","Type":"ContainerDied","Data":"76301e4af0178a1b0e21f0a5668a7b7bf09097fe5bb055a33ce3d01d8a3ab8c0"}
Sep 14 02:53:20.349370 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:20.349050    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-ddbbff75f-ddr76" event={"ID":"9f2aa931-4930-419f-8078-13ff29d6a8bc","Type":"ContainerDied","Data":"0825c900b41efc518d35ac819db174fee1154591ac3cd297eaaf573a54e9f0dd"}
Sep 14 02:53:20.349370 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:20.349079    2577 scope.go:117] "RemoveContainer" containerID="76301e4af0178a1b0e21f0a5668a7b7bf09097fe5bb055a33ce3d01d8a3ab8c0"
Sep 14 02:53:20.367565 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:20.367540    2577 scope.go:117] "RemoveContainer" containerID="76301e4af0178a1b0e21f0a5668a7b7bf09097fe5bb055a33ce3d01d8a3ab8c0"
Sep 14 02:53:20.367906 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:53:20.367870    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76301e4af0178a1b0e21f0a5668a7b7bf09097fe5bb055a33ce3d01d8a3ab8c0\": container with ID starting with 76301e4af0178a1b0e21f0a5668a7b7bf09097fe5bb055a33ce3d01d8a3ab8c0 not found: ID does not exist" containerID="76301e4af0178a1b0e21f0a5668a7b7bf09097fe5bb055a33ce3d01d8a3ab8c0"
Sep 14 02:53:20.368032 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:20.367911    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76301e4af0178a1b0e21f0a5668a7b7bf09097fe5bb055a33ce3d01d8a3ab8c0"} err="failed to get container status \"76301e4af0178a1b0e21f0a5668a7b7bf09097fe5bb055a33ce3d01d8a3ab8c0\": rpc error: code = NotFound desc = could not find container \"76301e4af0178a1b0e21f0a5668a7b7bf09097fe5bb055a33ce3d01d8a3ab8c0\": container with ID starting with 76301e4af0178a1b0e21f0a5668a7b7bf09097fe5bb055a33ce3d01d8a3ab8c0 not found: ID does not exist"
Sep 14 02:53:20.369640 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:20.369618    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-ddbbff75f-ddr76"]
Sep 14 02:53:20.373865 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:20.373847    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-ddbbff75f-ddr76"]
Sep 14 02:53:21.616257 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:21.616228    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f2aa931-4930-419f-8078-13ff29d6a8bc" path="/var/lib/kubelet/pods/9f2aa931-4930-419f-8078-13ff29d6a8bc/volumes"
Sep 14 02:53:29.385955 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:29.385910    2577 generic.go:356] "Generic (PLEG): container finished" podID="c10dbb6e-709f-46d7-af54-39fe56f4720e" containerID="c87803b32a8e80cc3632bb3f008b807586b32b4154e53d8b53338ca6308c474d" exitCode=0
Sep 14 02:53:29.386383 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:29.385991    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-s8847" event={"ID":"c10dbb6e-709f-46d7-af54-39fe56f4720e","Type":"ContainerDied","Data":"c87803b32a8e80cc3632bb3f008b807586b32b4154e53d8b53338ca6308c474d"}
Sep 14 02:53:29.386383 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:29.386364    2577 scope.go:117] "RemoveContainer" containerID="c87803b32a8e80cc3632bb3f008b807586b32b4154e53d8b53338ca6308c474d"
Sep 14 02:53:30.388050 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:30.388014    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-557bbd7fcd-sb9qd" podUID="1553f21c-eabf-4881-89b0-8957cc55d52b" containerName="console" containerID="cri-o://05a182cbcd7715294eef8efae32c5d970d6d5b8ac676bca223c61a5fc709d8e8" gracePeriod=15
Sep 14 02:53:30.392368 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:30.392345    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-s8847" event={"ID":"c10dbb6e-709f-46d7-af54-39fe56f4720e","Type":"ContainerStarted","Data":"f356cc2c2bee92e9190fd15cd6d60e90862daf77400c030bdaf3c33437e03230"}
Sep 14 02:53:30.628201 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:30.628176    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-557bbd7fcd-sb9qd"
Sep 14 02:53:30.720174 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:30.720110    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1553f21c-eabf-4881-89b0-8957cc55d52b-oauth-serving-cert\") pod \"1553f21c-eabf-4881-89b0-8957cc55d52b\" (UID: \"1553f21c-eabf-4881-89b0-8957cc55d52b\") "
Sep 14 02:53:30.720174 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:30.720139    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5m65j\" (UniqueName: \"kubernetes.io/projected/1553f21c-eabf-4881-89b0-8957cc55d52b-kube-api-access-5m65j\") pod \"1553f21c-eabf-4881-89b0-8957cc55d52b\" (UID: \"1553f21c-eabf-4881-89b0-8957cc55d52b\") "
Sep 14 02:53:30.720174 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:30.720171    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1553f21c-eabf-4881-89b0-8957cc55d52b-console-serving-cert\") pod \"1553f21c-eabf-4881-89b0-8957cc55d52b\" (UID: \"1553f21c-eabf-4881-89b0-8957cc55d52b\") "
Sep 14 02:53:30.720433 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:30.720328    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1553f21c-eabf-4881-89b0-8957cc55d52b-console-config\") pod \"1553f21c-eabf-4881-89b0-8957cc55d52b\" (UID: \"1553f21c-eabf-4881-89b0-8957cc55d52b\") "
Sep 14 02:53:30.720433 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:30.720370    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1553f21c-eabf-4881-89b0-8957cc55d52b-console-oauth-config\") pod \"1553f21c-eabf-4881-89b0-8957cc55d52b\" (UID: \"1553f21c-eabf-4881-89b0-8957cc55d52b\") "
Sep 14 02:53:30.720544 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:30.720447    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1553f21c-eabf-4881-89b0-8957cc55d52b-service-ca\") pod \"1553f21c-eabf-4881-89b0-8957cc55d52b\" (UID: \"1553f21c-eabf-4881-89b0-8957cc55d52b\") "
Sep 14 02:53:30.720645 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:30.720621    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1553f21c-eabf-4881-89b0-8957cc55d52b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1553f21c-eabf-4881-89b0-8957cc55d52b" (UID: "1553f21c-eabf-4881-89b0-8957cc55d52b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 02:53:30.720712 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:30.720693    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1553f21c-eabf-4881-89b0-8957cc55d52b-trusted-ca-bundle\") pod \"1553f21c-eabf-4881-89b0-8957cc55d52b\" (UID: \"1553f21c-eabf-4881-89b0-8957cc55d52b\") "
Sep 14 02:53:30.720764 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:30.720716    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1553f21c-eabf-4881-89b0-8957cc55d52b-console-config" (OuterVolumeSpecName: "console-config") pod "1553f21c-eabf-4881-89b0-8957cc55d52b" (UID: "1553f21c-eabf-4881-89b0-8957cc55d52b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 02:53:30.720764 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:30.720747    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1553f21c-eabf-4881-89b0-8957cc55d52b-service-ca" (OuterVolumeSpecName: "service-ca") pod "1553f21c-eabf-4881-89b0-8957cc55d52b" (UID: "1553f21c-eabf-4881-89b0-8957cc55d52b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 02:53:30.720956 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:30.720939    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1553f21c-eabf-4881-89b0-8957cc55d52b-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:53:30.721014 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:30.720971    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1553f21c-eabf-4881-89b0-8957cc55d52b-oauth-serving-cert\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:53:30.721014 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:30.720985    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1553f21c-eabf-4881-89b0-8957cc55d52b-console-config\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:53:30.721107 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:30.721025    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1553f21c-eabf-4881-89b0-8957cc55d52b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1553f21c-eabf-4881-89b0-8957cc55d52b" (UID: "1553f21c-eabf-4881-89b0-8957cc55d52b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 02:53:30.722360 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:30.722332    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1553f21c-eabf-4881-89b0-8957cc55d52b-kube-api-access-5m65j" (OuterVolumeSpecName: "kube-api-access-5m65j") pod "1553f21c-eabf-4881-89b0-8957cc55d52b" (UID: "1553f21c-eabf-4881-89b0-8957cc55d52b"). InnerVolumeSpecName "kube-api-access-5m65j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:53:30.722460 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:30.722398    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1553f21c-eabf-4881-89b0-8957cc55d52b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1553f21c-eabf-4881-89b0-8957cc55d52b" (UID: "1553f21c-eabf-4881-89b0-8957cc55d52b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 02:53:30.722629 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:30.722612    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1553f21c-eabf-4881-89b0-8957cc55d52b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1553f21c-eabf-4881-89b0-8957cc55d52b" (UID: "1553f21c-eabf-4881-89b0-8957cc55d52b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 02:53:30.822080 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:30.822059    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1553f21c-eabf-4881-89b0-8957cc55d52b-console-serving-cert\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:53:30.822080 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:30.822079    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1553f21c-eabf-4881-89b0-8957cc55d52b-console-oauth-config\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:53:30.822199 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:30.822089    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1553f21c-eabf-4881-89b0-8957cc55d52b-trusted-ca-bundle\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:53:30.822199 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:30.822097    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5m65j\" (UniqueName: \"kubernetes.io/projected/1553f21c-eabf-4881-89b0-8957cc55d52b-kube-api-access-5m65j\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:53:31.398589 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:31.398558    2577 generic.go:356] "Generic (PLEG): container finished" podID="1553f21c-eabf-4881-89b0-8957cc55d52b" containerID="05a182cbcd7715294eef8efae32c5d970d6d5b8ac676bca223c61a5fc709d8e8" exitCode=0
Sep 14 02:53:31.398913 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:31.398610    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-557bbd7fcd-sb9qd"
Sep 14 02:53:31.398913 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:31.398657    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-557bbd7fcd-sb9qd" event={"ID":"1553f21c-eabf-4881-89b0-8957cc55d52b","Type":"ContainerDied","Data":"05a182cbcd7715294eef8efae32c5d970d6d5b8ac676bca223c61a5fc709d8e8"}
Sep 14 02:53:31.398913 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:31.398719    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-557bbd7fcd-sb9qd" event={"ID":"1553f21c-eabf-4881-89b0-8957cc55d52b","Type":"ContainerDied","Data":"314a903306e9937b47f0d274312935fcff78012e16ece982c0f44fd3006372ea"}
Sep 14 02:53:31.398913 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:31.398742    2577 scope.go:117] "RemoveContainer" containerID="05a182cbcd7715294eef8efae32c5d970d6d5b8ac676bca223c61a5fc709d8e8"
Sep 14 02:53:31.408374 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:31.408348    2577 scope.go:117] "RemoveContainer" containerID="05a182cbcd7715294eef8efae32c5d970d6d5b8ac676bca223c61a5fc709d8e8"
Sep 14 02:53:31.408698 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:53:31.408675    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05a182cbcd7715294eef8efae32c5d970d6d5b8ac676bca223c61a5fc709d8e8\": container with ID starting with 05a182cbcd7715294eef8efae32c5d970d6d5b8ac676bca223c61a5fc709d8e8 not found: ID does not exist" containerID="05a182cbcd7715294eef8efae32c5d970d6d5b8ac676bca223c61a5fc709d8e8"
Sep 14 02:53:31.408774 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:31.408706    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05a182cbcd7715294eef8efae32c5d970d6d5b8ac676bca223c61a5fc709d8e8"} err="failed to get container status \"05a182cbcd7715294eef8efae32c5d970d6d5b8ac676bca223c61a5fc709d8e8\": rpc error: code = NotFound desc = could not find container \"05a182cbcd7715294eef8efae32c5d970d6d5b8ac676bca223c61a5fc709d8e8\": container with ID starting with 05a182cbcd7715294eef8efae32c5d970d6d5b8ac676bca223c61a5fc709d8e8 not found: ID does not exist"
Sep 14 02:53:31.420371 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:31.420347    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-557bbd7fcd-sb9qd"]
Sep 14 02:53:31.424260 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:31.424232    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-557bbd7fcd-sb9qd"]
Sep 14 02:53:31.615997 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:53:31.615973    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1553f21c-eabf-4881-89b0-8957cc55d52b" path="/var/lib/kubelet/pods/1553f21c-eabf-4881-89b0-8957cc55d52b/volumes"
Sep 14 02:54:06.113248 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:06.113218    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-55c47f6d67-mhxtv"]
Sep 14 02:54:06.113696 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:06.113514    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f2aa931-4930-419f-8078-13ff29d6a8bc" containerName="console"
Sep 14 02:54:06.113696 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:06.113528    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f2aa931-4930-419f-8078-13ff29d6a8bc" containerName="console"
Sep 14 02:54:06.113696 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:06.113547    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1553f21c-eabf-4881-89b0-8957cc55d52b" containerName="console"
Sep 14 02:54:06.113696 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:06.113553    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1553f21c-eabf-4881-89b0-8957cc55d52b" containerName="console"
Sep 14 02:54:06.113696 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:06.113599    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f2aa931-4930-419f-8078-13ff29d6a8bc" containerName="console"
Sep 14 02:54:06.113696 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:06.113606    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1553f21c-eabf-4881-89b0-8957cc55d52b" containerName="console"
Sep 14 02:54:06.118283 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:06.118249    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55c47f6d67-mhxtv"
Sep 14 02:54:06.133134 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:06.133114    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55c47f6d67-mhxtv"]
Sep 14 02:54:06.271418 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:06.271400    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e4a51488-03a5-44a8-a6dd-6956392df22f-console-config\") pod \"console-55c47f6d67-mhxtv\" (UID: \"e4a51488-03a5-44a8-a6dd-6956392df22f\") " pod="openshift-console/console-55c47f6d67-mhxtv"
Sep 14 02:54:06.271529 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:06.271426    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e4a51488-03a5-44a8-a6dd-6956392df22f-console-serving-cert\") pod \"console-55c47f6d67-mhxtv\" (UID: \"e4a51488-03a5-44a8-a6dd-6956392df22f\") " pod="openshift-console/console-55c47f6d67-mhxtv"
Sep 14 02:54:06.271529 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:06.271454    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4a51488-03a5-44a8-a6dd-6956392df22f-trusted-ca-bundle\") pod \"console-55c47f6d67-mhxtv\" (UID: \"e4a51488-03a5-44a8-a6dd-6956392df22f\") " pod="openshift-console/console-55c47f6d67-mhxtv"
Sep 14 02:54:06.271529 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:06.271483    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e4a51488-03a5-44a8-a6dd-6956392df22f-service-ca\") pod \"console-55c47f6d67-mhxtv\" (UID: \"e4a51488-03a5-44a8-a6dd-6956392df22f\") " pod="openshift-console/console-55c47f6d67-mhxtv"
Sep 14 02:54:06.271633 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:06.271529    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e4a51488-03a5-44a8-a6dd-6956392df22f-oauth-serving-cert\") pod \"console-55c47f6d67-mhxtv\" (UID: \"e4a51488-03a5-44a8-a6dd-6956392df22f\") " pod="openshift-console/console-55c47f6d67-mhxtv"
Sep 14 02:54:06.271633 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:06.271568    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e4a51488-03a5-44a8-a6dd-6956392df22f-console-oauth-config\") pod \"console-55c47f6d67-mhxtv\" (UID: \"e4a51488-03a5-44a8-a6dd-6956392df22f\") " pod="openshift-console/console-55c47f6d67-mhxtv"
Sep 14 02:54:06.271633 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:06.271594    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4947\" (UniqueName: \"kubernetes.io/projected/e4a51488-03a5-44a8-a6dd-6956392df22f-kube-api-access-l4947\") pod \"console-55c47f6d67-mhxtv\" (UID: \"e4a51488-03a5-44a8-a6dd-6956392df22f\") " pod="openshift-console/console-55c47f6d67-mhxtv"
Sep 14 02:54:06.371984 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:06.371927    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4a51488-03a5-44a8-a6dd-6956392df22f-trusted-ca-bundle\") pod \"console-55c47f6d67-mhxtv\" (UID: \"e4a51488-03a5-44a8-a6dd-6956392df22f\") " pod="openshift-console/console-55c47f6d67-mhxtv"
Sep 14 02:54:06.371984 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:06.371960    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e4a51488-03a5-44a8-a6dd-6956392df22f-service-ca\") pod \"console-55c47f6d67-mhxtv\" (UID: \"e4a51488-03a5-44a8-a6dd-6956392df22f\") " pod="openshift-console/console-55c47f6d67-mhxtv"
Sep 14 02:54:06.371984 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:06.371983    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e4a51488-03a5-44a8-a6dd-6956392df22f-oauth-serving-cert\") pod \"console-55c47f6d67-mhxtv\" (UID: \"e4a51488-03a5-44a8-a6dd-6956392df22f\") " pod="openshift-console/console-55c47f6d67-mhxtv"
Sep 14 02:54:06.372156 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:06.372005    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e4a51488-03a5-44a8-a6dd-6956392df22f-console-oauth-config\") pod \"console-55c47f6d67-mhxtv\" (UID: \"e4a51488-03a5-44a8-a6dd-6956392df22f\") " pod="openshift-console/console-55c47f6d67-mhxtv"
Sep 14 02:54:06.372156 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:06.372129    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4947\" (UniqueName: \"kubernetes.io/projected/e4a51488-03a5-44a8-a6dd-6956392df22f-kube-api-access-l4947\") pod \"console-55c47f6d67-mhxtv\" (UID: \"e4a51488-03a5-44a8-a6dd-6956392df22f\") " pod="openshift-console/console-55c47f6d67-mhxtv"
Sep 14 02:54:06.372250 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:06.372174    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e4a51488-03a5-44a8-a6dd-6956392df22f-console-config\") pod \"console-55c47f6d67-mhxtv\" (UID: \"e4a51488-03a5-44a8-a6dd-6956392df22f\") " pod="openshift-console/console-55c47f6d67-mhxtv"
Sep 14 02:54:06.372250 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:06.372192    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e4a51488-03a5-44a8-a6dd-6956392df22f-console-serving-cert\") pod \"console-55c47f6d67-mhxtv\" (UID: \"e4a51488-03a5-44a8-a6dd-6956392df22f\") " pod="openshift-console/console-55c47f6d67-mhxtv"
Sep 14 02:54:06.372795 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:06.372772    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e4a51488-03a5-44a8-a6dd-6956392df22f-oauth-serving-cert\") pod \"console-55c47f6d67-mhxtv\" (UID: \"e4a51488-03a5-44a8-a6dd-6956392df22f\") " pod="openshift-console/console-55c47f6d67-mhxtv"
Sep 14 02:54:06.372920 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:06.372897    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e4a51488-03a5-44a8-a6dd-6956392df22f-console-config\") pod \"console-55c47f6d67-mhxtv\" (UID: \"e4a51488-03a5-44a8-a6dd-6956392df22f\") " pod="openshift-console/console-55c47f6d67-mhxtv"
Sep 14 02:54:06.372992 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:06.372877    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4a51488-03a5-44a8-a6dd-6956392df22f-trusted-ca-bundle\") pod \"console-55c47f6d67-mhxtv\" (UID: \"e4a51488-03a5-44a8-a6dd-6956392df22f\") " pod="openshift-console/console-55c47f6d67-mhxtv"
Sep 14 02:54:06.372992 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:06.372785    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e4a51488-03a5-44a8-a6dd-6956392df22f-service-ca\") pod \"console-55c47f6d67-mhxtv\" (UID: \"e4a51488-03a5-44a8-a6dd-6956392df22f\") " pod="openshift-console/console-55c47f6d67-mhxtv"
Sep 14 02:54:06.374404 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:06.374386    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e4a51488-03a5-44a8-a6dd-6956392df22f-console-oauth-config\") pod \"console-55c47f6d67-mhxtv\" (UID: \"e4a51488-03a5-44a8-a6dd-6956392df22f\") " pod="openshift-console/console-55c47f6d67-mhxtv"
Sep 14 02:54:06.374593 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:06.374575    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e4a51488-03a5-44a8-a6dd-6956392df22f-console-serving-cert\") pod \"console-55c47f6d67-mhxtv\" (UID: \"e4a51488-03a5-44a8-a6dd-6956392df22f\") " pod="openshift-console/console-55c47f6d67-mhxtv"
Sep 14 02:54:06.383068 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:06.383045    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4947\" (UniqueName: \"kubernetes.io/projected/e4a51488-03a5-44a8-a6dd-6956392df22f-kube-api-access-l4947\") pod \"console-55c47f6d67-mhxtv\" (UID: \"e4a51488-03a5-44a8-a6dd-6956392df22f\") " pod="openshift-console/console-55c47f6d67-mhxtv"
Sep 14 02:54:06.427436 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:06.427403    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55c47f6d67-mhxtv"
Sep 14 02:54:06.707454 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:06.707374    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55c47f6d67-mhxtv"]
Sep 14 02:54:06.710982 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:54:06.710950    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4a51488_03a5_44a8_a6dd_6956392df22f.slice/crio-e1962bce5a75264a9ba314c5781f9d55ba772e32bf1907bd2f2254d32deacaee WatchSource:0}: Error finding container e1962bce5a75264a9ba314c5781f9d55ba772e32bf1907bd2f2254d32deacaee: Status 404 returned error can't find the container with id e1962bce5a75264a9ba314c5781f9d55ba772e32bf1907bd2f2254d32deacaee
Sep 14 02:54:07.538001 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:07.537963    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55c47f6d67-mhxtv" event={"ID":"e4a51488-03a5-44a8-a6dd-6956392df22f","Type":"ContainerStarted","Data":"35b24006644a2a836969623afca0da8ae63dee194f83a1d4bb64f3b16ac790ee"}
Sep 14 02:54:07.538386 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:07.538020    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55c47f6d67-mhxtv" event={"ID":"e4a51488-03a5-44a8-a6dd-6956392df22f","Type":"ContainerStarted","Data":"e1962bce5a75264a9ba314c5781f9d55ba772e32bf1907bd2f2254d32deacaee"}
Sep 14 02:54:07.563378 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:07.563333    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-55c47f6d67-mhxtv" podStartSLOduration=1.5633183160000002 podStartE2EDuration="1.563318316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:54:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:54:07.562018435 +0000 UTC m=+196.539996458" watchObservedRunningTime="2026-09-14 02:54:07.563318316 +0000 UTC m=+196.541296345"
Sep 14 02:54:16.427733 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:16.427701    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-55c47f6d67-mhxtv"
Sep 14 02:54:16.428193 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:16.427754    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-55c47f6d67-mhxtv"
Sep 14 02:54:16.433024 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:16.433001    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-55c47f6d67-mhxtv"
Sep 14 02:54:16.574906 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:16.574884    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-55c47f6d67-mhxtv"
Sep 14 02:54:16.628177 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:16.628146    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5587bfd74c-2jrfx"]
Sep 14 02:54:41.647937 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:41.647896    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5587bfd74c-2jrfx" podUID="8bd31446-bd3b-4eee-93ef-e31d2d1210f7" containerName="console" containerID="cri-o://92398199369a334762a2aa9480968722294bcc5c8687beb4812f4922251df6a2" gracePeriod=15
Sep 14 02:54:41.887925 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:41.887903    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5587bfd74c-2jrfx"
Sep 14 02:54:41.923821 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:41.923762    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-29f59\" (UniqueName: \"kubernetes.io/projected/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-kube-api-access-29f59\") pod \"8bd31446-bd3b-4eee-93ef-e31d2d1210f7\" (UID: \"8bd31446-bd3b-4eee-93ef-e31d2d1210f7\") "
Sep 14 02:54:41.923821 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:41.923797    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-oauth-serving-cert\") pod \"8bd31446-bd3b-4eee-93ef-e31d2d1210f7\" (UID: \"8bd31446-bd3b-4eee-93ef-e31d2d1210f7\") "
Sep 14 02:54:41.924004 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:41.923829    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-console-serving-cert\") pod \"8bd31446-bd3b-4eee-93ef-e31d2d1210f7\" (UID: \"8bd31446-bd3b-4eee-93ef-e31d2d1210f7\") "
Sep 14 02:54:41.924004 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:41.923870    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-console-config\") pod \"8bd31446-bd3b-4eee-93ef-e31d2d1210f7\" (UID: \"8bd31446-bd3b-4eee-93ef-e31d2d1210f7\") "
Sep 14 02:54:41.924004 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:41.923907    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-console-oauth-config\") pod \"8bd31446-bd3b-4eee-93ef-e31d2d1210f7\" (UID: \"8bd31446-bd3b-4eee-93ef-e31d2d1210f7\") "
Sep 14 02:54:41.924004 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:41.923970    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-service-ca\") pod \"8bd31446-bd3b-4eee-93ef-e31d2d1210f7\" (UID: \"8bd31446-bd3b-4eee-93ef-e31d2d1210f7\") "
Sep 14 02:54:41.924004 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:41.923997    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-trusted-ca-bundle\") pod \"8bd31446-bd3b-4eee-93ef-e31d2d1210f7\" (UID: \"8bd31446-bd3b-4eee-93ef-e31d2d1210f7\") "
Sep 14 02:54:41.924300 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:41.924235    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8bd31446-bd3b-4eee-93ef-e31d2d1210f7" (UID: "8bd31446-bd3b-4eee-93ef-e31d2d1210f7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 02:54:41.924383 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:41.924287    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-console-config" (OuterVolumeSpecName: "console-config") pod "8bd31446-bd3b-4eee-93ef-e31d2d1210f7" (UID: "8bd31446-bd3b-4eee-93ef-e31d2d1210f7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 02:54:41.924655 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:41.924610    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8bd31446-bd3b-4eee-93ef-e31d2d1210f7" (UID: "8bd31446-bd3b-4eee-93ef-e31d2d1210f7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 02:54:41.924732 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:41.924696    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-service-ca" (OuterVolumeSpecName: "service-ca") pod "8bd31446-bd3b-4eee-93ef-e31d2d1210f7" (UID: "8bd31446-bd3b-4eee-93ef-e31d2d1210f7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 02:54:41.926092 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:41.926062    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8bd31446-bd3b-4eee-93ef-e31d2d1210f7" (UID: "8bd31446-bd3b-4eee-93ef-e31d2d1210f7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 02:54:41.926172 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:41.926111    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-kube-api-access-29f59" (OuterVolumeSpecName: "kube-api-access-29f59") pod "8bd31446-bd3b-4eee-93ef-e31d2d1210f7" (UID: "8bd31446-bd3b-4eee-93ef-e31d2d1210f7"). InnerVolumeSpecName "kube-api-access-29f59". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:54:41.926405 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:41.926385    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8bd31446-bd3b-4eee-93ef-e31d2d1210f7" (UID: "8bd31446-bd3b-4eee-93ef-e31d2d1210f7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 02:54:42.024946 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:42.024913    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29f59\" (UniqueName: \"kubernetes.io/projected/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-kube-api-access-29f59\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:54:42.024946 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:42.024944    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-oauth-serving-cert\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:54:42.024946 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:42.024955    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-console-serving-cert\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:54:42.025113 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:42.024964    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-console-config\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:54:42.025113 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:42.024973    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-console-oauth-config\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:54:42.025113 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:42.024981    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:54:42.025113 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:42.024989    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bd31446-bd3b-4eee-93ef-e31d2d1210f7-trusted-ca-bundle\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:54:42.672369 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:42.672337    2577 generic.go:356] "Generic (PLEG): container finished" podID="8bd31446-bd3b-4eee-93ef-e31d2d1210f7" containerID="92398199369a334762a2aa9480968722294bcc5c8687beb4812f4922251df6a2" exitCode=0
Sep 14 02:54:42.672732 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:42.672395    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5587bfd74c-2jrfx"
Sep 14 02:54:42.672732 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:42.672427    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5587bfd74c-2jrfx" event={"ID":"8bd31446-bd3b-4eee-93ef-e31d2d1210f7","Type":"ContainerDied","Data":"92398199369a334762a2aa9480968722294bcc5c8687beb4812f4922251df6a2"}
Sep 14 02:54:42.672732 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:42.672481    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5587bfd74c-2jrfx" event={"ID":"8bd31446-bd3b-4eee-93ef-e31d2d1210f7","Type":"ContainerDied","Data":"01f4373aa51d8b9603377d297163721925c33a14170ab0ed2656f207e54f1204"}
Sep 14 02:54:42.672732 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:42.672514    2577 scope.go:117] "RemoveContainer" containerID="92398199369a334762a2aa9480968722294bcc5c8687beb4812f4922251df6a2"
Sep 14 02:54:42.681752 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:42.681731    2577 scope.go:117] "RemoveContainer" containerID="92398199369a334762a2aa9480968722294bcc5c8687beb4812f4922251df6a2"
Sep 14 02:54:42.682065 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:54:42.682037    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92398199369a334762a2aa9480968722294bcc5c8687beb4812f4922251df6a2\": container with ID starting with 92398199369a334762a2aa9480968722294bcc5c8687beb4812f4922251df6a2 not found: ID does not exist" containerID="92398199369a334762a2aa9480968722294bcc5c8687beb4812f4922251df6a2"
Sep 14 02:54:42.682181 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:42.682068    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92398199369a334762a2aa9480968722294bcc5c8687beb4812f4922251df6a2"} err="failed to get container status \"92398199369a334762a2aa9480968722294bcc5c8687beb4812f4922251df6a2\": rpc error: code = NotFound desc = could not find container \"92398199369a334762a2aa9480968722294bcc5c8687beb4812f4922251df6a2\": container with ID starting with 92398199369a334762a2aa9480968722294bcc5c8687beb4812f4922251df6a2 not found: ID does not exist"
Sep 14 02:54:42.695906 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:42.695878    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5587bfd74c-2jrfx"]
Sep 14 02:54:42.702315 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:42.702293    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5587bfd74c-2jrfx"]
Sep 14 02:54:43.616114 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:43.616083    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bd31446-bd3b-4eee-93ef-e31d2d1210f7" path="/var/lib/kubelet/pods/8bd31446-bd3b-4eee-93ef-e31d2d1210f7/volumes"
Sep 14 02:54:59.619511 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:59.619473    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfj5x"]
Sep 14 02:54:59.619907 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:59.619777    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bd31446-bd3b-4eee-93ef-e31d2d1210f7" containerName="console"
Sep 14 02:54:59.619907 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:59.619789    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bd31446-bd3b-4eee-93ef-e31d2d1210f7" containerName="console"
Sep 14 02:54:59.619907 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:59.619852    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bd31446-bd3b-4eee-93ef-e31d2d1210f7" containerName="console"
Sep 14 02:54:59.624223 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:59.624202    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfj5x"
Sep 14 02:54:59.628637 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:59.628216    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 14 02:54:59.628637 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:59.628289    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 14 02:54:59.628637 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:59.628534    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zrfw6\""
Sep 14 02:54:59.635783 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:59.635728    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfj5x"]
Sep 14 02:54:59.652857 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:59.652829    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-452qr\" (UniqueName: \"kubernetes.io/projected/1512810e-d5c1-4b67-a6eb-dbd94650ce89-kube-api-access-452qr\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfj5x\" (UID: \"1512810e-d5c1-4b67-a6eb-dbd94650ce89\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfj5x"
Sep 14 02:54:59.652967 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:59.652947    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1512810e-d5c1-4b67-a6eb-dbd94650ce89-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfj5x\" (UID: \"1512810e-d5c1-4b67-a6eb-dbd94650ce89\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfj5x"
Sep 14 02:54:59.653021 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:59.652987    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1512810e-d5c1-4b67-a6eb-dbd94650ce89-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfj5x\" (UID: \"1512810e-d5c1-4b67-a6eb-dbd94650ce89\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfj5x"
Sep 14 02:54:59.754133 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:59.754106    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-452qr\" (UniqueName: \"kubernetes.io/projected/1512810e-d5c1-4b67-a6eb-dbd94650ce89-kube-api-access-452qr\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfj5x\" (UID: \"1512810e-d5c1-4b67-a6eb-dbd94650ce89\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfj5x"
Sep 14 02:54:59.754229 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:59.754154    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1512810e-d5c1-4b67-a6eb-dbd94650ce89-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfj5x\" (UID: \"1512810e-d5c1-4b67-a6eb-dbd94650ce89\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfj5x"
Sep 14 02:54:59.754315 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:59.754295    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1512810e-d5c1-4b67-a6eb-dbd94650ce89-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfj5x\" (UID: \"1512810e-d5c1-4b67-a6eb-dbd94650ce89\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfj5x"
Sep 14 02:54:59.754549 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:59.754529    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1512810e-d5c1-4b67-a6eb-dbd94650ce89-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfj5x\" (UID: \"1512810e-d5c1-4b67-a6eb-dbd94650ce89\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfj5x"
Sep 14 02:54:59.754588 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:59.754558    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1512810e-d5c1-4b67-a6eb-dbd94650ce89-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfj5x\" (UID: \"1512810e-d5c1-4b67-a6eb-dbd94650ce89\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfj5x"
Sep 14 02:54:59.764355 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:59.764336    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-452qr\" (UniqueName: \"kubernetes.io/projected/1512810e-d5c1-4b67-a6eb-dbd94650ce89-kube-api-access-452qr\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfj5x\" (UID: \"1512810e-d5c1-4b67-a6eb-dbd94650ce89\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfj5x"
Sep 14 02:54:59.939987 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:54:59.939927    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfj5x"
Sep 14 02:55:00.217374 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:00.217299    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfj5x"]
Sep 14 02:55:00.220642 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:55:00.220605    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1512810e_d5c1_4b67_a6eb_dbd94650ce89.slice/crio-00582f17bc70d9d40925a2474f748e7b3171c52b80c68765ab0ac15e7c14011e WatchSource:0}: Error finding container 00582f17bc70d9d40925a2474f748e7b3171c52b80c68765ab0ac15e7c14011e: Status 404 returned error can't find the container with id 00582f17bc70d9d40925a2474f748e7b3171c52b80c68765ab0ac15e7c14011e
Sep 14 02:55:00.743020 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:00.742988    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfj5x" event={"ID":"1512810e-d5c1-4b67-a6eb-dbd94650ce89","Type":"ContainerStarted","Data":"00582f17bc70d9d40925a2474f748e7b3171c52b80c68765ab0ac15e7c14011e"}
Sep 14 02:55:06.770928 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:06.770900    2577 generic.go:356] "Generic (PLEG): container finished" podID="1512810e-d5c1-4b67-a6eb-dbd94650ce89" containerID="962825f15a0afff4c33a2bfbebd616896babdc9cb2ff7b594a1ba1499322fe7f" exitCode=0
Sep 14 02:55:06.771264 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:06.770990    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfj5x" event={"ID":"1512810e-d5c1-4b67-a6eb-dbd94650ce89","Type":"ContainerDied","Data":"962825f15a0afff4c33a2bfbebd616896babdc9cb2ff7b594a1ba1499322fe7f"}
Sep 14 02:55:09.786217 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:09.786184    2577 generic.go:356] "Generic (PLEG): container finished" podID="1512810e-d5c1-4b67-a6eb-dbd94650ce89" containerID="a456acb543f4b4b5bf6277dafe857d1ca46b333d58f1fb23392ad91d1d97e552" exitCode=0
Sep 14 02:55:09.786656 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:09.786291    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfj5x" event={"ID":"1512810e-d5c1-4b67-a6eb-dbd94650ce89","Type":"ContainerDied","Data":"a456acb543f4b4b5bf6277dafe857d1ca46b333d58f1fb23392ad91d1d97e552"}
Sep 14 02:55:16.816549 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:16.816517    2577 generic.go:356] "Generic (PLEG): container finished" podID="1512810e-d5c1-4b67-a6eb-dbd94650ce89" containerID="bf027206df25e65fe456f0cf8d1e2e9529936ab9b65627f0cbf7c1df01c3134c" exitCode=0
Sep 14 02:55:16.816886 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:16.816600    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfj5x" event={"ID":"1512810e-d5c1-4b67-a6eb-dbd94650ce89","Type":"ContainerDied","Data":"bf027206df25e65fe456f0cf8d1e2e9529936ab9b65627f0cbf7c1df01c3134c"}
Sep 14 02:55:17.938622 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:17.938598    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfj5x"
Sep 14 02:55:17.987447 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:17.987426    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-452qr\" (UniqueName: \"kubernetes.io/projected/1512810e-d5c1-4b67-a6eb-dbd94650ce89-kube-api-access-452qr\") pod \"1512810e-d5c1-4b67-a6eb-dbd94650ce89\" (UID: \"1512810e-d5c1-4b67-a6eb-dbd94650ce89\") "
Sep 14 02:55:17.987553 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:17.987458    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1512810e-d5c1-4b67-a6eb-dbd94650ce89-bundle\") pod \"1512810e-d5c1-4b67-a6eb-dbd94650ce89\" (UID: \"1512810e-d5c1-4b67-a6eb-dbd94650ce89\") "
Sep 14 02:55:17.987596 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:17.987548    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1512810e-d5c1-4b67-a6eb-dbd94650ce89-util\") pod \"1512810e-d5c1-4b67-a6eb-dbd94650ce89\" (UID: \"1512810e-d5c1-4b67-a6eb-dbd94650ce89\") "
Sep 14 02:55:17.988081 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:17.988055    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1512810e-d5c1-4b67-a6eb-dbd94650ce89-bundle" (OuterVolumeSpecName: "bundle") pod "1512810e-d5c1-4b67-a6eb-dbd94650ce89" (UID: "1512810e-d5c1-4b67-a6eb-dbd94650ce89"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:55:17.989545 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:17.989524    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1512810e-d5c1-4b67-a6eb-dbd94650ce89-kube-api-access-452qr" (OuterVolumeSpecName: "kube-api-access-452qr") pod "1512810e-d5c1-4b67-a6eb-dbd94650ce89" (UID: "1512810e-d5c1-4b67-a6eb-dbd94650ce89"). InnerVolumeSpecName "kube-api-access-452qr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:55:17.991871 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:17.991848    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1512810e-d5c1-4b67-a6eb-dbd94650ce89-util" (OuterVolumeSpecName: "util") pod "1512810e-d5c1-4b67-a6eb-dbd94650ce89" (UID: "1512810e-d5c1-4b67-a6eb-dbd94650ce89"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:55:18.088587 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:18.088540    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1512810e-d5c1-4b67-a6eb-dbd94650ce89-util\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:55:18.088587 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:18.088559    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-452qr\" (UniqueName: \"kubernetes.io/projected/1512810e-d5c1-4b67-a6eb-dbd94650ce89-kube-api-access-452qr\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:55:18.088587 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:18.088569    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1512810e-d5c1-4b67-a6eb-dbd94650ce89-bundle\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:55:18.825007 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:18.824972    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfj5x" event={"ID":"1512810e-d5c1-4b67-a6eb-dbd94650ce89","Type":"ContainerDied","Data":"00582f17bc70d9d40925a2474f748e7b3171c52b80c68765ab0ac15e7c14011e"}
Sep 14 02:55:18.825194 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:18.825030    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="00582f17bc70d9d40925a2474f748e7b3171c52b80c68765ab0ac15e7c14011e"
Sep 14 02:55:18.825194 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:18.824989    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xfj5x"
Sep 14 02:55:28.420089 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:28.420045    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpdkrn"]
Sep 14 02:55:28.420614 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:28.420378    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1512810e-d5c1-4b67-a6eb-dbd94650ce89" containerName="util"
Sep 14 02:55:28.420614 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:28.420391    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1512810e-d5c1-4b67-a6eb-dbd94650ce89" containerName="util"
Sep 14 02:55:28.420614 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:28.420401    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1512810e-d5c1-4b67-a6eb-dbd94650ce89" containerName="pull"
Sep 14 02:55:28.420614 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:28.420407    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1512810e-d5c1-4b67-a6eb-dbd94650ce89" containerName="pull"
Sep 14 02:55:28.420614 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:28.420414    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1512810e-d5c1-4b67-a6eb-dbd94650ce89" containerName="extract"
Sep 14 02:55:28.420614 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:28.420420    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1512810e-d5c1-4b67-a6eb-dbd94650ce89" containerName="extract"
Sep 14 02:55:28.420614 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:28.420480    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1512810e-d5c1-4b67-a6eb-dbd94650ce89" containerName="extract"
Sep 14 02:55:28.424109 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:28.424083    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpdkrn"
Sep 14 02:55:28.426796 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:28.426767    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 14 02:55:28.427023 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:28.426867    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zrfw6\""
Sep 14 02:55:28.427858 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:28.427835    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 14 02:55:28.482778 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:28.482751    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpdkrn"]
Sep 14 02:55:28.567223 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:28.567198    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3b54e3c7-2330-4e68-a95d-c8c0ddc8df80-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpdkrn\" (UID: \"3b54e3c7-2330-4e68-a95d-c8c0ddc8df80\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpdkrn"
Sep 14 02:55:28.567360 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:28.567232    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3b54e3c7-2330-4e68-a95d-c8c0ddc8df80-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpdkrn\" (UID: \"3b54e3c7-2330-4e68-a95d-c8c0ddc8df80\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpdkrn"
Sep 14 02:55:28.567360 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:28.567254    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7zm8\" (UniqueName: \"kubernetes.io/projected/3b54e3c7-2330-4e68-a95d-c8c0ddc8df80-kube-api-access-x7zm8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpdkrn\" (UID: \"3b54e3c7-2330-4e68-a95d-c8c0ddc8df80\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpdkrn"
Sep 14 02:55:28.667963 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:28.667929    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7zm8\" (UniqueName: \"kubernetes.io/projected/3b54e3c7-2330-4e68-a95d-c8c0ddc8df80-kube-api-access-x7zm8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpdkrn\" (UID: \"3b54e3c7-2330-4e68-a95d-c8c0ddc8df80\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpdkrn"
Sep 14 02:55:28.668086 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:28.668024    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3b54e3c7-2330-4e68-a95d-c8c0ddc8df80-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpdkrn\" (UID: \"3b54e3c7-2330-4e68-a95d-c8c0ddc8df80\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpdkrn"
Sep 14 02:55:28.668086 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:28.668069    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3b54e3c7-2330-4e68-a95d-c8c0ddc8df80-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpdkrn\" (UID: \"3b54e3c7-2330-4e68-a95d-c8c0ddc8df80\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpdkrn"
Sep 14 02:55:28.668487 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:28.668461    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3b54e3c7-2330-4e68-a95d-c8c0ddc8df80-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpdkrn\" (UID: \"3b54e3c7-2330-4e68-a95d-c8c0ddc8df80\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpdkrn"
Sep 14 02:55:28.668578 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:28.668463    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3b54e3c7-2330-4e68-a95d-c8c0ddc8df80-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpdkrn\" (UID: \"3b54e3c7-2330-4e68-a95d-c8c0ddc8df80\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpdkrn"
Sep 14 02:55:28.693124 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:28.693071    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7zm8\" (UniqueName: \"kubernetes.io/projected/3b54e3c7-2330-4e68-a95d-c8c0ddc8df80-kube-api-access-x7zm8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpdkrn\" (UID: \"3b54e3c7-2330-4e68-a95d-c8c0ddc8df80\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpdkrn"
Sep 14 02:55:28.734521 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:28.734497    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpdkrn"
Sep 14 02:55:29.036302 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:29.036208    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpdkrn"]
Sep 14 02:55:29.040320 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:55:29.040290    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b54e3c7_2330_4e68_a95d_c8c0ddc8df80.slice/crio-46849be4db57bd8e484b9bd4cda2f622901994e932e415a89119d6cac84b39b3 WatchSource:0}: Error finding container 46849be4db57bd8e484b9bd4cda2f622901994e932e415a89119d6cac84b39b3: Status 404 returned error can't find the container with id 46849be4db57bd8e484b9bd4cda2f622901994e932e415a89119d6cac84b39b3
Sep 14 02:55:29.869846 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:29.869806    2577 generic.go:356] "Generic (PLEG): container finished" podID="3b54e3c7-2330-4e68-a95d-c8c0ddc8df80" containerID="02af70f270e7d48d257d006c536aa43beac6ab1eabf470f7a1f660e63789c939" exitCode=0
Sep 14 02:55:29.870211 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:29.869893    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpdkrn" event={"ID":"3b54e3c7-2330-4e68-a95d-c8c0ddc8df80","Type":"ContainerDied","Data":"02af70f270e7d48d257d006c536aa43beac6ab1eabf470f7a1f660e63789c939"}
Sep 14 02:55:29.870211 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:29.869977    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpdkrn" event={"ID":"3b54e3c7-2330-4e68-a95d-c8c0ddc8df80","Type":"ContainerStarted","Data":"46849be4db57bd8e484b9bd4cda2f622901994e932e415a89119d6cac84b39b3"}
Sep 14 02:55:32.883791 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:32.883756    2577 generic.go:356] "Generic (PLEG): container finished" podID="3b54e3c7-2330-4e68-a95d-c8c0ddc8df80" containerID="181916dcdb5bac5ed7add67ffc616da3d0f384dce9217a39a57e63f38bb7e728" exitCode=0
Sep 14 02:55:32.884141 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:32.883845    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpdkrn" event={"ID":"3b54e3c7-2330-4e68-a95d-c8c0ddc8df80","Type":"ContainerDied","Data":"181916dcdb5bac5ed7add67ffc616da3d0f384dce9217a39a57e63f38bb7e728"}
Sep 14 02:55:33.890080 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:33.890037    2577 generic.go:356] "Generic (PLEG): container finished" podID="3b54e3c7-2330-4e68-a95d-c8c0ddc8df80" containerID="c421b6e619d68e03e740bb2971dbc132c34851c7574990e39fb913ad69d249c2" exitCode=0
Sep 14 02:55:33.890532 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:33.890094    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpdkrn" event={"ID":"3b54e3c7-2330-4e68-a95d-c8c0ddc8df80","Type":"ContainerDied","Data":"c421b6e619d68e03e740bb2971dbc132c34851c7574990e39fb913ad69d249c2"}
Sep 14 02:55:35.023506 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:35.023478    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpdkrn"
Sep 14 02:55:35.117183 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:35.117157    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3b54e3c7-2330-4e68-a95d-c8c0ddc8df80-util\") pod \"3b54e3c7-2330-4e68-a95d-c8c0ddc8df80\" (UID: \"3b54e3c7-2330-4e68-a95d-c8c0ddc8df80\") "
Sep 14 02:55:35.117331 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:35.117225    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7zm8\" (UniqueName: \"kubernetes.io/projected/3b54e3c7-2330-4e68-a95d-c8c0ddc8df80-kube-api-access-x7zm8\") pod \"3b54e3c7-2330-4e68-a95d-c8c0ddc8df80\" (UID: \"3b54e3c7-2330-4e68-a95d-c8c0ddc8df80\") "
Sep 14 02:55:35.117331 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:35.117251    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3b54e3c7-2330-4e68-a95d-c8c0ddc8df80-bundle\") pod \"3b54e3c7-2330-4e68-a95d-c8c0ddc8df80\" (UID: \"3b54e3c7-2330-4e68-a95d-c8c0ddc8df80\") "
Sep 14 02:55:35.117661 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:35.117635    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b54e3c7-2330-4e68-a95d-c8c0ddc8df80-bundle" (OuterVolumeSpecName: "bundle") pod "3b54e3c7-2330-4e68-a95d-c8c0ddc8df80" (UID: "3b54e3c7-2330-4e68-a95d-c8c0ddc8df80"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:55:35.119307 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:35.119282    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b54e3c7-2330-4e68-a95d-c8c0ddc8df80-kube-api-access-x7zm8" (OuterVolumeSpecName: "kube-api-access-x7zm8") pod "3b54e3c7-2330-4e68-a95d-c8c0ddc8df80" (UID: "3b54e3c7-2330-4e68-a95d-c8c0ddc8df80"). InnerVolumeSpecName "kube-api-access-x7zm8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:55:35.121411 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:35.121390    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b54e3c7-2330-4e68-a95d-c8c0ddc8df80-util" (OuterVolumeSpecName: "util") pod "3b54e3c7-2330-4e68-a95d-c8c0ddc8df80" (UID: "3b54e3c7-2330-4e68-a95d-c8c0ddc8df80"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:55:35.218310 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:35.218246    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7zm8\" (UniqueName: \"kubernetes.io/projected/3b54e3c7-2330-4e68-a95d-c8c0ddc8df80-kube-api-access-x7zm8\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:55:35.218310 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:35.218279    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3b54e3c7-2330-4e68-a95d-c8c0ddc8df80-bundle\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:55:35.218310 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:35.218289    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3b54e3c7-2330-4e68-a95d-c8c0ddc8df80-util\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:55:35.900583 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:35.900521    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpdkrn"
Sep 14 02:55:35.900583 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:35.900549    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpdkrn" event={"ID":"3b54e3c7-2330-4e68-a95d-c8c0ddc8df80","Type":"ContainerDied","Data":"46849be4db57bd8e484b9bd4cda2f622901994e932e415a89119d6cac84b39b3"}
Sep 14 02:55:35.900713 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:35.900602    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="46849be4db57bd8e484b9bd4cda2f622901994e932e415a89119d6cac84b39b3"
Sep 14 02:55:38.986202 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:38.986168    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-6ds7s"]
Sep 14 02:55:38.986599 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:38.986487    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b54e3c7-2330-4e68-a95d-c8c0ddc8df80" containerName="pull"
Sep 14 02:55:38.986599 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:38.986501    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b54e3c7-2330-4e68-a95d-c8c0ddc8df80" containerName="pull"
Sep 14 02:55:38.986599 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:38.986516    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b54e3c7-2330-4e68-a95d-c8c0ddc8df80" containerName="extract"
Sep 14 02:55:38.986599 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:38.986522    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b54e3c7-2330-4e68-a95d-c8c0ddc8df80" containerName="extract"
Sep 14 02:55:38.986599 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:38.986528    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b54e3c7-2330-4e68-a95d-c8c0ddc8df80" containerName="util"
Sep 14 02:55:38.986599 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:38.986533    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b54e3c7-2330-4e68-a95d-c8c0ddc8df80" containerName="util"
Sep 14 02:55:38.986599 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:38.986586    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b54e3c7-2330-4e68-a95d-c8c0ddc8df80" containerName="extract"
Sep 14 02:55:38.990536 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:38.990519    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-6ds7s"
Sep 14 02:55:38.993647 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:38.993622    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 14 02:55:38.994572 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:38.994554    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-cm4pq\""
Sep 14 02:55:38.994681 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:38.994620    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 14 02:55:39.002586 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:39.002563    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-6ds7s"]
Sep 14 02:55:39.145765 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:39.145739    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lkn5\" (UniqueName: \"kubernetes.io/projected/9a0a2adf-bfd7-4545-9d1c-23e3203cc447-kube-api-access-8lkn5\") pod \"cert-manager-55b4d954d5-6ds7s\" (UID: \"9a0a2adf-bfd7-4545-9d1c-23e3203cc447\") " pod="cert-manager/cert-manager-55b4d954d5-6ds7s"
Sep 14 02:55:39.145879 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:39.145800    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9a0a2adf-bfd7-4545-9d1c-23e3203cc447-bound-sa-token\") pod \"cert-manager-55b4d954d5-6ds7s\" (UID: \"9a0a2adf-bfd7-4545-9d1c-23e3203cc447\") " pod="cert-manager/cert-manager-55b4d954d5-6ds7s"
Sep 14 02:55:39.246396 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:39.246332    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9a0a2adf-bfd7-4545-9d1c-23e3203cc447-bound-sa-token\") pod \"cert-manager-55b4d954d5-6ds7s\" (UID: \"9a0a2adf-bfd7-4545-9d1c-23e3203cc447\") " pod="cert-manager/cert-manager-55b4d954d5-6ds7s"
Sep 14 02:55:39.246396 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:39.246375    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lkn5\" (UniqueName: \"kubernetes.io/projected/9a0a2adf-bfd7-4545-9d1c-23e3203cc447-kube-api-access-8lkn5\") pod \"cert-manager-55b4d954d5-6ds7s\" (UID: \"9a0a2adf-bfd7-4545-9d1c-23e3203cc447\") " pod="cert-manager/cert-manager-55b4d954d5-6ds7s"
Sep 14 02:55:39.258029 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:39.257999    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lkn5\" (UniqueName: \"kubernetes.io/projected/9a0a2adf-bfd7-4545-9d1c-23e3203cc447-kube-api-access-8lkn5\") pod \"cert-manager-55b4d954d5-6ds7s\" (UID: \"9a0a2adf-bfd7-4545-9d1c-23e3203cc447\") " pod="cert-manager/cert-manager-55b4d954d5-6ds7s"
Sep 14 02:55:39.262016 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:39.261990    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9a0a2adf-bfd7-4545-9d1c-23e3203cc447-bound-sa-token\") pod \"cert-manager-55b4d954d5-6ds7s\" (UID: \"9a0a2adf-bfd7-4545-9d1c-23e3203cc447\") " pod="cert-manager/cert-manager-55b4d954d5-6ds7s"
Sep 14 02:55:39.311626 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:39.311601    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-6ds7s"
Sep 14 02:55:39.591653 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:39.591583    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-6ds7s"]
Sep 14 02:55:39.594980 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:55:39.594950    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a0a2adf_bfd7_4545_9d1c_23e3203cc447.slice/crio-92bb88075e76f7af4ced6c46fae172f50e05f7ce3da74d5ad308a6ed32c750d0 WatchSource:0}: Error finding container 92bb88075e76f7af4ced6c46fae172f50e05f7ce3da74d5ad308a6ed32c750d0: Status 404 returned error can't find the container with id 92bb88075e76f7af4ced6c46fae172f50e05f7ce3da74d5ad308a6ed32c750d0
Sep 14 02:55:39.917210 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:39.917141    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-6ds7s" event={"ID":"9a0a2adf-bfd7-4545-9d1c-23e3203cc447","Type":"ContainerStarted","Data":"92bb88075e76f7af4ced6c46fae172f50e05f7ce3da74d5ad308a6ed32c750d0"}
Sep 14 02:55:42.931718 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:42.931677    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-6ds7s" event={"ID":"9a0a2adf-bfd7-4545-9d1c-23e3203cc447","Type":"ContainerStarted","Data":"b7b8d259bb2b3af0a0f811ea01d2fea9dd6cd88ab392f9d1b183bbbe7aad6b39"}
Sep 14 02:55:51.444462 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:55:51.444430    2577 kubelet.go:1648] "Image garbage collection succeeded"
Sep 14 02:56:04.881979 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:04.881923    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-6ds7s" podStartSLOduration=24.14598551 podStartE2EDuration="26.881905229s" totalImagesPullingTime="2.735919719s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:55:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:55:42.959881807 +0000 UTC m=+291.937859836" watchObservedRunningTime="2026-09-14 02:56:04.881905229 +0000 UTC m=+313.859883258"
Sep 14 02:56:04.883370 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:04.882832    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd8rth"]
Sep 14 02:56:04.884875 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:04.884860    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd8rth"
Sep 14 02:56:04.887482 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:04.887460    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 14 02:56:04.887482 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:04.887473    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zrfw6\""
Sep 14 02:56:04.888438 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:04.888417    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 14 02:56:04.895747 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:04.895725    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd8rth"]
Sep 14 02:56:05.032053 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:05.032019    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxhsd\" (UniqueName: \"kubernetes.io/projected/c64d7873-c713-48b7-b5cb-eb56dbf90308-kube-api-access-zxhsd\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd8rth\" (UID: \"c64d7873-c713-48b7-b5cb-eb56dbf90308\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd8rth"
Sep 14 02:56:05.032216 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:05.032060    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c64d7873-c713-48b7-b5cb-eb56dbf90308-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd8rth\" (UID: \"c64d7873-c713-48b7-b5cb-eb56dbf90308\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd8rth"
Sep 14 02:56:05.032216 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:05.032093    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c64d7873-c713-48b7-b5cb-eb56dbf90308-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd8rth\" (UID: \"c64d7873-c713-48b7-b5cb-eb56dbf90308\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd8rth"
Sep 14 02:56:05.132821 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:05.132755    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxhsd\" (UniqueName: \"kubernetes.io/projected/c64d7873-c713-48b7-b5cb-eb56dbf90308-kube-api-access-zxhsd\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd8rth\" (UID: \"c64d7873-c713-48b7-b5cb-eb56dbf90308\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd8rth"
Sep 14 02:56:05.132821 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:05.132790    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c64d7873-c713-48b7-b5cb-eb56dbf90308-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd8rth\" (UID: \"c64d7873-c713-48b7-b5cb-eb56dbf90308\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd8rth"
Sep 14 02:56:05.132977 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:05.132838    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c64d7873-c713-48b7-b5cb-eb56dbf90308-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd8rth\" (UID: \"c64d7873-c713-48b7-b5cb-eb56dbf90308\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd8rth"
Sep 14 02:56:05.133098 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:05.133081    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c64d7873-c713-48b7-b5cb-eb56dbf90308-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd8rth\" (UID: \"c64d7873-c713-48b7-b5cb-eb56dbf90308\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd8rth"
Sep 14 02:56:05.133163 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:05.133115    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c64d7873-c713-48b7-b5cb-eb56dbf90308-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd8rth\" (UID: \"c64d7873-c713-48b7-b5cb-eb56dbf90308\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd8rth"
Sep 14 02:56:05.142035 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:05.142007    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxhsd\" (UniqueName: \"kubernetes.io/projected/c64d7873-c713-48b7-b5cb-eb56dbf90308-kube-api-access-zxhsd\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd8rth\" (UID: \"c64d7873-c713-48b7-b5cb-eb56dbf90308\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd8rth"
Sep 14 02:56:05.193316 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:05.193292    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd8rth"
Sep 14 02:56:05.460533 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:05.460399    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd8rth"]
Sep 14 02:56:05.462840 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:56:05.462812    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc64d7873_c713_48b7_b5cb_eb56dbf90308.slice/crio-74681f4427c2ef2398c384718a550e8a661578f751bf81974b9b94ac2f07eaf1 WatchSource:0}: Error finding container 74681f4427c2ef2398c384718a550e8a661578f751bf81974b9b94ac2f07eaf1: Status 404 returned error can't find the container with id 74681f4427c2ef2398c384718a550e8a661578f751bf81974b9b94ac2f07eaf1
Sep 14 02:56:06.022802 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:06.022767    2577 generic.go:356] "Generic (PLEG): container finished" podID="c64d7873-c713-48b7-b5cb-eb56dbf90308" containerID="d8a100fb01f1f8be7e01f0c1ef36b517266cafc7707c2f0d4b27c152e1604a50" exitCode=0
Sep 14 02:56:06.023173 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:06.022857    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd8rth" event={"ID":"c64d7873-c713-48b7-b5cb-eb56dbf90308","Type":"ContainerDied","Data":"d8a100fb01f1f8be7e01f0c1ef36b517266cafc7707c2f0d4b27c152e1604a50"}
Sep 14 02:56:06.023173 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:06.022909    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd8rth" event={"ID":"c64d7873-c713-48b7-b5cb-eb56dbf90308","Type":"ContainerStarted","Data":"74681f4427c2ef2398c384718a550e8a661578f751bf81974b9b94ac2f07eaf1"}
Sep 14 02:56:06.023245 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:06.023201    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 02:56:07.028284 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:07.028241    2577 generic.go:356] "Generic (PLEG): container finished" podID="c64d7873-c713-48b7-b5cb-eb56dbf90308" containerID="046ecc552f05b92737bb198e81a0b525a2aa050a86fe31680f2a1144f3763dd8" exitCode=0
Sep 14 02:56:07.028577 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:07.028300    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd8rth" event={"ID":"c64d7873-c713-48b7-b5cb-eb56dbf90308","Type":"ContainerDied","Data":"046ecc552f05b92737bb198e81a0b525a2aa050a86fe31680f2a1144f3763dd8"}
Sep 14 02:56:08.035249 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:08.035214    2577 generic.go:356] "Generic (PLEG): container finished" podID="c64d7873-c713-48b7-b5cb-eb56dbf90308" containerID="72e5de12a51038f7cc102b43e7515f0c41a4aa1db8110f9256f25b1ec6537fee" exitCode=0
Sep 14 02:56:08.035611 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:08.035302    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd8rth" event={"ID":"c64d7873-c713-48b7-b5cb-eb56dbf90308","Type":"ContainerDied","Data":"72e5de12a51038f7cc102b43e7515f0c41a4aa1db8110f9256f25b1ec6537fee"}
Sep 14 02:56:09.163230 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:09.163204    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd8rth"
Sep 14 02:56:09.264201 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:09.264172    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zxhsd\" (UniqueName: \"kubernetes.io/projected/c64d7873-c713-48b7-b5cb-eb56dbf90308-kube-api-access-zxhsd\") pod \"c64d7873-c713-48b7-b5cb-eb56dbf90308\" (UID: \"c64d7873-c713-48b7-b5cb-eb56dbf90308\") "
Sep 14 02:56:09.264341 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:09.264245    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c64d7873-c713-48b7-b5cb-eb56dbf90308-bundle\") pod \"c64d7873-c713-48b7-b5cb-eb56dbf90308\" (UID: \"c64d7873-c713-48b7-b5cb-eb56dbf90308\") "
Sep 14 02:56:09.264341 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:09.264264    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c64d7873-c713-48b7-b5cb-eb56dbf90308-util\") pod \"c64d7873-c713-48b7-b5cb-eb56dbf90308\" (UID: \"c64d7873-c713-48b7-b5cb-eb56dbf90308\") "
Sep 14 02:56:09.264935 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:09.264911    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c64d7873-c713-48b7-b5cb-eb56dbf90308-bundle" (OuterVolumeSpecName: "bundle") pod "c64d7873-c713-48b7-b5cb-eb56dbf90308" (UID: "c64d7873-c713-48b7-b5cb-eb56dbf90308"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:56:09.266145 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:09.266123    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c64d7873-c713-48b7-b5cb-eb56dbf90308-kube-api-access-zxhsd" (OuterVolumeSpecName: "kube-api-access-zxhsd") pod "c64d7873-c713-48b7-b5cb-eb56dbf90308" (UID: "c64d7873-c713-48b7-b5cb-eb56dbf90308"). InnerVolumeSpecName "kube-api-access-zxhsd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:56:09.269505 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:09.269472    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c64d7873-c713-48b7-b5cb-eb56dbf90308-util" (OuterVolumeSpecName: "util") pod "c64d7873-c713-48b7-b5cb-eb56dbf90308" (UID: "c64d7873-c713-48b7-b5cb-eb56dbf90308"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:56:09.365539 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:09.365471    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxhsd\" (UniqueName: \"kubernetes.io/projected/c64d7873-c713-48b7-b5cb-eb56dbf90308-kube-api-access-zxhsd\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:56:09.365539 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:09.365508    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c64d7873-c713-48b7-b5cb-eb56dbf90308-bundle\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:56:09.365539 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:09.365518    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c64d7873-c713-48b7-b5cb-eb56dbf90308-util\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:56:10.049696 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:10.049672    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd8rth"
Sep 14 02:56:10.049696 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:10.049679    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd8rth" event={"ID":"c64d7873-c713-48b7-b5cb-eb56dbf90308","Type":"ContainerDied","Data":"74681f4427c2ef2398c384718a550e8a661578f751bf81974b9b94ac2f07eaf1"}
Sep 14 02:56:10.049855 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:10.049724    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="74681f4427c2ef2398c384718a550e8a661578f751bf81974b9b94ac2f07eaf1"
Sep 14 02:56:19.090074 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:19.090040    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mvjz6"]
Sep 14 02:56:19.090454 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:19.090329    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c64d7873-c713-48b7-b5cb-eb56dbf90308" containerName="extract"
Sep 14 02:56:19.090454 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:19.090341    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c64d7873-c713-48b7-b5cb-eb56dbf90308" containerName="extract"
Sep 14 02:56:19.090454 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:19.090359    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c64d7873-c713-48b7-b5cb-eb56dbf90308" containerName="util"
Sep 14 02:56:19.090454 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:19.090365    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c64d7873-c713-48b7-b5cb-eb56dbf90308" containerName="util"
Sep 14 02:56:19.090454 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:19.090373    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c64d7873-c713-48b7-b5cb-eb56dbf90308" containerName="pull"
Sep 14 02:56:19.090454 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:19.090378    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c64d7873-c713-48b7-b5cb-eb56dbf90308" containerName="pull"
Sep 14 02:56:19.090454 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:19.090435    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c64d7873-c713-48b7-b5cb-eb56dbf90308" containerName="extract"
Sep 14 02:56:19.093399 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:19.093383    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mvjz6"
Sep 14 02:56:19.096986 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:19.096964    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zrfw6\""
Sep 14 02:56:19.097111 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:19.097039    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 14 02:56:19.100001 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:19.099973    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 14 02:56:19.105902 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:19.105884    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mvjz6"]
Sep 14 02:56:19.130242 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:19.130218    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e3730e58-3cc0-4604-aab3-304bdd601e7b-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mvjz6\" (UID: \"e3730e58-3cc0-4604-aab3-304bdd601e7b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mvjz6"
Sep 14 02:56:19.130370 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:19.130257    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m992m\" (UniqueName: \"kubernetes.io/projected/e3730e58-3cc0-4604-aab3-304bdd601e7b-kube-api-access-m992m\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mvjz6\" (UID: \"e3730e58-3cc0-4604-aab3-304bdd601e7b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mvjz6"
Sep 14 02:56:19.130437 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:19.130392    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e3730e58-3cc0-4604-aab3-304bdd601e7b-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mvjz6\" (UID: \"e3730e58-3cc0-4604-aab3-304bdd601e7b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mvjz6"
Sep 14 02:56:19.231096 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:19.231069    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m992m\" (UniqueName: \"kubernetes.io/projected/e3730e58-3cc0-4604-aab3-304bdd601e7b-kube-api-access-m992m\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mvjz6\" (UID: \"e3730e58-3cc0-4604-aab3-304bdd601e7b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mvjz6"
Sep 14 02:56:19.231194 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:19.231120    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e3730e58-3cc0-4604-aab3-304bdd601e7b-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mvjz6\" (UID: \"e3730e58-3cc0-4604-aab3-304bdd601e7b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mvjz6"
Sep 14 02:56:19.231194 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:19.231185    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e3730e58-3cc0-4604-aab3-304bdd601e7b-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mvjz6\" (UID: \"e3730e58-3cc0-4604-aab3-304bdd601e7b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mvjz6"
Sep 14 02:56:19.231509 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:19.231490    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e3730e58-3cc0-4604-aab3-304bdd601e7b-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mvjz6\" (UID: \"e3730e58-3cc0-4604-aab3-304bdd601e7b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mvjz6"
Sep 14 02:56:19.231577 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:19.231526    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e3730e58-3cc0-4604-aab3-304bdd601e7b-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mvjz6\" (UID: \"e3730e58-3cc0-4604-aab3-304bdd601e7b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mvjz6"
Sep 14 02:56:19.265686 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:19.265664    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m992m\" (UniqueName: \"kubernetes.io/projected/e3730e58-3cc0-4604-aab3-304bdd601e7b-kube-api-access-m992m\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mvjz6\" (UID: \"e3730e58-3cc0-4604-aab3-304bdd601e7b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mvjz6"
Sep 14 02:56:19.402330 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:19.402280    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mvjz6"
Sep 14 02:56:19.679246 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:19.679222    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mvjz6"]
Sep 14 02:56:19.681725 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:56:19.681700    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3730e58_3cc0_4604_aab3_304bdd601e7b.slice/crio-ae8091b8a636299e9570a73e6e22e12de8fe236057fa5a7676795e255b989cb6 WatchSource:0}: Error finding container ae8091b8a636299e9570a73e6e22e12de8fe236057fa5a7676795e255b989cb6: Status 404 returned error can't find the container with id ae8091b8a636299e9570a73e6e22e12de8fe236057fa5a7676795e255b989cb6
Sep 14 02:56:20.091806 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:20.091773    2577 generic.go:356] "Generic (PLEG): container finished" podID="e3730e58-3cc0-4604-aab3-304bdd601e7b" containerID="ac1d087cdfa476b8f83cec4e533e4d37e32dfee1405e3b1725775e57764e8f8f" exitCode=0
Sep 14 02:56:20.092128 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:20.091859    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mvjz6" event={"ID":"e3730e58-3cc0-4604-aab3-304bdd601e7b","Type":"ContainerDied","Data":"ac1d087cdfa476b8f83cec4e533e4d37e32dfee1405e3b1725775e57764e8f8f"}
Sep 14 02:56:20.092128 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:20.091907    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mvjz6" event={"ID":"e3730e58-3cc0-4604-aab3-304bdd601e7b","Type":"ContainerStarted","Data":"ae8091b8a636299e9570a73e6e22e12de8fe236057fa5a7676795e255b989cb6"}
Sep 14 02:56:21.097500 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:21.097473    2577 generic.go:356] "Generic (PLEG): container finished" podID="e3730e58-3cc0-4604-aab3-304bdd601e7b" containerID="34266dda8d747780da025e3e8fe96b417341b8a85348ad252cdbe7502015c999" exitCode=0
Sep 14 02:56:21.097802 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:21.097533    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mvjz6" event={"ID":"e3730e58-3cc0-4604-aab3-304bdd601e7b","Type":"ContainerDied","Data":"34266dda8d747780da025e3e8fe96b417341b8a85348ad252cdbe7502015c999"}
Sep 14 02:56:21.589384 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:21.589356    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f647699c6-58rls"]
Sep 14 02:56:21.592519 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:21.592503    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-58rls"
Sep 14 02:56:21.595091 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:21.595071    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 14 02:56:21.595564 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:21.595550    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 14 02:56:21.595643 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:21.595582    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 14 02:56:21.595701 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:21.595660    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 14 02:56:21.595809 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:21.595790    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-q2bns\""
Sep 14 02:56:21.617192 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:21.617167    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f647699c6-58rls"]
Sep 14 02:56:21.646774 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:21.646745    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f9f740b2-f51d-42b7-9867-a864ac7e05ee-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f647699c6-58rls\" (UID: \"f9f740b2-f51d-42b7-9867-a864ac7e05ee\") " pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-58rls"
Sep 14 02:56:21.646867 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:21.646787    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f9f740b2-f51d-42b7-9867-a864ac7e05ee-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f647699c6-58rls\" (UID: \"f9f740b2-f51d-42b7-9867-a864ac7e05ee\") " pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-58rls"
Sep 14 02:56:21.646917 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:21.646874    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f9f740b2-f51d-42b7-9867-a864ac7e05ee-manifests\") pod \"opendatahub-operator-controller-manager-6f647699c6-58rls\" (UID: \"f9f740b2-f51d-42b7-9867-a864ac7e05ee\") " pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-58rls"
Sep 14 02:56:21.646917 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:21.646898    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpnmw\" (UniqueName: \"kubernetes.io/projected/f9f740b2-f51d-42b7-9867-a864ac7e05ee-kube-api-access-gpnmw\") pod \"opendatahub-operator-controller-manager-6f647699c6-58rls\" (UID: \"f9f740b2-f51d-42b7-9867-a864ac7e05ee\") " pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-58rls"
Sep 14 02:56:21.647007 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:21.646919    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f9f740b2-f51d-42b7-9867-a864ac7e05ee-tmp\") pod \"opendatahub-operator-controller-manager-6f647699c6-58rls\" (UID: \"f9f740b2-f51d-42b7-9867-a864ac7e05ee\") " pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-58rls"
Sep 14 02:56:21.748081 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:21.748055    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f9f740b2-f51d-42b7-9867-a864ac7e05ee-tmp\") pod \"opendatahub-operator-controller-manager-6f647699c6-58rls\" (UID: \"f9f740b2-f51d-42b7-9867-a864ac7e05ee\") " pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-58rls"
Sep 14 02:56:21.748195 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:21.748108    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f9f740b2-f51d-42b7-9867-a864ac7e05ee-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f647699c6-58rls\" (UID: \"f9f740b2-f51d-42b7-9867-a864ac7e05ee\") " pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-58rls"
Sep 14 02:56:21.748195 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:21.748141    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f9f740b2-f51d-42b7-9867-a864ac7e05ee-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f647699c6-58rls\" (UID: \"f9f740b2-f51d-42b7-9867-a864ac7e05ee\") " pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-58rls"
Sep 14 02:56:21.748330 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:21.748210    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f9f740b2-f51d-42b7-9867-a864ac7e05ee-manifests\") pod \"opendatahub-operator-controller-manager-6f647699c6-58rls\" (UID: \"f9f740b2-f51d-42b7-9867-a864ac7e05ee\") " pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-58rls"
Sep 14 02:56:21.748330 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:21.748239    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpnmw\" (UniqueName: \"kubernetes.io/projected/f9f740b2-f51d-42b7-9867-a864ac7e05ee-kube-api-access-gpnmw\") pod \"opendatahub-operator-controller-manager-6f647699c6-58rls\" (UID: \"f9f740b2-f51d-42b7-9867-a864ac7e05ee\") " pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-58rls"
Sep 14 02:56:21.748552 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:21.748532    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f9f740b2-f51d-42b7-9867-a864ac7e05ee-manifests\") pod \"opendatahub-operator-controller-manager-6f647699c6-58rls\" (UID: \"f9f740b2-f51d-42b7-9867-a864ac7e05ee\") " pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-58rls"
Sep 14 02:56:21.750286 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:21.750243    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f9f740b2-f51d-42b7-9867-a864ac7e05ee-tmp\") pod \"opendatahub-operator-controller-manager-6f647699c6-58rls\" (UID: \"f9f740b2-f51d-42b7-9867-a864ac7e05ee\") " pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-58rls"
Sep 14 02:56:21.750495 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:21.750478    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f9f740b2-f51d-42b7-9867-a864ac7e05ee-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f647699c6-58rls\" (UID: \"f9f740b2-f51d-42b7-9867-a864ac7e05ee\") " pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-58rls"
Sep 14 02:56:21.750571 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:21.750553    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f9f740b2-f51d-42b7-9867-a864ac7e05ee-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f647699c6-58rls\" (UID: \"f9f740b2-f51d-42b7-9867-a864ac7e05ee\") " pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-58rls"
Sep 14 02:56:21.756904 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:21.756876    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpnmw\" (UniqueName: \"kubernetes.io/projected/f9f740b2-f51d-42b7-9867-a864ac7e05ee-kube-api-access-gpnmw\") pod \"opendatahub-operator-controller-manager-6f647699c6-58rls\" (UID: \"f9f740b2-f51d-42b7-9867-a864ac7e05ee\") " pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-58rls"
Sep 14 02:56:21.902500 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:21.902451    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-58rls"
Sep 14 02:56:22.104024 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:22.103997    2577 generic.go:356] "Generic (PLEG): container finished" podID="e3730e58-3cc0-4604-aab3-304bdd601e7b" containerID="56d8a9007bf48c04cb440f8d5258719443e83b8209b07a0fda71437b6d4b427a" exitCode=0
Sep 14 02:56:22.104364 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:22.104091    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mvjz6" event={"ID":"e3730e58-3cc0-4604-aab3-304bdd601e7b","Type":"ContainerDied","Data":"56d8a9007bf48c04cb440f8d5258719443e83b8209b07a0fda71437b6d4b427a"}
Sep 14 02:56:22.190332 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:22.190302    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f647699c6-58rls"]
Sep 14 02:56:22.193069 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:56:22.193039    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9f740b2_f51d_42b7_9867_a864ac7e05ee.slice/crio-fa652e37c92786d4adf92ff3e78b7572213866b08f856fd99112f5cae402e704 WatchSource:0}: Error finding container fa652e37c92786d4adf92ff3e78b7572213866b08f856fd99112f5cae402e704: Status 404 returned error can't find the container with id fa652e37c92786d4adf92ff3e78b7572213866b08f856fd99112f5cae402e704
Sep 14 02:56:23.117055 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:23.117001    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-58rls" event={"ID":"f9f740b2-f51d-42b7-9867-a864ac7e05ee","Type":"ContainerStarted","Data":"fa652e37c92786d4adf92ff3e78b7572213866b08f856fd99112f5cae402e704"}
Sep 14 02:56:23.262143 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:23.262102    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mvjz6"
Sep 14 02:56:23.359708 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:23.359663    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e3730e58-3cc0-4604-aab3-304bdd601e7b-util\") pod \"e3730e58-3cc0-4604-aab3-304bdd601e7b\" (UID: \"e3730e58-3cc0-4604-aab3-304bdd601e7b\") "
Sep 14 02:56:23.359898 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:23.359781    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e3730e58-3cc0-4604-aab3-304bdd601e7b-bundle\") pod \"e3730e58-3cc0-4604-aab3-304bdd601e7b\" (UID: \"e3730e58-3cc0-4604-aab3-304bdd601e7b\") "
Sep 14 02:56:23.359898 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:23.359818    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m992m\" (UniqueName: \"kubernetes.io/projected/e3730e58-3cc0-4604-aab3-304bdd601e7b-kube-api-access-m992m\") pod \"e3730e58-3cc0-4604-aab3-304bdd601e7b\" (UID: \"e3730e58-3cc0-4604-aab3-304bdd601e7b\") "
Sep 14 02:56:23.361019 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:23.360952    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3730e58-3cc0-4604-aab3-304bdd601e7b-bundle" (OuterVolumeSpecName: "bundle") pod "e3730e58-3cc0-4604-aab3-304bdd601e7b" (UID: "e3730e58-3cc0-4604-aab3-304bdd601e7b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:56:23.363962 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:23.363932    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3730e58-3cc0-4604-aab3-304bdd601e7b-kube-api-access-m992m" (OuterVolumeSpecName: "kube-api-access-m992m") pod "e3730e58-3cc0-4604-aab3-304bdd601e7b" (UID: "e3730e58-3cc0-4604-aab3-304bdd601e7b"). InnerVolumeSpecName "kube-api-access-m992m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:56:23.368485 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:23.368428    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3730e58-3cc0-4604-aab3-304bdd601e7b-util" (OuterVolumeSpecName: "util") pod "e3730e58-3cc0-4604-aab3-304bdd601e7b" (UID: "e3730e58-3cc0-4604-aab3-304bdd601e7b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:56:23.461284 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:23.461246    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e3730e58-3cc0-4604-aab3-304bdd601e7b-bundle\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:56:23.461398 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:23.461299    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m992m\" (UniqueName: \"kubernetes.io/projected/e3730e58-3cc0-4604-aab3-304bdd601e7b-kube-api-access-m992m\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:56:23.461398 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:23.461317    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e3730e58-3cc0-4604-aab3-304bdd601e7b-util\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:56:24.124955 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:24.124925    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mvjz6"
Sep 14 02:56:24.125442 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:24.124920    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mvjz6" event={"ID":"e3730e58-3cc0-4604-aab3-304bdd601e7b","Type":"ContainerDied","Data":"ae8091b8a636299e9570a73e6e22e12de8fe236057fa5a7676795e255b989cb6"}
Sep 14 02:56:24.125442 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:24.125062    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae8091b8a636299e9570a73e6e22e12de8fe236057fa5a7676795e255b989cb6"
Sep 14 02:56:26.137760 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:26.137718    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-58rls" event={"ID":"f9f740b2-f51d-42b7-9867-a864ac7e05ee","Type":"ContainerStarted","Data":"daae1b4c51a9c38e12a35892ca5ac58687135849d3089ad9712cbcbea6e4534a"}
Sep 14 02:56:27.144063 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:27.144025    2577 generic.go:356] "Generic (PLEG): container finished" podID="f9f740b2-f51d-42b7-9867-a864ac7e05ee" containerID="daae1b4c51a9c38e12a35892ca5ac58687135849d3089ad9712cbcbea6e4534a" exitCode=0
Sep 14 02:56:27.144523 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:27.144125    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-58rls" event={"ID":"f9f740b2-f51d-42b7-9867-a864ac7e05ee","Type":"ContainerDied","Data":"daae1b4c51a9c38e12a35892ca5ac58687135849d3089ad9712cbcbea6e4534a"}
Sep 14 02:56:28.150829 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:28.150744    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-58rls" event={"ID":"f9f740b2-f51d-42b7-9867-a864ac7e05ee","Type":"ContainerStarted","Data":"67b46c454b5c2c4519fe584e49bbb6978a8edc5bfa96f81138f233c5de86e186"}
Sep 14 02:56:28.151287 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:28.150850    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-58rls"
Sep 14 02:56:28.180393 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:28.180347    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-58rls" podStartSLOduration=2.459954825 podStartE2EDuration="7.180332545s" totalImagesPullingTime="3.716505475s" totalInitContainerRuntime="1.003872245s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:56:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:56:28.177933346 +0000 UTC m=+337.155911375" watchObservedRunningTime="2026-09-14 02:56:28.180332545 +0000 UTC m=+337.158310574"
Sep 14 02:56:39.158136 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:39.158093    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6f647699c6-58rls"
Sep 14 02:56:41.489345 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:41.489303    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-66f955bb57-d45jp"]
Sep 14 02:56:41.489736 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:41.489631    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3730e58-3cc0-4604-aab3-304bdd601e7b" containerName="util"
Sep 14 02:56:41.489736 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:41.489643    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3730e58-3cc0-4604-aab3-304bdd601e7b" containerName="util"
Sep 14 02:56:41.489736 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:41.489653    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3730e58-3cc0-4604-aab3-304bdd601e7b" containerName="pull"
Sep 14 02:56:41.489736 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:41.489658    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3730e58-3cc0-4604-aab3-304bdd601e7b" containerName="pull"
Sep 14 02:56:41.489736 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:41.489664    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3730e58-3cc0-4604-aab3-304bdd601e7b" containerName="extract"
Sep 14 02:56:41.489736 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:41.489671    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3730e58-3cc0-4604-aab3-304bdd601e7b" containerName="extract"
Sep 14 02:56:41.489736 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:41.489725    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3730e58-3cc0-4604-aab3-304bdd601e7b" containerName="extract"
Sep 14 02:56:41.499726 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:41.499700    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-66f955bb57-d45jp"
Sep 14 02:56:41.517820 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:41.517789    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 14 02:56:41.517950 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:41.517840    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 14 02:56:41.519690 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:41.519664    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-66f955bb57-d45jp"]
Sep 14 02:56:41.595461 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:41.595437    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b30e897-08f8-4bcd-a435-380fcd3ce905-tls-certs\") pod \"kube-auth-proxy-66f955bb57-d45jp\" (UID: \"1b30e897-08f8-4bcd-a435-380fcd3ce905\") " pod="openshift-ingress/kube-auth-proxy-66f955bb57-d45jp"
Sep 14 02:56:41.595552 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:41.595481    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b30e897-08f8-4bcd-a435-380fcd3ce905-tmp\") pod \"kube-auth-proxy-66f955bb57-d45jp\" (UID: \"1b30e897-08f8-4bcd-a435-380fcd3ce905\") " pod="openshift-ingress/kube-auth-proxy-66f955bb57-d45jp"
Sep 14 02:56:41.595552 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:41.595514    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5gbw\" (UniqueName: \"kubernetes.io/projected/1b30e897-08f8-4bcd-a435-380fcd3ce905-kube-api-access-v5gbw\") pod \"kube-auth-proxy-66f955bb57-d45jp\" (UID: \"1b30e897-08f8-4bcd-a435-380fcd3ce905\") " pod="openshift-ingress/kube-auth-proxy-66f955bb57-d45jp"
Sep 14 02:56:41.696136 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:41.696110    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b30e897-08f8-4bcd-a435-380fcd3ce905-tmp\") pod \"kube-auth-proxy-66f955bb57-d45jp\" (UID: \"1b30e897-08f8-4bcd-a435-380fcd3ce905\") " pod="openshift-ingress/kube-auth-proxy-66f955bb57-d45jp"
Sep 14 02:56:41.696230 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:41.696151    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5gbw\" (UniqueName: \"kubernetes.io/projected/1b30e897-08f8-4bcd-a435-380fcd3ce905-kube-api-access-v5gbw\") pod \"kube-auth-proxy-66f955bb57-d45jp\" (UID: \"1b30e897-08f8-4bcd-a435-380fcd3ce905\") " pod="openshift-ingress/kube-auth-proxy-66f955bb57-d45jp"
Sep 14 02:56:41.696230 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:41.696179    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b30e897-08f8-4bcd-a435-380fcd3ce905-tls-certs\") pod \"kube-auth-proxy-66f955bb57-d45jp\" (UID: \"1b30e897-08f8-4bcd-a435-380fcd3ce905\") " pod="openshift-ingress/kube-auth-proxy-66f955bb57-d45jp"
Sep 14 02:56:41.698446 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:41.698420    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b30e897-08f8-4bcd-a435-380fcd3ce905-tmp\") pod \"kube-auth-proxy-66f955bb57-d45jp\" (UID: \"1b30e897-08f8-4bcd-a435-380fcd3ce905\") " pod="openshift-ingress/kube-auth-proxy-66f955bb57-d45jp"
Sep 14 02:56:41.698698 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:41.698679    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b30e897-08f8-4bcd-a435-380fcd3ce905-tls-certs\") pod \"kube-auth-proxy-66f955bb57-d45jp\" (UID: \"1b30e897-08f8-4bcd-a435-380fcd3ce905\") " pod="openshift-ingress/kube-auth-proxy-66f955bb57-d45jp"
Sep 14 02:56:41.708473 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:41.708450    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5gbw\" (UniqueName: \"kubernetes.io/projected/1b30e897-08f8-4bcd-a435-380fcd3ce905-kube-api-access-v5gbw\") pod \"kube-auth-proxy-66f955bb57-d45jp\" (UID: \"1b30e897-08f8-4bcd-a435-380fcd3ce905\") " pod="openshift-ingress/kube-auth-proxy-66f955bb57-d45jp"
Sep 14 02:56:41.809230 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:41.809180    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-66f955bb57-d45jp"
Sep 14 02:56:42.081573 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:42.081549    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-66f955bb57-d45jp"]
Sep 14 02:56:42.083222 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:56:42.083189    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b30e897_08f8_4bcd_a435_380fcd3ce905.slice/crio-0c759af50ca2b0e3cbd345007f18b7b67659708bfb6a08de3fb503b4101052ee WatchSource:0}: Error finding container 0c759af50ca2b0e3cbd345007f18b7b67659708bfb6a08de3fb503b4101052ee: Status 404 returned error can't find the container with id 0c759af50ca2b0e3cbd345007f18b7b67659708bfb6a08de3fb503b4101052ee
Sep 14 02:56:42.220183 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:42.220156    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-66f955bb57-d45jp" event={"ID":"1b30e897-08f8-4bcd-a435-380fcd3ce905","Type":"ContainerStarted","Data":"0c759af50ca2b0e3cbd345007f18b7b67659708bfb6a08de3fb503b4101052ee"}
Sep 14 02:56:43.227806 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:43.227769    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5"]
Sep 14 02:56:43.230209 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:43.230189    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5"
Sep 14 02:56:43.232823 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:43.232792    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 14 02:56:43.232823 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:43.232814    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 14 02:56:43.232972 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:43.232830    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 14 02:56:43.232972 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:43.232792    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 14 02:56:43.233080 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:43.233061    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-pncbl\""
Sep 14 02:56:43.245137 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:43.245113    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5"]
Sep 14 02:56:43.306164 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:43.306134    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dfec4394-69dc-4a1d-9421-913fb3eb3213-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-cqtx5\" (UID: \"dfec4394-69dc-4a1d-9421-913fb3eb3213\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5"
Sep 14 02:56:43.306318 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:43.306174    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dfec4394-69dc-4a1d-9421-913fb3eb3213-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-cqtx5\" (UID: \"dfec4394-69dc-4a1d-9421-913fb3eb3213\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5"
Sep 14 02:56:43.306318 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:43.306203    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dfec4394-69dc-4a1d-9421-913fb3eb3213-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-cqtx5\" (UID: \"dfec4394-69dc-4a1d-9421-913fb3eb3213\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5"
Sep 14 02:56:43.306318 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:43.306255    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dfec4394-69dc-4a1d-9421-913fb3eb3213-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-cqtx5\" (UID: \"dfec4394-69dc-4a1d-9421-913fb3eb3213\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5"
Sep 14 02:56:43.306318 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:43.306308    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzzcr\" (UniqueName: \"kubernetes.io/projected/dfec4394-69dc-4a1d-9421-913fb3eb3213-kube-api-access-gzzcr\") pod \"istiod-openshift-gateway-7b7995d5cd-cqtx5\" (UID: \"dfec4394-69dc-4a1d-9421-913fb3eb3213\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5"
Sep 14 02:56:43.306552 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:43.306353    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dfec4394-69dc-4a1d-9421-913fb3eb3213-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-cqtx5\" (UID: \"dfec4394-69dc-4a1d-9421-913fb3eb3213\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5"
Sep 14 02:56:43.306552 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:43.306372    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dfec4394-69dc-4a1d-9421-913fb3eb3213-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-cqtx5\" (UID: \"dfec4394-69dc-4a1d-9421-913fb3eb3213\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5"
Sep 14 02:56:43.406901 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:43.406864    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dfec4394-69dc-4a1d-9421-913fb3eb3213-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-cqtx5\" (UID: \"dfec4394-69dc-4a1d-9421-913fb3eb3213\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5"
Sep 14 02:56:43.407043 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:43.406993    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dfec4394-69dc-4a1d-9421-913fb3eb3213-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-cqtx5\" (UID: \"dfec4394-69dc-4a1d-9421-913fb3eb3213\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5"
Sep 14 02:56:43.407043 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:43.407025    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dfec4394-69dc-4a1d-9421-913fb3eb3213-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-cqtx5\" (UID: \"dfec4394-69dc-4a1d-9421-913fb3eb3213\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5"
Sep 14 02:56:43.407169 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:43.407061    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dfec4394-69dc-4a1d-9421-913fb3eb3213-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-cqtx5\" (UID: \"dfec4394-69dc-4a1d-9421-913fb3eb3213\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5"
Sep 14 02:56:43.407533 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:43.407499    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzzcr\" (UniqueName: \"kubernetes.io/projected/dfec4394-69dc-4a1d-9421-913fb3eb3213-kube-api-access-gzzcr\") pod \"istiod-openshift-gateway-7b7995d5cd-cqtx5\" (UID: \"dfec4394-69dc-4a1d-9421-913fb3eb3213\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5"
Sep 14 02:56:43.407659 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:43.407555    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dfec4394-69dc-4a1d-9421-913fb3eb3213-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-cqtx5\" (UID: \"dfec4394-69dc-4a1d-9421-913fb3eb3213\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5"
Sep 14 02:56:43.407659 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:43.407588    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dfec4394-69dc-4a1d-9421-913fb3eb3213-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-cqtx5\" (UID: \"dfec4394-69dc-4a1d-9421-913fb3eb3213\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5"
Sep 14 02:56:43.408324 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:43.408257    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dfec4394-69dc-4a1d-9421-913fb3eb3213-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-cqtx5\" (UID: \"dfec4394-69dc-4a1d-9421-913fb3eb3213\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5"
Sep 14 02:56:43.409725 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:43.409682    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dfec4394-69dc-4a1d-9421-913fb3eb3213-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-cqtx5\" (UID: \"dfec4394-69dc-4a1d-9421-913fb3eb3213\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5"
Sep 14 02:56:43.410492 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:43.410464    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dfec4394-69dc-4a1d-9421-913fb3eb3213-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-cqtx5\" (UID: \"dfec4394-69dc-4a1d-9421-913fb3eb3213\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5"
Sep 14 02:56:43.411088 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:43.411067    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dfec4394-69dc-4a1d-9421-913fb3eb3213-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-cqtx5\" (UID: \"dfec4394-69dc-4a1d-9421-913fb3eb3213\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5"
Sep 14 02:56:43.411165 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:43.411092    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dfec4394-69dc-4a1d-9421-913fb3eb3213-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-cqtx5\" (UID: \"dfec4394-69dc-4a1d-9421-913fb3eb3213\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5"
Sep 14 02:56:43.416500 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:43.416460    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dfec4394-69dc-4a1d-9421-913fb3eb3213-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-cqtx5\" (UID: \"dfec4394-69dc-4a1d-9421-913fb3eb3213\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5"
Sep 14 02:56:43.418133 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:43.418090    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzzcr\" (UniqueName: \"kubernetes.io/projected/dfec4394-69dc-4a1d-9421-913fb3eb3213-kube-api-access-gzzcr\") pod \"istiod-openshift-gateway-7b7995d5cd-cqtx5\" (UID: \"dfec4394-69dc-4a1d-9421-913fb3eb3213\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5"
Sep 14 02:56:43.541804 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:43.541728    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5"
Sep 14 02:56:44.981782 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:44.981751    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5"]
Sep 14 02:56:44.984596 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:56:44.984560    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfec4394_69dc_4a1d_9421_913fb3eb3213.slice/crio-062d2132acc15a25bb44d9becc9ad89cb4d7ce1fbdbb39c4d87592172f022221 WatchSource:0}: Error finding container 062d2132acc15a25bb44d9becc9ad89cb4d7ce1fbdbb39c4d87592172f022221: Status 404 returned error can't find the container with id 062d2132acc15a25bb44d9becc9ad89cb4d7ce1fbdbb39c4d87592172f022221
Sep 14 02:56:45.237006 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:45.236918    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-66f955bb57-d45jp" event={"ID":"1b30e897-08f8-4bcd-a435-380fcd3ce905","Type":"ContainerStarted","Data":"b69474ed20ac046ed1cd90b0968fc4246d336af59cbb10cdfae70a3e1a101b3f"}
Sep 14 02:56:45.238359 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:45.238329    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5" event={"ID":"dfec4394-69dc-4a1d-9421-913fb3eb3213","Type":"ContainerStarted","Data":"062d2132acc15a25bb44d9becc9ad89cb4d7ce1fbdbb39c4d87592172f022221"}
Sep 14 02:56:45.254977 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:45.254919    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-66f955bb57-d45jp" podStartSLOduration=1.574710655 podStartE2EDuration="4.254900969s" totalImagesPullingTime="2.680190314s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:56:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:56:45.253988601 +0000 UTC m=+354.231966641" watchObservedRunningTime="2026-09-14 02:56:45.254900969 +0000 UTC m=+354.232878997"
Sep 14 02:56:47.565917 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:47.565878    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 14 02:56:47.566163 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:47.565945    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 14 02:56:48.255105 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:48.255067    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5" event={"ID":"dfec4394-69dc-4a1d-9421-913fb3eb3213","Type":"ContainerStarted","Data":"bc6e5f590f514775408447431eac76613b378c17f30eea37b5a4008305e43a16"}
Sep 14 02:56:48.255303 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:48.255227    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5"
Sep 14 02:56:48.256957 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:48.256925    2577 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-cqtx5 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 14 02:56:48.257082 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:48.257000    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5" podUID="dfec4394-69dc-4a1d-9421-913fb3eb3213" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 02:56:49.260539 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:49.260513    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5"
Sep 14 02:56:49.290382 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:49.290321    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5" podStartSLOduration=3.711277818 podStartE2EDuration="6.290307305s" totalImagesPullingTime="2.579029487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:56:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:56:48.284985688 +0000 UTC m=+357.262963718" watchObservedRunningTime="2026-09-14 02:56:49.290307305 +0000 UTC m=+358.268285334"
Sep 14 02:56:52.253812 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:52.253771    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-6c2d2"]
Sep 14 02:56:52.259496 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:52.259467    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-6c2d2"
Sep 14 02:56:52.262107 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:52.262082    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-zqknl\""
Sep 14 02:56:52.262730 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:52.262710    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 14 02:56:52.268491 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:52.268126    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-6c2d2"]
Sep 14 02:56:52.375418 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:52.375386    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4afcf0b4-40de-4320-8239-222e77a6ebdc-cert\") pod \"odh-model-controller-7998c67dc4-6c2d2\" (UID: \"4afcf0b4-40de-4320-8239-222e77a6ebdc\") " pod="opendatahub/odh-model-controller-7998c67dc4-6c2d2"
Sep 14 02:56:52.375418 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:52.375419    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw7x2\" (UniqueName: \"kubernetes.io/projected/4afcf0b4-40de-4320-8239-222e77a6ebdc-kube-api-access-hw7x2\") pod \"odh-model-controller-7998c67dc4-6c2d2\" (UID: \"4afcf0b4-40de-4320-8239-222e77a6ebdc\") " pod="opendatahub/odh-model-controller-7998c67dc4-6c2d2"
Sep 14 02:56:52.476210 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:52.476172    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4afcf0b4-40de-4320-8239-222e77a6ebdc-cert\") pod \"odh-model-controller-7998c67dc4-6c2d2\" (UID: \"4afcf0b4-40de-4320-8239-222e77a6ebdc\") " pod="opendatahub/odh-model-controller-7998c67dc4-6c2d2"
Sep 14 02:56:52.476210 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:52.476209    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hw7x2\" (UniqueName: \"kubernetes.io/projected/4afcf0b4-40de-4320-8239-222e77a6ebdc-kube-api-access-hw7x2\") pod \"odh-model-controller-7998c67dc4-6c2d2\" (UID: \"4afcf0b4-40de-4320-8239-222e77a6ebdc\") " pod="opendatahub/odh-model-controller-7998c67dc4-6c2d2"
Sep 14 02:56:52.476440 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:56:52.476353    2577 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 14 02:56:52.476440 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:56:52.476425    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4afcf0b4-40de-4320-8239-222e77a6ebdc-cert podName:4afcf0b4-40de-4320-8239-222e77a6ebdc nodeName:}" failed. No retries permitted until 2026-09-14 02:56:52.976406059 +0000 UTC m=+361.954384080 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4afcf0b4-40de-4320-8239-222e77a6ebdc-cert") pod "odh-model-controller-7998c67dc4-6c2d2" (UID: "4afcf0b4-40de-4320-8239-222e77a6ebdc") : secret "odh-model-controller-webhook-cert" not found
Sep 14 02:56:52.496463 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:52.496429    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw7x2\" (UniqueName: \"kubernetes.io/projected/4afcf0b4-40de-4320-8239-222e77a6ebdc-kube-api-access-hw7x2\") pod \"odh-model-controller-7998c67dc4-6c2d2\" (UID: \"4afcf0b4-40de-4320-8239-222e77a6ebdc\") " pod="opendatahub/odh-model-controller-7998c67dc4-6c2d2"
Sep 14 02:56:52.917753 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:52.917722    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-ghwhx"]
Sep 14 02:56:52.921199 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:52.921176    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-ghwhx"
Sep 14 02:56:52.923753 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:52.923727    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 14 02:56:52.934987 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:52.934963    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-ghwhx"]
Sep 14 02:56:52.980051 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:52.980027    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4afcf0b4-40de-4320-8239-222e77a6ebdc-cert\") pod \"odh-model-controller-7998c67dc4-6c2d2\" (UID: \"4afcf0b4-40de-4320-8239-222e77a6ebdc\") " pod="opendatahub/odh-model-controller-7998c67dc4-6c2d2"
Sep 14 02:56:52.980205 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:52.980079    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rv9bv\" (UniqueName: \"kubernetes.io/projected/398f036d-5636-4f09-b52c-c21ecbaf33f5-kube-api-access-rv9bv\") pod \"model-serving-api-54d89bd5fc-ghwhx\" (UID: \"398f036d-5636-4f09-b52c-c21ecbaf33f5\") " pod="opendatahub/model-serving-api-54d89bd5fc-ghwhx"
Sep 14 02:56:52.980205 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:52.980099    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/398f036d-5636-4f09-b52c-c21ecbaf33f5-tls-certs\") pod \"model-serving-api-54d89bd5fc-ghwhx\" (UID: \"398f036d-5636-4f09-b52c-c21ecbaf33f5\") " pod="opendatahub/model-serving-api-54d89bd5fc-ghwhx"
Sep 14 02:56:52.982317 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:52.982298    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4afcf0b4-40de-4320-8239-222e77a6ebdc-cert\") pod \"odh-model-controller-7998c67dc4-6c2d2\" (UID: \"4afcf0b4-40de-4320-8239-222e77a6ebdc\") " pod="opendatahub/odh-model-controller-7998c67dc4-6c2d2"
Sep 14 02:56:53.080524 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:53.080495    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rv9bv\" (UniqueName: \"kubernetes.io/projected/398f036d-5636-4f09-b52c-c21ecbaf33f5-kube-api-access-rv9bv\") pod \"model-serving-api-54d89bd5fc-ghwhx\" (UID: \"398f036d-5636-4f09-b52c-c21ecbaf33f5\") " pod="opendatahub/model-serving-api-54d89bd5fc-ghwhx"
Sep 14 02:56:53.080617 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:53.080528    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/398f036d-5636-4f09-b52c-c21ecbaf33f5-tls-certs\") pod \"model-serving-api-54d89bd5fc-ghwhx\" (UID: \"398f036d-5636-4f09-b52c-c21ecbaf33f5\") " pod="opendatahub/model-serving-api-54d89bd5fc-ghwhx"
Sep 14 02:56:53.080658 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:56:53.080643    2577 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 14 02:56:53.080696 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:56:53.080691    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/398f036d-5636-4f09-b52c-c21ecbaf33f5-tls-certs podName:398f036d-5636-4f09-b52c-c21ecbaf33f5 nodeName:}" failed. No retries permitted until 2026-09-14 02:56:53.58067776 +0000 UTC m=+362.558655766 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/398f036d-5636-4f09-b52c-c21ecbaf33f5-tls-certs") pod "model-serving-api-54d89bd5fc-ghwhx" (UID: "398f036d-5636-4f09-b52c-c21ecbaf33f5") : secret "model-serving-api-tls" not found
Sep 14 02:56:53.093183 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:53.093146    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rv9bv\" (UniqueName: \"kubernetes.io/projected/398f036d-5636-4f09-b52c-c21ecbaf33f5-kube-api-access-rv9bv\") pod \"model-serving-api-54d89bd5fc-ghwhx\" (UID: \"398f036d-5636-4f09-b52c-c21ecbaf33f5\") " pod="opendatahub/model-serving-api-54d89bd5fc-ghwhx"
Sep 14 02:56:53.174293 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:53.174231    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-6c2d2"
Sep 14 02:56:53.445374 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:53.445348    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-6c2d2"]
Sep 14 02:56:53.447479 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:56:53.447454    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4afcf0b4_40de_4320_8239_222e77a6ebdc.slice/crio-698da98de34f9f4cd21ba08d58c3e2c7c166048621fc02c430f068f962d54dc7 WatchSource:0}: Error finding container 698da98de34f9f4cd21ba08d58c3e2c7c166048621fc02c430f068f962d54dc7: Status 404 returned error can't find the container with id 698da98de34f9f4cd21ba08d58c3e2c7c166048621fc02c430f068f962d54dc7
Sep 14 02:56:53.584006 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:53.583981    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/398f036d-5636-4f09-b52c-c21ecbaf33f5-tls-certs\") pod \"model-serving-api-54d89bd5fc-ghwhx\" (UID: \"398f036d-5636-4f09-b52c-c21ecbaf33f5\") " pod="opendatahub/model-serving-api-54d89bd5fc-ghwhx"
Sep 14 02:56:53.586408 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:53.586378    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/398f036d-5636-4f09-b52c-c21ecbaf33f5-tls-certs\") pod \"model-serving-api-54d89bd5fc-ghwhx\" (UID: \"398f036d-5636-4f09-b52c-c21ecbaf33f5\") " pod="opendatahub/model-serving-api-54d89bd5fc-ghwhx"
Sep 14 02:56:53.832036 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:53.832009    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-ghwhx"
Sep 14 02:56:54.113247 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:54.113175    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-ghwhx"]
Sep 14 02:56:54.116957 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:56:54.116926    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod398f036d_5636_4f09_b52c_c21ecbaf33f5.slice/crio-befca0133d726e8e0894a4131ad6875e5b2522a2c6bd54c3e6f3a66ba50454ff WatchSource:0}: Error finding container befca0133d726e8e0894a4131ad6875e5b2522a2c6bd54c3e6f3a66ba50454ff: Status 404 returned error can't find the container with id befca0133d726e8e0894a4131ad6875e5b2522a2c6bd54c3e6f3a66ba50454ff
Sep 14 02:56:54.287710 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:54.287672    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-ghwhx" event={"ID":"398f036d-5636-4f09-b52c-c21ecbaf33f5","Type":"ContainerStarted","Data":"befca0133d726e8e0894a4131ad6875e5b2522a2c6bd54c3e6f3a66ba50454ff"}
Sep 14 02:56:54.289113 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:54.289083    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-6c2d2" event={"ID":"4afcf0b4-40de-4320-8239-222e77a6ebdc","Type":"ContainerStarted","Data":"698da98de34f9f4cd21ba08d58c3e2c7c166048621fc02c430f068f962d54dc7"}
Sep 14 02:56:57.310915 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:57.310870    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-ghwhx" event={"ID":"398f036d-5636-4f09-b52c-c21ecbaf33f5","Type":"ContainerStarted","Data":"e70770f046862fee5a93515b2be37ea69919f927597ab5d0472772feac2c0bf1"}
Sep 14 02:56:57.311453 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:57.310966    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-ghwhx"
Sep 14 02:56:57.312838 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:57.312803    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-6c2d2" event={"ID":"4afcf0b4-40de-4320-8239-222e77a6ebdc","Type":"ContainerStarted","Data":"7fe50046a9c1c10ba51598939c998886d5f31a4c0d5682a06f0a7c835ea166ad"}
Sep 14 02:56:57.312999 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:57.312857    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-6c2d2"
Sep 14 02:56:57.330464 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:57.330401    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-ghwhx" podStartSLOduration=3.03247162 podStartE2EDuration="5.330384053s" totalImagesPullingTime="2.297912433s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:56:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:56:57.327544809 +0000 UTC m=+366.305522839" watchObservedRunningTime="2026-09-14 02:56:57.330384053 +0000 UTC m=+366.308362084"
Sep 14 02:56:57.346565 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:57.346513    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-6c2d2" podStartSLOduration=2.375778068 podStartE2EDuration="5.34649426s" totalImagesPullingTime="2.970716192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:56:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:56:57.344736493 +0000 UTC m=+366.322714522" watchObservedRunningTime="2026-09-14 02:56:57.34649426 +0000 UTC m=+366.324472290"
Sep 14 02:56:59.350136 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:59.350101    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-fnd68"]
Sep 14 02:56:59.353545 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:59.353522    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fnd68"
Sep 14 02:56:59.374581 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:59.374558    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-fnd68"]
Sep 14 02:56:59.435029 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:59.435002    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z26wn\" (UniqueName: \"kubernetes.io/projected/7dd41e91-99ac-4015-9f0f-c9423b47fe3f-kube-api-access-z26wn\") pod \"istiod-openshift-gateway-5677ccb49b-fnd68\" (UID: \"7dd41e91-99ac-4015-9f0f-c9423b47fe3f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fnd68"
Sep 14 02:56:59.435195 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:59.435033    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dd41e91-99ac-4015-9f0f-c9423b47fe3f-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-fnd68\" (UID: \"7dd41e91-99ac-4015-9f0f-c9423b47fe3f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fnd68"
Sep 14 02:56:59.435195 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:59.435060    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7dd41e91-99ac-4015-9f0f-c9423b47fe3f-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-fnd68\" (UID: \"7dd41e91-99ac-4015-9f0f-c9423b47fe3f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fnd68"
Sep 14 02:56:59.435195 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:59.435081    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7dd41e91-99ac-4015-9f0f-c9423b47fe3f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-fnd68\" (UID: \"7dd41e91-99ac-4015-9f0f-c9423b47fe3f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fnd68"
Sep 14 02:56:59.435195 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:59.435147    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7dd41e91-99ac-4015-9f0f-c9423b47fe3f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-fnd68\" (UID: \"7dd41e91-99ac-4015-9f0f-c9423b47fe3f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fnd68"
Sep 14 02:56:59.435195 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:59.435184    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dd41e91-99ac-4015-9f0f-c9423b47fe3f-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-fnd68\" (UID: \"7dd41e91-99ac-4015-9f0f-c9423b47fe3f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fnd68"
Sep 14 02:56:59.435479 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:59.435204    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7dd41e91-99ac-4015-9f0f-c9423b47fe3f-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-fnd68\" (UID: \"7dd41e91-99ac-4015-9f0f-c9423b47fe3f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fnd68"
Sep 14 02:56:59.536256 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:59.536228    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dd41e91-99ac-4015-9f0f-c9423b47fe3f-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-fnd68\" (UID: \"7dd41e91-99ac-4015-9f0f-c9423b47fe3f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fnd68"
Sep 14 02:56:59.536379 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:59.536295    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7dd41e91-99ac-4015-9f0f-c9423b47fe3f-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-fnd68\" (UID: \"7dd41e91-99ac-4015-9f0f-c9423b47fe3f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fnd68"
Sep 14 02:56:59.536379 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:59.536349    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z26wn\" (UniqueName: \"kubernetes.io/projected/7dd41e91-99ac-4015-9f0f-c9423b47fe3f-kube-api-access-z26wn\") pod \"istiod-openshift-gateway-5677ccb49b-fnd68\" (UID: \"7dd41e91-99ac-4015-9f0f-c9423b47fe3f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fnd68"
Sep 14 02:56:59.536465 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:59.536447    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dd41e91-99ac-4015-9f0f-c9423b47fe3f-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-fnd68\" (UID: \"7dd41e91-99ac-4015-9f0f-c9423b47fe3f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fnd68"
Sep 14 02:56:59.536502 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:59.536488    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7dd41e91-99ac-4015-9f0f-c9423b47fe3f-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-fnd68\" (UID: \"7dd41e91-99ac-4015-9f0f-c9423b47fe3f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fnd68"
Sep 14 02:56:59.536538 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:59.536514    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7dd41e91-99ac-4015-9f0f-c9423b47fe3f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-fnd68\" (UID: \"7dd41e91-99ac-4015-9f0f-c9423b47fe3f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fnd68"
Sep 14 02:56:59.536576 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:59.536561    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7dd41e91-99ac-4015-9f0f-c9423b47fe3f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-fnd68\" (UID: \"7dd41e91-99ac-4015-9f0f-c9423b47fe3f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fnd68"
Sep 14 02:56:59.537215 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:59.537188    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7dd41e91-99ac-4015-9f0f-c9423b47fe3f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-fnd68\" (UID: \"7dd41e91-99ac-4015-9f0f-c9423b47fe3f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fnd68"
Sep 14 02:56:59.538673 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:59.538643    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7dd41e91-99ac-4015-9f0f-c9423b47fe3f-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-fnd68\" (UID: \"7dd41e91-99ac-4015-9f0f-c9423b47fe3f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fnd68"
Sep 14 02:56:59.538861 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:59.538802    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7dd41e91-99ac-4015-9f0f-c9423b47fe3f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-fnd68\" (UID: \"7dd41e91-99ac-4015-9f0f-c9423b47fe3f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fnd68"
Sep 14 02:56:59.539057 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:59.539035    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dd41e91-99ac-4015-9f0f-c9423b47fe3f-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-fnd68\" (UID: \"7dd41e91-99ac-4015-9f0f-c9423b47fe3f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fnd68"
Sep 14 02:56:59.539115 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:59.539066    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7dd41e91-99ac-4015-9f0f-c9423b47fe3f-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-fnd68\" (UID: \"7dd41e91-99ac-4015-9f0f-c9423b47fe3f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fnd68"
Sep 14 02:56:59.545507 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:59.545484    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dd41e91-99ac-4015-9f0f-c9423b47fe3f-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-fnd68\" (UID: \"7dd41e91-99ac-4015-9f0f-c9423b47fe3f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fnd68"
Sep 14 02:56:59.545733 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:59.545714    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z26wn\" (UniqueName: \"kubernetes.io/projected/7dd41e91-99ac-4015-9f0f-c9423b47fe3f-kube-api-access-z26wn\") pod \"istiod-openshift-gateway-5677ccb49b-fnd68\" (UID: \"7dd41e91-99ac-4015-9f0f-c9423b47fe3f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fnd68"
Sep 14 02:56:59.662415 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:59.662365    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fnd68"
Sep 14 02:56:59.951452 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:59.951387    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-fnd68"]
Sep 14 02:56:59.954729 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:56:59.954701    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dd41e91_99ac_4015_9f0f_c9423b47fe3f.slice/crio-4762a06349ed8ca27ff90d879831b67c499e5a5b90119616f60d809c6e43ae63 WatchSource:0}: Error finding container 4762a06349ed8ca27ff90d879831b67c499e5a5b90119616f60d809c6e43ae63: Status 404 returned error can't find the container with id 4762a06349ed8ca27ff90d879831b67c499e5a5b90119616f60d809c6e43ae63
Sep 14 02:56:59.956755 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:59.956723    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 14 02:56:59.956853 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:56:59.956788    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236196Ki","pods":"250"}
Sep 14 02:57:00.329895 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:00.329866    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fnd68" event={"ID":"7dd41e91-99ac-4015-9f0f-c9423b47fe3f","Type":"ContainerStarted","Data":"5da37074cb548c9fc288a85dbf923daf41d06893da811a95a3f580f45383a722"}
Sep 14 02:57:00.330072 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:00.329910    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fnd68" event={"ID":"7dd41e91-99ac-4015-9f0f-c9423b47fe3f","Type":"ContainerStarted","Data":"4762a06349ed8ca27ff90d879831b67c499e5a5b90119616f60d809c6e43ae63"}
Sep 14 02:57:00.330072 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:00.329951    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fnd68"
Sep 14 02:57:00.440503 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:00.440449    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fnd68" podStartSLOduration=1.440429495 podStartE2EDuration="1.440429495s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:56:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:57:00.432977651 +0000 UTC m=+369.410955691" watchObservedRunningTime="2026-09-14 02:57:00.440429495 +0000 UTC m=+369.418407524"
Sep 14 02:57:01.337407 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:01.337371    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fnd68"
Sep 14 02:57:01.405675 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:01.405635    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5"]
Sep 14 02:57:01.405896 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:01.405855    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5" podUID="dfec4394-69dc-4a1d-9421-913fb3eb3213" containerName="discovery" containerID="cri-o://bc6e5f590f514775408447431eac76613b378c17f30eea37b5a4008305e43a16" gracePeriod=30
Sep 14 02:57:01.648927 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:01.648903    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5"
Sep 14 02:57:01.753798 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:01.753760    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dfec4394-69dc-4a1d-9421-913fb3eb3213-istio-token\") pod \"dfec4394-69dc-4a1d-9421-913fb3eb3213\" (UID: \"dfec4394-69dc-4a1d-9421-913fb3eb3213\") "
Sep 14 02:57:01.753979 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:01.753824    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dfec4394-69dc-4a1d-9421-913fb3eb3213-istio-csr-dns-cert\") pod \"dfec4394-69dc-4a1d-9421-913fb3eb3213\" (UID: \"dfec4394-69dc-4a1d-9421-913fb3eb3213\") "
Sep 14 02:57:01.753979 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:01.753874    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gzzcr\" (UniqueName: \"kubernetes.io/projected/dfec4394-69dc-4a1d-9421-913fb3eb3213-kube-api-access-gzzcr\") pod \"dfec4394-69dc-4a1d-9421-913fb3eb3213\" (UID: \"dfec4394-69dc-4a1d-9421-913fb3eb3213\") "
Sep 14 02:57:01.753979 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:01.753892    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dfec4394-69dc-4a1d-9421-913fb3eb3213-cacerts\") pod \"dfec4394-69dc-4a1d-9421-913fb3eb3213\" (UID: \"dfec4394-69dc-4a1d-9421-913fb3eb3213\") "
Sep 14 02:57:01.753979 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:01.753908    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dfec4394-69dc-4a1d-9421-913fb3eb3213-istio-csr-ca-configmap\") pod \"dfec4394-69dc-4a1d-9421-913fb3eb3213\" (UID: \"dfec4394-69dc-4a1d-9421-913fb3eb3213\") "
Sep 14 02:57:01.753979 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:01.753950    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dfec4394-69dc-4a1d-9421-913fb3eb3213-istio-kubeconfig\") pod \"dfec4394-69dc-4a1d-9421-913fb3eb3213\" (UID: \"dfec4394-69dc-4a1d-9421-913fb3eb3213\") "
Sep 14 02:57:01.754181 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:01.754021    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dfec4394-69dc-4a1d-9421-913fb3eb3213-local-certs\") pod \"dfec4394-69dc-4a1d-9421-913fb3eb3213\" (UID: \"dfec4394-69dc-4a1d-9421-913fb3eb3213\") "
Sep 14 02:57:01.754422 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:01.754386    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dfec4394-69dc-4a1d-9421-913fb3eb3213-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "dfec4394-69dc-4a1d-9421-913fb3eb3213" (UID: "dfec4394-69dc-4a1d-9421-913fb3eb3213"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 02:57:01.754713 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:01.754673    2577 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dfec4394-69dc-4a1d-9421-913fb3eb3213-istio-csr-ca-configmap\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:57:01.756635 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:01.756595    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfec4394-69dc-4a1d-9421-913fb3eb3213-local-certs" (OuterVolumeSpecName: "local-certs") pod "dfec4394-69dc-4a1d-9421-913fb3eb3213" (UID: "dfec4394-69dc-4a1d-9421-913fb3eb3213"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:57:01.756737 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:01.756647    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfec4394-69dc-4a1d-9421-913fb3eb3213-istio-token" (OuterVolumeSpecName: "istio-token") pod "dfec4394-69dc-4a1d-9421-913fb3eb3213" (UID: "dfec4394-69dc-4a1d-9421-913fb3eb3213"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:57:01.756737 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:01.756662    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfec4394-69dc-4a1d-9421-913fb3eb3213-kube-api-access-gzzcr" (OuterVolumeSpecName: "kube-api-access-gzzcr") pod "dfec4394-69dc-4a1d-9421-913fb3eb3213" (UID: "dfec4394-69dc-4a1d-9421-913fb3eb3213"). InnerVolumeSpecName "kube-api-access-gzzcr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:57:01.756737 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:01.756650    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfec4394-69dc-4a1d-9421-913fb3eb3213-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "dfec4394-69dc-4a1d-9421-913fb3eb3213" (UID: "dfec4394-69dc-4a1d-9421-913fb3eb3213"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 02:57:01.756737 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:01.756695    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfec4394-69dc-4a1d-9421-913fb3eb3213-cacerts" (OuterVolumeSpecName: "cacerts") pod "dfec4394-69dc-4a1d-9421-913fb3eb3213" (UID: "dfec4394-69dc-4a1d-9421-913fb3eb3213"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 02:57:01.756737 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:01.756712    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfec4394-69dc-4a1d-9421-913fb3eb3213-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "dfec4394-69dc-4a1d-9421-913fb3eb3213" (UID: "dfec4394-69dc-4a1d-9421-913fb3eb3213"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 02:57:01.856019 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:01.855993    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dfec4394-69dc-4a1d-9421-913fb3eb3213-istio-token\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:57:01.856019 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:01.856015    2577 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dfec4394-69dc-4a1d-9421-913fb3eb3213-istio-csr-dns-cert\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:57:01.856155 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:01.856024    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzzcr\" (UniqueName: \"kubernetes.io/projected/dfec4394-69dc-4a1d-9421-913fb3eb3213-kube-api-access-gzzcr\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:57:01.856155 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:01.856033    2577 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dfec4394-69dc-4a1d-9421-913fb3eb3213-cacerts\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:57:01.856155 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:01.856044    2577 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dfec4394-69dc-4a1d-9421-913fb3eb3213-istio-kubeconfig\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:57:01.856155 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:01.856054    2577 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dfec4394-69dc-4a1d-9421-913fb3eb3213-local-certs\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:57:02.341547 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:02.341516    2577 generic.go:356] "Generic (PLEG): container finished" podID="dfec4394-69dc-4a1d-9421-913fb3eb3213" containerID="bc6e5f590f514775408447431eac76613b378c17f30eea37b5a4008305e43a16" exitCode=0
Sep 14 02:57:02.341706 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:02.341559    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5"
Sep 14 02:57:02.341706 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:02.341595    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5" event={"ID":"dfec4394-69dc-4a1d-9421-913fb3eb3213","Type":"ContainerDied","Data":"bc6e5f590f514775408447431eac76613b378c17f30eea37b5a4008305e43a16"}
Sep 14 02:57:02.341706 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:02.341641    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5" event={"ID":"dfec4394-69dc-4a1d-9421-913fb3eb3213","Type":"ContainerDied","Data":"062d2132acc15a25bb44d9becc9ad89cb4d7ce1fbdbb39c4d87592172f022221"}
Sep 14 02:57:02.341706 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:02.341664    2577 scope.go:117] "RemoveContainer" containerID="bc6e5f590f514775408447431eac76613b378c17f30eea37b5a4008305e43a16"
Sep 14 02:57:02.350773 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:02.350758    2577 scope.go:117] "RemoveContainer" containerID="bc6e5f590f514775408447431eac76613b378c17f30eea37b5a4008305e43a16"
Sep 14 02:57:02.351026 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:57:02.351008    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc6e5f590f514775408447431eac76613b378c17f30eea37b5a4008305e43a16\": container with ID starting with bc6e5f590f514775408447431eac76613b378c17f30eea37b5a4008305e43a16 not found: ID does not exist" containerID="bc6e5f590f514775408447431eac76613b378c17f30eea37b5a4008305e43a16"
Sep 14 02:57:02.351064 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:02.351035    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc6e5f590f514775408447431eac76613b378c17f30eea37b5a4008305e43a16"} err="failed to get container status \"bc6e5f590f514775408447431eac76613b378c17f30eea37b5a4008305e43a16\": rpc error: code = NotFound desc = could not find container \"bc6e5f590f514775408447431eac76613b378c17f30eea37b5a4008305e43a16\": container with ID starting with bc6e5f590f514775408447431eac76613b378c17f30eea37b5a4008305e43a16 not found: ID does not exist"
Sep 14 02:57:02.379670 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:02.379644    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5"]
Sep 14 02:57:02.386603 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:02.386583    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5"]
Sep 14 02:57:02.638162 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:02.638086    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-nnzpk"]
Sep 14 02:57:02.638440 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:02.638425    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfec4394-69dc-4a1d-9421-913fb3eb3213" containerName="discovery"
Sep 14 02:57:02.638440 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:02.638440    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfec4394-69dc-4a1d-9421-913fb3eb3213" containerName="discovery"
Sep 14 02:57:02.638585 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:02.638499    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfec4394-69dc-4a1d-9421-913fb3eb3213" containerName="discovery"
Sep 14 02:57:02.642595 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:02.642573    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nnzpk"
Sep 14 02:57:02.662494 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:02.662470    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 14 02:57:02.662776 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:02.662692    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-8qdn6\""
Sep 14 02:57:02.715071 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:02.715049    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-nnzpk"]
Sep 14 02:57:02.763545 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:02.763509    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2dkl\" (UniqueName: \"kubernetes.io/projected/824cabb4-f00b-405c-a42f-8c8d1a7d19bd-kube-api-access-r2dkl\") pod \"llmisvc-controller-manager-777c6bd7d5-nnzpk\" (UID: \"824cabb4-f00b-405c-a42f-8c8d1a7d19bd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nnzpk"
Sep 14 02:57:02.763674 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:02.763547    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/824cabb4-f00b-405c-a42f-8c8d1a7d19bd-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-nnzpk\" (UID: \"824cabb4-f00b-405c-a42f-8c8d1a7d19bd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nnzpk"
Sep 14 02:57:02.864611 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:02.864579    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2dkl\" (UniqueName: \"kubernetes.io/projected/824cabb4-f00b-405c-a42f-8c8d1a7d19bd-kube-api-access-r2dkl\") pod \"llmisvc-controller-manager-777c6bd7d5-nnzpk\" (UID: \"824cabb4-f00b-405c-a42f-8c8d1a7d19bd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nnzpk"
Sep 14 02:57:02.864739 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:02.864617    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/824cabb4-f00b-405c-a42f-8c8d1a7d19bd-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-nnzpk\" (UID: \"824cabb4-f00b-405c-a42f-8c8d1a7d19bd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nnzpk"
Sep 14 02:57:02.867016 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:02.866995    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/824cabb4-f00b-405c-a42f-8c8d1a7d19bd-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-nnzpk\" (UID: \"824cabb4-f00b-405c-a42f-8c8d1a7d19bd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nnzpk"
Sep 14 02:57:02.896446 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:02.896393    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2dkl\" (UniqueName: \"kubernetes.io/projected/824cabb4-f00b-405c-a42f-8c8d1a7d19bd-kube-api-access-r2dkl\") pod \"llmisvc-controller-manager-777c6bd7d5-nnzpk\" (UID: \"824cabb4-f00b-405c-a42f-8c8d1a7d19bd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nnzpk"
Sep 14 02:57:02.951370 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:02.951351    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nnzpk"
Sep 14 02:57:03.300927 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:03.300894    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-nnzpk"]
Sep 14 02:57:03.303695 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:57:03.303669    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod824cabb4_f00b_405c_a42f_8c8d1a7d19bd.slice/crio-09e7cb06f733f2714b95d565699fd5ff3d5393d78650f3155c9669cedb859e00 WatchSource:0}: Error finding container 09e7cb06f733f2714b95d565699fd5ff3d5393d78650f3155c9669cedb859e00: Status 404 returned error can't find the container with id 09e7cb06f733f2714b95d565699fd5ff3d5393d78650f3155c9669cedb859e00
Sep 14 02:57:03.347590 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:03.347560    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nnzpk" event={"ID":"824cabb4-f00b-405c-a42f-8c8d1a7d19bd","Type":"ContainerStarted","Data":"09e7cb06f733f2714b95d565699fd5ff3d5393d78650f3155c9669cedb859e00"}
Sep 14 02:57:03.616956 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:03.616883    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfec4394-69dc-4a1d-9421-913fb3eb3213" path="/var/lib/kubelet/pods/dfec4394-69dc-4a1d-9421-913fb3eb3213/volumes"
Sep 14 02:57:06.365694 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:06.365657    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nnzpk" event={"ID":"824cabb4-f00b-405c-a42f-8c8d1a7d19bd","Type":"ContainerStarted","Data":"34e1d38f0a11a958cd714448c9bda4e3dca978ccacc9fa6d23accd8b7838bb5a"}
Sep 14 02:57:06.366109 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:06.365716    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nnzpk"
Sep 14 02:57:06.400070 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:06.400024    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nnzpk" podStartSLOduration=2.336838283 podStartE2EDuration="4.4000098s" totalImagesPullingTime="2.063171517s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:57:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:57:06.398382108 +0000 UTC m=+375.376360136" watchObservedRunningTime="2026-09-14 02:57:06.4000098 +0000 UTC m=+375.377987828"
Sep 14 02:57:08.321038 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:08.321010    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-6c2d2"
Sep 14 02:57:08.322859 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:08.322839    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-ghwhx"
Sep 14 02:57:37.373211 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:37.373174    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nnzpk"
Sep 14 02:57:39.569220 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:39.569186    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-57cf94574b-b8kq5"]
Sep 14 02:57:39.588305 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:39.588254    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57cf94574b-b8kq5"
Sep 14 02:57:39.588770 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:39.588745    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57cf94574b-b8kq5"]
Sep 14 02:57:39.729965 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:39.729928    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d58f253a-1db9-42a9-ab18-786f2024a97f-service-ca\") pod \"console-57cf94574b-b8kq5\" (UID: \"d58f253a-1db9-42a9-ab18-786f2024a97f\") " pod="openshift-console/console-57cf94574b-b8kq5"
Sep 14 02:57:39.730125 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:39.729985    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d58f253a-1db9-42a9-ab18-786f2024a97f-console-serving-cert\") pod \"console-57cf94574b-b8kq5\" (UID: \"d58f253a-1db9-42a9-ab18-786f2024a97f\") " pod="openshift-console/console-57cf94574b-b8kq5"
Sep 14 02:57:39.730125 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:39.730047    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d58f253a-1db9-42a9-ab18-786f2024a97f-oauth-serving-cert\") pod \"console-57cf94574b-b8kq5\" (UID: \"d58f253a-1db9-42a9-ab18-786f2024a97f\") " pod="openshift-console/console-57cf94574b-b8kq5"
Sep 14 02:57:39.730125 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:39.730076    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d58f253a-1db9-42a9-ab18-786f2024a97f-console-config\") pod \"console-57cf94574b-b8kq5\" (UID: \"d58f253a-1db9-42a9-ab18-786f2024a97f\") " pod="openshift-console/console-57cf94574b-b8kq5"
Sep 14 02:57:39.730125 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:39.730110    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d58f253a-1db9-42a9-ab18-786f2024a97f-trusted-ca-bundle\") pod \"console-57cf94574b-b8kq5\" (UID: \"d58f253a-1db9-42a9-ab18-786f2024a97f\") " pod="openshift-console/console-57cf94574b-b8kq5"
Sep 14 02:57:39.730388 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:39.730178    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d58f253a-1db9-42a9-ab18-786f2024a97f-console-oauth-config\") pod \"console-57cf94574b-b8kq5\" (UID: \"d58f253a-1db9-42a9-ab18-786f2024a97f\") " pod="openshift-console/console-57cf94574b-b8kq5"
Sep 14 02:57:39.730388 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:39.730206    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jc48b\" (UniqueName: \"kubernetes.io/projected/d58f253a-1db9-42a9-ab18-786f2024a97f-kube-api-access-jc48b\") pod \"console-57cf94574b-b8kq5\" (UID: \"d58f253a-1db9-42a9-ab18-786f2024a97f\") " pod="openshift-console/console-57cf94574b-b8kq5"
Sep 14 02:57:39.831322 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:39.831248    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d58f253a-1db9-42a9-ab18-786f2024a97f-console-oauth-config\") pod \"console-57cf94574b-b8kq5\" (UID: \"d58f253a-1db9-42a9-ab18-786f2024a97f\") " pod="openshift-console/console-57cf94574b-b8kq5"
Sep 14 02:57:39.831322 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:39.831305    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jc48b\" (UniqueName: \"kubernetes.io/projected/d58f253a-1db9-42a9-ab18-786f2024a97f-kube-api-access-jc48b\") pod \"console-57cf94574b-b8kq5\" (UID: \"d58f253a-1db9-42a9-ab18-786f2024a97f\") " pod="openshift-console/console-57cf94574b-b8kq5"
Sep 14 02:57:39.831615 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:39.831350    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d58f253a-1db9-42a9-ab18-786f2024a97f-service-ca\") pod \"console-57cf94574b-b8kq5\" (UID: \"d58f253a-1db9-42a9-ab18-786f2024a97f\") " pod="openshift-console/console-57cf94574b-b8kq5"
Sep 14 02:57:39.831615 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:39.831384    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d58f253a-1db9-42a9-ab18-786f2024a97f-console-serving-cert\") pod \"console-57cf94574b-b8kq5\" (UID: \"d58f253a-1db9-42a9-ab18-786f2024a97f\") " pod="openshift-console/console-57cf94574b-b8kq5"
Sep 14 02:57:39.831615 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:39.831440    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d58f253a-1db9-42a9-ab18-786f2024a97f-oauth-serving-cert\") pod \"console-57cf94574b-b8kq5\" (UID: \"d58f253a-1db9-42a9-ab18-786f2024a97f\") " pod="openshift-console/console-57cf94574b-b8kq5"
Sep 14 02:57:39.831615 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:39.831467    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d58f253a-1db9-42a9-ab18-786f2024a97f-console-config\") pod \"console-57cf94574b-b8kq5\" (UID: \"d58f253a-1db9-42a9-ab18-786f2024a97f\") " pod="openshift-console/console-57cf94574b-b8kq5"
Sep 14 02:57:39.831615 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:39.831503    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d58f253a-1db9-42a9-ab18-786f2024a97f-trusted-ca-bundle\") pod \"console-57cf94574b-b8kq5\" (UID: \"d58f253a-1db9-42a9-ab18-786f2024a97f\") " pod="openshift-console/console-57cf94574b-b8kq5"
Sep 14 02:57:39.832342 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:39.832308    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d58f253a-1db9-42a9-ab18-786f2024a97f-service-ca\") pod \"console-57cf94574b-b8kq5\" (UID: \"d58f253a-1db9-42a9-ab18-786f2024a97f\") " pod="openshift-console/console-57cf94574b-b8kq5"
Sep 14 02:57:39.832475 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:39.832372    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d58f253a-1db9-42a9-ab18-786f2024a97f-console-config\") pod \"console-57cf94574b-b8kq5\" (UID: \"d58f253a-1db9-42a9-ab18-786f2024a97f\") " pod="openshift-console/console-57cf94574b-b8kq5"
Sep 14 02:57:39.832475 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:39.832372    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d58f253a-1db9-42a9-ab18-786f2024a97f-oauth-serving-cert\") pod \"console-57cf94574b-b8kq5\" (UID: \"d58f253a-1db9-42a9-ab18-786f2024a97f\") " pod="openshift-console/console-57cf94574b-b8kq5"
Sep 14 02:57:39.832598 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:39.832580    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d58f253a-1db9-42a9-ab18-786f2024a97f-trusted-ca-bundle\") pod \"console-57cf94574b-b8kq5\" (UID: \"d58f253a-1db9-42a9-ab18-786f2024a97f\") " pod="openshift-console/console-57cf94574b-b8kq5"
Sep 14 02:57:39.834116 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:39.834093    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d58f253a-1db9-42a9-ab18-786f2024a97f-console-serving-cert\") pod \"console-57cf94574b-b8kq5\" (UID: \"d58f253a-1db9-42a9-ab18-786f2024a97f\") " pod="openshift-console/console-57cf94574b-b8kq5"
Sep 14 02:57:39.834217 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:39.834170    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d58f253a-1db9-42a9-ab18-786f2024a97f-console-oauth-config\") pod \"console-57cf94574b-b8kq5\" (UID: \"d58f253a-1db9-42a9-ab18-786f2024a97f\") " pod="openshift-console/console-57cf94574b-b8kq5"
Sep 14 02:57:39.840171 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:39.840143    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jc48b\" (UniqueName: \"kubernetes.io/projected/d58f253a-1db9-42a9-ab18-786f2024a97f-kube-api-access-jc48b\") pod \"console-57cf94574b-b8kq5\" (UID: \"d58f253a-1db9-42a9-ab18-786f2024a97f\") " pod="openshift-console/console-57cf94574b-b8kq5"
Sep 14 02:57:39.897731 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:39.897706    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57cf94574b-b8kq5"
Sep 14 02:57:40.178101 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:40.178075    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57cf94574b-b8kq5"]
Sep 14 02:57:40.180749 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:57:40.180722    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd58f253a_1db9_42a9_ab18_786f2024a97f.slice/crio-6d52312277f112ca1c8e90ed24de8a5c684b309e18098732ea48f5d82bfb4fae WatchSource:0}: Error finding container 6d52312277f112ca1c8e90ed24de8a5c684b309e18098732ea48f5d82bfb4fae: Status 404 returned error can't find the container with id 6d52312277f112ca1c8e90ed24de8a5c684b309e18098732ea48f5d82bfb4fae
Sep 14 02:57:40.525415 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:40.525378    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57cf94574b-b8kq5" event={"ID":"d58f253a-1db9-42a9-ab18-786f2024a97f","Type":"ContainerStarted","Data":"39aa38ebed34f142cb04b9858c42a549110e04ca94377f9ca923e39d361d0555"}
Sep 14 02:57:40.525654 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:40.525464    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57cf94574b-b8kq5" event={"ID":"d58f253a-1db9-42a9-ab18-786f2024a97f","Type":"ContainerStarted","Data":"6d52312277f112ca1c8e90ed24de8a5c684b309e18098732ea48f5d82bfb4fae"}
Sep 14 02:57:40.544347 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:40.544300    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-57cf94574b-b8kq5" podStartSLOduration=1.544258929 podStartE2EDuration="1.544258929s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:57:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:57:40.543785055 +0000 UTC m=+409.521763085" watchObservedRunningTime="2026-09-14 02:57:40.544258929 +0000 UTC m=+409.522236957"
Sep 14 02:57:49.898151 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:49.898116    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-57cf94574b-b8kq5"
Sep 14 02:57:49.898712 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:49.898212    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-57cf94574b-b8kq5"
Sep 14 02:57:49.903051 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:49.903021    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-57cf94574b-b8kq5"
Sep 14 02:57:50.580733 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:50.580701    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-57cf94574b-b8kq5"
Sep 14 02:57:50.673498 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:50.673461    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55c47f6d67-mhxtv"]
Sep 14 02:57:58.208718 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.208673    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gtzwn"]
Sep 14 02:57:58.213659 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.213639    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gtzwn"
Sep 14 02:57:58.216613 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.216586    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 14 02:57:58.217561 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.217537    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zrfw6\""
Sep 14 02:57:58.217686 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.217567    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 14 02:57:58.222310 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.222259    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gtzwn"]
Sep 14 02:57:58.263782 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.263758    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdgcn\" (UniqueName: \"kubernetes.io/projected/bc36e9f5-28d1-47fb-86ab-843cc31c1698-kube-api-access-kdgcn\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gtzwn\" (UID: \"bc36e9f5-28d1-47fb-86ab-843cc31c1698\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gtzwn"
Sep 14 02:57:58.263899 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.263792    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc36e9f5-28d1-47fb-86ab-843cc31c1698-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gtzwn\" (UID: \"bc36e9f5-28d1-47fb-86ab-843cc31c1698\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gtzwn"
Sep 14 02:57:58.263899 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.263830    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc36e9f5-28d1-47fb-86ab-843cc31c1698-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gtzwn\" (UID: \"bc36e9f5-28d1-47fb-86ab-843cc31c1698\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gtzwn"
Sep 14 02:57:58.293053 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.293024    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avqln8"]
Sep 14 02:57:58.296641 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.296624    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avqln8"
Sep 14 02:57:58.307113 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.307092    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avqln8"]
Sep 14 02:57:58.364913 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.364891    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/31b714a4-b440-42a6-9b05-5249cfc755d0-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avqln8\" (UID: \"31b714a4-b440-42a6-9b05-5249cfc755d0\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avqln8"
Sep 14 02:57:58.365016 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.364921    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdgcn\" (UniqueName: \"kubernetes.io/projected/bc36e9f5-28d1-47fb-86ab-843cc31c1698-kube-api-access-kdgcn\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gtzwn\" (UID: \"bc36e9f5-28d1-47fb-86ab-843cc31c1698\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gtzwn"
Sep 14 02:57:58.365016 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.364945    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc36e9f5-28d1-47fb-86ab-843cc31c1698-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gtzwn\" (UID: \"bc36e9f5-28d1-47fb-86ab-843cc31c1698\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gtzwn"
Sep 14 02:57:58.365016 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.364971    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c74bz\" (UniqueName: \"kubernetes.io/projected/31b714a4-b440-42a6-9b05-5249cfc755d0-kube-api-access-c74bz\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avqln8\" (UID: \"31b714a4-b440-42a6-9b05-5249cfc755d0\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avqln8"
Sep 14 02:57:58.365016 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.365008    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc36e9f5-28d1-47fb-86ab-843cc31c1698-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gtzwn\" (UID: \"bc36e9f5-28d1-47fb-86ab-843cc31c1698\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gtzwn"
Sep 14 02:57:58.365236 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.365028    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/31b714a4-b440-42a6-9b05-5249cfc755d0-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avqln8\" (UID: \"31b714a4-b440-42a6-9b05-5249cfc755d0\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avqln8"
Sep 14 02:57:58.365353 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.365335    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc36e9f5-28d1-47fb-86ab-843cc31c1698-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gtzwn\" (UID: \"bc36e9f5-28d1-47fb-86ab-843cc31c1698\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gtzwn"
Sep 14 02:57:58.365413 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.365366    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc36e9f5-28d1-47fb-86ab-843cc31c1698-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gtzwn\" (UID: \"bc36e9f5-28d1-47fb-86ab-843cc31c1698\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gtzwn"
Sep 14 02:57:58.373816 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.373789    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdgcn\" (UniqueName: \"kubernetes.io/projected/bc36e9f5-28d1-47fb-86ab-843cc31c1698-kube-api-access-kdgcn\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gtzwn\" (UID: \"bc36e9f5-28d1-47fb-86ab-843cc31c1698\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gtzwn"
Sep 14 02:57:58.398876 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.398854    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dl9hb"]
Sep 14 02:57:58.402146 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.402130    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dl9hb"
Sep 14 02:57:58.412011 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.411988    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dl9hb"]
Sep 14 02:57:58.465830 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.465781    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/31b714a4-b440-42a6-9b05-5249cfc755d0-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avqln8\" (UID: \"31b714a4-b440-42a6-9b05-5249cfc755d0\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avqln8"
Sep 14 02:57:58.465830 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.465809    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d0aaff3a-006d-419f-a123-36bd4ce2be5b-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dl9hb\" (UID: \"d0aaff3a-006d-419f-a123-36bd4ce2be5b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dl9hb"
Sep 14 02:57:58.465943 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.465844    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs86z\" (UniqueName: \"kubernetes.io/projected/d0aaff3a-006d-419f-a123-36bd4ce2be5b-kube-api-access-bs86z\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dl9hb\" (UID: \"d0aaff3a-006d-419f-a123-36bd4ce2be5b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dl9hb"
Sep 14 02:57:58.465943 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.465863    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/31b714a4-b440-42a6-9b05-5249cfc755d0-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avqln8\" (UID: \"31b714a4-b440-42a6-9b05-5249cfc755d0\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avqln8"
Sep 14 02:57:58.465943 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.465886    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d0aaff3a-006d-419f-a123-36bd4ce2be5b-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dl9hb\" (UID: \"d0aaff3a-006d-419f-a123-36bd4ce2be5b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dl9hb"
Sep 14 02:57:58.465943 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.465913    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c74bz\" (UniqueName: \"kubernetes.io/projected/31b714a4-b440-42a6-9b05-5249cfc755d0-kube-api-access-c74bz\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avqln8\" (UID: \"31b714a4-b440-42a6-9b05-5249cfc755d0\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avqln8"
Sep 14 02:57:58.466117 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.466100    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/31b714a4-b440-42a6-9b05-5249cfc755d0-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avqln8\" (UID: \"31b714a4-b440-42a6-9b05-5249cfc755d0\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avqln8"
Sep 14 02:57:58.466164 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.466146    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/31b714a4-b440-42a6-9b05-5249cfc755d0-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avqln8\" (UID: \"31b714a4-b440-42a6-9b05-5249cfc755d0\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avqln8"
Sep 14 02:57:58.476977 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.476953    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c74bz\" (UniqueName: \"kubernetes.io/projected/31b714a4-b440-42a6-9b05-5249cfc755d0-kube-api-access-c74bz\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avqln8\" (UID: \"31b714a4-b440-42a6-9b05-5249cfc755d0\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avqln8"
Sep 14 02:57:58.495612 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.495582    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqfmzl"]
Sep 14 02:57:58.498975 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.498957    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqfmzl"
Sep 14 02:57:58.508196 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.508171    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqfmzl"]
Sep 14 02:57:58.525146 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.525126    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gtzwn"
Sep 14 02:57:58.566439 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.566414    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a68025f3-4c29-4dfb-b031-c435ae2e4c1b-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqfmzl\" (UID: \"a68025f3-4c29-4dfb-b031-c435ae2e4c1b\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqfmzl"
Sep 14 02:57:58.566562 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.566457    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d0aaff3a-006d-419f-a123-36bd4ce2be5b-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dl9hb\" (UID: \"d0aaff3a-006d-419f-a123-36bd4ce2be5b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dl9hb"
Sep 14 02:57:58.566562 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.566536    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bs86z\" (UniqueName: \"kubernetes.io/projected/d0aaff3a-006d-419f-a123-36bd4ce2be5b-kube-api-access-bs86z\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dl9hb\" (UID: \"d0aaff3a-006d-419f-a123-36bd4ce2be5b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dl9hb"
Sep 14 02:57:58.566673 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.566588    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjngz\" (UniqueName: \"kubernetes.io/projected/a68025f3-4c29-4dfb-b031-c435ae2e4c1b-kube-api-access-tjngz\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqfmzl\" (UID: \"a68025f3-4c29-4dfb-b031-c435ae2e4c1b\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqfmzl"
Sep 14 02:57:58.566673 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.566619    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d0aaff3a-006d-419f-a123-36bd4ce2be5b-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dl9hb\" (UID: \"d0aaff3a-006d-419f-a123-36bd4ce2be5b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dl9hb"
Sep 14 02:57:58.566779 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.566673    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a68025f3-4c29-4dfb-b031-c435ae2e4c1b-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqfmzl\" (UID: \"a68025f3-4c29-4dfb-b031-c435ae2e4c1b\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqfmzl"
Sep 14 02:57:58.566834 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.566818    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d0aaff3a-006d-419f-a123-36bd4ce2be5b-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dl9hb\" (UID: \"d0aaff3a-006d-419f-a123-36bd4ce2be5b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dl9hb"
Sep 14 02:57:58.566916 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.566895    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d0aaff3a-006d-419f-a123-36bd4ce2be5b-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dl9hb\" (UID: \"d0aaff3a-006d-419f-a123-36bd4ce2be5b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dl9hb"
Sep 14 02:57:58.579175 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.579151    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs86z\" (UniqueName: \"kubernetes.io/projected/d0aaff3a-006d-419f-a123-36bd4ce2be5b-kube-api-access-bs86z\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dl9hb\" (UID: \"d0aaff3a-006d-419f-a123-36bd4ce2be5b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dl9hb"
Sep 14 02:57:58.605562 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.605537    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avqln8"
Sep 14 02:57:58.668116 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.668089    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjngz\" (UniqueName: \"kubernetes.io/projected/a68025f3-4c29-4dfb-b031-c435ae2e4c1b-kube-api-access-tjngz\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqfmzl\" (UID: \"a68025f3-4c29-4dfb-b031-c435ae2e4c1b\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqfmzl"
Sep 14 02:57:58.668210 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.668133    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a68025f3-4c29-4dfb-b031-c435ae2e4c1b-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqfmzl\" (UID: \"a68025f3-4c29-4dfb-b031-c435ae2e4c1b\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqfmzl"
Sep 14 02:57:58.668210 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.668192    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a68025f3-4c29-4dfb-b031-c435ae2e4c1b-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqfmzl\" (UID: \"a68025f3-4c29-4dfb-b031-c435ae2e4c1b\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqfmzl"
Sep 14 02:57:58.668616 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.668595    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a68025f3-4c29-4dfb-b031-c435ae2e4c1b-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqfmzl\" (UID: \"a68025f3-4c29-4dfb-b031-c435ae2e4c1b\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqfmzl"
Sep 14 02:57:58.668841 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.668824    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a68025f3-4c29-4dfb-b031-c435ae2e4c1b-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqfmzl\" (UID: \"a68025f3-4c29-4dfb-b031-c435ae2e4c1b\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqfmzl"
Sep 14 02:57:58.678208 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.678183    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjngz\" (UniqueName: \"kubernetes.io/projected/a68025f3-4c29-4dfb-b031-c435ae2e4c1b-kube-api-access-tjngz\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqfmzl\" (UID: \"a68025f3-4c29-4dfb-b031-c435ae2e4c1b\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqfmzl"
Sep 14 02:57:58.710825 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.710804    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dl9hb"
Sep 14 02:57:58.803929 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.803905    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gtzwn"]
Sep 14 02:57:58.805609 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:57:58.805580    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc36e9f5_28d1_47fb_86ab_843cc31c1698.slice/crio-717e50dab9c92ba3a6ba3b802732063f4c34e2e5705762af51260f05e6952ec6 WatchSource:0}: Error finding container 717e50dab9c92ba3a6ba3b802732063f4c34e2e5705762af51260f05e6952ec6: Status 404 returned error can't find the container with id 717e50dab9c92ba3a6ba3b802732063f4c34e2e5705762af51260f05e6952ec6
Sep 14 02:57:58.807367 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.807350    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqfmzl"
Sep 14 02:57:58.887330 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.887285    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avqln8"]
Sep 14 02:57:58.887687 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:57:58.887654    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31b714a4_b440_42a6_9b05_5249cfc755d0.slice/crio-659ff75a917bc828b5de3e14f5ae16b32cab8efa3c5d75278db02f7fdcfa5493 WatchSource:0}: Error finding container 659ff75a917bc828b5de3e14f5ae16b32cab8efa3c5d75278db02f7fdcfa5493: Status 404 returned error can't find the container with id 659ff75a917bc828b5de3e14f5ae16b32cab8efa3c5d75278db02f7fdcfa5493
Sep 14 02:57:58.987044 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:58.987021    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dl9hb"]
Sep 14 02:57:58.988995 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:57:58.988974    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0aaff3a_006d_419f_a123_36bd4ce2be5b.slice/crio-08f2ae6278c5b001e309743ae825ca5193a01c0a48389cb42ad8c79ea1421719 WatchSource:0}: Error finding container 08f2ae6278c5b001e309743ae825ca5193a01c0a48389cb42ad8c79ea1421719: Status 404 returned error can't find the container with id 08f2ae6278c5b001e309743ae825ca5193a01c0a48389cb42ad8c79ea1421719
Sep 14 02:57:59.086973 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:59.086947    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqfmzl"]
Sep 14 02:57:59.141863 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:57:59.141763    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda68025f3_4c29_4dfb_b031_c435ae2e4c1b.slice/crio-0af5ae3f890b5bdf3f295d9cceaa84b40b9bd1293cbd45e3542fe626b294ea41 WatchSource:0}: Error finding container 0af5ae3f890b5bdf3f295d9cceaa84b40b9bd1293cbd45e3542fe626b294ea41: Status 404 returned error can't find the container with id 0af5ae3f890b5bdf3f295d9cceaa84b40b9bd1293cbd45e3542fe626b294ea41
Sep 14 02:57:59.622904 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:59.622866    2577 generic.go:356] "Generic (PLEG): container finished" podID="31b714a4-b440-42a6-9b05-5249cfc755d0" containerID="9dbd026d47b6562f8422622ca570f7ddcae29d59169042d45f424ee040897003" exitCode=0
Sep 14 02:57:59.623365 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:59.622950    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avqln8" event={"ID":"31b714a4-b440-42a6-9b05-5249cfc755d0","Type":"ContainerDied","Data":"9dbd026d47b6562f8422622ca570f7ddcae29d59169042d45f424ee040897003"}
Sep 14 02:57:59.623365 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:59.622997    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avqln8" event={"ID":"31b714a4-b440-42a6-9b05-5249cfc755d0","Type":"ContainerStarted","Data":"659ff75a917bc828b5de3e14f5ae16b32cab8efa3c5d75278db02f7fdcfa5493"}
Sep 14 02:57:59.624942 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:59.624910    2577 generic.go:356] "Generic (PLEG): container finished" podID="bc36e9f5-28d1-47fb-86ab-843cc31c1698" containerID="da4c3e12c405f3f42cbfe316042c7aa1a535fba3e7472e69938542e612ebde16" exitCode=0
Sep 14 02:57:59.625107 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:59.624964    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gtzwn" event={"ID":"bc36e9f5-28d1-47fb-86ab-843cc31c1698","Type":"ContainerDied","Data":"da4c3e12c405f3f42cbfe316042c7aa1a535fba3e7472e69938542e612ebde16"}
Sep 14 02:57:59.625107 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:59.625040    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gtzwn" event={"ID":"bc36e9f5-28d1-47fb-86ab-843cc31c1698","Type":"ContainerStarted","Data":"717e50dab9c92ba3a6ba3b802732063f4c34e2e5705762af51260f05e6952ec6"}
Sep 14 02:57:59.626955 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:59.626933    2577 generic.go:356] "Generic (PLEG): container finished" podID="d0aaff3a-006d-419f-a123-36bd4ce2be5b" containerID="45abfe91e3ef90dd83c753346f5f2d329fda38f5564c066f880c1fe5ae082a53" exitCode=0
Sep 14 02:57:59.627035 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:59.626989    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dl9hb" event={"ID":"d0aaff3a-006d-419f-a123-36bd4ce2be5b","Type":"ContainerDied","Data":"45abfe91e3ef90dd83c753346f5f2d329fda38f5564c066f880c1fe5ae082a53"}
Sep 14 02:57:59.627084 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:59.627035    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dl9hb" event={"ID":"d0aaff3a-006d-419f-a123-36bd4ce2be5b","Type":"ContainerStarted","Data":"08f2ae6278c5b001e309743ae825ca5193a01c0a48389cb42ad8c79ea1421719"}
Sep 14 02:57:59.629189 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:59.629161    2577 generic.go:356] "Generic (PLEG): container finished" podID="a68025f3-4c29-4dfb-b031-c435ae2e4c1b" containerID="d8a03fff0cf82f913a72415cda794e2e1334fbdaf174f78cb705ecbb94c50579" exitCode=0
Sep 14 02:57:59.629313 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:59.629203    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqfmzl" event={"ID":"a68025f3-4c29-4dfb-b031-c435ae2e4c1b","Type":"ContainerDied","Data":"d8a03fff0cf82f913a72415cda794e2e1334fbdaf174f78cb705ecbb94c50579"}
Sep 14 02:57:59.629313 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:57:59.629253    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqfmzl" event={"ID":"a68025f3-4c29-4dfb-b031-c435ae2e4c1b","Type":"ContainerStarted","Data":"0af5ae3f890b5bdf3f295d9cceaa84b40b9bd1293cbd45e3542fe626b294ea41"}
Sep 14 02:58:00.636627 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:00.636599    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqfmzl" event={"ID":"a68025f3-4c29-4dfb-b031-c435ae2e4c1b","Type":"ContainerStarted","Data":"cea4caafa6d8c8fcbb29e7e8ce8f6f420cc861e0afa04cc90dde3d4185a477b8"}
Sep 14 02:58:00.638463 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:00.638435    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gtzwn" event={"ID":"bc36e9f5-28d1-47fb-86ab-843cc31c1698","Type":"ContainerStarted","Data":"406c570b321918b14297eb8dffc4275b9fd714fd4ba4384bc759f618e2838c6c"}
Sep 14 02:58:01.645129 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:01.645096    2577 generic.go:356] "Generic (PLEG): container finished" podID="31b714a4-b440-42a6-9b05-5249cfc755d0" containerID="47b5b93f8fe9f3689a0e5ffa51e471fb8ab8a6b402c91be83e9384c86cf6f811" exitCode=0
Sep 14 02:58:01.645650 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:01.645179    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avqln8" event={"ID":"31b714a4-b440-42a6-9b05-5249cfc755d0","Type":"ContainerDied","Data":"47b5b93f8fe9f3689a0e5ffa51e471fb8ab8a6b402c91be83e9384c86cf6f811"}
Sep 14 02:58:01.647021 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:01.646998    2577 generic.go:356] "Generic (PLEG): container finished" podID="bc36e9f5-28d1-47fb-86ab-843cc31c1698" containerID="406c570b321918b14297eb8dffc4275b9fd714fd4ba4384bc759f618e2838c6c" exitCode=0
Sep 14 02:58:01.647143 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:01.647083    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gtzwn" event={"ID":"bc36e9f5-28d1-47fb-86ab-843cc31c1698","Type":"ContainerDied","Data":"406c570b321918b14297eb8dffc4275b9fd714fd4ba4384bc759f618e2838c6c"}
Sep 14 02:58:01.651397 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:01.651241    2577 generic.go:356] "Generic (PLEG): container finished" podID="d0aaff3a-006d-419f-a123-36bd4ce2be5b" containerID="54171df9ab59501add14b3eed79690dd9b50ee21e3f1a794c6a5eacdf273e59f" exitCode=0
Sep 14 02:58:01.651397 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:01.651354    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dl9hb" event={"ID":"d0aaff3a-006d-419f-a123-36bd4ce2be5b","Type":"ContainerDied","Data":"54171df9ab59501add14b3eed79690dd9b50ee21e3f1a794c6a5eacdf273e59f"}
Sep 14 02:58:01.653705 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:01.653687    2577 generic.go:356] "Generic (PLEG): container finished" podID="a68025f3-4c29-4dfb-b031-c435ae2e4c1b" containerID="cea4caafa6d8c8fcbb29e7e8ce8f6f420cc861e0afa04cc90dde3d4185a477b8" exitCode=0
Sep 14 02:58:01.653795 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:01.653769    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqfmzl" event={"ID":"a68025f3-4c29-4dfb-b031-c435ae2e4c1b","Type":"ContainerDied","Data":"cea4caafa6d8c8fcbb29e7e8ce8f6f420cc861e0afa04cc90dde3d4185a477b8"}
Sep 14 02:58:02.660450 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:02.660419    2577 generic.go:356] "Generic (PLEG): container finished" podID="31b714a4-b440-42a6-9b05-5249cfc755d0" containerID="87c6fc6c2bf8e1bfc0b087b2952bbdd8d8d766e34691f6c9c8f58e0bda79582c" exitCode=0
Sep 14 02:58:02.660865 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:02.660501    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avqln8" event={"ID":"31b714a4-b440-42a6-9b05-5249cfc755d0","Type":"ContainerDied","Data":"87c6fc6c2bf8e1bfc0b087b2952bbdd8d8d766e34691f6c9c8f58e0bda79582c"}
Sep 14 02:58:02.662532 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:02.662508    2577 generic.go:356] "Generic (PLEG): container finished" podID="bc36e9f5-28d1-47fb-86ab-843cc31c1698" containerID="7eb92288ef73f54306183266df85f5b465f511e5d1e1094afcd595d4ed326370" exitCode=0
Sep 14 02:58:02.662659 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:02.662543    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gtzwn" event={"ID":"bc36e9f5-28d1-47fb-86ab-843cc31c1698","Type":"ContainerDied","Data":"7eb92288ef73f54306183266df85f5b465f511e5d1e1094afcd595d4ed326370"}
Sep 14 02:58:02.664638 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:02.664613    2577 generic.go:356] "Generic (PLEG): container finished" podID="d0aaff3a-006d-419f-a123-36bd4ce2be5b" containerID="825da13f85014c0056b77855125b01945366788146d56902f96b03a5a5a51129" exitCode=0
Sep 14 02:58:02.664732 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:02.664695    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dl9hb" event={"ID":"d0aaff3a-006d-419f-a123-36bd4ce2be5b","Type":"ContainerDied","Data":"825da13f85014c0056b77855125b01945366788146d56902f96b03a5a5a51129"}
Sep 14 02:58:02.666674 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:02.666656    2577 generic.go:356] "Generic (PLEG): container finished" podID="a68025f3-4c29-4dfb-b031-c435ae2e4c1b" containerID="fcc803bf150b410e0f90c2c634d6bcdbf464243a38d911c5703c98c3a4d9c913" exitCode=0
Sep 14 02:58:02.666775 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:02.666692    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqfmzl" event={"ID":"a68025f3-4c29-4dfb-b031-c435ae2e4c1b","Type":"ContainerDied","Data":"fcc803bf150b410e0f90c2c634d6bcdbf464243a38d911c5703c98c3a4d9c913"}
Sep 14 02:58:03.861450 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:03.861424    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gtzwn"
Sep 14 02:58:03.864426 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:03.864406    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqfmzl"
Sep 14 02:58:03.895057 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:03.895029    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avqln8"
Sep 14 02:58:03.898122 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:03.898101    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dl9hb"
Sep 14 02:58:03.909510 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:03.909486    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/31b714a4-b440-42a6-9b05-5249cfc755d0-util\") pod \"31b714a4-b440-42a6-9b05-5249cfc755d0\" (UID: \"31b714a4-b440-42a6-9b05-5249cfc755d0\") "
Sep 14 02:58:03.909589 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:03.909531    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a68025f3-4c29-4dfb-b031-c435ae2e4c1b-bundle\") pod \"a68025f3-4c29-4dfb-b031-c435ae2e4c1b\" (UID: \"a68025f3-4c29-4dfb-b031-c435ae2e4c1b\") "
Sep 14 02:58:03.909589 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:03.909559    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d0aaff3a-006d-419f-a123-36bd4ce2be5b-util\") pod \"d0aaff3a-006d-419f-a123-36bd4ce2be5b\" (UID: \"d0aaff3a-006d-419f-a123-36bd4ce2be5b\") "
Sep 14 02:58:03.909589 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:03.909585    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc36e9f5-28d1-47fb-86ab-843cc31c1698-util\") pod \"bc36e9f5-28d1-47fb-86ab-843cc31c1698\" (UID: \"bc36e9f5-28d1-47fb-86ab-843cc31c1698\") "
Sep 14 02:58:03.909738 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:03.909604    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a68025f3-4c29-4dfb-b031-c435ae2e4c1b-util\") pod \"a68025f3-4c29-4dfb-b031-c435ae2e4c1b\" (UID: \"a68025f3-4c29-4dfb-b031-c435ae2e4c1b\") "
Sep 14 02:58:03.909738 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:03.909635    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c74bz\" (UniqueName: \"kubernetes.io/projected/31b714a4-b440-42a6-9b05-5249cfc755d0-kube-api-access-c74bz\") pod \"31b714a4-b440-42a6-9b05-5249cfc755d0\" (UID: \"31b714a4-b440-42a6-9b05-5249cfc755d0\") "
Sep 14 02:58:03.909738 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:03.909665    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/31b714a4-b440-42a6-9b05-5249cfc755d0-bundle\") pod \"31b714a4-b440-42a6-9b05-5249cfc755d0\" (UID: \"31b714a4-b440-42a6-9b05-5249cfc755d0\") "
Sep 14 02:58:03.909738 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:03.909714    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdgcn\" (UniqueName: \"kubernetes.io/projected/bc36e9f5-28d1-47fb-86ab-843cc31c1698-kube-api-access-kdgcn\") pod \"bc36e9f5-28d1-47fb-86ab-843cc31c1698\" (UID: \"bc36e9f5-28d1-47fb-86ab-843cc31c1698\") "
Sep 14 02:58:03.909738 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:03.909737    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bs86z\" (UniqueName: \"kubernetes.io/projected/d0aaff3a-006d-419f-a123-36bd4ce2be5b-kube-api-access-bs86z\") pod \"d0aaff3a-006d-419f-a123-36bd4ce2be5b\" (UID: \"d0aaff3a-006d-419f-a123-36bd4ce2be5b\") "
Sep 14 02:58:03.909993 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:03.909765    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d0aaff3a-006d-419f-a123-36bd4ce2be5b-bundle\") pod \"d0aaff3a-006d-419f-a123-36bd4ce2be5b\" (UID: \"d0aaff3a-006d-419f-a123-36bd4ce2be5b\") "
Sep 14 02:58:03.909993 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:03.909789    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tjngz\" (UniqueName: \"kubernetes.io/projected/a68025f3-4c29-4dfb-b031-c435ae2e4c1b-kube-api-access-tjngz\") pod \"a68025f3-4c29-4dfb-b031-c435ae2e4c1b\" (UID: \"a68025f3-4c29-4dfb-b031-c435ae2e4c1b\") "
Sep 14 02:58:03.909993 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:03.909846    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc36e9f5-28d1-47fb-86ab-843cc31c1698-bundle\") pod \"bc36e9f5-28d1-47fb-86ab-843cc31c1698\" (UID: \"bc36e9f5-28d1-47fb-86ab-843cc31c1698\") "
Sep 14 02:58:03.910531 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:03.910504    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc36e9f5-28d1-47fb-86ab-843cc31c1698-bundle" (OuterVolumeSpecName: "bundle") pod "bc36e9f5-28d1-47fb-86ab-843cc31c1698" (UID: "bc36e9f5-28d1-47fb-86ab-843cc31c1698"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:58:03.910695 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:03.910668    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a68025f3-4c29-4dfb-b031-c435ae2e4c1b-bundle" (OuterVolumeSpecName: "bundle") pod "a68025f3-4c29-4dfb-b031-c435ae2e4c1b" (UID: "a68025f3-4c29-4dfb-b031-c435ae2e4c1b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:58:03.911452 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:03.911426    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31b714a4-b440-42a6-9b05-5249cfc755d0-bundle" (OuterVolumeSpecName: "bundle") pod "31b714a4-b440-42a6-9b05-5249cfc755d0" (UID: "31b714a4-b440-42a6-9b05-5249cfc755d0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:58:03.912436 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:03.912408    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0aaff3a-006d-419f-a123-36bd4ce2be5b-bundle" (OuterVolumeSpecName: "bundle") pod "d0aaff3a-006d-419f-a123-36bd4ce2be5b" (UID: "d0aaff3a-006d-419f-a123-36bd4ce2be5b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:58:03.914020 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:03.913978    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31b714a4-b440-42a6-9b05-5249cfc755d0-kube-api-access-c74bz" (OuterVolumeSpecName: "kube-api-access-c74bz") pod "31b714a4-b440-42a6-9b05-5249cfc755d0" (UID: "31b714a4-b440-42a6-9b05-5249cfc755d0"). InnerVolumeSpecName "kube-api-access-c74bz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:58:03.914284 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:03.914241    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0aaff3a-006d-419f-a123-36bd4ce2be5b-kube-api-access-bs86z" (OuterVolumeSpecName: "kube-api-access-bs86z") pod "d0aaff3a-006d-419f-a123-36bd4ce2be5b" (UID: "d0aaff3a-006d-419f-a123-36bd4ce2be5b"). InnerVolumeSpecName "kube-api-access-bs86z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:58:03.914717 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:03.914695    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a68025f3-4c29-4dfb-b031-c435ae2e4c1b-kube-api-access-tjngz" (OuterVolumeSpecName: "kube-api-access-tjngz") pod "a68025f3-4c29-4dfb-b031-c435ae2e4c1b" (UID: "a68025f3-4c29-4dfb-b031-c435ae2e4c1b"). InnerVolumeSpecName "kube-api-access-tjngz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:58:03.915138 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:03.915102    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc36e9f5-28d1-47fb-86ab-843cc31c1698-kube-api-access-kdgcn" (OuterVolumeSpecName: "kube-api-access-kdgcn") pod "bc36e9f5-28d1-47fb-86ab-843cc31c1698" (UID: "bc36e9f5-28d1-47fb-86ab-843cc31c1698"). InnerVolumeSpecName "kube-api-access-kdgcn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:58:03.916675 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:03.916634    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc36e9f5-28d1-47fb-86ab-843cc31c1698-util" (OuterVolumeSpecName: "util") pod "bc36e9f5-28d1-47fb-86ab-843cc31c1698" (UID: "bc36e9f5-28d1-47fb-86ab-843cc31c1698"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:58:03.919132 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:03.919105    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0aaff3a-006d-419f-a123-36bd4ce2be5b-util" (OuterVolumeSpecName: "util") pod "d0aaff3a-006d-419f-a123-36bd4ce2be5b" (UID: "d0aaff3a-006d-419f-a123-36bd4ce2be5b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:58:03.919341 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:03.919318    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a68025f3-4c29-4dfb-b031-c435ae2e4c1b-util" (OuterVolumeSpecName: "util") pod "a68025f3-4c29-4dfb-b031-c435ae2e4c1b" (UID: "a68025f3-4c29-4dfb-b031-c435ae2e4c1b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:58:03.923519 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:03.923336    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31b714a4-b440-42a6-9b05-5249cfc755d0-util" (OuterVolumeSpecName: "util") pod "31b714a4-b440-42a6-9b05-5249cfc755d0" (UID: "31b714a4-b440-42a6-9b05-5249cfc755d0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:58:04.010917 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:04.010892    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdgcn\" (UniqueName: \"kubernetes.io/projected/bc36e9f5-28d1-47fb-86ab-843cc31c1698-kube-api-access-kdgcn\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:58:04.010917 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:04.010915    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bs86z\" (UniqueName: \"kubernetes.io/projected/d0aaff3a-006d-419f-a123-36bd4ce2be5b-kube-api-access-bs86z\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:58:04.011057 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:04.010924    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d0aaff3a-006d-419f-a123-36bd4ce2be5b-bundle\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:58:04.011057 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:04.010933    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjngz\" (UniqueName: \"kubernetes.io/projected/a68025f3-4c29-4dfb-b031-c435ae2e4c1b-kube-api-access-tjngz\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:58:04.011057 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:04.010941    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc36e9f5-28d1-47fb-86ab-843cc31c1698-bundle\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:58:04.011057 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:04.010949    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/31b714a4-b440-42a6-9b05-5249cfc755d0-util\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:58:04.011057 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:04.010957    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a68025f3-4c29-4dfb-b031-c435ae2e4c1b-bundle\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:58:04.011057 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:04.010964    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d0aaff3a-006d-419f-a123-36bd4ce2be5b-util\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:58:04.011057 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:04.010971    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc36e9f5-28d1-47fb-86ab-843cc31c1698-util\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:58:04.011057 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:04.010978    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a68025f3-4c29-4dfb-b031-c435ae2e4c1b-util\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:58:04.011057 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:04.010985    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c74bz\" (UniqueName: \"kubernetes.io/projected/31b714a4-b440-42a6-9b05-5249cfc755d0-kube-api-access-c74bz\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:58:04.011057 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:04.010992    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/31b714a4-b440-42a6-9b05-5249cfc755d0-bundle\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:58:04.679504 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:04.679475    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avqln8"
Sep 14 02:58:04.679683 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:04.679475    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847avqln8" event={"ID":"31b714a4-b440-42a6-9b05-5249cfc755d0","Type":"ContainerDied","Data":"659ff75a917bc828b5de3e14f5ae16b32cab8efa3c5d75278db02f7fdcfa5493"}
Sep 14 02:58:04.679683 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:04.679594    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="659ff75a917bc828b5de3e14f5ae16b32cab8efa3c5d75278db02f7fdcfa5493"
Sep 14 02:58:04.681497 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:04.681469    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gtzwn" event={"ID":"bc36e9f5-28d1-47fb-86ab-843cc31c1698","Type":"ContainerDied","Data":"717e50dab9c92ba3a6ba3b802732063f4c34e2e5705762af51260f05e6952ec6"}
Sep 14 02:58:04.681649 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:04.681511    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="717e50dab9c92ba3a6ba3b802732063f4c34e2e5705762af51260f05e6952ec6"
Sep 14 02:58:04.681649 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:04.681475    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7gtzwn"
Sep 14 02:58:04.683578 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:04.683559    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dl9hb"
Sep 14 02:58:04.683692 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:04.683574    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dl9hb" event={"ID":"d0aaff3a-006d-419f-a123-36bd4ce2be5b","Type":"ContainerDied","Data":"08f2ae6278c5b001e309743ae825ca5193a01c0a48389cb42ad8c79ea1421719"}
Sep 14 02:58:04.683692 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:04.683610    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="08f2ae6278c5b001e309743ae825ca5193a01c0a48389cb42ad8c79ea1421719"
Sep 14 02:58:04.685528 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:04.685511    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqfmzl"
Sep 14 02:58:04.685528 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:04.685513    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqfmzl" event={"ID":"a68025f3-4c29-4dfb-b031-c435ae2e4c1b","Type":"ContainerDied","Data":"0af5ae3f890b5bdf3f295d9cceaa84b40b9bd1293cbd45e3542fe626b294ea41"}
Sep 14 02:58:04.685656 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:04.685551    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0af5ae3f890b5bdf3f295d9cceaa84b40b9bd1293cbd45e3542fe626b294ea41"
Sep 14 02:58:11.479624 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:11.479594    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-qffx5"]
Sep 14 02:58:11.480006 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:11.479915    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0aaff3a-006d-419f-a123-36bd4ce2be5b" containerName="pull"
Sep 14 02:58:11.480006 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:11.479926    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0aaff3a-006d-419f-a123-36bd4ce2be5b" containerName="pull"
Sep 14 02:58:11.480006 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:11.479938    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc36e9f5-28d1-47fb-86ab-843cc31c1698" containerName="pull"
Sep 14 02:58:11.480006 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:11.479943    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc36e9f5-28d1-47fb-86ab-843cc31c1698" containerName="pull"
Sep 14 02:58:11.480006 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:11.479952    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31b714a4-b440-42a6-9b05-5249cfc755d0" containerName="pull"
Sep 14 02:58:11.480006 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:11.479957    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="31b714a4-b440-42a6-9b05-5249cfc755d0" containerName="pull"
Sep 14 02:58:11.480006 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:11.479963    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a68025f3-4c29-4dfb-b031-c435ae2e4c1b" containerName="util"
Sep 14 02:58:11.480006 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:11.479968    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a68025f3-4c29-4dfb-b031-c435ae2e4c1b" containerName="util"
Sep 14 02:58:11.480006 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:11.479978    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31b714a4-b440-42a6-9b05-5249cfc755d0" containerName="util"
Sep 14 02:58:11.480006 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:11.479983    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="31b714a4-b440-42a6-9b05-5249cfc755d0" containerName="util"
Sep 14 02:58:11.480006 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:11.479989    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a68025f3-4c29-4dfb-b031-c435ae2e4c1b" containerName="pull"
Sep 14 02:58:11.480006 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:11.479994    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a68025f3-4c29-4dfb-b031-c435ae2e4c1b" containerName="pull"
Sep 14 02:58:11.480006 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:11.480000    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc36e9f5-28d1-47fb-86ab-843cc31c1698" containerName="util"
Sep 14 02:58:11.480006 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:11.480005    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc36e9f5-28d1-47fb-86ab-843cc31c1698" containerName="util"
Sep 14 02:58:11.480006 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:11.480010    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc36e9f5-28d1-47fb-86ab-843cc31c1698" containerName="extract"
Sep 14 02:58:11.480006 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:11.480015    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc36e9f5-28d1-47fb-86ab-843cc31c1698" containerName="extract"
Sep 14 02:58:11.480471 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:11.480021    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31b714a4-b440-42a6-9b05-5249cfc755d0" containerName="extract"
Sep 14 02:58:11.480471 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:11.480026    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="31b714a4-b440-42a6-9b05-5249cfc755d0" containerName="extract"
Sep 14 02:58:11.480471 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:11.480032    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0aaff3a-006d-419f-a123-36bd4ce2be5b" containerName="extract"
Sep 14 02:58:11.480471 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:11.480037    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0aaff3a-006d-419f-a123-36bd4ce2be5b" containerName="extract"
Sep 14 02:58:11.480471 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:11.480042    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a68025f3-4c29-4dfb-b031-c435ae2e4c1b" containerName="extract"
Sep 14 02:58:11.480471 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:11.480048    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a68025f3-4c29-4dfb-b031-c435ae2e4c1b" containerName="extract"
Sep 14 02:58:11.480471 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:11.480055    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0aaff3a-006d-419f-a123-36bd4ce2be5b" containerName="util"
Sep 14 02:58:11.480471 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:11.480060    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0aaff3a-006d-419f-a123-36bd4ce2be5b" containerName="util"
Sep 14 02:58:11.480471 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:11.480121    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="31b714a4-b440-42a6-9b05-5249cfc755d0" containerName="extract"
Sep 14 02:58:11.480471 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:11.480130    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a68025f3-4c29-4dfb-b031-c435ae2e4c1b" containerName="extract"
Sep 14 02:58:11.480471 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:11.480136    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0aaff3a-006d-419f-a123-36bd4ce2be5b" containerName="extract"
Sep 14 02:58:11.480471 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:11.480142    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc36e9f5-28d1-47fb-86ab-843cc31c1698" containerName="extract"
Sep 14 02:58:11.484728 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:11.484708    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-qffx5"
Sep 14 02:58:11.499619 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:11.499599    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 14 02:58:11.499704 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:11.499601    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 14 02:58:11.499704 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:11.499602    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-96h56\""
Sep 14 02:58:11.513503 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:11.513478    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-qffx5"]
Sep 14 02:58:11.570276 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:11.570244    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fckrc\" (UniqueName: \"kubernetes.io/projected/e7236ab6-c434-4535-9e0e-6e65b90a1237-kube-api-access-fckrc\") pod \"limitador-operator-controller-manager-74495c69f8-qffx5\" (UID: \"e7236ab6-c434-4535-9e0e-6e65b90a1237\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-qffx5"
Sep 14 02:58:11.671447 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:11.671424    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fckrc\" (UniqueName: \"kubernetes.io/projected/e7236ab6-c434-4535-9e0e-6e65b90a1237-kube-api-access-fckrc\") pod \"limitador-operator-controller-manager-74495c69f8-qffx5\" (UID: \"e7236ab6-c434-4535-9e0e-6e65b90a1237\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-qffx5"
Sep 14 02:58:11.685776 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:11.685751    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fckrc\" (UniqueName: \"kubernetes.io/projected/e7236ab6-c434-4535-9e0e-6e65b90a1237-kube-api-access-fckrc\") pod \"limitador-operator-controller-manager-74495c69f8-qffx5\" (UID: \"e7236ab6-c434-4535-9e0e-6e65b90a1237\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-qffx5"
Sep 14 02:58:11.794414 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:11.794396    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-qffx5"
Sep 14 02:58:12.095467 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:12.095399    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-qffx5"]
Sep 14 02:58:12.102334 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:58:12.101448    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7236ab6_c434_4535_9e0e_6e65b90a1237.slice/crio-33fb2e1d7146b2b63074e230e4c8b97b2d760fc423ecffda0da23e9900a2d587 WatchSource:0}: Error finding container 33fb2e1d7146b2b63074e230e4c8b97b2d760fc423ecffda0da23e9900a2d587: Status 404 returned error can't find the container with id 33fb2e1d7146b2b63074e230e4c8b97b2d760fc423ecffda0da23e9900a2d587
Sep 14 02:58:12.724075 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:12.724046    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-qffx5" event={"ID":"e7236ab6-c434-4535-9e0e-6e65b90a1237","Type":"ContainerStarted","Data":"33fb2e1d7146b2b63074e230e4c8b97b2d760fc423ecffda0da23e9900a2d587"}
Sep 14 02:58:14.739540 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:14.739504    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-qffx5" event={"ID":"e7236ab6-c434-4535-9e0e-6e65b90a1237","Type":"ContainerStarted","Data":"850f05d77b0620d57bbfbf89c0399e499b6162fe735c561d63d40f990340155b"}
Sep 14 02:58:14.739862 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:14.739588    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-qffx5"
Sep 14 02:58:14.773531 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:14.773487    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-qffx5" podStartSLOduration=1.3328466350000001 podStartE2EDuration="3.773472674s" totalImagesPullingTime="2.440626039s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:58:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:58:14.772605963 +0000 UTC m=+443.750583986" watchObservedRunningTime="2026-09-14 02:58:14.773472674 +0000 UTC m=+443.751450703"
Sep 14 02:58:15.692772 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:15.692732    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-55c47f6d67-mhxtv" podUID="e4a51488-03a5-44a8-a6dd-6956392df22f" containerName="console" containerID="cri-o://35b24006644a2a836969623afca0da8ae63dee194f83a1d4bb64f3b16ac790ee" gracePeriod=15
Sep 14 02:58:15.938689 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:15.938666    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55c47f6d67-mhxtv"
Sep 14 02:58:16.007999 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.007978    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e4a51488-03a5-44a8-a6dd-6956392df22f-console-config\") pod \"e4a51488-03a5-44a8-a6dd-6956392df22f\" (UID: \"e4a51488-03a5-44a8-a6dd-6956392df22f\") "
Sep 14 02:58:16.008105 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.008007    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e4a51488-03a5-44a8-a6dd-6956392df22f-service-ca\") pod \"e4a51488-03a5-44a8-a6dd-6956392df22f\" (UID: \"e4a51488-03a5-44a8-a6dd-6956392df22f\") "
Sep 14 02:58:16.008105 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.008062    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e4a51488-03a5-44a8-a6dd-6956392df22f-oauth-serving-cert\") pod \"e4a51488-03a5-44a8-a6dd-6956392df22f\" (UID: \"e4a51488-03a5-44a8-a6dd-6956392df22f\") "
Sep 14 02:58:16.008189 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.008166    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e4a51488-03a5-44a8-a6dd-6956392df22f-console-serving-cert\") pod \"e4a51488-03a5-44a8-a6dd-6956392df22f\" (UID: \"e4a51488-03a5-44a8-a6dd-6956392df22f\") "
Sep 14 02:58:16.008242 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.008215    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l4947\" (UniqueName: \"kubernetes.io/projected/e4a51488-03a5-44a8-a6dd-6956392df22f-kube-api-access-l4947\") pod \"e4a51488-03a5-44a8-a6dd-6956392df22f\" (UID: \"e4a51488-03a5-44a8-a6dd-6956392df22f\") "
Sep 14 02:58:16.008242 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.008233    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e4a51488-03a5-44a8-a6dd-6956392df22f-console-oauth-config\") pod \"e4a51488-03a5-44a8-a6dd-6956392df22f\" (UID: \"e4a51488-03a5-44a8-a6dd-6956392df22f\") "
Sep 14 02:58:16.008371 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.008281    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4a51488-03a5-44a8-a6dd-6956392df22f-trusted-ca-bundle\") pod \"e4a51488-03a5-44a8-a6dd-6956392df22f\" (UID: \"e4a51488-03a5-44a8-a6dd-6956392df22f\") "
Sep 14 02:58:16.008422 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.008390    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4a51488-03a5-44a8-a6dd-6956392df22f-console-config" (OuterVolumeSpecName: "console-config") pod "e4a51488-03a5-44a8-a6dd-6956392df22f" (UID: "e4a51488-03a5-44a8-a6dd-6956392df22f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 02:58:16.008477 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.008417    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4a51488-03a5-44a8-a6dd-6956392df22f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e4a51488-03a5-44a8-a6dd-6956392df22f" (UID: "e4a51488-03a5-44a8-a6dd-6956392df22f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 02:58:16.008477 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.008433    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4a51488-03a5-44a8-a6dd-6956392df22f-service-ca" (OuterVolumeSpecName: "service-ca") pod "e4a51488-03a5-44a8-a6dd-6956392df22f" (UID: "e4a51488-03a5-44a8-a6dd-6956392df22f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 02:58:16.008732 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.008711    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e4a51488-03a5-44a8-a6dd-6956392df22f-console-config\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:58:16.008805 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.008734    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e4a51488-03a5-44a8-a6dd-6956392df22f-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:58:16.008805 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.008747    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e4a51488-03a5-44a8-a6dd-6956392df22f-oauth-serving-cert\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:58:16.008805 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.008785    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4a51488-03a5-44a8-a6dd-6956392df22f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e4a51488-03a5-44a8-a6dd-6956392df22f" (UID: "e4a51488-03a5-44a8-a6dd-6956392df22f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 02:58:16.010261 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.010233    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4a51488-03a5-44a8-a6dd-6956392df22f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e4a51488-03a5-44a8-a6dd-6956392df22f" (UID: "e4a51488-03a5-44a8-a6dd-6956392df22f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 02:58:16.010353 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.010297    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4a51488-03a5-44a8-a6dd-6956392df22f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e4a51488-03a5-44a8-a6dd-6956392df22f" (UID: "e4a51488-03a5-44a8-a6dd-6956392df22f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 02:58:16.010474 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.010339    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4a51488-03a5-44a8-a6dd-6956392df22f-kube-api-access-l4947" (OuterVolumeSpecName: "kube-api-access-l4947") pod "e4a51488-03a5-44a8-a6dd-6956392df22f" (UID: "e4a51488-03a5-44a8-a6dd-6956392df22f"). InnerVolumeSpecName "kube-api-access-l4947". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:58:16.079841 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.079818    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-km4tk"]
Sep 14 02:58:16.080162 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.080150    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4a51488-03a5-44a8-a6dd-6956392df22f" containerName="console"
Sep 14 02:58:16.080253 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.080163    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4a51488-03a5-44a8-a6dd-6956392df22f" containerName="console"
Sep 14 02:58:16.080253 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.080244    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4a51488-03a5-44a8-a6dd-6956392df22f" containerName="console"
Sep 14 02:58:16.083387 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.083372    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-km4tk"
Sep 14 02:58:16.091436 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.091418    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-c96wl\""
Sep 14 02:58:16.091600 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.091582    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 14 02:58:16.109523 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.109499    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e4a51488-03a5-44a8-a6dd-6956392df22f-console-serving-cert\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:58:16.109601 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.109526    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4947\" (UniqueName: \"kubernetes.io/projected/e4a51488-03a5-44a8-a6dd-6956392df22f-kube-api-access-l4947\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:58:16.109601 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.109542    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e4a51488-03a5-44a8-a6dd-6956392df22f-console-oauth-config\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:58:16.109601 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.109555    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4a51488-03a5-44a8-a6dd-6956392df22f-trusted-ca-bundle\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:58:16.109837 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.109817    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-km4tk"]
Sep 14 02:58:16.210840 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.210812    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xx7v\" (UniqueName: \"kubernetes.io/projected/31e6517f-03ef-4862-bde0-71d4293f96c0-kube-api-access-9xx7v\") pod \"dns-operator-controller-manager-7b84cd7c9c-km4tk\" (UID: \"31e6517f-03ef-4862-bde0-71d4293f96c0\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-km4tk"
Sep 14 02:58:16.311392 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.311335    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xx7v\" (UniqueName: \"kubernetes.io/projected/31e6517f-03ef-4862-bde0-71d4293f96c0-kube-api-access-9xx7v\") pod \"dns-operator-controller-manager-7b84cd7c9c-km4tk\" (UID: \"31e6517f-03ef-4862-bde0-71d4293f96c0\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-km4tk"
Sep 14 02:58:16.335964 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.335940    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xx7v\" (UniqueName: \"kubernetes.io/projected/31e6517f-03ef-4862-bde0-71d4293f96c0-kube-api-access-9xx7v\") pod \"dns-operator-controller-manager-7b84cd7c9c-km4tk\" (UID: \"31e6517f-03ef-4862-bde0-71d4293f96c0\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-km4tk"
Sep 14 02:58:16.393074 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.393050    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-km4tk"
Sep 14 02:58:16.672933 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.672904    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-km4tk"]
Sep 14 02:58:16.674370 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:58:16.674347    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31e6517f_03ef_4862_bde0_71d4293f96c0.slice/crio-5db89c8d5efa4c216f5299f605a258274961359afcd45bfc6f041e37281e5e77 WatchSource:0}: Error finding container 5db89c8d5efa4c216f5299f605a258274961359afcd45bfc6f041e37281e5e77: Status 404 returned error can't find the container with id 5db89c8d5efa4c216f5299f605a258274961359afcd45bfc6f041e37281e5e77
Sep 14 02:58:16.750827 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.750797    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-km4tk" event={"ID":"31e6517f-03ef-4862-bde0-71d4293f96c0","Type":"ContainerStarted","Data":"5db89c8d5efa4c216f5299f605a258274961359afcd45bfc6f041e37281e5e77"}
Sep 14 02:58:16.752035 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.752015    2577 generic.go:356] "Generic (PLEG): container finished" podID="e4a51488-03a5-44a8-a6dd-6956392df22f" containerID="35b24006644a2a836969623afca0da8ae63dee194f83a1d4bb64f3b16ac790ee" exitCode=0
Sep 14 02:58:16.752121 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.752053    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55c47f6d67-mhxtv" event={"ID":"e4a51488-03a5-44a8-a6dd-6956392df22f","Type":"ContainerDied","Data":"35b24006644a2a836969623afca0da8ae63dee194f83a1d4bb64f3b16ac790ee"}
Sep 14 02:58:16.752121 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.752059    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55c47f6d67-mhxtv"
Sep 14 02:58:16.752121 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.752080    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55c47f6d67-mhxtv" event={"ID":"e4a51488-03a5-44a8-a6dd-6956392df22f","Type":"ContainerDied","Data":"e1962bce5a75264a9ba314c5781f9d55ba772e32bf1907bd2f2254d32deacaee"}
Sep 14 02:58:16.752121 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.752098    2577 scope.go:117] "RemoveContainer" containerID="35b24006644a2a836969623afca0da8ae63dee194f83a1d4bb64f3b16ac790ee"
Sep 14 02:58:16.760410 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.760391    2577 scope.go:117] "RemoveContainer" containerID="35b24006644a2a836969623afca0da8ae63dee194f83a1d4bb64f3b16ac790ee"
Sep 14 02:58:16.760674 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:58:16.760647    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35b24006644a2a836969623afca0da8ae63dee194f83a1d4bb64f3b16ac790ee\": container with ID starting with 35b24006644a2a836969623afca0da8ae63dee194f83a1d4bb64f3b16ac790ee not found: ID does not exist" containerID="35b24006644a2a836969623afca0da8ae63dee194f83a1d4bb64f3b16ac790ee"
Sep 14 02:58:16.760727 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.760675    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35b24006644a2a836969623afca0da8ae63dee194f83a1d4bb64f3b16ac790ee"} err="failed to get container status \"35b24006644a2a836969623afca0da8ae63dee194f83a1d4bb64f3b16ac790ee\": rpc error: code = NotFound desc = could not find container \"35b24006644a2a836969623afca0da8ae63dee194f83a1d4bb64f3b16ac790ee\": container with ID starting with 35b24006644a2a836969623afca0da8ae63dee194f83a1d4bb64f3b16ac790ee not found: ID does not exist"
Sep 14 02:58:16.776636 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.776615    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55c47f6d67-mhxtv"]
Sep 14 02:58:16.791321 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:16.787499    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-55c47f6d67-mhxtv"]
Sep 14 02:58:17.618336 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:17.618304    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4a51488-03a5-44a8-a6dd-6956392df22f" path="/var/lib/kubelet/pods/e4a51488-03a5-44a8-a6dd-6956392df22f/volumes"
Sep 14 02:58:18.765818 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:18.765722    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-km4tk" event={"ID":"31e6517f-03ef-4862-bde0-71d4293f96c0","Type":"ContainerStarted","Data":"54c28ac7dacced1f4787570b973388c96f068a0e2f0b13f168a884f3e72ef785"}
Sep 14 02:58:18.765818 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:18.765794    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-km4tk"
Sep 14 02:58:18.791629 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:18.791585    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-km4tk" podStartSLOduration=0.961518842 podStartE2EDuration="2.79157078s" totalImagesPullingTime="1.830051938s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:58:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:58:18.789109226 +0000 UTC m=+447.767087270" watchObservedRunningTime="2026-09-14 02:58:18.79157078 +0000 UTC m=+447.769548809"
Sep 14 02:58:20.197768 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:20.197739    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gf9cg"]
Sep 14 02:58:20.201239 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:20.201223    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gf9cg"
Sep 14 02:58:20.204633 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:20.204618    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-hd99t\""
Sep 14 02:58:20.219205 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:20.219181    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gf9cg"]
Sep 14 02:58:20.237836 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:20.237807    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/adbfd619-db36-4a0d-a741-5b2bef706f2a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-gf9cg\" (UID: \"adbfd619-db36-4a0d-a741-5b2bef706f2a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gf9cg"
Sep 14 02:58:20.237947 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:20.237889    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4crf\" (UniqueName: \"kubernetes.io/projected/adbfd619-db36-4a0d-a741-5b2bef706f2a-kube-api-access-s4crf\") pod \"kuadrant-operator-controller-manager-76f8f686bf-gf9cg\" (UID: \"adbfd619-db36-4a0d-a741-5b2bef706f2a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gf9cg"
Sep 14 02:58:20.338281 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:20.338241    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/adbfd619-db36-4a0d-a741-5b2bef706f2a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-gf9cg\" (UID: \"adbfd619-db36-4a0d-a741-5b2bef706f2a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gf9cg"
Sep 14 02:58:20.338372 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:20.338322    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4crf\" (UniqueName: \"kubernetes.io/projected/adbfd619-db36-4a0d-a741-5b2bef706f2a-kube-api-access-s4crf\") pod \"kuadrant-operator-controller-manager-76f8f686bf-gf9cg\" (UID: \"adbfd619-db36-4a0d-a741-5b2bef706f2a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gf9cg"
Sep 14 02:58:20.338586 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:20.338570    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/adbfd619-db36-4a0d-a741-5b2bef706f2a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-gf9cg\" (UID: \"adbfd619-db36-4a0d-a741-5b2bef706f2a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gf9cg"
Sep 14 02:58:20.355208 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:20.355184    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4crf\" (UniqueName: \"kubernetes.io/projected/adbfd619-db36-4a0d-a741-5b2bef706f2a-kube-api-access-s4crf\") pod \"kuadrant-operator-controller-manager-76f8f686bf-gf9cg\" (UID: \"adbfd619-db36-4a0d-a741-5b2bef706f2a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gf9cg"
Sep 14 02:58:20.510718 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:20.510695    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gf9cg"
Sep 14 02:58:20.794540 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:20.794513    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gf9cg"]
Sep 14 02:58:20.795721 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:58:20.795699    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadbfd619_db36_4a0d_a741_5b2bef706f2a.slice/crio-de4de36fb6d0361b4b98c5a957ed2d5618ab4c9494c255e623cce56153bdd244 WatchSource:0}: Error finding container de4de36fb6d0361b4b98c5a957ed2d5618ab4c9494c255e623cce56153bdd244: Status 404 returned error can't find the container with id de4de36fb6d0361b4b98c5a957ed2d5618ab4c9494c255e623cce56153bdd244
Sep 14 02:58:21.783309 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:21.783259    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gf9cg" event={"ID":"adbfd619-db36-4a0d-a741-5b2bef706f2a","Type":"ContainerStarted","Data":"de4de36fb6d0361b4b98c5a957ed2d5618ab4c9494c255e623cce56153bdd244"}
Sep 14 02:58:24.802108 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:24.802078    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gf9cg" event={"ID":"adbfd619-db36-4a0d-a741-5b2bef706f2a","Type":"ContainerStarted","Data":"dab54c181c0d0f26ef01e61a2c8e885eeb7d1927d7a5028df08d548e2907bd5c"}
Sep 14 02:58:24.802493 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:24.802212    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gf9cg"
Sep 14 02:58:24.829049 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:24.828992    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gf9cg" podStartSLOduration=0.986969011 podStartE2EDuration="4.828972461s" totalImagesPullingTime="3.84200345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:58:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:58:24.82817732 +0000 UTC m=+453.806155349" watchObservedRunningTime="2026-09-14 02:58:24.828972461 +0000 UTC m=+453.806950493"
Sep 14 02:58:25.746746 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:25.746714    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-qffx5"
Sep 14 02:58:29.773744 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:29.773698    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-km4tk"
Sep 14 02:58:35.810597 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:35.810554    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gf9cg"
Sep 14 02:58:38.377736 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.377689    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gf9cg"]
Sep 14 02:58:38.378239 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.377945    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gf9cg" podUID="adbfd619-db36-4a0d-a741-5b2bef706f2a" containerName="manager" containerID="cri-o://dab54c181c0d0f26ef01e61a2c8e885eeb7d1927d7a5028df08d548e2907bd5c" gracePeriod=2
Sep 14 02:58:38.380680 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:58:38.380635    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-gf9cg\" is forbidden: User \"system:node:ip-10-0-141-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-180.ec2.internal' and this object" podUID="adbfd619-db36-4a0d-a741-5b2bef706f2a" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gf9cg"
Sep 14 02:58:38.383084 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.382980    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gf9cg"]
Sep 14 02:58:38.398731 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.398697    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-qffx5"]
Sep 14 02:58:38.398972 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.398946    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-qffx5" podUID="e7236ab6-c434-4535-9e0e-6e65b90a1237" containerName="manager" containerID="cri-o://850f05d77b0620d57bbfbf89c0399e499b6162fe735c561d63d40f990340155b" gracePeriod=2
Sep 14 02:58:38.419797 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.419757    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-qffx5"]
Sep 14 02:58:38.422364 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.422341    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9rr6z"]
Sep 14 02:58:38.422943 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.422763    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adbfd619-db36-4a0d-a741-5b2bef706f2a" containerName="manager"
Sep 14 02:58:38.422943 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.422784    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="adbfd619-db36-4a0d-a741-5b2bef706f2a" containerName="manager"
Sep 14 02:58:38.422943 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.422810    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7236ab6-c434-4535-9e0e-6e65b90a1237" containerName="manager"
Sep 14 02:58:38.422943 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.422819    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7236ab6-c434-4535-9e0e-6e65b90a1237" containerName="manager"
Sep 14 02:58:38.422943 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.422900    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="adbfd619-db36-4a0d-a741-5b2bef706f2a" containerName="manager"
Sep 14 02:58:38.422943 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.422910    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7236ab6-c434-4535-9e0e-6e65b90a1237" containerName="manager"
Sep 14 02:58:38.425971 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.425953    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9rr6z"
Sep 14 02:58:38.430689 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:58:38.430653    2577 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"e7236ab6-c434-4535-9e0e-6e65b90a1237\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-74495c69f8-qffx5\": pods \"limitador-operator-controller-manager-74495c69f8-qffx5\" not found" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-qffx5"
Sep 14 02:58:38.432916 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:58:38.432889    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-gf9cg\" is forbidden: User \"system:node:ip-10-0-141-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-180.ec2.internal' and this object" podUID="adbfd619-db36-4a0d-a741-5b2bef706f2a" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gf9cg"
Sep 14 02:58:38.438503 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.438386    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-7p4ps"]
Sep 14 02:58:38.438722 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:58:38.438687    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-gf9cg\" is forbidden: User \"system:node:ip-10-0-141-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-180.ec2.internal' and this object" podUID="adbfd619-db36-4a0d-a741-5b2bef706f2a" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gf9cg"
Sep 14 02:58:38.442720 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.442696    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9rr6z"]
Sep 14 02:58:38.442819 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.442775    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7p4ps"
Sep 14 02:58:38.463795 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.463772    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-7p4ps"]
Sep 14 02:58:38.473877 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:58:38.473848    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-qffx5\" is forbidden: User \"system:node:ip-10-0-141-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-180.ec2.internal' and this object" podUID="e7236ab6-c434-4535-9e0e-6e65b90a1237" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-qffx5"
Sep 14 02:58:38.476429 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:58:38.476397    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-gf9cg\" is forbidden: User \"system:node:ip-10-0-141-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-180.ec2.internal' and this object" podUID="adbfd619-db36-4a0d-a741-5b2bef706f2a" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gf9cg"
Sep 14 02:58:38.499666 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:58:38.499642    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-qffx5\" is forbidden: User \"system:node:ip-10-0-141-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-180.ec2.internal' and this object" podUID="e7236ab6-c434-4535-9e0e-6e65b90a1237" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-qffx5"
Sep 14 02:58:38.580846 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.580814    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ch6xf\" (UniqueName: \"kubernetes.io/projected/af73e240-e395-4da4-9a24-e53f77daef4e-kube-api-access-ch6xf\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9rr6z\" (UID: \"af73e240-e395-4da4-9a24-e53f77daef4e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9rr6z"
Sep 14 02:58:38.580980 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.580859    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggg8l\" (UniqueName: \"kubernetes.io/projected/ed8c4796-f1d6-43f3-b956-4f0a43fcaec9-kube-api-access-ggg8l\") pod \"limitador-operator-controller-manager-74495c69f8-7p4ps\" (UID: \"ed8c4796-f1d6-43f3-b956-4f0a43fcaec9\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7p4ps"
Sep 14 02:58:38.580980 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.580883    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af73e240-e395-4da4-9a24-e53f77daef4e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9rr6z\" (UID: \"af73e240-e395-4da4-9a24-e53f77daef4e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9rr6z"
Sep 14 02:58:38.639599 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.639547    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gf9cg"
Sep 14 02:58:38.641713 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:58:38.641690    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-qffx5\" is forbidden: User \"system:node:ip-10-0-141-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-180.ec2.internal' and this object" podUID="e7236ab6-c434-4535-9e0e-6e65b90a1237" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-qffx5"
Sep 14 02:58:38.643753 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:58:38.643727    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-gf9cg\" is forbidden: User \"system:node:ip-10-0-141-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-180.ec2.internal' and this object" podUID="adbfd619-db36-4a0d-a741-5b2bef706f2a" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gf9cg"
Sep 14 02:58:38.661496 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.661477    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-qffx5"
Sep 14 02:58:38.663805 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:58:38.663785    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-qffx5\" is forbidden: User \"system:node:ip-10-0-141-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-180.ec2.internal' and this object" podUID="e7236ab6-c434-4535-9e0e-6e65b90a1237" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-qffx5"
Sep 14 02:58:38.668061 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:58:38.668039    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-gf9cg\" is forbidden: User \"system:node:ip-10-0-141-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-180.ec2.internal' and this object" podUID="adbfd619-db36-4a0d-a741-5b2bef706f2a" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gf9cg"
Sep 14 02:58:38.681950 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.681925    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ch6xf\" (UniqueName: \"kubernetes.io/projected/af73e240-e395-4da4-9a24-e53f77daef4e-kube-api-access-ch6xf\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9rr6z\" (UID: \"af73e240-e395-4da4-9a24-e53f77daef4e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9rr6z"
Sep 14 02:58:38.682019 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.681956    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggg8l\" (UniqueName: \"kubernetes.io/projected/ed8c4796-f1d6-43f3-b956-4f0a43fcaec9-kube-api-access-ggg8l\") pod \"limitador-operator-controller-manager-74495c69f8-7p4ps\" (UID: \"ed8c4796-f1d6-43f3-b956-4f0a43fcaec9\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7p4ps"
Sep 14 02:58:38.682019 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.681979    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af73e240-e395-4da4-9a24-e53f77daef4e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9rr6z\" (UID: \"af73e240-e395-4da4-9a24-e53f77daef4e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9rr6z"
Sep 14 02:58:38.682296 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.682264    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af73e240-e395-4da4-9a24-e53f77daef4e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9rr6z\" (UID: \"af73e240-e395-4da4-9a24-e53f77daef4e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9rr6z"
Sep 14 02:58:38.696723 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.696697    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ch6xf\" (UniqueName: \"kubernetes.io/projected/af73e240-e395-4da4-9a24-e53f77daef4e-kube-api-access-ch6xf\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9rr6z\" (UID: \"af73e240-e395-4da4-9a24-e53f77daef4e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9rr6z"
Sep 14 02:58:38.696806 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.696697    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggg8l\" (UniqueName: \"kubernetes.io/projected/ed8c4796-f1d6-43f3-b956-4f0a43fcaec9-kube-api-access-ggg8l\") pod \"limitador-operator-controller-manager-74495c69f8-7p4ps\" (UID: \"ed8c4796-f1d6-43f3-b956-4f0a43fcaec9\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7p4ps"
Sep 14 02:58:38.783126 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.783104    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s4crf\" (UniqueName: \"kubernetes.io/projected/adbfd619-db36-4a0d-a741-5b2bef706f2a-kube-api-access-s4crf\") pod \"adbfd619-db36-4a0d-a741-5b2bef706f2a\" (UID: \"adbfd619-db36-4a0d-a741-5b2bef706f2a\") "
Sep 14 02:58:38.783220 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.783144    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fckrc\" (UniqueName: \"kubernetes.io/projected/e7236ab6-c434-4535-9e0e-6e65b90a1237-kube-api-access-fckrc\") pod \"e7236ab6-c434-4535-9e0e-6e65b90a1237\" (UID: \"e7236ab6-c434-4535-9e0e-6e65b90a1237\") "
Sep 14 02:58:38.783220 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.783195    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/adbfd619-db36-4a0d-a741-5b2bef706f2a-extensions-socket-volume\") pod \"adbfd619-db36-4a0d-a741-5b2bef706f2a\" (UID: \"adbfd619-db36-4a0d-a741-5b2bef706f2a\") "
Sep 14 02:58:38.783743 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.783716    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adbfd619-db36-4a0d-a741-5b2bef706f2a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "adbfd619-db36-4a0d-a741-5b2bef706f2a" (UID: "adbfd619-db36-4a0d-a741-5b2bef706f2a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:58:38.785006 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.784984    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adbfd619-db36-4a0d-a741-5b2bef706f2a-kube-api-access-s4crf" (OuterVolumeSpecName: "kube-api-access-s4crf") pod "adbfd619-db36-4a0d-a741-5b2bef706f2a" (UID: "adbfd619-db36-4a0d-a741-5b2bef706f2a"). InnerVolumeSpecName "kube-api-access-s4crf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:58:38.785113 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.785092    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7236ab6-c434-4535-9e0e-6e65b90a1237-kube-api-access-fckrc" (OuterVolumeSpecName: "kube-api-access-fckrc") pod "e7236ab6-c434-4535-9e0e-6e65b90a1237" (UID: "e7236ab6-c434-4535-9e0e-6e65b90a1237"). InnerVolumeSpecName "kube-api-access-fckrc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:58:38.826372 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.826349    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9rr6z"
Sep 14 02:58:38.832344 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.832319    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7p4ps"
Sep 14 02:58:38.883996 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.883944    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s4crf\" (UniqueName: \"kubernetes.io/projected/adbfd619-db36-4a0d-a741-5b2bef706f2a-kube-api-access-s4crf\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:58:38.883996 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.883975    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fckrc\" (UniqueName: \"kubernetes.io/projected/e7236ab6-c434-4535-9e0e-6e65b90a1237-kube-api-access-fckrc\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:58:38.883996 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.883992    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/adbfd619-db36-4a0d-a741-5b2bef706f2a-extensions-socket-volume\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:58:38.901433 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.900742    2577 generic.go:356] "Generic (PLEG): container finished" podID="e7236ab6-c434-4535-9e0e-6e65b90a1237" containerID="850f05d77b0620d57bbfbf89c0399e499b6162fe735c561d63d40f990340155b" exitCode=0
Sep 14 02:58:38.901433 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.900927    2577 scope.go:117] "RemoveContainer" containerID="850f05d77b0620d57bbfbf89c0399e499b6162fe735c561d63d40f990340155b"
Sep 14 02:58:38.901433 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.901099    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-qffx5"
Sep 14 02:58:38.909127 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:58:38.908836    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-qffx5\" is forbidden: User \"system:node:ip-10-0-141-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-180.ec2.internal' and this object" podUID="e7236ab6-c434-4535-9e0e-6e65b90a1237" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-qffx5"
Sep 14 02:58:38.912905 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:58:38.912872    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-gf9cg\" is forbidden: User \"system:node:ip-10-0-141-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-180.ec2.internal' and this object" podUID="adbfd619-db36-4a0d-a741-5b2bef706f2a" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gf9cg"
Sep 14 02:58:38.913632 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.913353    2577 generic.go:356] "Generic (PLEG): container finished" podID="adbfd619-db36-4a0d-a741-5b2bef706f2a" containerID="dab54c181c0d0f26ef01e61a2c8e885eeb7d1927d7a5028df08d548e2907bd5c" exitCode=0
Sep 14 02:58:38.913632 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.913484    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gf9cg"
Sep 14 02:58:38.916383 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:58:38.916353    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-qffx5\" is forbidden: User \"system:node:ip-10-0-141-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-180.ec2.internal' and this object" podUID="e7236ab6-c434-4535-9e0e-6e65b90a1237" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-qffx5"
Sep 14 02:58:38.923579 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:58:38.923427    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-gf9cg\" is forbidden: User \"system:node:ip-10-0-141-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-180.ec2.internal' and this object" podUID="adbfd619-db36-4a0d-a741-5b2bef706f2a" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gf9cg"
Sep 14 02:58:38.927754 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:58:38.927621    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-qffx5\" is forbidden: User \"system:node:ip-10-0-141-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-180.ec2.internal' and this object" podUID="e7236ab6-c434-4535-9e0e-6e65b90a1237" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-qffx5"
Sep 14 02:58:38.929968 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:58:38.929943    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-gf9cg\" is forbidden: User \"system:node:ip-10-0-141-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-180.ec2.internal' and this object" podUID="adbfd619-db36-4a0d-a741-5b2bef706f2a" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gf9cg"
Sep 14 02:58:38.933452 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.933393    2577 scope.go:117] "RemoveContainer" containerID="850f05d77b0620d57bbfbf89c0399e499b6162fe735c561d63d40f990340155b"
Sep 14 02:58:38.934080 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:58:38.933687    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"850f05d77b0620d57bbfbf89c0399e499b6162fe735c561d63d40f990340155b\": container with ID starting with 850f05d77b0620d57bbfbf89c0399e499b6162fe735c561d63d40f990340155b not found: ID does not exist" containerID="850f05d77b0620d57bbfbf89c0399e499b6162fe735c561d63d40f990340155b"
Sep 14 02:58:38.934080 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.933725    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"850f05d77b0620d57bbfbf89c0399e499b6162fe735c561d63d40f990340155b"} err="failed to get container status \"850f05d77b0620d57bbfbf89c0399e499b6162fe735c561d63d40f990340155b\": rpc error: code = NotFound desc = could not find container \"850f05d77b0620d57bbfbf89c0399e499b6162fe735c561d63d40f990340155b\": container with ID starting with 850f05d77b0620d57bbfbf89c0399e499b6162fe735c561d63d40f990340155b not found: ID does not exist"
Sep 14 02:58:38.934080 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.933744    2577 scope.go:117] "RemoveContainer" containerID="dab54c181c0d0f26ef01e61a2c8e885eeb7d1927d7a5028df08d548e2907bd5c"
Sep 14 02:58:38.942401 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:58:38.942323    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-qffx5\" is forbidden: User \"system:node:ip-10-0-141-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-180.ec2.internal' and this object" podUID="e7236ab6-c434-4535-9e0e-6e65b90a1237" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-qffx5"
Sep 14 02:58:38.944139 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.944116    2577 scope.go:117] "RemoveContainer" containerID="dab54c181c0d0f26ef01e61a2c8e885eeb7d1927d7a5028df08d548e2907bd5c"
Sep 14 02:58:38.944295 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:58:38.944237    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-gf9cg\" is forbidden: User \"system:node:ip-10-0-141-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-180.ec2.internal' and this object" podUID="adbfd619-db36-4a0d-a741-5b2bef706f2a" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gf9cg"
Sep 14 02:58:38.944499 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:58:38.944438    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dab54c181c0d0f26ef01e61a2c8e885eeb7d1927d7a5028df08d548e2907bd5c\": container with ID starting with dab54c181c0d0f26ef01e61a2c8e885eeb7d1927d7a5028df08d548e2907bd5c not found: ID does not exist" containerID="dab54c181c0d0f26ef01e61a2c8e885eeb7d1927d7a5028df08d548e2907bd5c"
Sep 14 02:58:38.944499 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:38.944469    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dab54c181c0d0f26ef01e61a2c8e885eeb7d1927d7a5028df08d548e2907bd5c"} err="failed to get container status \"dab54c181c0d0f26ef01e61a2c8e885eeb7d1927d7a5028df08d548e2907bd5c\": rpc error: code = NotFound desc = could not find container \"dab54c181c0d0f26ef01e61a2c8e885eeb7d1927d7a5028df08d548e2907bd5c\": container with ID starting with dab54c181c0d0f26ef01e61a2c8e885eeb7d1927d7a5028df08d548e2907bd5c not found: ID does not exist"
Sep 14 02:58:39.137983 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:39.137957    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9rr6z"]
Sep 14 02:58:39.141407 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:58:39.141382    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf73e240_e395_4da4_9a24_e53f77daef4e.slice/crio-4b8a338981714f78bbdb50156adea5636dca38053f24e74d2e12aba23a441ca3 WatchSource:0}: Error finding container 4b8a338981714f78bbdb50156adea5636dca38053f24e74d2e12aba23a441ca3: Status 404 returned error can't find the container with id 4b8a338981714f78bbdb50156adea5636dca38053f24e74d2e12aba23a441ca3
Sep 14 02:58:39.167187 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:39.167168    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-7p4ps"]
Sep 14 02:58:39.182331 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:58:39.182308    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded8c4796_f1d6_43f3_b956_4f0a43fcaec9.slice/crio-ecf2d665a7d52d95bd228ca490771e6060b31788876ea0a4908a61c04dd9cd07 WatchSource:0}: Error finding container ecf2d665a7d52d95bd228ca490771e6060b31788876ea0a4908a61c04dd9cd07: Status 404 returned error can't find the container with id ecf2d665a7d52d95bd228ca490771e6060b31788876ea0a4908a61c04dd9cd07
Sep 14 02:58:39.619487 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:39.619445    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="adbfd619-db36-4a0d-a741-5b2bef706f2a" path="/var/lib/kubelet/pods/adbfd619-db36-4a0d-a741-5b2bef706f2a/volumes"
Sep 14 02:58:39.619970 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:39.619935    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7236ab6-c434-4535-9e0e-6e65b90a1237" path="/var/lib/kubelet/pods/e7236ab6-c434-4535-9e0e-6e65b90a1237/volumes"
Sep 14 02:58:39.924797 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:39.924703    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7p4ps" event={"ID":"ed8c4796-f1d6-43f3-b956-4f0a43fcaec9","Type":"ContainerStarted","Data":"ba76cd44b9a962a39dc9fa677e059df1a870591a7703c5c7daa26db54fcb3ff8"}
Sep 14 02:58:39.924797 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:39.924769    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7p4ps" event={"ID":"ed8c4796-f1d6-43f3-b956-4f0a43fcaec9","Type":"ContainerStarted","Data":"ecf2d665a7d52d95bd228ca490771e6060b31788876ea0a4908a61c04dd9cd07"}
Sep 14 02:58:39.925037 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:39.924936    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7p4ps"
Sep 14 02:58:39.927185 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:39.927156    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9rr6z" event={"ID":"af73e240-e395-4da4-9a24-e53f77daef4e","Type":"ContainerStarted","Data":"7d6e0bd5694eac4d9d3baae4ea55a2b963104fb65e0f17eada6e5b4bec1464ed"}
Sep 14 02:58:39.927349 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:39.927199    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9rr6z" event={"ID":"af73e240-e395-4da4-9a24-e53f77daef4e","Type":"ContainerStarted","Data":"4b8a338981714f78bbdb50156adea5636dca38053f24e74d2e12aba23a441ca3"}
Sep 14 02:58:39.927349 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:39.927307    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9rr6z"
Sep 14 02:58:39.961492 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:39.961449    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7p4ps" podStartSLOduration=1.961433522 podStartE2EDuration="1.961433522s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:58:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:58:39.960416 +0000 UTC m=+468.938394034" watchObservedRunningTime="2026-09-14 02:58:39.961433522 +0000 UTC m=+468.939411551"
Sep 14 02:58:39.997079 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:39.997040    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9rr6z" podStartSLOduration=1.997028367 podStartE2EDuration="1.997028367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:58:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:58:39.994658772 +0000 UTC m=+468.972636780" watchObservedRunningTime="2026-09-14 02:58:39.997028367 +0000 UTC m=+468.975006396"
Sep 14 02:58:50.935115 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:50.935075    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9rr6z"
Sep 14 02:58:50.935694 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:50.935360    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7p4ps"
Sep 14 02:58:55.958486 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:55.958440    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9rr6z"]
Sep 14 02:58:55.958992 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:55.958712    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9rr6z" podUID="af73e240-e395-4da4-9a24-e53f77daef4e" containerName="manager" containerID="cri-o://7d6e0bd5694eac4d9d3baae4ea55a2b963104fb65e0f17eada6e5b4bec1464ed" gracePeriod=10
Sep 14 02:58:56.203954 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:56.203929    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9rr6z"
Sep 14 02:58:56.314982 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:56.314953    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af73e240-e395-4da4-9a24-e53f77daef4e-extensions-socket-volume\") pod \"af73e240-e395-4da4-9a24-e53f77daef4e\" (UID: \"af73e240-e395-4da4-9a24-e53f77daef4e\") "
Sep 14 02:58:56.315125 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:56.315004    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ch6xf\" (UniqueName: \"kubernetes.io/projected/af73e240-e395-4da4-9a24-e53f77daef4e-kube-api-access-ch6xf\") pod \"af73e240-e395-4da4-9a24-e53f77daef4e\" (UID: \"af73e240-e395-4da4-9a24-e53f77daef4e\") "
Sep 14 02:58:56.315339 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:56.315316    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af73e240-e395-4da4-9a24-e53f77daef4e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "af73e240-e395-4da4-9a24-e53f77daef4e" (UID: "af73e240-e395-4da4-9a24-e53f77daef4e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:58:56.317086 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:56.317065    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af73e240-e395-4da4-9a24-e53f77daef4e-kube-api-access-ch6xf" (OuterVolumeSpecName: "kube-api-access-ch6xf") pod "af73e240-e395-4da4-9a24-e53f77daef4e" (UID: "af73e240-e395-4da4-9a24-e53f77daef4e"). InnerVolumeSpecName "kube-api-access-ch6xf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:58:56.415921 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:56.415892    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af73e240-e395-4da4-9a24-e53f77daef4e-extensions-socket-volume\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:58:56.415921 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:56.415920    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ch6xf\" (UniqueName: \"kubernetes.io/projected/af73e240-e395-4da4-9a24-e53f77daef4e-kube-api-access-ch6xf\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:58:57.025003 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:57.024963    2577 generic.go:356] "Generic (PLEG): container finished" podID="af73e240-e395-4da4-9a24-e53f77daef4e" containerID="7d6e0bd5694eac4d9d3baae4ea55a2b963104fb65e0f17eada6e5b4bec1464ed" exitCode=0
Sep 14 02:58:57.025511 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:57.025019    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9rr6z"
Sep 14 02:58:57.025511 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:57.025059    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9rr6z" event={"ID":"af73e240-e395-4da4-9a24-e53f77daef4e","Type":"ContainerDied","Data":"7d6e0bd5694eac4d9d3baae4ea55a2b963104fb65e0f17eada6e5b4bec1464ed"}
Sep 14 02:58:57.025511 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:57.025123    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9rr6z" event={"ID":"af73e240-e395-4da4-9a24-e53f77daef4e","Type":"ContainerDied","Data":"4b8a338981714f78bbdb50156adea5636dca38053f24e74d2e12aba23a441ca3"}
Sep 14 02:58:57.025511 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:57.025148    2577 scope.go:117] "RemoveContainer" containerID="7d6e0bd5694eac4d9d3baae4ea55a2b963104fb65e0f17eada6e5b4bec1464ed"
Sep 14 02:58:57.036114 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:57.036093    2577 scope.go:117] "RemoveContainer" containerID="7d6e0bd5694eac4d9d3baae4ea55a2b963104fb65e0f17eada6e5b4bec1464ed"
Sep 14 02:58:57.036480 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:58:57.036456    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d6e0bd5694eac4d9d3baae4ea55a2b963104fb65e0f17eada6e5b4bec1464ed\": container with ID starting with 7d6e0bd5694eac4d9d3baae4ea55a2b963104fb65e0f17eada6e5b4bec1464ed not found: ID does not exist" containerID="7d6e0bd5694eac4d9d3baae4ea55a2b963104fb65e0f17eada6e5b4bec1464ed"
Sep 14 02:58:57.036552 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:57.036490    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d6e0bd5694eac4d9d3baae4ea55a2b963104fb65e0f17eada6e5b4bec1464ed"} err="failed to get container status \"7d6e0bd5694eac4d9d3baae4ea55a2b963104fb65e0f17eada6e5b4bec1464ed\": rpc error: code = NotFound desc = could not find container \"7d6e0bd5694eac4d9d3baae4ea55a2b963104fb65e0f17eada6e5b4bec1464ed\": container with ID starting with 7d6e0bd5694eac4d9d3baae4ea55a2b963104fb65e0f17eada6e5b4bec1464ed not found: ID does not exist"
Sep 14 02:58:57.057508 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:57.057481    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9rr6z"]
Sep 14 02:58:57.065461 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:57.065435    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9rr6z"]
Sep 14 02:58:57.617595 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:58:57.617566    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af73e240-e395-4da4-9a24-e53f77daef4e" path="/var/lib/kubelet/pods/af73e240-e395-4da4-9a24-e53f77daef4e/volumes"
Sep 14 02:59:14.585169 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:14.585138    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-kj94l"]
Sep 14 02:59:14.585568 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:14.585529    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af73e240-e395-4da4-9a24-e53f77daef4e" containerName="manager"
Sep 14 02:59:14.585568 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:14.585542    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="af73e240-e395-4da4-9a24-e53f77daef4e" containerName="manager"
Sep 14 02:59:14.585642 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:14.585597    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="af73e240-e395-4da4-9a24-e53f77daef4e" containerName="manager"
Sep 14 02:59:14.588448 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:14.588430    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-kj94l"
Sep 14 02:59:14.591055 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:14.591033    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-wmm8d\""
Sep 14 02:59:14.591350 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:14.591331    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 14 02:59:14.597984 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:14.597962    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-kj94l"]
Sep 14 02:59:14.646126 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:14.646098    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2c94f7f8-c195-4889-b88d-a865fcaddcd1-config-file\") pod \"limitador-limitador-8588749977-kj94l\" (UID: \"2c94f7f8-c195-4889-b88d-a865fcaddcd1\") " pod="kuadrant-system/limitador-limitador-8588749977-kj94l"
Sep 14 02:59:14.646310 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:14.646177    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp5q5\" (UniqueName: \"kubernetes.io/projected/2c94f7f8-c195-4889-b88d-a865fcaddcd1-kube-api-access-gp5q5\") pod \"limitador-limitador-8588749977-kj94l\" (UID: \"2c94f7f8-c195-4889-b88d-a865fcaddcd1\") " pod="kuadrant-system/limitador-limitador-8588749977-kj94l"
Sep 14 02:59:14.677690 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:14.677658    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-kj94l"]
Sep 14 02:59:14.747704 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:14.747678    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2c94f7f8-c195-4889-b88d-a865fcaddcd1-config-file\") pod \"limitador-limitador-8588749977-kj94l\" (UID: \"2c94f7f8-c195-4889-b88d-a865fcaddcd1\") " pod="kuadrant-system/limitador-limitador-8588749977-kj94l"
Sep 14 02:59:14.747808 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:14.747724    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gp5q5\" (UniqueName: \"kubernetes.io/projected/2c94f7f8-c195-4889-b88d-a865fcaddcd1-kube-api-access-gp5q5\") pod \"limitador-limitador-8588749977-kj94l\" (UID: \"2c94f7f8-c195-4889-b88d-a865fcaddcd1\") " pod="kuadrant-system/limitador-limitador-8588749977-kj94l"
Sep 14 02:59:14.748359 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:14.748343    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2c94f7f8-c195-4889-b88d-a865fcaddcd1-config-file\") pod \"limitador-limitador-8588749977-kj94l\" (UID: \"2c94f7f8-c195-4889-b88d-a865fcaddcd1\") " pod="kuadrant-system/limitador-limitador-8588749977-kj94l"
Sep 14 02:59:14.755825 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:14.755807    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp5q5\" (UniqueName: \"kubernetes.io/projected/2c94f7f8-c195-4889-b88d-a865fcaddcd1-kube-api-access-gp5q5\") pod \"limitador-limitador-8588749977-kj94l\" (UID: \"2c94f7f8-c195-4889-b88d-a865fcaddcd1\") " pod="kuadrant-system/limitador-limitador-8588749977-kj94l"
Sep 14 02:59:14.900203 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:14.900147    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-kj94l"
Sep 14 02:59:15.173535 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:15.173454    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-kj94l"]
Sep 14 02:59:15.176127 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:59:15.176099    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c94f7f8_c195_4889_b88d_a865fcaddcd1.slice/crio-bf7cef57bdba0cdfd06a185624bb54daa195073cb7573d6746e49b1620ce086c WatchSource:0}: Error finding container bf7cef57bdba0cdfd06a185624bb54daa195073cb7573d6746e49b1620ce086c: Status 404 returned error can't find the container with id bf7cef57bdba0cdfd06a185624bb54daa195073cb7573d6746e49b1620ce086c
Sep 14 02:59:15.533517 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:15.533486    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-wrtzq"]
Sep 14 02:59:15.536602 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:15.536584    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-wrtzq"
Sep 14 02:59:15.539231 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:15.539211    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-gwwx2\""
Sep 14 02:59:15.542904 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:15.542882    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-wrtzq"]
Sep 14 02:59:15.555444 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:15.555421    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vw5pm\" (UniqueName: \"kubernetes.io/projected/ddf3326e-2bee-441f-bebd-731feb91410f-kube-api-access-vw5pm\") pod \"authorino-59c974bd48-wrtzq\" (UID: \"ddf3326e-2bee-441f-bebd-731feb91410f\") " pod="kuadrant-system/authorino-59c974bd48-wrtzq"
Sep 14 02:59:15.655892 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:15.655865    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vw5pm\" (UniqueName: \"kubernetes.io/projected/ddf3326e-2bee-441f-bebd-731feb91410f-kube-api-access-vw5pm\") pod \"authorino-59c974bd48-wrtzq\" (UID: \"ddf3326e-2bee-441f-bebd-731feb91410f\") " pod="kuadrant-system/authorino-59c974bd48-wrtzq"
Sep 14 02:59:15.665984 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:15.665955    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vw5pm\" (UniqueName: \"kubernetes.io/projected/ddf3326e-2bee-441f-bebd-731feb91410f-kube-api-access-vw5pm\") pod \"authorino-59c974bd48-wrtzq\" (UID: \"ddf3326e-2bee-441f-bebd-731feb91410f\") " pod="kuadrant-system/authorino-59c974bd48-wrtzq"
Sep 14 02:59:15.847165 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:15.847099    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-wrtzq"
Sep 14 02:59:16.118007 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:16.117977    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-wrtzq"]
Sep 14 02:59:16.119456 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:59:16.119419    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podddf3326e_2bee_441f_bebd_731feb91410f.slice/crio-447d3cffbc5d78e7a32a6bf04d3a34f166cb0ee0336271084780a3677244ddb0 WatchSource:0}: Error finding container 447d3cffbc5d78e7a32a6bf04d3a34f166cb0ee0336271084780a3677244ddb0: Status 404 returned error can't find the container with id 447d3cffbc5d78e7a32a6bf04d3a34f166cb0ee0336271084780a3677244ddb0
Sep 14 02:59:16.125926 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:16.125887    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-kj94l" event={"ID":"2c94f7f8-c195-4889-b88d-a865fcaddcd1","Type":"ContainerStarted","Data":"bf7cef57bdba0cdfd06a185624bb54daa195073cb7573d6746e49b1620ce086c"}
Sep 14 02:59:17.134022 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:17.133973    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-wrtzq" event={"ID":"ddf3326e-2bee-441f-bebd-731feb91410f","Type":"ContainerStarted","Data":"447d3cffbc5d78e7a32a6bf04d3a34f166cb0ee0336271084780a3677244ddb0"}
Sep 14 02:59:17.136008 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:17.135973    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-kj94l" event={"ID":"2c94f7f8-c195-4889-b88d-a865fcaddcd1","Type":"ContainerStarted","Data":"e7ead882ecbcc8cb8edead2806236c95f550e4da41f87834b15029448619fb0a"}
Sep 14 02:59:17.136156 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:17.136078    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-kj94l"
Sep 14 02:59:17.156289 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:17.156196    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-kj94l" podStartSLOduration=1.912827963 podStartE2EDuration="3.156178514s" totalImagesPullingTime="1.243350551s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:59:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:59:17.155384881 +0000 UTC m=+506.133362910" watchObservedRunningTime="2026-09-14 02:59:17.156178514 +0000 UTC m=+506.134156544"
Sep 14 02:59:18.144073 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:18.143987    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-wrtzq" event={"ID":"ddf3326e-2bee-441f-bebd-731feb91410f","Type":"ContainerStarted","Data":"1fa5f8c0227cd150389903d94fb2ee8625a2e95a72d677287fb2a97205813fd0"}
Sep 14 02:59:18.159363 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:18.159314    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-wrtzq" podStartSLOduration=1.5100617459999999 podStartE2EDuration="3.159298778s" totalImagesPullingTime="1.649237032s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:59:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:59:18.158465715 +0000 UTC m=+507.136443745" watchObservedRunningTime="2026-09-14 02:59:18.159298778 +0000 UTC m=+507.137276807"
Sep 14 02:59:28.145593 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:28.145542    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-kj94l"
Sep 14 02:59:28.566449 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:28.566405    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-kj94l"]
Sep 14 02:59:28.566667 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:28.566585    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-kj94l" podUID="2c94f7f8-c195-4889-b88d-a865fcaddcd1" containerName="limitador" containerID="cri-o://e7ead882ecbcc8cb8edead2806236c95f550e4da41f87834b15029448619fb0a" gracePeriod=30
Sep 14 02:59:29.109060 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:29.109034    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-kj94l"
Sep 14 02:59:29.164320 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:29.164239    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gp5q5\" (UniqueName: \"kubernetes.io/projected/2c94f7f8-c195-4889-b88d-a865fcaddcd1-kube-api-access-gp5q5\") pod \"2c94f7f8-c195-4889-b88d-a865fcaddcd1\" (UID: \"2c94f7f8-c195-4889-b88d-a865fcaddcd1\") "
Sep 14 02:59:29.164320 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:29.164301    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2c94f7f8-c195-4889-b88d-a865fcaddcd1-config-file\") pod \"2c94f7f8-c195-4889-b88d-a865fcaddcd1\" (UID: \"2c94f7f8-c195-4889-b88d-a865fcaddcd1\") "
Sep 14 02:59:29.164639 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:29.164613    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c94f7f8-c195-4889-b88d-a865fcaddcd1-config-file" (OuterVolumeSpecName: "config-file") pod "2c94f7f8-c195-4889-b88d-a865fcaddcd1" (UID: "2c94f7f8-c195-4889-b88d-a865fcaddcd1"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 02:59:29.166160 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:29.166141    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c94f7f8-c195-4889-b88d-a865fcaddcd1-kube-api-access-gp5q5" (OuterVolumeSpecName: "kube-api-access-gp5q5") pod "2c94f7f8-c195-4889-b88d-a865fcaddcd1" (UID: "2c94f7f8-c195-4889-b88d-a865fcaddcd1"). InnerVolumeSpecName "kube-api-access-gp5q5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:59:29.208958 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:29.208930    2577 generic.go:356] "Generic (PLEG): container finished" podID="2c94f7f8-c195-4889-b88d-a865fcaddcd1" containerID="e7ead882ecbcc8cb8edead2806236c95f550e4da41f87834b15029448619fb0a" exitCode=0
Sep 14 02:59:29.209072 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:29.208976    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-kj94l"
Sep 14 02:59:29.209072 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:29.208987    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-kj94l" event={"ID":"2c94f7f8-c195-4889-b88d-a865fcaddcd1","Type":"ContainerDied","Data":"e7ead882ecbcc8cb8edead2806236c95f550e4da41f87834b15029448619fb0a"}
Sep 14 02:59:29.209072 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:29.209051    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-kj94l" event={"ID":"2c94f7f8-c195-4889-b88d-a865fcaddcd1","Type":"ContainerDied","Data":"bf7cef57bdba0cdfd06a185624bb54daa195073cb7573d6746e49b1620ce086c"}
Sep 14 02:59:29.209227 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:29.209073    2577 scope.go:117] "RemoveContainer" containerID="e7ead882ecbcc8cb8edead2806236c95f550e4da41f87834b15029448619fb0a"
Sep 14 02:59:29.217677 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:29.217660    2577 scope.go:117] "RemoveContainer" containerID="e7ead882ecbcc8cb8edead2806236c95f550e4da41f87834b15029448619fb0a"
Sep 14 02:59:29.217956 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:59:29.217938    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7ead882ecbcc8cb8edead2806236c95f550e4da41f87834b15029448619fb0a\": container with ID starting with e7ead882ecbcc8cb8edead2806236c95f550e4da41f87834b15029448619fb0a not found: ID does not exist" containerID="e7ead882ecbcc8cb8edead2806236c95f550e4da41f87834b15029448619fb0a"
Sep 14 02:59:29.218005 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:29.217964    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7ead882ecbcc8cb8edead2806236c95f550e4da41f87834b15029448619fb0a"} err="failed to get container status \"e7ead882ecbcc8cb8edead2806236c95f550e4da41f87834b15029448619fb0a\": rpc error: code = NotFound desc = could not find container \"e7ead882ecbcc8cb8edead2806236c95f550e4da41f87834b15029448619fb0a\": container with ID starting with e7ead882ecbcc8cb8edead2806236c95f550e4da41f87834b15029448619fb0a not found: ID does not exist"
Sep 14 02:59:29.235720 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:29.235695    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-kj94l"]
Sep 14 02:59:29.240119 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:29.240100    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-kj94l"]
Sep 14 02:59:29.265446 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:29.265424    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gp5q5\" (UniqueName: \"kubernetes.io/projected/2c94f7f8-c195-4889-b88d-a865fcaddcd1-kube-api-access-gp5q5\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:59:29.265446 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:29.265443    2577 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2c94f7f8-c195-4889-b88d-a865fcaddcd1-config-file\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:59:29.617397 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:29.617369    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c94f7f8-c195-4889-b88d-a865fcaddcd1" path="/var/lib/kubelet/pods/2c94f7f8-c195-4889-b88d-a865fcaddcd1/volumes"
Sep 14 02:59:30.997863 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:30.997831    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-m59jj"]
Sep 14 02:59:30.998290 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:30.998172    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c94f7f8-c195-4889-b88d-a865fcaddcd1" containerName="limitador"
Sep 14 02:59:30.998290 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:30.998183    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c94f7f8-c195-4889-b88d-a865fcaddcd1" containerName="limitador"
Sep 14 02:59:30.998290 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:30.998257    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c94f7f8-c195-4889-b88d-a865fcaddcd1" containerName="limitador"
Sep 14 02:59:31.002634 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:31.002615    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-m59jj"
Sep 14 02:59:31.006286 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:31.006230    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 14 02:59:31.006417 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:31.006359    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 14 02:59:31.007295 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:31.007261    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-776zd\""
Sep 14 02:59:31.007387 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:31.007326    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 02:59:31.012639 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:31.012613    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-m59jj"]
Sep 14 02:59:31.079395 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:31.079371    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e1c91692-6335-491f-a9c8-03cfb0a3b79c-data\") pod \"postgres-69fb4594cf-m59jj\" (UID: \"e1c91692-6335-491f-a9c8-03cfb0a3b79c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-m59jj"
Sep 14 02:59:31.079493 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:31.079461    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzn8v\" (UniqueName: \"kubernetes.io/projected/e1c91692-6335-491f-a9c8-03cfb0a3b79c-kube-api-access-zzn8v\") pod \"postgres-69fb4594cf-m59jj\" (UID: \"e1c91692-6335-491f-a9c8-03cfb0a3b79c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-m59jj"
Sep 14 02:59:31.179794 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:31.179770    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzn8v\" (UniqueName: \"kubernetes.io/projected/e1c91692-6335-491f-a9c8-03cfb0a3b79c-kube-api-access-zzn8v\") pod \"postgres-69fb4594cf-m59jj\" (UID: \"e1c91692-6335-491f-a9c8-03cfb0a3b79c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-m59jj"
Sep 14 02:59:31.179886 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:31.179808    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e1c91692-6335-491f-a9c8-03cfb0a3b79c-data\") pod \"postgres-69fb4594cf-m59jj\" (UID: \"e1c91692-6335-491f-a9c8-03cfb0a3b79c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-m59jj"
Sep 14 02:59:31.180113 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:31.180097    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e1c91692-6335-491f-a9c8-03cfb0a3b79c-data\") pod \"postgres-69fb4594cf-m59jj\" (UID: \"e1c91692-6335-491f-a9c8-03cfb0a3b79c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-m59jj"
Sep 14 02:59:31.189363 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:31.189337    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzn8v\" (UniqueName: \"kubernetes.io/projected/e1c91692-6335-491f-a9c8-03cfb0a3b79c-kube-api-access-zzn8v\") pod \"postgres-69fb4594cf-m59jj\" (UID: \"e1c91692-6335-491f-a9c8-03cfb0a3b79c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-m59jj"
Sep 14 02:59:31.313920 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:31.313867    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-m59jj"
Sep 14 02:59:31.589223 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:31.589197    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-m59jj"]
Sep 14 02:59:31.590951 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:59:31.590917    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1c91692_6335_491f_a9c8_03cfb0a3b79c.slice/crio-061b8bd39b1cb949a586f6a2d141f4da6311c55ded5839cd7d080825c6afcce1 WatchSource:0}: Error finding container 061b8bd39b1cb949a586f6a2d141f4da6311c55ded5839cd7d080825c6afcce1: Status 404 returned error can't find the container with id 061b8bd39b1cb949a586f6a2d141f4da6311c55ded5839cd7d080825c6afcce1
Sep 14 02:59:32.229845 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:32.229811    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-m59jj" event={"ID":"e1c91692-6335-491f-a9c8-03cfb0a3b79c","Type":"ContainerStarted","Data":"061b8bd39b1cb949a586f6a2d141f4da6311c55ded5839cd7d080825c6afcce1"}
Sep 14 02:59:37.260348 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:37.260304    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-m59jj" event={"ID":"e1c91692-6335-491f-a9c8-03cfb0a3b79c","Type":"ContainerStarted","Data":"44ee7ce7bdc6f41efe11f37c86c93aed928cbf7d3415d108a1e91c8bfdb5726e"}
Sep 14 02:59:37.260705 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:37.260370    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-m59jj"
Sep 14 02:59:37.280244 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:37.280201    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-m59jj" podStartSLOduration=2.630062808 podStartE2EDuration="7.280186907s" totalImagesPullingTime="4.650124099s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:59:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:59:37.278372516 +0000 UTC m=+526.256350545" watchObservedRunningTime="2026-09-14 02:59:37.280186907 +0000 UTC m=+526.258164970"
Sep 14 02:59:43.296457 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:43.296423    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-m59jj"
Sep 14 02:59:43.942107 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:43.942029    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-ch7fd"]
Sep 14 02:59:43.949851 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:43.949808    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-ch7fd"
Sep 14 02:59:43.953843 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:43.953817    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 14 02:59:43.963330 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:43.963299    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-ch7fd"]
Sep 14 02:59:44.087501 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:44.087471    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-db4cx\" (UniqueName: \"kubernetes.io/projected/f216c171-61b0-436f-805f-9ea5b01e422a-kube-api-access-db4cx\") pod \"authorino-6d4dfc649f-ch7fd\" (UID: \"f216c171-61b0-436f-805f-9ea5b01e422a\") " pod="kuadrant-system/authorino-6d4dfc649f-ch7fd"
Sep 14 02:59:44.087501 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:44.087504    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f216c171-61b0-436f-805f-9ea5b01e422a-tls-cert\") pod \"authorino-6d4dfc649f-ch7fd\" (UID: \"f216c171-61b0-436f-805f-9ea5b01e422a\") " pod="kuadrant-system/authorino-6d4dfc649f-ch7fd"
Sep 14 02:59:44.135905 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:44.135876    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-ch7fd"]
Sep 14 02:59:44.136097 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:59:44.136074    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-db4cx tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-ch7fd" podUID="f216c171-61b0-436f-805f-9ea5b01e422a"
Sep 14 02:59:44.188497 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:44.188471    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-db4cx\" (UniqueName: \"kubernetes.io/projected/f216c171-61b0-436f-805f-9ea5b01e422a-kube-api-access-db4cx\") pod \"authorino-6d4dfc649f-ch7fd\" (UID: \"f216c171-61b0-436f-805f-9ea5b01e422a\") " pod="kuadrant-system/authorino-6d4dfc649f-ch7fd"
Sep 14 02:59:44.188594 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:44.188505    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f216c171-61b0-436f-805f-9ea5b01e422a-tls-cert\") pod \"authorino-6d4dfc649f-ch7fd\" (UID: \"f216c171-61b0-436f-805f-9ea5b01e422a\") " pod="kuadrant-system/authorino-6d4dfc649f-ch7fd"
Sep 14 02:59:44.191063 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:44.191040    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f216c171-61b0-436f-805f-9ea5b01e422a-tls-cert\") pod \"authorino-6d4dfc649f-ch7fd\" (UID: \"f216c171-61b0-436f-805f-9ea5b01e422a\") " pod="kuadrant-system/authorino-6d4dfc649f-ch7fd"
Sep 14 02:59:44.207842 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:44.207783    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-db4cx\" (UniqueName: \"kubernetes.io/projected/f216c171-61b0-436f-805f-9ea5b01e422a-kube-api-access-db4cx\") pod \"authorino-6d4dfc649f-ch7fd\" (UID: \"f216c171-61b0-436f-805f-9ea5b01e422a\") " pod="kuadrant-system/authorino-6d4dfc649f-ch7fd"
Sep 14 02:59:44.299083 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:44.299054    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-ch7fd"
Sep 14 02:59:44.307520 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:44.307499    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-ch7fd"
Sep 14 02:59:44.389990 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:44.389964    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f216c171-61b0-436f-805f-9ea5b01e422a-tls-cert\") pod \"f216c171-61b0-436f-805f-9ea5b01e422a\" (UID: \"f216c171-61b0-436f-805f-9ea5b01e422a\") "
Sep 14 02:59:44.390094 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:44.390002    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-db4cx\" (UniqueName: \"kubernetes.io/projected/f216c171-61b0-436f-805f-9ea5b01e422a-kube-api-access-db4cx\") pod \"f216c171-61b0-436f-805f-9ea5b01e422a\" (UID: \"f216c171-61b0-436f-805f-9ea5b01e422a\") "
Sep 14 02:59:44.392134 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:44.392100    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f216c171-61b0-436f-805f-9ea5b01e422a-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "f216c171-61b0-436f-805f-9ea5b01e422a" (UID: "f216c171-61b0-436f-805f-9ea5b01e422a"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 02:59:44.392251 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:44.392227    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f216c171-61b0-436f-805f-9ea5b01e422a-kube-api-access-db4cx" (OuterVolumeSpecName: "kube-api-access-db4cx") pod "f216c171-61b0-436f-805f-9ea5b01e422a" (UID: "f216c171-61b0-436f-805f-9ea5b01e422a"). InnerVolumeSpecName "kube-api-access-db4cx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:59:44.491157 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:44.491134    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-db4cx\" (UniqueName: \"kubernetes.io/projected/f216c171-61b0-436f-805f-9ea5b01e422a-kube-api-access-db4cx\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:59:44.491255 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:44.491158    2577 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f216c171-61b0-436f-805f-9ea5b01e422a-tls-cert\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:59:45.305041 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:45.305012    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-ch7fd"
Sep 14 02:59:45.339212 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:45.339184    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-ch7fd"]
Sep 14 02:59:45.343542 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:45.343514    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-ch7fd"]
Sep 14 02:59:45.617353 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:45.617295    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f216c171-61b0-436f-805f-9ea5b01e422a" path="/var/lib/kubelet/pods/f216c171-61b0-436f-805f-9ea5b01e422a/volumes"
Sep 14 02:59:46.212902 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:46.212873    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-wrtzq"]
Sep 14 02:59:46.213067 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:46.213047    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-wrtzq" podUID="ddf3326e-2bee-441f-bebd-731feb91410f" containerName="authorino" containerID="cri-o://1fa5f8c0227cd150389903d94fb2ee8625a2e95a72d677287fb2a97205813fd0" gracePeriod=30
Sep 14 02:59:46.504250 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:46.504225    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-wrtzq"
Sep 14 02:59:46.607424 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:46.607392    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vw5pm\" (UniqueName: \"kubernetes.io/projected/ddf3326e-2bee-441f-bebd-731feb91410f-kube-api-access-vw5pm\") pod \"ddf3326e-2bee-441f-bebd-731feb91410f\" (UID: \"ddf3326e-2bee-441f-bebd-731feb91410f\") "
Sep 14 02:59:46.609299 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:46.609280    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddf3326e-2bee-441f-bebd-731feb91410f-kube-api-access-vw5pm" (OuterVolumeSpecName: "kube-api-access-vw5pm") pod "ddf3326e-2bee-441f-bebd-731feb91410f" (UID: "ddf3326e-2bee-441f-bebd-731feb91410f"). InnerVolumeSpecName "kube-api-access-vw5pm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:59:46.708591 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:46.708567    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vw5pm\" (UniqueName: \"kubernetes.io/projected/ddf3326e-2bee-441f-bebd-731feb91410f-kube-api-access-vw5pm\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 02:59:47.316888 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:47.316860    2577 generic.go:356] "Generic (PLEG): container finished" podID="ddf3326e-2bee-441f-bebd-731feb91410f" containerID="1fa5f8c0227cd150389903d94fb2ee8625a2e95a72d677287fb2a97205813fd0" exitCode=0
Sep 14 02:59:47.317015 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:47.316909    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-wrtzq"
Sep 14 02:59:47.317015 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:47.316906    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-wrtzq" event={"ID":"ddf3326e-2bee-441f-bebd-731feb91410f","Type":"ContainerDied","Data":"1fa5f8c0227cd150389903d94fb2ee8625a2e95a72d677287fb2a97205813fd0"}
Sep 14 02:59:47.317092 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:47.317019    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-wrtzq" event={"ID":"ddf3326e-2bee-441f-bebd-731feb91410f","Type":"ContainerDied","Data":"447d3cffbc5d78e7a32a6bf04d3a34f166cb0ee0336271084780a3677244ddb0"}
Sep 14 02:59:47.317092 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:47.317047    2577 scope.go:117] "RemoveContainer" containerID="1fa5f8c0227cd150389903d94fb2ee8625a2e95a72d677287fb2a97205813fd0"
Sep 14 02:59:47.326609 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:47.326588    2577 scope.go:117] "RemoveContainer" containerID="1fa5f8c0227cd150389903d94fb2ee8625a2e95a72d677287fb2a97205813fd0"
Sep 14 02:59:47.326905 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:59:47.326883    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fa5f8c0227cd150389903d94fb2ee8625a2e95a72d677287fb2a97205813fd0\": container with ID starting with 1fa5f8c0227cd150389903d94fb2ee8625a2e95a72d677287fb2a97205813fd0 not found: ID does not exist" containerID="1fa5f8c0227cd150389903d94fb2ee8625a2e95a72d677287fb2a97205813fd0"
Sep 14 02:59:47.326959 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:47.326914    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fa5f8c0227cd150389903d94fb2ee8625a2e95a72d677287fb2a97205813fd0"} err="failed to get container status \"1fa5f8c0227cd150389903d94fb2ee8625a2e95a72d677287fb2a97205813fd0\": rpc error: code = NotFound desc = could not find container \"1fa5f8c0227cd150389903d94fb2ee8625a2e95a72d677287fb2a97205813fd0\": container with ID starting with 1fa5f8c0227cd150389903d94fb2ee8625a2e95a72d677287fb2a97205813fd0 not found: ID does not exist"
Sep 14 02:59:47.340101 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:47.340076    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-wrtzq"]
Sep 14 02:59:47.342141 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:47.342120    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-wrtzq"]
Sep 14 02:59:47.617328 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:47.617259    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddf3326e-2bee-441f-bebd-731feb91410f" path="/var/lib/kubelet/pods/ddf3326e-2bee-441f-bebd-731feb91410f/volumes"
Sep 14 02:59:51.034058 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:51.034023    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-98f5d4d4f-znj7h"]
Sep 14 02:59:51.034485 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:51.034432    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddf3326e-2bee-441f-bebd-731feb91410f" containerName="authorino"
Sep 14 02:59:51.034485 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:51.034447    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddf3326e-2bee-441f-bebd-731feb91410f" containerName="authorino"
Sep 14 02:59:51.034562 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:51.034511    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddf3326e-2bee-441f-bebd-731feb91410f" containerName="authorino"
Sep 14 02:59:51.040205 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:51.040184    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-98f5d4d4f-znj7h"
Sep 14 02:59:51.044037 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:51.044012    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-98f5d4d4f-znj7h"]
Sep 14 02:59:51.044708 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:51.044685    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 14 02:59:51.044827 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:51.044708    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 14 02:59:51.045985 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:51.045713    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-ktqbd\""
Sep 14 02:59:51.045985 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:51.045723    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 14 02:59:51.141523 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:51.141501    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c79e0e11-261c-4934-8cc7-4492042ea3ea-metrics-certs\") pod \"maas-controller-98f5d4d4f-znj7h\" (UID: \"c79e0e11-261c-4934-8cc7-4492042ea3ea\") " pod="opendatahub/maas-controller-98f5d4d4f-znj7h"
Sep 14 02:59:51.141620 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:51.141534    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c79e0e11-261c-4934-8cc7-4492042ea3ea-webhook-cert\") pod \"maas-controller-98f5d4d4f-znj7h\" (UID: \"c79e0e11-261c-4934-8cc7-4492042ea3ea\") " pod="opendatahub/maas-controller-98f5d4d4f-znj7h"
Sep 14 02:59:51.141620 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:51.141570    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bg27m\" (UniqueName: \"kubernetes.io/projected/c79e0e11-261c-4934-8cc7-4492042ea3ea-kube-api-access-bg27m\") pod \"maas-controller-98f5d4d4f-znj7h\" (UID: \"c79e0e11-261c-4934-8cc7-4492042ea3ea\") " pod="opendatahub/maas-controller-98f5d4d4f-znj7h"
Sep 14 02:59:51.199847 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:51.199819    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-64b88c4d64-hpbg2"]
Sep 14 02:59:51.202134 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:51.202114    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64b88c4d64-hpbg2"
Sep 14 02:59:51.215455 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:51.215433    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64b88c4d64-hpbg2"]
Sep 14 02:59:51.242604 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:51.242581    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c79e0e11-261c-4934-8cc7-4492042ea3ea-metrics-certs\") pod \"maas-controller-98f5d4d4f-znj7h\" (UID: \"c79e0e11-261c-4934-8cc7-4492042ea3ea\") " pod="opendatahub/maas-controller-98f5d4d4f-znj7h"
Sep 14 02:59:51.242688 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:51.242612    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c79e0e11-261c-4934-8cc7-4492042ea3ea-webhook-cert\") pod \"maas-controller-98f5d4d4f-znj7h\" (UID: \"c79e0e11-261c-4934-8cc7-4492042ea3ea\") " pod="opendatahub/maas-controller-98f5d4d4f-znj7h"
Sep 14 02:59:51.242688 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:51.242656    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bg27m\" (UniqueName: \"kubernetes.io/projected/c79e0e11-261c-4934-8cc7-4492042ea3ea-kube-api-access-bg27m\") pod \"maas-controller-98f5d4d4f-znj7h\" (UID: \"c79e0e11-261c-4934-8cc7-4492042ea3ea\") " pod="opendatahub/maas-controller-98f5d4d4f-znj7h"
Sep 14 02:59:51.242770 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:59:51.242742    2577 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 14 02:59:51.242824 ip-10-0-141-180 kubenswrapper[2577]: E0914 02:59:51.242811    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c79e0e11-261c-4934-8cc7-4492042ea3ea-webhook-cert podName:c79e0e11-261c-4934-8cc7-4492042ea3ea nodeName:}" failed. No retries permitted until 2026-09-14 02:59:51.742793168 +0000 UTC m=+540.720771230 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/c79e0e11-261c-4934-8cc7-4492042ea3ea-webhook-cert") pod "maas-controller-98f5d4d4f-znj7h" (UID: "c79e0e11-261c-4934-8cc7-4492042ea3ea") : secret "maas-controller-webhook-cert" not found
Sep 14 02:59:51.245023 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:51.244998    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c79e0e11-261c-4934-8cc7-4492042ea3ea-metrics-certs\") pod \"maas-controller-98f5d4d4f-znj7h\" (UID: \"c79e0e11-261c-4934-8cc7-4492042ea3ea\") " pod="opendatahub/maas-controller-98f5d4d4f-znj7h"
Sep 14 02:59:51.252999 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:51.252975    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bg27m\" (UniqueName: \"kubernetes.io/projected/c79e0e11-261c-4934-8cc7-4492042ea3ea-kube-api-access-bg27m\") pod \"maas-controller-98f5d4d4f-znj7h\" (UID: \"c79e0e11-261c-4934-8cc7-4492042ea3ea\") " pod="opendatahub/maas-controller-98f5d4d4f-znj7h"
Sep 14 02:59:51.343879 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:51.343826    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7810aee1-af2a-4025-ad7c-0544aa7c49a9-webhook-cert\") pod \"maas-controller-64b88c4d64-hpbg2\" (UID: \"7810aee1-af2a-4025-ad7c-0544aa7c49a9\") " pod="opendatahub/maas-controller-64b88c4d64-hpbg2"
Sep 14 02:59:51.343879 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:51.343853    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8wqx\" (UniqueName: \"kubernetes.io/projected/7810aee1-af2a-4025-ad7c-0544aa7c49a9-kube-api-access-l8wqx\") pod \"maas-controller-64b88c4d64-hpbg2\" (UID: \"7810aee1-af2a-4025-ad7c-0544aa7c49a9\") " pod="opendatahub/maas-controller-64b88c4d64-hpbg2"
Sep 14 02:59:51.343879 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:51.343872    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7810aee1-af2a-4025-ad7c-0544aa7c49a9-metrics-certs\") pod \"maas-controller-64b88c4d64-hpbg2\" (UID: \"7810aee1-af2a-4025-ad7c-0544aa7c49a9\") " pod="opendatahub/maas-controller-64b88c4d64-hpbg2"
Sep 14 02:59:51.444617 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:51.444598    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7810aee1-af2a-4025-ad7c-0544aa7c49a9-webhook-cert\") pod \"maas-controller-64b88c4d64-hpbg2\" (UID: \"7810aee1-af2a-4025-ad7c-0544aa7c49a9\") " pod="opendatahub/maas-controller-64b88c4d64-hpbg2"
Sep 14 02:59:51.444700 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:51.444625    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8wqx\" (UniqueName: \"kubernetes.io/projected/7810aee1-af2a-4025-ad7c-0544aa7c49a9-kube-api-access-l8wqx\") pod \"maas-controller-64b88c4d64-hpbg2\" (UID: \"7810aee1-af2a-4025-ad7c-0544aa7c49a9\") " pod="opendatahub/maas-controller-64b88c4d64-hpbg2"
Sep 14 02:59:51.444700 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:51.444644    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7810aee1-af2a-4025-ad7c-0544aa7c49a9-metrics-certs\") pod \"maas-controller-64b88c4d64-hpbg2\" (UID: \"7810aee1-af2a-4025-ad7c-0544aa7c49a9\") " pod="opendatahub/maas-controller-64b88c4d64-hpbg2"
Sep 14 02:59:51.446878 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:51.446854    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7810aee1-af2a-4025-ad7c-0544aa7c49a9-metrics-certs\") pod \"maas-controller-64b88c4d64-hpbg2\" (UID: \"7810aee1-af2a-4025-ad7c-0544aa7c49a9\") " pod="opendatahub/maas-controller-64b88c4d64-hpbg2"
Sep 14 02:59:51.446956 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:51.446876    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7810aee1-af2a-4025-ad7c-0544aa7c49a9-webhook-cert\") pod \"maas-controller-64b88c4d64-hpbg2\" (UID: \"7810aee1-af2a-4025-ad7c-0544aa7c49a9\") " pod="opendatahub/maas-controller-64b88c4d64-hpbg2"
Sep 14 02:59:51.454993 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:51.454971    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8wqx\" (UniqueName: \"kubernetes.io/projected/7810aee1-af2a-4025-ad7c-0544aa7c49a9-kube-api-access-l8wqx\") pod \"maas-controller-64b88c4d64-hpbg2\" (UID: \"7810aee1-af2a-4025-ad7c-0544aa7c49a9\") " pod="opendatahub/maas-controller-64b88c4d64-hpbg2"
Sep 14 02:59:51.514726 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:51.514709    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-ktqbd\""
Sep 14 02:59:51.521953 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:51.521938    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64b88c4d64-hpbg2"
Sep 14 02:59:51.747524 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:51.747500    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c79e0e11-261c-4934-8cc7-4492042ea3ea-webhook-cert\") pod \"maas-controller-98f5d4d4f-znj7h\" (UID: \"c79e0e11-261c-4934-8cc7-4492042ea3ea\") " pod="opendatahub/maas-controller-98f5d4d4f-znj7h"
Sep 14 02:59:51.749690 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:51.749667    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c79e0e11-261c-4934-8cc7-4492042ea3ea-webhook-cert\") pod \"maas-controller-98f5d4d4f-znj7h\" (UID: \"c79e0e11-261c-4934-8cc7-4492042ea3ea\") " pod="opendatahub/maas-controller-98f5d4d4f-znj7h"
Sep 14 02:59:51.799232 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:51.798696    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64b88c4d64-hpbg2"]
Sep 14 02:59:51.801500 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:59:51.801472    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7810aee1_af2a_4025_ad7c_0544aa7c49a9.slice/crio-2e4d7611a46eb34ab76a6a4a35f87a41c99fc038e1617abb605e1e1afd28d40d WatchSource:0}: Error finding container 2e4d7611a46eb34ab76a6a4a35f87a41c99fc038e1617abb605e1e1afd28d40d: Status 404 returned error can't find the container with id 2e4d7611a46eb34ab76a6a4a35f87a41c99fc038e1617abb605e1e1afd28d40d
Sep 14 02:59:51.951778 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:51.951752    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-98f5d4d4f-znj7h"
Sep 14 02:59:52.232294 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:52.232248    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-98f5d4d4f-znj7h"]
Sep 14 02:59:52.233055 ip-10-0-141-180 kubenswrapper[2577]: W0914 02:59:52.233022    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc79e0e11_261c_4934_8cc7_4492042ea3ea.slice/crio-e4d46dbf80e080f05dd68375d00711b72203bfd0b11d484fa57a6768636e7041 WatchSource:0}: Error finding container e4d46dbf80e080f05dd68375d00711b72203bfd0b11d484fa57a6768636e7041: Status 404 returned error can't find the container with id e4d46dbf80e080f05dd68375d00711b72203bfd0b11d484fa57a6768636e7041
Sep 14 02:59:52.349507 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:52.349431    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64b88c4d64-hpbg2" event={"ID":"7810aee1-af2a-4025-ad7c-0544aa7c49a9","Type":"ContainerStarted","Data":"2e4d7611a46eb34ab76a6a4a35f87a41c99fc038e1617abb605e1e1afd28d40d"}
Sep 14 02:59:52.351140 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:52.351111    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-98f5d4d4f-znj7h" event={"ID":"c79e0e11-261c-4934-8cc7-4492042ea3ea","Type":"ContainerStarted","Data":"e4d46dbf80e080f05dd68375d00711b72203bfd0b11d484fa57a6768636e7041"}
Sep 14 02:59:54.248125 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:54.248097    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 14 02:59:54.365484 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:54.365442    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64b88c4d64-hpbg2" event={"ID":"7810aee1-af2a-4025-ad7c-0544aa7c49a9","Type":"ContainerStarted","Data":"1303ec4303f9964db6bfc5ef6e66af9f5488e8c9aa9a2183ab36a62dd8b80a74"}
Sep 14 02:59:54.365666 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:54.365559    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-64b88c4d64-hpbg2"
Sep 14 02:59:54.367347 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:54.367323    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-98f5d4d4f-znj7h" event={"ID":"c79e0e11-261c-4934-8cc7-4492042ea3ea","Type":"ContainerStarted","Data":"39a02159338334c6a1eca3c1763727e76b011139631d74f4b39d07319f1869cf"}
Sep 14 02:59:54.367483 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:54.367440    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-98f5d4d4f-znj7h"
Sep 14 02:59:54.401105 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:54.401059    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-64b88c4d64-hpbg2" podStartSLOduration=0.960621912 podStartE2EDuration="3.401045993s" totalImagesPullingTime="2.440424081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:59:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:59:54.400361435 +0000 UTC m=+543.378339465" watchObservedRunningTime="2026-09-14 02:59:54.401045993 +0000 UTC m=+543.379024076"
Sep 14 02:59:54.434469 ip-10-0-141-180 kubenswrapper[2577]: I0914 02:59:54.434425    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-98f5d4d4f-znj7h" podStartSLOduration=2.418768161 podStartE2EDuration="4.434411171s" totalImagesPullingTime="2.01564301s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:59:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:59:54.433585817 +0000 UTC m=+543.411563846" watchObservedRunningTime="2026-09-14 02:59:54.434411171 +0000 UTC m=+543.412389200"
Sep 14 03:00:05.375861 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:05.375833    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-98f5d4d4f-znj7h"
Sep 14 03:00:05.376257 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:05.375896    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-64b88c4d64-hpbg2"
Sep 14 03:00:05.477823 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:05.477790    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-98f5d4d4f-znj7h"]
Sep 14 03:00:05.478027 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:05.478000    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-98f5d4d4f-znj7h" podUID="c79e0e11-261c-4934-8cc7-4492042ea3ea" containerName="manager" containerID="cri-o://39a02159338334c6a1eca3c1763727e76b011139631d74f4b39d07319f1869cf" gracePeriod=30
Sep 14 03:00:05.723799 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:05.723773    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-98f5d4d4f-znj7h"
Sep 14 03:00:05.764974 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:05.764943    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c79e0e11-261c-4934-8cc7-4492042ea3ea-webhook-cert\") pod \"c79e0e11-261c-4934-8cc7-4492042ea3ea\" (UID: \"c79e0e11-261c-4934-8cc7-4492042ea3ea\") "
Sep 14 03:00:05.765099 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:05.765038    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bg27m\" (UniqueName: \"kubernetes.io/projected/c79e0e11-261c-4934-8cc7-4492042ea3ea-kube-api-access-bg27m\") pod \"c79e0e11-261c-4934-8cc7-4492042ea3ea\" (UID: \"c79e0e11-261c-4934-8cc7-4492042ea3ea\") "
Sep 14 03:00:05.765099 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:05.765085    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c79e0e11-261c-4934-8cc7-4492042ea3ea-metrics-certs\") pod \"c79e0e11-261c-4934-8cc7-4492042ea3ea\" (UID: \"c79e0e11-261c-4934-8cc7-4492042ea3ea\") "
Sep 14 03:00:05.767417 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:05.767389    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c79e0e11-261c-4934-8cc7-4492042ea3ea-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "c79e0e11-261c-4934-8cc7-4492042ea3ea" (UID: "c79e0e11-261c-4934-8cc7-4492042ea3ea"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:00:05.767554 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:05.767435    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c79e0e11-261c-4934-8cc7-4492042ea3ea-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c79e0e11-261c-4934-8cc7-4492042ea3ea" (UID: "c79e0e11-261c-4934-8cc7-4492042ea3ea"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:00:05.767629 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:05.767515    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c79e0e11-261c-4934-8cc7-4492042ea3ea-kube-api-access-bg27m" (OuterVolumeSpecName: "kube-api-access-bg27m") pod "c79e0e11-261c-4934-8cc7-4492042ea3ea" (UID: "c79e0e11-261c-4934-8cc7-4492042ea3ea"). InnerVolumeSpecName "kube-api-access-bg27m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:00:05.866302 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:05.866246    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bg27m\" (UniqueName: \"kubernetes.io/projected/c79e0e11-261c-4934-8cc7-4492042ea3ea-kube-api-access-bg27m\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:00:05.866302 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:05.866297    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c79e0e11-261c-4934-8cc7-4492042ea3ea-metrics-certs\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:00:05.866302 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:05.866306    2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c79e0e11-261c-4934-8cc7-4492042ea3ea-webhook-cert\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:00:06.434025 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:06.433989    2577 generic.go:356] "Generic (PLEG): container finished" podID="c79e0e11-261c-4934-8cc7-4492042ea3ea" containerID="39a02159338334c6a1eca3c1763727e76b011139631d74f4b39d07319f1869cf" exitCode=0
Sep 14 03:00:06.434400 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:06.434030    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-98f5d4d4f-znj7h"
Sep 14 03:00:06.434400 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:06.434041    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-98f5d4d4f-znj7h" event={"ID":"c79e0e11-261c-4934-8cc7-4492042ea3ea","Type":"ContainerDied","Data":"39a02159338334c6a1eca3c1763727e76b011139631d74f4b39d07319f1869cf"}
Sep 14 03:00:06.434400 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:06.434090    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-98f5d4d4f-znj7h" event={"ID":"c79e0e11-261c-4934-8cc7-4492042ea3ea","Type":"ContainerDied","Data":"e4d46dbf80e080f05dd68375d00711b72203bfd0b11d484fa57a6768636e7041"}
Sep 14 03:00:06.434400 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:06.434113    2577 scope.go:117] "RemoveContainer" containerID="39a02159338334c6a1eca3c1763727e76b011139631d74f4b39d07319f1869cf"
Sep 14 03:00:06.444101 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:06.444084    2577 scope.go:117] "RemoveContainer" containerID="39a02159338334c6a1eca3c1763727e76b011139631d74f4b39d07319f1869cf"
Sep 14 03:00:06.444388 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:00:06.444371    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39a02159338334c6a1eca3c1763727e76b011139631d74f4b39d07319f1869cf\": container with ID starting with 39a02159338334c6a1eca3c1763727e76b011139631d74f4b39d07319f1869cf not found: ID does not exist" containerID="39a02159338334c6a1eca3c1763727e76b011139631d74f4b39d07319f1869cf"
Sep 14 03:00:06.444439 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:06.444397    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39a02159338334c6a1eca3c1763727e76b011139631d74f4b39d07319f1869cf"} err="failed to get container status \"39a02159338334c6a1eca3c1763727e76b011139631d74f4b39d07319f1869cf\": rpc error: code = NotFound desc = could not find container \"39a02159338334c6a1eca3c1763727e76b011139631d74f4b39d07319f1869cf\": container with ID starting with 39a02159338334c6a1eca3c1763727e76b011139631d74f4b39d07319f1869cf not found: ID does not exist"
Sep 14 03:00:06.459344 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:06.459321    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-98f5d4d4f-znj7h"]
Sep 14 03:00:06.461470 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:06.461453    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-98f5d4d4f-znj7h"]
Sep 14 03:00:07.616441 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:07.616412    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c79e0e11-261c-4934-8cc7-4492042ea3ea" path="/var/lib/kubelet/pods/c79e0e11-261c-4934-8cc7-4492042ea3ea/volumes"
Sep 14 03:00:27.759542 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:27.759507    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-w6xsp"]
Sep 14 03:00:27.760004 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:27.759883    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c79e0e11-261c-4934-8cc7-4492042ea3ea" containerName="manager"
Sep 14 03:00:27.760004 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:27.759895    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c79e0e11-261c-4934-8cc7-4492042ea3ea" containerName="manager"
Sep 14 03:00:27.760004 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:27.759967    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c79e0e11-261c-4934-8cc7-4492042ea3ea" containerName="manager"
Sep 14 03:00:27.765632 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:27.765616    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-w6xsp"
Sep 14 03:00:27.768569 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:27.768551    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 14 03:00:27.769577 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:27.769559    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 14 03:00:27.769657 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:27.769592    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vckj5\""
Sep 14 03:00:27.786674 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:27.786651    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-w6xsp"]
Sep 14 03:00:27.835065 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:27.835042    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31d8f6c7-b07f-4544-bbf5-067a7297b6cc-metrics-certs\") pod \"maas-api-59f6566bbc-w6xsp\" (UID: \"31d8f6c7-b07f-4544-bbf5-067a7297b6cc\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-w6xsp"
Sep 14 03:00:27.835161 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:27.835084    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rn4q\" (UniqueName: \"kubernetes.io/projected/31d8f6c7-b07f-4544-bbf5-067a7297b6cc-kube-api-access-7rn4q\") pod \"maas-api-59f6566bbc-w6xsp\" (UID: \"31d8f6c7-b07f-4544-bbf5-067a7297b6cc\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-w6xsp"
Sep 14 03:00:27.835161 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:27.835105    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31d8f6c7-b07f-4544-bbf5-067a7297b6cc-maas-api-tls\") pod \"maas-api-59f6566bbc-w6xsp\" (UID: \"31d8f6c7-b07f-4544-bbf5-067a7297b6cc\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-w6xsp"
Sep 14 03:00:27.835238 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:27.835161    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31d8f6c7-b07f-4544-bbf5-067a7297b6cc-openshift-service-ca\") pod \"maas-api-59f6566bbc-w6xsp\" (UID: \"31d8f6c7-b07f-4544-bbf5-067a7297b6cc\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-w6xsp"
Sep 14 03:00:27.935865 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:27.935838    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31d8f6c7-b07f-4544-bbf5-067a7297b6cc-metrics-certs\") pod \"maas-api-59f6566bbc-w6xsp\" (UID: \"31d8f6c7-b07f-4544-bbf5-067a7297b6cc\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-w6xsp"
Sep 14 03:00:27.935961 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:27.935880    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rn4q\" (UniqueName: \"kubernetes.io/projected/31d8f6c7-b07f-4544-bbf5-067a7297b6cc-kube-api-access-7rn4q\") pod \"maas-api-59f6566bbc-w6xsp\" (UID: \"31d8f6c7-b07f-4544-bbf5-067a7297b6cc\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-w6xsp"
Sep 14 03:00:27.935961 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:27.935900    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31d8f6c7-b07f-4544-bbf5-067a7297b6cc-maas-api-tls\") pod \"maas-api-59f6566bbc-w6xsp\" (UID: \"31d8f6c7-b07f-4544-bbf5-067a7297b6cc\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-w6xsp"
Sep 14 03:00:27.936044 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:27.935957    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31d8f6c7-b07f-4544-bbf5-067a7297b6cc-openshift-service-ca\") pod \"maas-api-59f6566bbc-w6xsp\" (UID: \"31d8f6c7-b07f-4544-bbf5-067a7297b6cc\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-w6xsp"
Sep 14 03:00:27.936615 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:27.936594    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31d8f6c7-b07f-4544-bbf5-067a7297b6cc-openshift-service-ca\") pod \"maas-api-59f6566bbc-w6xsp\" (UID: \"31d8f6c7-b07f-4544-bbf5-067a7297b6cc\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-w6xsp"
Sep 14 03:00:27.938286 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:27.938238    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31d8f6c7-b07f-4544-bbf5-067a7297b6cc-metrics-certs\") pod \"maas-api-59f6566bbc-w6xsp\" (UID: \"31d8f6c7-b07f-4544-bbf5-067a7297b6cc\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-w6xsp"
Sep 14 03:00:27.938365 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:27.938349    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31d8f6c7-b07f-4544-bbf5-067a7297b6cc-maas-api-tls\") pod \"maas-api-59f6566bbc-w6xsp\" (UID: \"31d8f6c7-b07f-4544-bbf5-067a7297b6cc\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-w6xsp"
Sep 14 03:00:27.971225 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:27.971196    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rn4q\" (UniqueName: \"kubernetes.io/projected/31d8f6c7-b07f-4544-bbf5-067a7297b6cc-kube-api-access-7rn4q\") pod \"maas-api-59f6566bbc-w6xsp\" (UID: \"31d8f6c7-b07f-4544-bbf5-067a7297b6cc\") " pod="odh-ai-gateway-infra/maas-api-59f6566bbc-w6xsp"
Sep 14 03:00:28.075742 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:28.075688    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-w6xsp"
Sep 14 03:00:28.365343 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:28.365253    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-w6xsp"]
Sep 14 03:00:28.368148 ip-10-0-141-180 kubenswrapper[2577]: W0914 03:00:28.368116    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31d8f6c7_b07f_4544_bbf5_067a7297b6cc.slice/crio-9c373cca057b25b96790808cb119f420149fea390b7a1b7e8516fdf1b091d956 WatchSource:0}: Error finding container 9c373cca057b25b96790808cb119f420149fea390b7a1b7e8516fdf1b091d956: Status 404 returned error can't find the container with id 9c373cca057b25b96790808cb119f420149fea390b7a1b7e8516fdf1b091d956
Sep 14 03:00:28.550632 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:28.550596    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-w6xsp" event={"ID":"31d8f6c7-b07f-4544-bbf5-067a7297b6cc","Type":"ContainerStarted","Data":"9c373cca057b25b96790808cb119f420149fea390b7a1b7e8516fdf1b091d956"}
Sep 14 03:00:28.767787 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:28.767756    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-t5rpm"]
Sep 14 03:00:28.771516 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:28.771500    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-t5rpm"
Sep 14 03:00:28.793411 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:28.793392    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 14 03:00:28.793845 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:28.793829    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-zhr5x\""
Sep 14 03:00:28.844875 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:28.844843    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0029d72-4477-4ffe-81e3-8f1fcdaaeae6-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-t5rpm\" (UID: \"f0029d72-4477-4ffe-81e3-8f1fcdaaeae6\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-t5rpm"
Sep 14 03:00:28.845028 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:28.844919    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nh9r4\" (UniqueName: \"kubernetes.io/projected/f0029d72-4477-4ffe-81e3-8f1fcdaaeae6-kube-api-access-nh9r4\") pod \"payload-pre-processing-59fbc47bf8-t5rpm\" (UID: \"f0029d72-4477-4ffe-81e3-8f1fcdaaeae6\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-t5rpm"
Sep 14 03:00:28.845415 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:28.845390    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-t5rpm"]
Sep 14 03:00:28.946191 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:28.946149    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0029d72-4477-4ffe-81e3-8f1fcdaaeae6-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-t5rpm\" (UID: \"f0029d72-4477-4ffe-81e3-8f1fcdaaeae6\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-t5rpm"
Sep 14 03:00:28.946191 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:28.946233    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nh9r4\" (UniqueName: \"kubernetes.io/projected/f0029d72-4477-4ffe-81e3-8f1fcdaaeae6-kube-api-access-nh9r4\") pod \"payload-pre-processing-59fbc47bf8-t5rpm\" (UID: \"f0029d72-4477-4ffe-81e3-8f1fcdaaeae6\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-t5rpm"
Sep 14 03:00:28.947346 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:28.947300    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0029d72-4477-4ffe-81e3-8f1fcdaaeae6-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-t5rpm\" (UID: \"f0029d72-4477-4ffe-81e3-8f1fcdaaeae6\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-t5rpm"
Sep 14 03:00:28.977287 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:28.977239    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-cr47g"]
Sep 14 03:00:28.979413 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:28.979389    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nh9r4\" (UniqueName: \"kubernetes.io/projected/f0029d72-4477-4ffe-81e3-8f1fcdaaeae6-kube-api-access-nh9r4\") pod \"payload-pre-processing-59fbc47bf8-t5rpm\" (UID: \"f0029d72-4477-4ffe-81e3-8f1fcdaaeae6\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-t5rpm"
Sep 14 03:00:28.981169 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:28.981151    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-cr47g"
Sep 14 03:00:29.047050 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:29.046982    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4v6q\" (UniqueName: \"kubernetes.io/projected/ae4337ee-b123-4455-b933-fc591ead6050-kube-api-access-k4v6q\") pod \"payload-processing-5b54b7fbb7-cr47g\" (UID: \"ae4337ee-b123-4455-b933-fc591ead6050\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-cr47g"
Sep 14 03:00:29.047170 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:29.047103    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae4337ee-b123-4455-b933-fc591ead6050-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-cr47g\" (UID: \"ae4337ee-b123-4455-b933-fc591ead6050\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-cr47g"
Sep 14 03:00:29.059721 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:29.059696    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-cr47g"]
Sep 14 03:00:29.084127 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:29.084090    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-t5rpm"
Sep 14 03:00:29.148658 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:29.148578    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae4337ee-b123-4455-b933-fc591ead6050-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-cr47g\" (UID: \"ae4337ee-b123-4455-b933-fc591ead6050\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-cr47g"
Sep 14 03:00:29.148753 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:29.148675    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4v6q\" (UniqueName: \"kubernetes.io/projected/ae4337ee-b123-4455-b933-fc591ead6050-kube-api-access-k4v6q\") pod \"payload-processing-5b54b7fbb7-cr47g\" (UID: \"ae4337ee-b123-4455-b933-fc591ead6050\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-cr47g"
Sep 14 03:00:29.149543 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:29.149515    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae4337ee-b123-4455-b933-fc591ead6050-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-cr47g\" (UID: \"ae4337ee-b123-4455-b933-fc591ead6050\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-cr47g"
Sep 14 03:00:29.176801 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:29.176774    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4v6q\" (UniqueName: \"kubernetes.io/projected/ae4337ee-b123-4455-b933-fc591ead6050-kube-api-access-k4v6q\") pod \"payload-processing-5b54b7fbb7-cr47g\" (UID: \"ae4337ee-b123-4455-b933-fc591ead6050\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-cr47g"
Sep 14 03:00:29.299603 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:29.299526    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-cr47g"
Sep 14 03:00:29.380629 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:29.380597    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-t5rpm"]
Sep 14 03:00:29.663224 ip-10-0-141-180 kubenswrapper[2577]: W0914 03:00:29.663145    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0029d72_4477_4ffe_81e3_8f1fcdaaeae6.slice/crio-d8cd58b51f7193441f1fa4fb2b957439d937a12a91bab4d492bef45aacc63cd0 WatchSource:0}: Error finding container d8cd58b51f7193441f1fa4fb2b957439d937a12a91bab4d492bef45aacc63cd0: Status 404 returned error can't find the container with id d8cd58b51f7193441f1fa4fb2b957439d937a12a91bab4d492bef45aacc63cd0
Sep 14 03:00:29.971644 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:29.971618    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-cr47g"]
Sep 14 03:00:29.973646 ip-10-0-141-180 kubenswrapper[2577]: W0914 03:00:29.973618    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae4337ee_b123_4455_b933_fc591ead6050.slice/crio-c003370d6768bdca39dbacd33fe726f74ec39289bcb1c50215105c9ff33eae19 WatchSource:0}: Error finding container c003370d6768bdca39dbacd33fe726f74ec39289bcb1c50215105c9ff33eae19: Status 404 returned error can't find the container with id c003370d6768bdca39dbacd33fe726f74ec39289bcb1c50215105c9ff33eae19
Sep 14 03:00:30.565494 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:30.565418    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-cr47g" event={"ID":"ae4337ee-b123-4455-b933-fc591ead6050","Type":"ContainerStarted","Data":"c003370d6768bdca39dbacd33fe726f74ec39289bcb1c50215105c9ff33eae19"}
Sep 14 03:00:30.570724 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:30.570641    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-t5rpm" event={"ID":"f0029d72-4477-4ffe-81e3-8f1fcdaaeae6","Type":"ContainerStarted","Data":"d8cd58b51f7193441f1fa4fb2b957439d937a12a91bab4d492bef45aacc63cd0"}
Sep 14 03:00:30.572964 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:30.572624    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-w6xsp" event={"ID":"31d8f6c7-b07f-4544-bbf5-067a7297b6cc","Type":"ContainerStarted","Data":"7f2a280b5481c9572fb65eb1d726096b4679e57972d2d2f0b9a53c03fa3aa9e9"}
Sep 14 03:00:30.572964 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:30.572777    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-w6xsp"
Sep 14 03:00:30.618624 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:30.618576    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-w6xsp" podStartSLOduration=2.274109577 podStartE2EDuration="3.6185613s" totalImagesPullingTime="1.344451723s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:00:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:00:30.618100331 +0000 UTC m=+579.596078363" watchObservedRunningTime="2026-09-14 03:00:30.6185613 +0000 UTC m=+579.596539329"
Sep 14 03:00:31.580640 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:31.580608    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-cr47g" event={"ID":"ae4337ee-b123-4455-b933-fc591ead6050","Type":"ContainerStarted","Data":"a1478be9cf99048114430b925efc3f6f2db31ae4037d11934fea1adbb597ef72"}
Sep 14 03:00:31.581089 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:31.580694    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-cr47g"
Sep 14 03:00:31.582309 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:31.582262    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-t5rpm" event={"ID":"f0029d72-4477-4ffe-81e3-8f1fcdaaeae6","Type":"ContainerStarted","Data":"bab0ec9bc9b268c116fe461da70d307c17bcfc13028cdd28c45f318ef227f683"}
Sep 14 03:00:31.582418 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:31.582405    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-t5rpm"
Sep 14 03:00:31.607174 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:31.607131    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-cr47g" podStartSLOduration=2.544848103 podStartE2EDuration="3.607116338s" totalImagesPullingTime="1.062268235s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:00:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:00:31.604892913 +0000 UTC m=+580.582870943" watchObservedRunningTime="2026-09-14 03:00:31.607116338 +0000 UTC m=+580.585094580"
Sep 14 03:00:31.637499 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:31.637447    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-t5rpm" podStartSLOduration=2.268391884 podStartE2EDuration="3.637430529s" totalImagesPullingTime="1.369038645s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:00:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:00:31.636100804 +0000 UTC m=+580.614078833" watchObservedRunningTime="2026-09-14 03:00:31.637430529 +0000 UTC m=+580.615408559"
Sep 14 03:00:36.588205 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:36.588169    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-w6xsp"
Sep 14 03:00:42.590459 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:42.590422    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-cr47g"
Sep 14 03:00:42.590966 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:00:42.590717    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-t5rpm"
Sep 14 03:01:07.995124 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:07.995082    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl"]
Sep 14 03:01:07.998602 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:07.998585    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl"
Sep 14 03:01:08.002586 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:08.002564    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 14 03:01:08.002586 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:08.002579    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 14 03:01:08.002792 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:08.002613    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 14 03:01:08.002792 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:08.002627    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-hcz4h\""
Sep 14 03:01:08.010123 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:08.010101    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl"]
Sep 14 03:01:08.151796 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:08.151754    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a5ddd59-1e11-4a17-b676-81603157f28b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl\" (UID: \"8a5ddd59-1e11-4a17-b676-81603157f28b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl"
Sep 14 03:01:08.151796 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:08.151802    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a5ddd59-1e11-4a17-b676-81603157f28b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl\" (UID: \"8a5ddd59-1e11-4a17-b676-81603157f28b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl"
Sep 14 03:01:08.151996 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:08.151828    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5np5l\" (UniqueName: \"kubernetes.io/projected/8a5ddd59-1e11-4a17-b676-81603157f28b-kube-api-access-5np5l\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl\" (UID: \"8a5ddd59-1e11-4a17-b676-81603157f28b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl"
Sep 14 03:01:08.151996 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:08.151899    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a5ddd59-1e11-4a17-b676-81603157f28b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl\" (UID: \"8a5ddd59-1e11-4a17-b676-81603157f28b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl"
Sep 14 03:01:08.151996 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:08.151937    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a5ddd59-1e11-4a17-b676-81603157f28b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl\" (UID: \"8a5ddd59-1e11-4a17-b676-81603157f28b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl"
Sep 14 03:01:08.151996 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:08.151954    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a5ddd59-1e11-4a17-b676-81603157f28b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl\" (UID: \"8a5ddd59-1e11-4a17-b676-81603157f28b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl"
Sep 14 03:01:08.253037 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:08.252962    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a5ddd59-1e11-4a17-b676-81603157f28b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl\" (UID: \"8a5ddd59-1e11-4a17-b676-81603157f28b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl"
Sep 14 03:01:08.253198 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:08.252998    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a5ddd59-1e11-4a17-b676-81603157f28b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl\" (UID: \"8a5ddd59-1e11-4a17-b676-81603157f28b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl"
Sep 14 03:01:08.253561 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:08.253532    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a5ddd59-1e11-4a17-b676-81603157f28b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl\" (UID: \"8a5ddd59-1e11-4a17-b676-81603157f28b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl"
Sep 14 03:01:08.253672 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:08.253641    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a5ddd59-1e11-4a17-b676-81603157f28b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl\" (UID: \"8a5ddd59-1e11-4a17-b676-81603157f28b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl"
Sep 14 03:01:08.253738 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:08.253645    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a5ddd59-1e11-4a17-b676-81603157f28b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl\" (UID: \"8a5ddd59-1e11-4a17-b676-81603157f28b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl"
Sep 14 03:01:08.257300 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:08.253903    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a5ddd59-1e11-4a17-b676-81603157f28b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl\" (UID: \"8a5ddd59-1e11-4a17-b676-81603157f28b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl"
Sep 14 03:01:08.257300 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:08.253965    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5np5l\" (UniqueName: \"kubernetes.io/projected/8a5ddd59-1e11-4a17-b676-81603157f28b-kube-api-access-5np5l\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl\" (UID: \"8a5ddd59-1e11-4a17-b676-81603157f28b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl"
Sep 14 03:01:08.257300 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:08.254046    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a5ddd59-1e11-4a17-b676-81603157f28b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl\" (UID: \"8a5ddd59-1e11-4a17-b676-81603157f28b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl"
Sep 14 03:01:08.257300 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:08.254397    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a5ddd59-1e11-4a17-b676-81603157f28b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl\" (UID: \"8a5ddd59-1e11-4a17-b676-81603157f28b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl"
Sep 14 03:01:08.257300 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:08.256107    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a5ddd59-1e11-4a17-b676-81603157f28b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl\" (UID: \"8a5ddd59-1e11-4a17-b676-81603157f28b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl"
Sep 14 03:01:08.257300 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:08.256648    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a5ddd59-1e11-4a17-b676-81603157f28b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl\" (UID: \"8a5ddd59-1e11-4a17-b676-81603157f28b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl"
Sep 14 03:01:08.263617 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:08.263594    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5np5l\" (UniqueName: \"kubernetes.io/projected/8a5ddd59-1e11-4a17-b676-81603157f28b-kube-api-access-5np5l\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl\" (UID: \"8a5ddd59-1e11-4a17-b676-81603157f28b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl"
Sep 14 03:01:08.309529 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:08.309506    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl"
Sep 14 03:01:08.595119 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:08.595091    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl"]
Sep 14 03:01:08.595250 ip-10-0-141-180 kubenswrapper[2577]: W0914 03:01:08.595089    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a5ddd59_1e11_4a17_b676_81603157f28b.slice/crio-1556528f6ea11659902880d1cffecb103692f26ae3e82d04a8934fe51ed8d901 WatchSource:0}: Error finding container 1556528f6ea11659902880d1cffecb103692f26ae3e82d04a8934fe51ed8d901: Status 404 returned error can't find the container with id 1556528f6ea11659902880d1cffecb103692f26ae3e82d04a8934fe51ed8d901
Sep 14 03:01:08.596604 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:08.596584    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 03:01:08.790426 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:08.790395    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl" event={"ID":"8a5ddd59-1e11-4a17-b676-81603157f28b","Type":"ContainerStarted","Data":"1556528f6ea11659902880d1cffecb103692f26ae3e82d04a8934fe51ed8d901"}
Sep 14 03:01:09.797866 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:09.797829    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl" event={"ID":"8a5ddd59-1e11-4a17-b676-81603157f28b","Type":"ContainerStarted","Data":"a625d5b57f5c275910cd81d0fa6fc7a42802faa70bbb2fe1c9b28aa030c842d7"}
Sep 14 03:01:09.838770 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:09.838725    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl" podStartSLOduration=1.76475947 podStartE2EDuration="2.838709768s" totalImagesPullingTime="1.073950298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:01:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:01:09.838591485 +0000 UTC m=+618.816569513" watchObservedRunningTime="2026-09-14 03:01:09.838709768 +0000 UTC m=+618.816687797"
Sep 14 03:01:18.309801 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:18.309716    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl"
Sep 14 03:01:18.309801 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:18.309786    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl"
Sep 14 03:01:18.323338 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:18.323314    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl"
Sep 14 03:01:18.857064 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:18.857037    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wvwrl"
Sep 14 03:01:35.016196 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:35.016161    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z"]
Sep 14 03:01:35.019997 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:35.019971    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z"
Sep 14 03:01:35.023386 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:35.023360    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 14 03:01:35.031229 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:35.031208    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z"]
Sep 14 03:01:35.066310 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:35.066258    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vn59\" (UniqueName: \"kubernetes.io/projected/59d1e016-94c4-4a02-90d0-0d8fba55c426-kube-api-access-5vn59\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z\" (UID: \"59d1e016-94c4-4a02-90d0-0d8fba55c426\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z"
Sep 14 03:01:35.066423 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:35.066319    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59d1e016-94c4-4a02-90d0-0d8fba55c426-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z\" (UID: \"59d1e016-94c4-4a02-90d0-0d8fba55c426\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z"
Sep 14 03:01:35.066423 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:35.066355    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59d1e016-94c4-4a02-90d0-0d8fba55c426-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z\" (UID: \"59d1e016-94c4-4a02-90d0-0d8fba55c426\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z"
Sep 14 03:01:35.066423 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:35.066371    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59d1e016-94c4-4a02-90d0-0d8fba55c426-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z\" (UID: \"59d1e016-94c4-4a02-90d0-0d8fba55c426\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z"
Sep 14 03:01:35.066423 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:35.066388    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59d1e016-94c4-4a02-90d0-0d8fba55c426-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z\" (UID: \"59d1e016-94c4-4a02-90d0-0d8fba55c426\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z"
Sep 14 03:01:35.066568 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:35.066493    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59d1e016-94c4-4a02-90d0-0d8fba55c426-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z\" (UID: \"59d1e016-94c4-4a02-90d0-0d8fba55c426\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z"
Sep 14 03:01:35.167336 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:35.167302    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vn59\" (UniqueName: \"kubernetes.io/projected/59d1e016-94c4-4a02-90d0-0d8fba55c426-kube-api-access-5vn59\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z\" (UID: \"59d1e016-94c4-4a02-90d0-0d8fba55c426\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z"
Sep 14 03:01:35.167467 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:35.167344    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59d1e016-94c4-4a02-90d0-0d8fba55c426-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z\" (UID: \"59d1e016-94c4-4a02-90d0-0d8fba55c426\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z"
Sep 14 03:01:35.167541 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:35.167519    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59d1e016-94c4-4a02-90d0-0d8fba55c426-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z\" (UID: \"59d1e016-94c4-4a02-90d0-0d8fba55c426\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z"
Sep 14 03:01:35.167608 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:35.167555    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59d1e016-94c4-4a02-90d0-0d8fba55c426-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z\" (UID: \"59d1e016-94c4-4a02-90d0-0d8fba55c426\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z"
Sep 14 03:01:35.167608 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:35.167572    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59d1e016-94c4-4a02-90d0-0d8fba55c426-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z\" (UID: \"59d1e016-94c4-4a02-90d0-0d8fba55c426\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z"
Sep 14 03:01:35.167710 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:35.167634    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59d1e016-94c4-4a02-90d0-0d8fba55c426-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z\" (UID: \"59d1e016-94c4-4a02-90d0-0d8fba55c426\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z"
Sep 14 03:01:35.167920 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:35.167901    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59d1e016-94c4-4a02-90d0-0d8fba55c426-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z\" (UID: \"59d1e016-94c4-4a02-90d0-0d8fba55c426\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z"
Sep 14 03:01:35.167989 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:35.167943    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59d1e016-94c4-4a02-90d0-0d8fba55c426-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z\" (UID: \"59d1e016-94c4-4a02-90d0-0d8fba55c426\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z"
Sep 14 03:01:35.167989 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:35.167956    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59d1e016-94c4-4a02-90d0-0d8fba55c426-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z\" (UID: \"59d1e016-94c4-4a02-90d0-0d8fba55c426\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z"
Sep 14 03:01:35.169794 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:35.169773    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59d1e016-94c4-4a02-90d0-0d8fba55c426-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z\" (UID: \"59d1e016-94c4-4a02-90d0-0d8fba55c426\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z"
Sep 14 03:01:35.169996 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:35.169975    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59d1e016-94c4-4a02-90d0-0d8fba55c426-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z\" (UID: \"59d1e016-94c4-4a02-90d0-0d8fba55c426\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z"
Sep 14 03:01:35.175542 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:35.175519    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vn59\" (UniqueName: \"kubernetes.io/projected/59d1e016-94c4-4a02-90d0-0d8fba55c426-kube-api-access-5vn59\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z\" (UID: \"59d1e016-94c4-4a02-90d0-0d8fba55c426\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z"
Sep 14 03:01:35.331432 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:35.331377    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z"
Sep 14 03:01:35.607896 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:35.607872    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z"]
Sep 14 03:01:35.608649 ip-10-0-141-180 kubenswrapper[2577]: W0914 03:01:35.608618    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59d1e016_94c4_4a02_90d0_0d8fba55c426.slice/crio-054a10761ff2343559f469a285d06ac7aa435b857015320e4ce9fd887cbc7ad9 WatchSource:0}: Error finding container 054a10761ff2343559f469a285d06ac7aa435b857015320e4ce9fd887cbc7ad9: Status 404 returned error can't find the container with id 054a10761ff2343559f469a285d06ac7aa435b857015320e4ce9fd887cbc7ad9
Sep 14 03:01:35.948432 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:35.948361    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z" event={"ID":"59d1e016-94c4-4a02-90d0-0d8fba55c426","Type":"ContainerStarted","Data":"bc6310c1f95485a3a4f5026780a17185523893d48e93345b3d00d1b9a34c5cb2"}
Sep 14 03:01:35.948432 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:35.948418    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z" event={"ID":"59d1e016-94c4-4a02-90d0-0d8fba55c426","Type":"ContainerStarted","Data":"054a10761ff2343559f469a285d06ac7aa435b857015320e4ce9fd887cbc7ad9"}
Sep 14 03:01:35.967048 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:35.967000    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z" podStartSLOduration=1.795623068 podStartE2EDuration="1.96698451s" totalImagesPullingTime="171.361442ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:01:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:01:35.966173103 +0000 UTC m=+644.944151134" watchObservedRunningTime="2026-09-14 03:01:35.96698451 +0000 UTC m=+644.944962597"
Sep 14 03:01:44.890512 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:44.890473    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-kbjlg"]
Sep 14 03:01:44.897019 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:44.896997    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kbjlg"
Sep 14 03:01:44.899860 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:44.899828    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 14 03:01:44.904073 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:44.904046    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-kbjlg"]
Sep 14 03:01:44.943751 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:44.943727    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41980a80-a472-4f18-82ad-c28f59683f77-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kbjlg\" (UID: \"41980a80-a472-4f18-82ad-c28f59683f77\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kbjlg"
Sep 14 03:01:44.943851 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:44.943758    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwjwf\" (UniqueName: \"kubernetes.io/projected/41980a80-a472-4f18-82ad-c28f59683f77-kube-api-access-xwjwf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kbjlg\" (UID: \"41980a80-a472-4f18-82ad-c28f59683f77\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kbjlg"
Sep 14 03:01:44.943851 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:44.943782    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41980a80-a472-4f18-82ad-c28f59683f77-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kbjlg\" (UID: \"41980a80-a472-4f18-82ad-c28f59683f77\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kbjlg"
Sep 14 03:01:44.943851 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:44.943810    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41980a80-a472-4f18-82ad-c28f59683f77-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kbjlg\" (UID: \"41980a80-a472-4f18-82ad-c28f59683f77\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kbjlg"
Sep 14 03:01:44.943961 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:44.943858    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41980a80-a472-4f18-82ad-c28f59683f77-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kbjlg\" (UID: \"41980a80-a472-4f18-82ad-c28f59683f77\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kbjlg"
Sep 14 03:01:44.943961 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:44.943901    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41980a80-a472-4f18-82ad-c28f59683f77-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kbjlg\" (UID: \"41980a80-a472-4f18-82ad-c28f59683f77\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kbjlg"
Sep 14 03:01:45.044966 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:45.044937    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41980a80-a472-4f18-82ad-c28f59683f77-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kbjlg\" (UID: \"41980a80-a472-4f18-82ad-c28f59683f77\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kbjlg"
Sep 14 03:01:45.045102 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:45.044985    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41980a80-a472-4f18-82ad-c28f59683f77-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kbjlg\" (UID: \"41980a80-a472-4f18-82ad-c28f59683f77\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kbjlg"
Sep 14 03:01:45.045102 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:45.045005    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwjwf\" (UniqueName: \"kubernetes.io/projected/41980a80-a472-4f18-82ad-c28f59683f77-kube-api-access-xwjwf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kbjlg\" (UID: \"41980a80-a472-4f18-82ad-c28f59683f77\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kbjlg"
Sep 14 03:01:45.045102 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:45.045036    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41980a80-a472-4f18-82ad-c28f59683f77-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kbjlg\" (UID: \"41980a80-a472-4f18-82ad-c28f59683f77\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kbjlg"
Sep 14 03:01:45.045102 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:45.045055    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41980a80-a472-4f18-82ad-c28f59683f77-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kbjlg\" (UID: \"41980a80-a472-4f18-82ad-c28f59683f77\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kbjlg"
Sep 14 03:01:45.045102 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:45.045082    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41980a80-a472-4f18-82ad-c28f59683f77-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kbjlg\" (UID: \"41980a80-a472-4f18-82ad-c28f59683f77\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kbjlg"
Sep 14 03:01:45.045410 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:45.045381    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41980a80-a472-4f18-82ad-c28f59683f77-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kbjlg\" (UID: \"41980a80-a472-4f18-82ad-c28f59683f77\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kbjlg"
Sep 14 03:01:45.045518 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:45.045497    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41980a80-a472-4f18-82ad-c28f59683f77-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kbjlg\" (UID: \"41980a80-a472-4f18-82ad-c28f59683f77\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kbjlg"
Sep 14 03:01:45.045573 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:45.045521    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41980a80-a472-4f18-82ad-c28f59683f77-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kbjlg\" (UID: \"41980a80-a472-4f18-82ad-c28f59683f77\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kbjlg"
Sep 14 03:01:45.047163 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:45.047136    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41980a80-a472-4f18-82ad-c28f59683f77-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kbjlg\" (UID: \"41980a80-a472-4f18-82ad-c28f59683f77\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kbjlg"
Sep 14 03:01:45.047491 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:45.047473    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41980a80-a472-4f18-82ad-c28f59683f77-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kbjlg\" (UID: \"41980a80-a472-4f18-82ad-c28f59683f77\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kbjlg"
Sep 14 03:01:45.054312 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:45.054293    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwjwf\" (UniqueName: \"kubernetes.io/projected/41980a80-a472-4f18-82ad-c28f59683f77-kube-api-access-xwjwf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kbjlg\" (UID: \"41980a80-a472-4f18-82ad-c28f59683f77\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kbjlg"
Sep 14 03:01:45.209336 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:45.209260    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kbjlg"
Sep 14 03:01:45.332447 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:45.332408    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z"
Sep 14 03:01:45.332592 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:45.332476    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z"
Sep 14 03:01:45.344534 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:45.344514    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z"
Sep 14 03:01:45.484036 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:45.484014    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-kbjlg"]
Sep 14 03:01:45.485618 ip-10-0-141-180 kubenswrapper[2577]: W0914 03:01:45.485591    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41980a80_a472_4f18_82ad_c28f59683f77.slice/crio-beacc4b74cd9dc553ed2f7f96e4c66ac88cba38cc44079df15fb80ac40322076 WatchSource:0}: Error finding container beacc4b74cd9dc553ed2f7f96e4c66ac88cba38cc44079df15fb80ac40322076: Status 404 returned error can't find the container with id beacc4b74cd9dc553ed2f7f96e4c66ac88cba38cc44079df15fb80ac40322076
Sep 14 03:01:46.006358 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:46.006325    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kbjlg" event={"ID":"41980a80-a472-4f18-82ad-c28f59683f77","Type":"ContainerStarted","Data":"d527c17df0f42c6fc209b2c662140fdc116b3b6b05f2a0b5ad06b7acef44fc7f"}
Sep 14 03:01:46.006837 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:46.006381    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kbjlg" event={"ID":"41980a80-a472-4f18-82ad-c28f59683f77","Type":"ContainerStarted","Data":"beacc4b74cd9dc553ed2f7f96e4c66ac88cba38cc44079df15fb80ac40322076"}
Sep 14 03:01:46.017854 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:46.017824    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-h5s7z"
Sep 14 03:01:46.032260 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:46.032215    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kbjlg" podStartSLOduration=1.8626959109999999 podStartE2EDuration="2.032200671s" totalImagesPullingTime="169.50476ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:01:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:01:46.028937235 +0000 UTC m=+655.006915265" watchObservedRunningTime="2026-09-14 03:01:46.032200671 +0000 UTC m=+655.010178699"
Sep 14 03:01:55.210347 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:55.210314    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kbjlg"
Sep 14 03:01:55.210773 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:55.210394    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kbjlg"
Sep 14 03:01:55.222611 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:55.222587    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kbjlg"
Sep 14 03:01:56.075276 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:01:56.075235    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kbjlg"
Sep 14 03:03:37.768672 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:37.768635    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-cr47g"]
Sep 14 03:03:37.769129 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:37.768830    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-5b54b7fbb7-cr47g" podUID="ae4337ee-b123-4455-b933-fc591ead6050" containerName="payload-processing" containerID="cri-o://a1478be9cf99048114430b925efc3f6f2db31ae4037d11934fea1adbb597ef72" gracePeriod=30
Sep 14 03:03:37.827052 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:37.827025    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-64b88c4d64-hpbg2"]
Sep 14 03:03:37.827255 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:37.827214    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-64b88c4d64-hpbg2" podUID="7810aee1-af2a-4025-ad7c-0544aa7c49a9" containerName="manager" containerID="cri-o://1303ec4303f9964db6bfc5ef6e66af9f5488e8c9aa9a2183ab36a62dd8b80a74" gracePeriod=30
Sep 14 03:03:38.090614 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.090579    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-cr47g"
Sep 14 03:03:38.092640 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.092619    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64b88c4d64-hpbg2"
Sep 14 03:03:38.121132 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.121106    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae4337ee-b123-4455-b933-fc591ead6050-plugins-config-volume\") pod \"ae4337ee-b123-4455-b933-fc591ead6050\" (UID: \"ae4337ee-b123-4455-b933-fc591ead6050\") "
Sep 14 03:03:38.121255 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.121142    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8wqx\" (UniqueName: \"kubernetes.io/projected/7810aee1-af2a-4025-ad7c-0544aa7c49a9-kube-api-access-l8wqx\") pod \"7810aee1-af2a-4025-ad7c-0544aa7c49a9\" (UID: \"7810aee1-af2a-4025-ad7c-0544aa7c49a9\") "
Sep 14 03:03:38.121255 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.121176    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7810aee1-af2a-4025-ad7c-0544aa7c49a9-webhook-cert\") pod \"7810aee1-af2a-4025-ad7c-0544aa7c49a9\" (UID: \"7810aee1-af2a-4025-ad7c-0544aa7c49a9\") "
Sep 14 03:03:38.121409 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.121348    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4v6q\" (UniqueName: \"kubernetes.io/projected/ae4337ee-b123-4455-b933-fc591ead6050-kube-api-access-k4v6q\") pod \"ae4337ee-b123-4455-b933-fc591ead6050\" (UID: \"ae4337ee-b123-4455-b933-fc591ead6050\") "
Sep 14 03:03:38.121476 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.121439    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7810aee1-af2a-4025-ad7c-0544aa7c49a9-metrics-certs\") pod \"7810aee1-af2a-4025-ad7c-0544aa7c49a9\" (UID: \"7810aee1-af2a-4025-ad7c-0544aa7c49a9\") "
Sep 14 03:03:38.121566 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.121530    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae4337ee-b123-4455-b933-fc591ead6050-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ae4337ee-b123-4455-b933-fc591ead6050" (UID: "ae4337ee-b123-4455-b933-fc591ead6050"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:03:38.121814 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.121793    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae4337ee-b123-4455-b933-fc591ead6050-plugins-config-volume\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:03:38.123290 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.123248    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7810aee1-af2a-4025-ad7c-0544aa7c49a9-kube-api-access-l8wqx" (OuterVolumeSpecName: "kube-api-access-l8wqx") pod "7810aee1-af2a-4025-ad7c-0544aa7c49a9" (UID: "7810aee1-af2a-4025-ad7c-0544aa7c49a9"). InnerVolumeSpecName "kube-api-access-l8wqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:03:38.123444 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.123415    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7810aee1-af2a-4025-ad7c-0544aa7c49a9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7810aee1-af2a-4025-ad7c-0544aa7c49a9" (UID: "7810aee1-af2a-4025-ad7c-0544aa7c49a9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:03:38.123716 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.123682    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7810aee1-af2a-4025-ad7c-0544aa7c49a9-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "7810aee1-af2a-4025-ad7c-0544aa7c49a9" (UID: "7810aee1-af2a-4025-ad7c-0544aa7c49a9"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:03:38.123716 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.123696    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae4337ee-b123-4455-b933-fc591ead6050-kube-api-access-k4v6q" (OuterVolumeSpecName: "kube-api-access-k4v6q") pod "ae4337ee-b123-4455-b933-fc591ead6050" (UID: "ae4337ee-b123-4455-b933-fc591ead6050"). InnerVolumeSpecName "kube-api-access-k4v6q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:03:38.188747 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.188715    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-t5rpm"]
Sep 14 03:03:38.188972 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.188948    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-t5rpm" podUID="f0029d72-4477-4ffe-81e3-8f1fcdaaeae6" containerName="payload-pre-processing" containerID="cri-o://bab0ec9bc9b268c116fe461da70d307c17bcfc13028cdd28c45f318ef227f683" gracePeriod=30
Sep 14 03:03:38.222992 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.222969    2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7810aee1-af2a-4025-ad7c-0544aa7c49a9-webhook-cert\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:03:38.223099 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.223024    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4v6q\" (UniqueName: \"kubernetes.io/projected/ae4337ee-b123-4455-b933-fc591ead6050-kube-api-access-k4v6q\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:03:38.223099 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.223042    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7810aee1-af2a-4025-ad7c-0544aa7c49a9-metrics-certs\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:03:38.223099 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.223057    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8wqx\" (UniqueName: \"kubernetes.io/projected/7810aee1-af2a-4025-ad7c-0544aa7c49a9-kube-api-access-l8wqx\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:03:38.424568 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.424538    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-t5rpm"
Sep 14 03:03:38.526242 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.526212    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nh9r4\" (UniqueName: \"kubernetes.io/projected/f0029d72-4477-4ffe-81e3-8f1fcdaaeae6-kube-api-access-nh9r4\") pod \"f0029d72-4477-4ffe-81e3-8f1fcdaaeae6\" (UID: \"f0029d72-4477-4ffe-81e3-8f1fcdaaeae6\") "
Sep 14 03:03:38.526392 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.526252    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0029d72-4477-4ffe-81e3-8f1fcdaaeae6-plugins-config-volume\") pod \"f0029d72-4477-4ffe-81e3-8f1fcdaaeae6\" (UID: \"f0029d72-4477-4ffe-81e3-8f1fcdaaeae6\") "
Sep 14 03:03:38.526671 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.526643    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0029d72-4477-4ffe-81e3-8f1fcdaaeae6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f0029d72-4477-4ffe-81e3-8f1fcdaaeae6" (UID: "f0029d72-4477-4ffe-81e3-8f1fcdaaeae6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:03:38.528257 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.528240    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0029d72-4477-4ffe-81e3-8f1fcdaaeae6-kube-api-access-nh9r4" (OuterVolumeSpecName: "kube-api-access-nh9r4") pod "f0029d72-4477-4ffe-81e3-8f1fcdaaeae6" (UID: "f0029d72-4477-4ffe-81e3-8f1fcdaaeae6"). InnerVolumeSpecName "kube-api-access-nh9r4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:03:38.623160 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.623103    2577 generic.go:356] "Generic (PLEG): container finished" podID="7810aee1-af2a-4025-ad7c-0544aa7c49a9" containerID="1303ec4303f9964db6bfc5ef6e66af9f5488e8c9aa9a2183ab36a62dd8b80a74" exitCode=0
Sep 14 03:03:38.623160 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.623141    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64b88c4d64-hpbg2"
Sep 14 03:03:38.623334 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.623189    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64b88c4d64-hpbg2" event={"ID":"7810aee1-af2a-4025-ad7c-0544aa7c49a9","Type":"ContainerDied","Data":"1303ec4303f9964db6bfc5ef6e66af9f5488e8c9aa9a2183ab36a62dd8b80a74"}
Sep 14 03:03:38.623334 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.623234    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64b88c4d64-hpbg2" event={"ID":"7810aee1-af2a-4025-ad7c-0544aa7c49a9","Type":"ContainerDied","Data":"2e4d7611a46eb34ab76a6a4a35f87a41c99fc038e1617abb605e1e1afd28d40d"}
Sep 14 03:03:38.623334 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.623257    2577 scope.go:117] "RemoveContainer" containerID="1303ec4303f9964db6bfc5ef6e66af9f5488e8c9aa9a2183ab36a62dd8b80a74"
Sep 14 03:03:38.625010 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.624984    2577 generic.go:356] "Generic (PLEG): container finished" podID="ae4337ee-b123-4455-b933-fc591ead6050" containerID="a1478be9cf99048114430b925efc3f6f2db31ae4037d11934fea1adbb597ef72" exitCode=0
Sep 14 03:03:38.625119 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.625031    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-cr47g"
Sep 14 03:03:38.625119 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.625080    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-cr47g" event={"ID":"ae4337ee-b123-4455-b933-fc591ead6050","Type":"ContainerDied","Data":"a1478be9cf99048114430b925efc3f6f2db31ae4037d11934fea1adbb597ef72"}
Sep 14 03:03:38.625382 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.625137    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-cr47g" event={"ID":"ae4337ee-b123-4455-b933-fc591ead6050","Type":"ContainerDied","Data":"c003370d6768bdca39dbacd33fe726f74ec39289bcb1c50215105c9ff33eae19"}
Sep 14 03:03:38.626793 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.626765    2577 generic.go:356] "Generic (PLEG): container finished" podID="f0029d72-4477-4ffe-81e3-8f1fcdaaeae6" containerID="bab0ec9bc9b268c116fe461da70d307c17bcfc13028cdd28c45f318ef227f683" exitCode=0
Sep 14 03:03:38.626914 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.626898    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-t5rpm" event={"ID":"f0029d72-4477-4ffe-81e3-8f1fcdaaeae6","Type":"ContainerDied","Data":"bab0ec9bc9b268c116fe461da70d307c17bcfc13028cdd28c45f318ef227f683"}
Sep 14 03:03:38.626987 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.626943    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-t5rpm" event={"ID":"f0029d72-4477-4ffe-81e3-8f1fcdaaeae6","Type":"ContainerDied","Data":"d8cd58b51f7193441f1fa4fb2b957439d937a12a91bab4d492bef45aacc63cd0"}
Sep 14 03:03:38.627043 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.627006    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-t5rpm"
Sep 14 03:03:38.627043 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.627016    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nh9r4\" (UniqueName: \"kubernetes.io/projected/f0029d72-4477-4ffe-81e3-8f1fcdaaeae6-kube-api-access-nh9r4\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:03:38.627043 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.627038    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0029d72-4477-4ffe-81e3-8f1fcdaaeae6-plugins-config-volume\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:03:38.635520 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.635496    2577 scope.go:117] "RemoveContainer" containerID="1303ec4303f9964db6bfc5ef6e66af9f5488e8c9aa9a2183ab36a62dd8b80a74"
Sep 14 03:03:38.635832 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:03:38.635813    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1303ec4303f9964db6bfc5ef6e66af9f5488e8c9aa9a2183ab36a62dd8b80a74\": container with ID starting with 1303ec4303f9964db6bfc5ef6e66af9f5488e8c9aa9a2183ab36a62dd8b80a74 not found: ID does not exist" containerID="1303ec4303f9964db6bfc5ef6e66af9f5488e8c9aa9a2183ab36a62dd8b80a74"
Sep 14 03:03:38.635891 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.635840    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1303ec4303f9964db6bfc5ef6e66af9f5488e8c9aa9a2183ab36a62dd8b80a74"} err="failed to get container status \"1303ec4303f9964db6bfc5ef6e66af9f5488e8c9aa9a2183ab36a62dd8b80a74\": rpc error: code = NotFound desc = could not find container \"1303ec4303f9964db6bfc5ef6e66af9f5488e8c9aa9a2183ab36a62dd8b80a74\": container with ID starting with 1303ec4303f9964db6bfc5ef6e66af9f5488e8c9aa9a2183ab36a62dd8b80a74 not found: ID does not exist"
Sep 14 03:03:38.635891 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.635856    2577 scope.go:117] "RemoveContainer" containerID="a1478be9cf99048114430b925efc3f6f2db31ae4037d11934fea1adbb597ef72"
Sep 14 03:03:38.644511 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.644493    2577 scope.go:117] "RemoveContainer" containerID="a1478be9cf99048114430b925efc3f6f2db31ae4037d11934fea1adbb597ef72"
Sep 14 03:03:38.644778 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:03:38.644756    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1478be9cf99048114430b925efc3f6f2db31ae4037d11934fea1adbb597ef72\": container with ID starting with a1478be9cf99048114430b925efc3f6f2db31ae4037d11934fea1adbb597ef72 not found: ID does not exist" containerID="a1478be9cf99048114430b925efc3f6f2db31ae4037d11934fea1adbb597ef72"
Sep 14 03:03:38.644834 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.644790    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1478be9cf99048114430b925efc3f6f2db31ae4037d11934fea1adbb597ef72"} err="failed to get container status \"a1478be9cf99048114430b925efc3f6f2db31ae4037d11934fea1adbb597ef72\": rpc error: code = NotFound desc = could not find container \"a1478be9cf99048114430b925efc3f6f2db31ae4037d11934fea1adbb597ef72\": container with ID starting with a1478be9cf99048114430b925efc3f6f2db31ae4037d11934fea1adbb597ef72 not found: ID does not exist"
Sep 14 03:03:38.644834 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.644811    2577 scope.go:117] "RemoveContainer" containerID="bab0ec9bc9b268c116fe461da70d307c17bcfc13028cdd28c45f318ef227f683"
Sep 14 03:03:38.648135 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.648113    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-64b88c4d64-hpbg2"]
Sep 14 03:03:38.653227 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.653206    2577 scope.go:117] "RemoveContainer" containerID="bab0ec9bc9b268c116fe461da70d307c17bcfc13028cdd28c45f318ef227f683"
Sep 14 03:03:38.653533 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:03:38.653507    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bab0ec9bc9b268c116fe461da70d307c17bcfc13028cdd28c45f318ef227f683\": container with ID starting with bab0ec9bc9b268c116fe461da70d307c17bcfc13028cdd28c45f318ef227f683 not found: ID does not exist" containerID="bab0ec9bc9b268c116fe461da70d307c17bcfc13028cdd28c45f318ef227f683"
Sep 14 03:03:38.653591 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.653532    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bab0ec9bc9b268c116fe461da70d307c17bcfc13028cdd28c45f318ef227f683"} err="failed to get container status \"bab0ec9bc9b268c116fe461da70d307c17bcfc13028cdd28c45f318ef227f683\": rpc error: code = NotFound desc = could not find container \"bab0ec9bc9b268c116fe461da70d307c17bcfc13028cdd28c45f318ef227f683\": container with ID starting with bab0ec9bc9b268c116fe461da70d307c17bcfc13028cdd28c45f318ef227f683 not found: ID does not exist"
Sep 14 03:03:38.654658 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.654633    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-64b88c4d64-hpbg2"]
Sep 14 03:03:38.665656 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.665635    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-t5rpm"]
Sep 14 03:03:38.669821 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.669798    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-t5rpm"]
Sep 14 03:03:38.679782 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.679761    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-cr47g"]
Sep 14 03:03:38.689216 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.689194    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-cr47g"]
Sep 14 03:03:38.878037 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.877973    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-64b88c4d64-xrsdd"]
Sep 14 03:03:38.878600 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.878574    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7810aee1-af2a-4025-ad7c-0544aa7c49a9" containerName="manager"
Sep 14 03:03:38.878600 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.878597    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7810aee1-af2a-4025-ad7c-0544aa7c49a9" containerName="manager"
Sep 14 03:03:38.878731 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.878608    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0029d72-4477-4ffe-81e3-8f1fcdaaeae6" containerName="payload-pre-processing"
Sep 14 03:03:38.878731 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.878617    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0029d72-4477-4ffe-81e3-8f1fcdaaeae6" containerName="payload-pre-processing"
Sep 14 03:03:38.878731 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.878640    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae4337ee-b123-4455-b933-fc591ead6050" containerName="payload-processing"
Sep 14 03:03:38.878731 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.878646    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae4337ee-b123-4455-b933-fc591ead6050" containerName="payload-processing"
Sep 14 03:03:38.878928 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.878737    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae4337ee-b123-4455-b933-fc591ead6050" containerName="payload-processing"
Sep 14 03:03:38.878928 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.878756    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7810aee1-af2a-4025-ad7c-0544aa7c49a9" containerName="manager"
Sep 14 03:03:38.878928 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.878768    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0029d72-4477-4ffe-81e3-8f1fcdaaeae6" containerName="payload-pre-processing"
Sep 14 03:03:38.883279 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.883252    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64b88c4d64-xrsdd"
Sep 14 03:03:38.886007 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.885986    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 14 03:03:38.886126 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.886105    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-ktqbd\""
Sep 14 03:03:38.886218 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.886150    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 14 03:03:38.886309 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.886226    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 14 03:03:38.899260 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.899236    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64b88c4d64-xrsdd"]
Sep 14 03:03:38.930683 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.930656    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdslx\" (UniqueName: \"kubernetes.io/projected/afeb4abf-2d9b-40d1-b896-c44607e40c34-kube-api-access-gdslx\") pod \"maas-controller-64b88c4d64-xrsdd\" (UID: \"afeb4abf-2d9b-40d1-b896-c44607e40c34\") " pod="opendatahub/maas-controller-64b88c4d64-xrsdd"
Sep 14 03:03:38.930784 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.930720    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/afeb4abf-2d9b-40d1-b896-c44607e40c34-webhook-cert\") pod \"maas-controller-64b88c4d64-xrsdd\" (UID: \"afeb4abf-2d9b-40d1-b896-c44607e40c34\") " pod="opendatahub/maas-controller-64b88c4d64-xrsdd"
Sep 14 03:03:38.930851 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:38.930833    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afeb4abf-2d9b-40d1-b896-c44607e40c34-metrics-certs\") pod \"maas-controller-64b88c4d64-xrsdd\" (UID: \"afeb4abf-2d9b-40d1-b896-c44607e40c34\") " pod="opendatahub/maas-controller-64b88c4d64-xrsdd"
Sep 14 03:03:39.031744 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:39.031718    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/afeb4abf-2d9b-40d1-b896-c44607e40c34-webhook-cert\") pod \"maas-controller-64b88c4d64-xrsdd\" (UID: \"afeb4abf-2d9b-40d1-b896-c44607e40c34\") " pod="opendatahub/maas-controller-64b88c4d64-xrsdd"
Sep 14 03:03:39.031838 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:39.031774    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afeb4abf-2d9b-40d1-b896-c44607e40c34-metrics-certs\") pod \"maas-controller-64b88c4d64-xrsdd\" (UID: \"afeb4abf-2d9b-40d1-b896-c44607e40c34\") " pod="opendatahub/maas-controller-64b88c4d64-xrsdd"
Sep 14 03:03:39.031838 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:39.031807    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdslx\" (UniqueName: \"kubernetes.io/projected/afeb4abf-2d9b-40d1-b896-c44607e40c34-kube-api-access-gdslx\") pod \"maas-controller-64b88c4d64-xrsdd\" (UID: \"afeb4abf-2d9b-40d1-b896-c44607e40c34\") " pod="opendatahub/maas-controller-64b88c4d64-xrsdd"
Sep 14 03:03:39.034138 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:39.034109    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afeb4abf-2d9b-40d1-b896-c44607e40c34-metrics-certs\") pod \"maas-controller-64b88c4d64-xrsdd\" (UID: \"afeb4abf-2d9b-40d1-b896-c44607e40c34\") " pod="opendatahub/maas-controller-64b88c4d64-xrsdd"
Sep 14 03:03:39.034221 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:39.034169    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/afeb4abf-2d9b-40d1-b896-c44607e40c34-webhook-cert\") pod \"maas-controller-64b88c4d64-xrsdd\" (UID: \"afeb4abf-2d9b-40d1-b896-c44607e40c34\") " pod="opendatahub/maas-controller-64b88c4d64-xrsdd"
Sep 14 03:03:39.040022 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:39.040003    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdslx\" (UniqueName: \"kubernetes.io/projected/afeb4abf-2d9b-40d1-b896-c44607e40c34-kube-api-access-gdslx\") pod \"maas-controller-64b88c4d64-xrsdd\" (UID: \"afeb4abf-2d9b-40d1-b896-c44607e40c34\") " pod="opendatahub/maas-controller-64b88c4d64-xrsdd"
Sep 14 03:03:39.192636 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:39.192575    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64b88c4d64-xrsdd"
Sep 14 03:03:39.467551 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:39.467524    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64b88c4d64-xrsdd"]
Sep 14 03:03:39.468724 ip-10-0-141-180 kubenswrapper[2577]: W0914 03:03:39.468694    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafeb4abf_2d9b_40d1_b896_c44607e40c34.slice/crio-19e1691be1cae313d91a0a9cddb2ab5aa4099506437e3cc97d0ca664235ec800 WatchSource:0}: Error finding container 19e1691be1cae313d91a0a9cddb2ab5aa4099506437e3cc97d0ca664235ec800: Status 404 returned error can't find the container with id 19e1691be1cae313d91a0a9cddb2ab5aa4099506437e3cc97d0ca664235ec800
Sep 14 03:03:39.617461 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:39.617436    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7810aee1-af2a-4025-ad7c-0544aa7c49a9" path="/var/lib/kubelet/pods/7810aee1-af2a-4025-ad7c-0544aa7c49a9/volumes"
Sep 14 03:03:39.617807 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:39.617792    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae4337ee-b123-4455-b933-fc591ead6050" path="/var/lib/kubelet/pods/ae4337ee-b123-4455-b933-fc591ead6050/volumes"
Sep 14 03:03:39.618147 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:39.618135    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0029d72-4477-4ffe-81e3-8f1fcdaaeae6" path="/var/lib/kubelet/pods/f0029d72-4477-4ffe-81e3-8f1fcdaaeae6/volumes"
Sep 14 03:03:39.634485 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:39.634461    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64b88c4d64-xrsdd" event={"ID":"afeb4abf-2d9b-40d1-b896-c44607e40c34","Type":"ContainerStarted","Data":"19e1691be1cae313d91a0a9cddb2ab5aa4099506437e3cc97d0ca664235ec800"}
Sep 14 03:03:40.643929 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:40.643892    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64b88c4d64-xrsdd" event={"ID":"afeb4abf-2d9b-40d1-b896-c44607e40c34","Type":"ContainerStarted","Data":"b2f9aeef6a8ee51a53048ee3a294fb1cfc32fcb2a116a78665cca3de126e1947"}
Sep 14 03:03:40.644353 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:40.643960    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-64b88c4d64-xrsdd"
Sep 14 03:03:40.662819 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:40.662771    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-64b88c4d64-xrsdd" podStartSLOduration=2.190905281 podStartE2EDuration="2.662755566s" totalImagesPullingTime="471.850285ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:03:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:03:40.6617014 +0000 UTC m=+769.639679429" watchObservedRunningTime="2026-09-14 03:03:40.662755566 +0000 UTC m=+769.640733594"
Sep 14 03:03:51.652390 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:51.652366    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-64b88c4d64-xrsdd"
Sep 14 03:03:55.272453 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:55.272410    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-74969cc5f8-nzb2n"]
Sep 14 03:03:55.277544 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:55.277516    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-74969cc5f8-nzb2n"
Sep 14 03:03:55.280063 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:55.280037    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 14 03:03:55.280204 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:55.280065    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-zhr5x\""
Sep 14 03:03:55.284536 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:55.284513    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-74969cc5f8-nzb2n"]
Sep 14 03:03:55.307031 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:55.306978    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-558d9b8847-hbpwz"]
Sep 14 03:03:55.310936 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:55.310915    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-558d9b8847-hbpwz"
Sep 14 03:03:55.320961 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:55.320936    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-558d9b8847-hbpwz"]
Sep 14 03:03:55.354909 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:55.354877    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22h2m\" (UniqueName: \"kubernetes.io/projected/9d22b886-4625-41cd-b7ed-bf3d0a25020e-kube-api-access-22h2m\") pod \"payload-pre-processing-558d9b8847-hbpwz\" (UID: \"9d22b886-4625-41cd-b7ed-bf3d0a25020e\") " pod="openshift-ingress/payload-pre-processing-558d9b8847-hbpwz"
Sep 14 03:03:55.355015 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:55.354938    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d22b886-4625-41cd-b7ed-bf3d0a25020e-plugins-config-volume\") pod \"payload-pre-processing-558d9b8847-hbpwz\" (UID: \"9d22b886-4625-41cd-b7ed-bf3d0a25020e\") " pod="openshift-ingress/payload-pre-processing-558d9b8847-hbpwz"
Sep 14 03:03:55.355015 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:55.354989    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx7vd\" (UniqueName: \"kubernetes.io/projected/02709d9b-915f-4d2b-8bd2-0b73f296f99b-kube-api-access-vx7vd\") pod \"payload-processing-74969cc5f8-nzb2n\" (UID: \"02709d9b-915f-4d2b-8bd2-0b73f296f99b\") " pod="openshift-ingress/payload-processing-74969cc5f8-nzb2n"
Sep 14 03:03:55.355120 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:55.355066    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02709d9b-915f-4d2b-8bd2-0b73f296f99b-plugins-config-volume\") pod \"payload-processing-74969cc5f8-nzb2n\" (UID: \"02709d9b-915f-4d2b-8bd2-0b73f296f99b\") " pod="openshift-ingress/payload-processing-74969cc5f8-nzb2n"
Sep 14 03:03:55.456323 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:55.456294    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx7vd\" (UniqueName: \"kubernetes.io/projected/02709d9b-915f-4d2b-8bd2-0b73f296f99b-kube-api-access-vx7vd\") pod \"payload-processing-74969cc5f8-nzb2n\" (UID: \"02709d9b-915f-4d2b-8bd2-0b73f296f99b\") " pod="openshift-ingress/payload-processing-74969cc5f8-nzb2n"
Sep 14 03:03:55.456422 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:55.456342    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02709d9b-915f-4d2b-8bd2-0b73f296f99b-plugins-config-volume\") pod \"payload-processing-74969cc5f8-nzb2n\" (UID: \"02709d9b-915f-4d2b-8bd2-0b73f296f99b\") " pod="openshift-ingress/payload-processing-74969cc5f8-nzb2n"
Sep 14 03:03:55.456422 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:55.456378    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22h2m\" (UniqueName: \"kubernetes.io/projected/9d22b886-4625-41cd-b7ed-bf3d0a25020e-kube-api-access-22h2m\") pod \"payload-pre-processing-558d9b8847-hbpwz\" (UID: \"9d22b886-4625-41cd-b7ed-bf3d0a25020e\") " pod="openshift-ingress/payload-pre-processing-558d9b8847-hbpwz"
Sep 14 03:03:55.456422 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:55.456418    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d22b886-4625-41cd-b7ed-bf3d0a25020e-plugins-config-volume\") pod \"payload-pre-processing-558d9b8847-hbpwz\" (UID: \"9d22b886-4625-41cd-b7ed-bf3d0a25020e\") " pod="openshift-ingress/payload-pre-processing-558d9b8847-hbpwz"
Sep 14 03:03:55.457182 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:55.457165    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d22b886-4625-41cd-b7ed-bf3d0a25020e-plugins-config-volume\") pod \"payload-pre-processing-558d9b8847-hbpwz\" (UID: \"9d22b886-4625-41cd-b7ed-bf3d0a25020e\") " pod="openshift-ingress/payload-pre-processing-558d9b8847-hbpwz"
Sep 14 03:03:55.457220 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:55.457179    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02709d9b-915f-4d2b-8bd2-0b73f296f99b-plugins-config-volume\") pod \"payload-processing-74969cc5f8-nzb2n\" (UID: \"02709d9b-915f-4d2b-8bd2-0b73f296f99b\") " pod="openshift-ingress/payload-processing-74969cc5f8-nzb2n"
Sep 14 03:03:55.464491 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:55.464467    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx7vd\" (UniqueName: \"kubernetes.io/projected/02709d9b-915f-4d2b-8bd2-0b73f296f99b-kube-api-access-vx7vd\") pod \"payload-processing-74969cc5f8-nzb2n\" (UID: \"02709d9b-915f-4d2b-8bd2-0b73f296f99b\") " pod="openshift-ingress/payload-processing-74969cc5f8-nzb2n"
Sep 14 03:03:55.464638 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:55.464619    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22h2m\" (UniqueName: \"kubernetes.io/projected/9d22b886-4625-41cd-b7ed-bf3d0a25020e-kube-api-access-22h2m\") pod \"payload-pre-processing-558d9b8847-hbpwz\" (UID: \"9d22b886-4625-41cd-b7ed-bf3d0a25020e\") " pod="openshift-ingress/payload-pre-processing-558d9b8847-hbpwz"
Sep 14 03:03:55.590794 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:55.590739    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-74969cc5f8-nzb2n"
Sep 14 03:03:55.623034 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:55.623013    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-558d9b8847-hbpwz"
Sep 14 03:03:55.868232 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:55.868209    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-74969cc5f8-nzb2n"]
Sep 14 03:03:55.869231 ip-10-0-141-180 kubenswrapper[2577]: W0914 03:03:55.869207    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02709d9b_915f_4d2b_8bd2_0b73f296f99b.slice/crio-c0ea1f4cb9e0a57ee186a3eab463d3bdcb649777ac5c085ab956cbde5818b1e2 WatchSource:0}: Error finding container c0ea1f4cb9e0a57ee186a3eab463d3bdcb649777ac5c085ab956cbde5818b1e2: Status 404 returned error can't find the container with id c0ea1f4cb9e0a57ee186a3eab463d3bdcb649777ac5c085ab956cbde5818b1e2
Sep 14 03:03:55.906425 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:55.906403    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-558d9b8847-hbpwz"]
Sep 14 03:03:55.907786 ip-10-0-141-180 kubenswrapper[2577]: W0914 03:03:55.907763    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d22b886_4625_41cd_b7ed_bf3d0a25020e.slice/crio-5c1591822a11742373176daf4e2e1c855a6cfc3b960abd7d695eecf9130b4556 WatchSource:0}: Error finding container 5c1591822a11742373176daf4e2e1c855a6cfc3b960abd7d695eecf9130b4556: Status 404 returned error can't find the container with id 5c1591822a11742373176daf4e2e1c855a6cfc3b960abd7d695eecf9130b4556
Sep 14 03:03:56.743098 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:56.743058    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-74969cc5f8-nzb2n" event={"ID":"02709d9b-915f-4d2b-8bd2-0b73f296f99b","Type":"ContainerStarted","Data":"c0ea1f4cb9e0a57ee186a3eab463d3bdcb649777ac5c085ab956cbde5818b1e2"}
Sep 14 03:03:56.745614 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:56.745585    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-558d9b8847-hbpwz" event={"ID":"9d22b886-4625-41cd-b7ed-bf3d0a25020e","Type":"ContainerStarted","Data":"5c1591822a11742373176daf4e2e1c855a6cfc3b960abd7d695eecf9130b4556"}
Sep 14 03:03:58.759198 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:58.759163    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-558d9b8847-hbpwz" event={"ID":"9d22b886-4625-41cd-b7ed-bf3d0a25020e","Type":"ContainerStarted","Data":"8a2cf78477ed1db04c136248b99a5351ae2844c682501edaf574406377296b4b"}
Sep 14 03:03:58.759690 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:58.759258    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-558d9b8847-hbpwz"
Sep 14 03:03:58.760987 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:58.760963    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-74969cc5f8-nzb2n" event={"ID":"02709d9b-915f-4d2b-8bd2-0b73f296f99b","Type":"ContainerStarted","Data":"b235fb894666ef195d014897d5947995e52f1b087465b4b497aaa55aac8a36e8"}
Sep 14 03:03:58.761123 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:58.761107    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-74969cc5f8-nzb2n"
Sep 14 03:03:58.791879 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:58.791836    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-74969cc5f8-nzb2n" podStartSLOduration=1.668951049 podStartE2EDuration="3.791822285s" totalImagesPullingTime="2.122871236s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:03:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:03:58.790428948 +0000 UTC m=+787.768406987" watchObservedRunningTime="2026-09-14 03:03:58.791822285 +0000 UTC m=+787.769800315"
Sep 14 03:03:58.792261 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:58.792238    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-558d9b8847-hbpwz" podStartSLOduration=1.702783697 podStartE2EDuration="3.792231832s" totalImagesPullingTime="2.089448135s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:03:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:03:58.776496014 +0000 UTC m=+787.754474043" watchObservedRunningTime="2026-09-14 03:03:58.792231832 +0000 UTC m=+787.770209862"
Sep 14 03:03:59.766955 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:59.766917    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-74969cc5f8-nzb2n" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:03:59.767374 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:03:59.767007    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-558d9b8847-hbpwz" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:04:09.768908 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:09.768877    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-74969cc5f8-nzb2n"
Sep 14 03:04:09.769309 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:09.768946    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-558d9b8847-hbpwz"
Sep 14 03:04:09.769358 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:09.769341    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-558d9b8847-hbpwz" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:04:09.769483 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:09.769462    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-74969cc5f8-nzb2n" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:04:42.521900 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:42.521864    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e467de-7746bc5466-262fs"]
Sep 14 03:04:42.525582 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:42.525566    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e467de-7746bc5466-262fs"
Sep 14 03:04:42.528830 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:42.528804    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-e467de-serving-cert\""
Sep 14 03:04:42.554962 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:42.554943    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15a7d6f1-7858-4371-800f-2d7da9308450-metrics-certs\") pod \"maas-api-e2e-inf-a-e467de-7746bc5466-262fs\" (UID: \"15a7d6f1-7858-4371-800f-2d7da9308450\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e467de-7746bc5466-262fs"
Sep 14 03:04:42.555062 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:42.554976    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15a7d6f1-7858-4371-800f-2d7da9308450-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e467de-7746bc5466-262fs\" (UID: \"15a7d6f1-7858-4371-800f-2d7da9308450\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e467de-7746bc5466-262fs"
Sep 14 03:04:42.555062 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:42.555005    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4l9g\" (UniqueName: \"kubernetes.io/projected/15a7d6f1-7858-4371-800f-2d7da9308450-kube-api-access-b4l9g\") pod \"maas-api-e2e-inf-a-e467de-7746bc5466-262fs\" (UID: \"15a7d6f1-7858-4371-800f-2d7da9308450\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e467de-7746bc5466-262fs"
Sep 14 03:04:42.555155 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:42.555070    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15a7d6f1-7858-4371-800f-2d7da9308450-maas-api-tls\") pod \"maas-api-e2e-inf-a-e467de-7746bc5466-262fs\" (UID: \"15a7d6f1-7858-4371-800f-2d7da9308450\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e467de-7746bc5466-262fs"
Sep 14 03:04:42.562622 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:42.562600    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e467de-7746bc5466-262fs"]
Sep 14 03:04:42.655480 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:42.655455    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15a7d6f1-7858-4371-800f-2d7da9308450-maas-api-tls\") pod \"maas-api-e2e-inf-a-e467de-7746bc5466-262fs\" (UID: \"15a7d6f1-7858-4371-800f-2d7da9308450\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e467de-7746bc5466-262fs"
Sep 14 03:04:42.655622 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:42.655506    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15a7d6f1-7858-4371-800f-2d7da9308450-metrics-certs\") pod \"maas-api-e2e-inf-a-e467de-7746bc5466-262fs\" (UID: \"15a7d6f1-7858-4371-800f-2d7da9308450\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e467de-7746bc5466-262fs"
Sep 14 03:04:42.655622 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:42.655543    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15a7d6f1-7858-4371-800f-2d7da9308450-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e467de-7746bc5466-262fs\" (UID: \"15a7d6f1-7858-4371-800f-2d7da9308450\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e467de-7746bc5466-262fs"
Sep 14 03:04:42.655622 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:04:42.655608    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-e467de-serving-cert: secret "maas-api-e2e-inf-a-e467de-serving-cert" not found
Sep 14 03:04:42.655799 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:42.655654    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4l9g\" (UniqueName: \"kubernetes.io/projected/15a7d6f1-7858-4371-800f-2d7da9308450-kube-api-access-b4l9g\") pod \"maas-api-e2e-inf-a-e467de-7746bc5466-262fs\" (UID: \"15a7d6f1-7858-4371-800f-2d7da9308450\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e467de-7746bc5466-262fs"
Sep 14 03:04:42.655799 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:04:42.655674    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15a7d6f1-7858-4371-800f-2d7da9308450-maas-api-tls podName:15a7d6f1-7858-4371-800f-2d7da9308450 nodeName:}" failed. No retries permitted until 2026-09-14 03:04:43.155653136 +0000 UTC m=+832.133631143 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/15a7d6f1-7858-4371-800f-2d7da9308450-maas-api-tls") pod "maas-api-e2e-inf-a-e467de-7746bc5466-262fs" (UID: "15a7d6f1-7858-4371-800f-2d7da9308450") : secret "maas-api-e2e-inf-a-e467de-serving-cert" not found
Sep 14 03:04:42.656220 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:42.656202    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15a7d6f1-7858-4371-800f-2d7da9308450-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e467de-7746bc5466-262fs\" (UID: \"15a7d6f1-7858-4371-800f-2d7da9308450\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e467de-7746bc5466-262fs"
Sep 14 03:04:42.657993 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:42.657977    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15a7d6f1-7858-4371-800f-2d7da9308450-metrics-certs\") pod \"maas-api-e2e-inf-a-e467de-7746bc5466-262fs\" (UID: \"15a7d6f1-7858-4371-800f-2d7da9308450\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e467de-7746bc5466-262fs"
Sep 14 03:04:42.680854 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:42.680835    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4l9g\" (UniqueName: \"kubernetes.io/projected/15a7d6f1-7858-4371-800f-2d7da9308450-kube-api-access-b4l9g\") pod \"maas-api-e2e-inf-a-e467de-7746bc5466-262fs\" (UID: \"15a7d6f1-7858-4371-800f-2d7da9308450\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e467de-7746bc5466-262fs"
Sep 14 03:04:43.159256 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:43.159222    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15a7d6f1-7858-4371-800f-2d7da9308450-maas-api-tls\") pod \"maas-api-e2e-inf-a-e467de-7746bc5466-262fs\" (UID: \"15a7d6f1-7858-4371-800f-2d7da9308450\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e467de-7746bc5466-262fs"
Sep 14 03:04:43.161805 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:43.161780    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15a7d6f1-7858-4371-800f-2d7da9308450-maas-api-tls\") pod \"maas-api-e2e-inf-a-e467de-7746bc5466-262fs\" (UID: \"15a7d6f1-7858-4371-800f-2d7da9308450\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e467de-7746bc5466-262fs"
Sep 14 03:04:43.436352 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:43.436255    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e467de-7746bc5466-262fs"
Sep 14 03:04:43.451212 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:43.451185    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-8mn6v"]
Sep 14 03:04:43.455046 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:43.455023    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-8mn6v"
Sep 14 03:04:43.458368 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:43.458344    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-e467de\""
Sep 14 03:04:43.458497 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:43.458429    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-e467de-dockercfg-ft29f\""
Sep 14 03:04:43.472612 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:43.472591    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-8mn6v"]
Sep 14 03:04:43.562239 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:43.562207    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa380a57-8e96-4c29-bc4a-d317d335297b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-8mn6v\" (UID: \"aa380a57-8e96-4c29-bc4a-d317d335297b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-8mn6v"
Sep 14 03:04:43.562604 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:43.562258    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tt22w\" (UniqueName: \"kubernetes.io/projected/aa380a57-8e96-4c29-bc4a-d317d335297b-kube-api-access-tt22w\") pod \"payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-8mn6v\" (UID: \"aa380a57-8e96-4c29-bc4a-d317d335297b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-8mn6v"
Sep 14 03:04:43.663164 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:43.663137    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa380a57-8e96-4c29-bc4a-d317d335297b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-8mn6v\" (UID: \"aa380a57-8e96-4c29-bc4a-d317d335297b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-8mn6v"
Sep 14 03:04:43.663338 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:43.663182    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tt22w\" (UniqueName: \"kubernetes.io/projected/aa380a57-8e96-4c29-bc4a-d317d335297b-kube-api-access-tt22w\") pod \"payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-8mn6v\" (UID: \"aa380a57-8e96-4c29-bc4a-d317d335297b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-8mn6v"
Sep 14 03:04:43.663856 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:43.663837    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa380a57-8e96-4c29-bc4a-d317d335297b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-8mn6v\" (UID: \"aa380a57-8e96-4c29-bc4a-d317d335297b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-8mn6v"
Sep 14 03:04:43.672640 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:43.672612    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tt22w\" (UniqueName: \"kubernetes.io/projected/aa380a57-8e96-4c29-bc4a-d317d335297b-kube-api-access-tt22w\") pod \"payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-8mn6v\" (UID: \"aa380a57-8e96-4c29-bc4a-d317d335297b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-8mn6v"
Sep 14 03:04:43.721114 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:43.721089    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e467de-7746bc5466-262fs"]
Sep 14 03:04:43.721960 ip-10-0-141-180 kubenswrapper[2577]: W0914 03:04:43.721930    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15a7d6f1_7858_4371_800f_2d7da9308450.slice/crio-bfc86a3c8b1598ad409295ed7e76626f27707c2c17045010e522743d73dda4c3 WatchSource:0}: Error finding container bfc86a3c8b1598ad409295ed7e76626f27707c2c17045010e522743d73dda4c3: Status 404 returned error can't find the container with id bfc86a3c8b1598ad409295ed7e76626f27707c2c17045010e522743d73dda4c3
Sep 14 03:04:43.773693 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:43.773670    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-8mn6v"
Sep 14 03:04:44.010142 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:44.010112    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e467de-7746bc5466-262fs" event={"ID":"15a7d6f1-7858-4371-800f-2d7da9308450","Type":"ContainerStarted","Data":"bfc86a3c8b1598ad409295ed7e76626f27707c2c17045010e522743d73dda4c3"}
Sep 14 03:04:44.058549 ip-10-0-141-180 kubenswrapper[2577]: W0914 03:04:44.058519    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa380a57_8e96_4c29_bc4a_d317d335297b.slice/crio-5ceb631dab5ac3bbd597f0ee367f2366fb0a8f97c60eee4ffe768fc0a39620ef WatchSource:0}: Error finding container 5ceb631dab5ac3bbd597f0ee367f2366fb0a8f97c60eee4ffe768fc0a39620ef: Status 404 returned error can't find the container with id 5ceb631dab5ac3bbd597f0ee367f2366fb0a8f97c60eee4ffe768fc0a39620ef
Sep 14 03:04:44.058935 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:44.058908    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-8mn6v"]
Sep 14 03:04:45.017422 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:45.017390    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-8mn6v" event={"ID":"aa380a57-8e96-4c29-bc4a-d317d335297b","Type":"ContainerStarted","Data":"d5026068dc54c823ec1bd8eaeeaa7f11d9d99b1bd0c83e839bf2fccdd1e1133c"}
Sep 14 03:04:45.017846 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:45.017443    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-8mn6v" event={"ID":"aa380a57-8e96-4c29-bc4a-d317d335297b","Type":"ContainerStarted","Data":"5ceb631dab5ac3bbd597f0ee367f2366fb0a8f97c60eee4ffe768fc0a39620ef"}
Sep 14 03:04:45.017846 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:45.017476    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-8mn6v"
Sep 14 03:04:45.019106 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:45.019080    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e467de-7746bc5466-262fs" event={"ID":"15a7d6f1-7858-4371-800f-2d7da9308450","Type":"ContainerStarted","Data":"07545cb69585f315eecca9474d95cd8538aa0fb4bab1b055e066d5b125e3b8b9"}
Sep 14 03:04:45.019218 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:45.019129    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e467de-7746bc5466-262fs"
Sep 14 03:04:45.051197 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:45.051154    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-8mn6v" podStartSLOduration=2.051138966 podStartE2EDuration="2.051138966s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:04:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:04:45.050956636 +0000 UTC m=+834.028934663" watchObservedRunningTime="2026-09-14 03:04:45.051138966 +0000 UTC m=+834.029116995"
Sep 14 03:04:45.099152 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:45.099114    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e467de-7746bc5466-262fs" podStartSLOduration=2.591555792 podStartE2EDuration="3.099101234s" totalImagesPullingTime="507.545442ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:04:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:04:45.095721119 +0000 UTC m=+834.073699147" watchObservedRunningTime="2026-09-14 03:04:45.099101234 +0000 UTC m=+834.077079263"
Sep 14 03:04:51.031605 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:51.031575    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e467de-7746bc5466-262fs"
Sep 14 03:04:52.216648 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:52.216612    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7"]
Sep 14 03:04:52.220365 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:52.220348    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7"
Sep 14 03:04:52.223785 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:52.223754    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-c4b97c19-serving-cert\""
Sep 14 03:04:52.234420 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:52.234393    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7"]
Sep 14 03:04:52.341750 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:52.341719    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17-openshift-service-ca\") pod \"maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7\" (UID: \"2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7"
Sep 14 03:04:52.341883 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:52.341756    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17-metrics-certs\") pod \"maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7\" (UID: \"2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7"
Sep 14 03:04:52.341883 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:52.341832    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17-maas-api-tls\") pod \"maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7\" (UID: \"2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7"
Sep 14 03:04:52.341883 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:52.341859    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9vll\" (UniqueName: \"kubernetes.io/projected/2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17-kube-api-access-z9vll\") pod \"maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7\" (UID: \"2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7"
Sep 14 03:04:52.442790 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:52.442756    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17-openshift-service-ca\") pod \"maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7\" (UID: \"2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7"
Sep 14 03:04:52.442941 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:52.442803    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17-metrics-certs\") pod \"maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7\" (UID: \"2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7"
Sep 14 03:04:52.442941 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:52.442878    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17-maas-api-tls\") pod \"maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7\" (UID: \"2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7"
Sep 14 03:04:52.442941 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:52.442907    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9vll\" (UniqueName: \"kubernetes.io/projected/2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17-kube-api-access-z9vll\") pod \"maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7\" (UID: \"2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7"
Sep 14 03:04:52.443131 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:04:52.442982    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-c4b97c19-serving-cert: secret "maas-api-e2e-ait-c4b97c19-serving-cert" not found
Sep 14 03:04:52.443131 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:04:52.443065    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17-maas-api-tls podName:2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17 nodeName:}" failed. No retries permitted until 2026-09-14 03:04:52.943042141 +0000 UTC m=+841.921020162 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17-maas-api-tls") pod "maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7" (UID: "2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17") : secret "maas-api-e2e-ait-c4b97c19-serving-cert" not found
Sep 14 03:04:52.443547 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:52.443522    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17-openshift-service-ca\") pod \"maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7\" (UID: \"2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7"
Sep 14 03:04:52.445374 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:52.445354    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17-metrics-certs\") pod \"maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7\" (UID: \"2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7"
Sep 14 03:04:52.453143 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:52.453109    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9vll\" (UniqueName: \"kubernetes.io/projected/2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17-kube-api-access-z9vll\") pod \"maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7\" (UID: \"2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7"
Sep 14 03:04:52.948058 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:52.948026    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17-maas-api-tls\") pod \"maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7\" (UID: \"2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7"
Sep 14 03:04:52.950313 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:52.950291    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17-maas-api-tls\") pod \"maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7\" (UID: \"2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7"
Sep 14 03:04:53.132215 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:53.132183    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7"
Sep 14 03:04:53.421558 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:53.421533    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7"]
Sep 14 03:04:53.423296 ip-10-0-141-180 kubenswrapper[2577]: W0914 03:04:53.423248    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b673aeb_46a4_4bf9_bfdb_6fcf0281ac17.slice/crio-5d14df7368dc234aced6d71eb84cc0635bd44887b7549648da7998dbc191254a WatchSource:0}: Error finding container 5d14df7368dc234aced6d71eb84cc0635bd44887b7549648da7998dbc191254a: Status 404 returned error can't find the container with id 5d14df7368dc234aced6d71eb84cc0635bd44887b7549648da7998dbc191254a
Sep 14 03:04:54.077486 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:54.077455    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7" event={"ID":"2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17","Type":"ContainerStarted","Data":"5d14df7368dc234aced6d71eb84cc0635bd44887b7549648da7998dbc191254a"}
Sep 14 03:04:55.085095 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:55.085057    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7" event={"ID":"2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17","Type":"ContainerStarted","Data":"52123b8b1c6e26dbb3f15a757cd67777c6d262fdd7b927c0080c1a8e13ee4bd5"}
Sep 14 03:04:55.085589 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:55.085196    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7"
Sep 14 03:04:55.104520 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:55.104474    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7" podStartSLOduration=2.472074456 podStartE2EDuration="3.104461617s" totalImagesPullingTime="632.387161ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:04:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:04:55.103012043 +0000 UTC m=+844.080990072" watchObservedRunningTime="2026-09-14 03:04:55.104461617 +0000 UTC m=+844.082439648"
Sep 14 03:04:56.027206 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:04:56.027172    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-8mn6v"
Sep 14 03:05:01.096890 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:01.096862    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7"
Sep 14 03:05:01.810586 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:01.810550    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl"]
Sep 14 03:05:01.814625 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:01.814598    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl"
Sep 14 03:05:01.817120 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:01.817099    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-bc1117-serving-cert\""
Sep 14 03:05:01.838996 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:01.838968    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl"]
Sep 14 03:05:01.922897 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:01.922864    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsqqz\" (UniqueName: \"kubernetes.io/projected/4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9-kube-api-access-dsqqz\") pod \"maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl\" (UID: \"4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl"
Sep 14 03:05:01.923079 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:01.922915    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9-metrics-certs\") pod \"maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl\" (UID: \"4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl"
Sep 14 03:05:01.923079 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:01.923024    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9-maas-api-tls\") pod \"maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl\" (UID: \"4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl"
Sep 14 03:05:01.923079 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:01.923059    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl\" (UID: \"4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl"
Sep 14 03:05:02.024356 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:02.024325    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9-metrics-certs\") pod \"maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl\" (UID: \"4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl"
Sep 14 03:05:02.024510 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:02.024378    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9-maas-api-tls\") pod \"maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl\" (UID: \"4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl"
Sep 14 03:05:02.024510 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:05:02.024453    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-bc1117-serving-cert: secret "maas-api-e2e-inf-b-bc1117-serving-cert" not found
Sep 14 03:05:02.024510 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:02.024495    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl\" (UID: \"4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl"
Sep 14 03:05:02.024681 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:05:02.024515    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9-maas-api-tls podName:4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9 nodeName:}" failed. No retries permitted until 2026-09-14 03:05:02.524498606 +0000 UTC m=+851.502476613 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9-maas-api-tls") pod "maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl" (UID: "4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9") : secret "maas-api-e2e-inf-b-bc1117-serving-cert" not found
Sep 14 03:05:02.024681 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:02.024616    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsqqz\" (UniqueName: \"kubernetes.io/projected/4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9-kube-api-access-dsqqz\") pod \"maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl\" (UID: \"4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl"
Sep 14 03:05:02.025168 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:02.025144    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl\" (UID: \"4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl"
Sep 14 03:05:02.026757 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:02.026733    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9-metrics-certs\") pod \"maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl\" (UID: \"4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl"
Sep 14 03:05:02.043187 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:02.043163    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsqqz\" (UniqueName: \"kubernetes.io/projected/4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9-kube-api-access-dsqqz\") pod \"maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl\" (UID: \"4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl"
Sep 14 03:05:02.529950 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:02.529916    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9-maas-api-tls\") pod \"maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl\" (UID: \"4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl"
Sep 14 03:05:02.532533 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:02.532502    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9-maas-api-tls\") pod \"maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl\" (UID: \"4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl"
Sep 14 03:05:02.725277 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:02.725230    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl"
Sep 14 03:05:03.040058 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:03.040012    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl"]
Sep 14 03:05:03.040339 ip-10-0-141-180 kubenswrapper[2577]: W0914 03:05:03.040310    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cdebe1c_131d_4e07_b6cb_d4df32d4c7b9.slice/crio-61ca711c5e167d191243ba415fb9e3177e2086ebe6be1fabe47a86dd1858d6e4 WatchSource:0}: Error finding container 61ca711c5e167d191243ba415fb9e3177e2086ebe6be1fabe47a86dd1858d6e4: Status 404 returned error can't find the container with id 61ca711c5e167d191243ba415fb9e3177e2086ebe6be1fabe47a86dd1858d6e4
Sep 14 03:05:03.135247 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:03.135216    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl" event={"ID":"4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9","Type":"ContainerStarted","Data":"61ca711c5e167d191243ba415fb9e3177e2086ebe6be1fabe47a86dd1858d6e4"}
Sep 14 03:05:04.142257 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:04.142219    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl" event={"ID":"4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9","Type":"ContainerStarted","Data":"a5fcc6ecce7213e034fc95d0a221eb5f22227ddc2275ce463207c8e0de10a6cf"}
Sep 14 03:05:04.142671 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:04.142340    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl"
Sep 14 03:05:04.182709 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:04.182654    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl" podStartSLOduration=2.799052371 podStartE2EDuration="3.182641327s" totalImagesPullingTime="383.588956ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:05:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:05:04.18037704 +0000 UTC m=+853.158355070" watchObservedRunningTime="2026-09-14 03:05:04.182641327 +0000 UTC m=+853.160619397"
Sep 14 03:05:10.153340 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:10.153314    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl"
Sep 14 03:05:21.615434 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:21.615393    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-558d9b8847-hbpwz" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:05:32.649370 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:32.649339    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7"]
Sep 14 03:05:32.649830 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:32.649519    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7" podUID="2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17" containerName="maas-api" containerID="cri-o://52123b8b1c6e26dbb3f15a757cd67777c6d262fdd7b927c0080c1a8e13ee4bd5" gracePeriod=30
Sep 14 03:05:32.912941 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:32.912886    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7"
Sep 14 03:05:32.989955 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:32.989924    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17-openshift-service-ca\") pod \"2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17\" (UID: \"2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17\") "
Sep 14 03:05:32.990092 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:32.989977    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17-metrics-certs\") pod \"2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17\" (UID: \"2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17\") "
Sep 14 03:05:32.990092 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:32.990014    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17-maas-api-tls\") pod \"2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17\" (UID: \"2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17\") "
Sep 14 03:05:32.990092 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:32.990038    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9vll\" (UniqueName: \"kubernetes.io/projected/2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17-kube-api-access-z9vll\") pod \"2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17\" (UID: \"2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17\") "
Sep 14 03:05:32.990356 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:32.990331    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17" (UID: "2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:05:32.992421 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:32.992387    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17-kube-api-access-z9vll" (OuterVolumeSpecName: "kube-api-access-z9vll") pod "2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17" (UID: "2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17"). InnerVolumeSpecName "kube-api-access-z9vll". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:05:32.992542 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:32.992429    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17" (UID: "2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:05:32.992542 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:32.992464    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17" (UID: "2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:05:33.091040 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:33.091010    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17-metrics-certs\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:05:33.091040 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:33.091033    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17-maas-api-tls\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:05:33.091040 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:33.091043    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9vll\" (UniqueName: \"kubernetes.io/projected/2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17-kube-api-access-z9vll\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:05:33.091303 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:33.091052    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17-openshift-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:05:33.324637 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:33.324605    2577 generic.go:356] "Generic (PLEG): container finished" podID="2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17" containerID="52123b8b1c6e26dbb3f15a757cd67777c6d262fdd7b927c0080c1a8e13ee4bd5" exitCode=0
Sep 14 03:05:33.324875 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:33.324680    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7"
Sep 14 03:05:33.325017 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:33.324694    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7" event={"ID":"2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17","Type":"ContainerDied","Data":"52123b8b1c6e26dbb3f15a757cd67777c6d262fdd7b927c0080c1a8e13ee4bd5"}
Sep 14 03:05:33.325110 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:33.325016    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7" event={"ID":"2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17","Type":"ContainerDied","Data":"5d14df7368dc234aced6d71eb84cc0635bd44887b7549648da7998dbc191254a"}
Sep 14 03:05:33.325110 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:33.325050    2577 scope.go:117] "RemoveContainer" containerID="52123b8b1c6e26dbb3f15a757cd67777c6d262fdd7b927c0080c1a8e13ee4bd5"
Sep 14 03:05:33.336121 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:33.336090    2577 scope.go:117] "RemoveContainer" containerID="52123b8b1c6e26dbb3f15a757cd67777c6d262fdd7b927c0080c1a8e13ee4bd5"
Sep 14 03:05:33.336506 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:05:33.336483    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52123b8b1c6e26dbb3f15a757cd67777c6d262fdd7b927c0080c1a8e13ee4bd5\": container with ID starting with 52123b8b1c6e26dbb3f15a757cd67777c6d262fdd7b927c0080c1a8e13ee4bd5 not found: ID does not exist" containerID="52123b8b1c6e26dbb3f15a757cd67777c6d262fdd7b927c0080c1a8e13ee4bd5"
Sep 14 03:05:33.336582 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:33.336513    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52123b8b1c6e26dbb3f15a757cd67777c6d262fdd7b927c0080c1a8e13ee4bd5"} err="failed to get container status \"52123b8b1c6e26dbb3f15a757cd67777c6d262fdd7b927c0080c1a8e13ee4bd5\": rpc error: code = NotFound desc = could not find container \"52123b8b1c6e26dbb3f15a757cd67777c6d262fdd7b927c0080c1a8e13ee4bd5\": container with ID starting with 52123b8b1c6e26dbb3f15a757cd67777c6d262fdd7b927c0080c1a8e13ee4bd5 not found: ID does not exist"
Sep 14 03:05:33.373990 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:33.373965    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7"]
Sep 14 03:05:33.385897 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:33.385874    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c4b97c19-8589f7555b-nfwp7"]
Sep 14 03:05:33.618182 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:33.618095    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17" path="/var/lib/kubelet/pods/2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17/volumes"
Sep 14 03:05:36.611684 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:36.611656    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-74969cc5f8-nzb2n" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:05:37.719921 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:37.719889    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp"]
Sep 14 03:05:37.720510 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:37.720489    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17" containerName="maas-api"
Sep 14 03:05:37.720605 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:37.720511    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17" containerName="maas-api"
Sep 14 03:05:37.720663 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:37.720646    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b673aeb-46a4-4bf9-bfdb-6fcf0281ac17" containerName="maas-api"
Sep 14 03:05:37.725292 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:37.725239    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp"
Sep 14 03:05:37.728011 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:37.727985    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-07ec96b6-serving-cert\""
Sep 14 03:05:37.730590 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:37.730569    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e3c34d5-a322-42bc-97fa-2f8a3477a198-maas-api-tls\") pod \"maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp\" (UID: \"7e3c34d5-a322-42bc-97fa-2f8a3477a198\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp"
Sep 14 03:05:37.730654 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:37.730599    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e3c34d5-a322-42bc-97fa-2f8a3477a198-openshift-service-ca\") pod \"maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp\" (UID: \"7e3c34d5-a322-42bc-97fa-2f8a3477a198\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp"
Sep 14 03:05:37.730654 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:37.730616    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e3c34d5-a322-42bc-97fa-2f8a3477a198-metrics-certs\") pod \"maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp\" (UID: \"7e3c34d5-a322-42bc-97fa-2f8a3477a198\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp"
Sep 14 03:05:37.730735 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:37.730720    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7zls\" (UniqueName: \"kubernetes.io/projected/7e3c34d5-a322-42bc-97fa-2f8a3477a198-kube-api-access-x7zls\") pod \"maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp\" (UID: \"7e3c34d5-a322-42bc-97fa-2f8a3477a198\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp"
Sep 14 03:05:37.739610 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:37.739583    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp"]
Sep 14 03:05:37.831418 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:37.831390    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7zls\" (UniqueName: \"kubernetes.io/projected/7e3c34d5-a322-42bc-97fa-2f8a3477a198-kube-api-access-x7zls\") pod \"maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp\" (UID: \"7e3c34d5-a322-42bc-97fa-2f8a3477a198\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp"
Sep 14 03:05:37.831527 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:37.831462    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e3c34d5-a322-42bc-97fa-2f8a3477a198-maas-api-tls\") pod \"maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp\" (UID: \"7e3c34d5-a322-42bc-97fa-2f8a3477a198\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp"
Sep 14 03:05:37.831527 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:37.831480    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e3c34d5-a322-42bc-97fa-2f8a3477a198-openshift-service-ca\") pod \"maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp\" (UID: \"7e3c34d5-a322-42bc-97fa-2f8a3477a198\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp"
Sep 14 03:05:37.831527 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:37.831496    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e3c34d5-a322-42bc-97fa-2f8a3477a198-metrics-certs\") pod \"maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp\" (UID: \"7e3c34d5-a322-42bc-97fa-2f8a3477a198\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp"
Sep 14 03:05:37.831642 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:05:37.831590    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-07ec96b6-serving-cert: secret "maas-api-e2e-migrate-07ec96b6-serving-cert" not found
Sep 14 03:05:37.831683 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:05:37.831654    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e3c34d5-a322-42bc-97fa-2f8a3477a198-maas-api-tls podName:7e3c34d5-a322-42bc-97fa-2f8a3477a198 nodeName:}" failed. No retries permitted until 2026-09-14 03:05:38.331637673 +0000 UTC m=+887.309615698 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7e3c34d5-a322-42bc-97fa-2f8a3477a198-maas-api-tls") pod "maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp" (UID: "7e3c34d5-a322-42bc-97fa-2f8a3477a198") : secret "maas-api-e2e-migrate-07ec96b6-serving-cert" not found
Sep 14 03:05:37.832219 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:37.832201    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e3c34d5-a322-42bc-97fa-2f8a3477a198-openshift-service-ca\") pod \"maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp\" (UID: \"7e3c34d5-a322-42bc-97fa-2f8a3477a198\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp"
Sep 14 03:05:37.833758 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:37.833741    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e3c34d5-a322-42bc-97fa-2f8a3477a198-metrics-certs\") pod \"maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp\" (UID: \"7e3c34d5-a322-42bc-97fa-2f8a3477a198\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp"
Sep 14 03:05:37.841419 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:37.841394    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7zls\" (UniqueName: \"kubernetes.io/projected/7e3c34d5-a322-42bc-97fa-2f8a3477a198-kube-api-access-x7zls\") pod \"maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp\" (UID: \"7e3c34d5-a322-42bc-97fa-2f8a3477a198\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp"
Sep 14 03:05:38.336364 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:38.336332    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e3c34d5-a322-42bc-97fa-2f8a3477a198-maas-api-tls\") pod \"maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp\" (UID: \"7e3c34d5-a322-42bc-97fa-2f8a3477a198\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp"
Sep 14 03:05:38.338830 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:38.338804    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e3c34d5-a322-42bc-97fa-2f8a3477a198-maas-api-tls\") pod \"maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp\" (UID: \"7e3c34d5-a322-42bc-97fa-2f8a3477a198\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp"
Sep 14 03:05:38.362427 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:38.362401    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp"
Sep 14 03:05:38.648506 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:38.648481    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp"]
Sep 14 03:05:38.650820 ip-10-0-141-180 kubenswrapper[2577]: W0914 03:05:38.650791    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e3c34d5_a322_42bc_97fa_2f8a3477a198.slice/crio-6a66f2e9c259e445c8f20b737aac57238f85108cdc3961b62bef7ebdb109f169 WatchSource:0}: Error finding container 6a66f2e9c259e445c8f20b737aac57238f85108cdc3961b62bef7ebdb109f169: Status 404 returned error can't find the container with id 6a66f2e9c259e445c8f20b737aac57238f85108cdc3961b62bef7ebdb109f169
Sep 14 03:05:39.372194 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:39.372114    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp" event={"ID":"7e3c34d5-a322-42bc-97fa-2f8a3477a198","Type":"ContainerStarted","Data":"9e09c942ea4548dfb04956420dec529f54b8e958ae95f7c6309e90f2550ac394"}
Sep 14 03:05:39.372610 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:39.372314    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp"
Sep 14 03:05:39.372610 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:39.372338    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp" event={"ID":"7e3c34d5-a322-42bc-97fa-2f8a3477a198","Type":"ContainerStarted","Data":"6a66f2e9c259e445c8f20b737aac57238f85108cdc3961b62bef7ebdb109f169"}
Sep 14 03:05:39.419136 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:39.419089    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp" podStartSLOduration=1.951002816 podStartE2EDuration="2.419076256s" totalImagesPullingTime="468.07344ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:05:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:05:39.417153326 +0000 UTC m=+888.395131354" watchObservedRunningTime="2026-09-14 03:05:39.419076256 +0000 UTC m=+888.397054284"
Sep 14 03:05:45.383614 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:05:45.383539    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp"
Sep 14 03:06:18.913031 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:18.912994    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp"]
Sep 14 03:06:18.915535 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:18.913238    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp" podUID="7e3c34d5-a322-42bc-97fa-2f8a3477a198" containerName="maas-api" containerID="cri-o://9e09c942ea4548dfb04956420dec529f54b8e958ae95f7c6309e90f2550ac394" gracePeriod=30
Sep 14 03:06:19.186706 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:19.186666    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp"
Sep 14 03:06:19.273530 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:19.273501    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e3c34d5-a322-42bc-97fa-2f8a3477a198-maas-api-tls\") pod \"7e3c34d5-a322-42bc-97fa-2f8a3477a198\" (UID: \"7e3c34d5-a322-42bc-97fa-2f8a3477a198\") "
Sep 14 03:06:19.273701 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:19.273541    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e3c34d5-a322-42bc-97fa-2f8a3477a198-openshift-service-ca\") pod \"7e3c34d5-a322-42bc-97fa-2f8a3477a198\" (UID: \"7e3c34d5-a322-42bc-97fa-2f8a3477a198\") "
Sep 14 03:06:19.273701 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:19.273608    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e3c34d5-a322-42bc-97fa-2f8a3477a198-metrics-certs\") pod \"7e3c34d5-a322-42bc-97fa-2f8a3477a198\" (UID: \"7e3c34d5-a322-42bc-97fa-2f8a3477a198\") "
Sep 14 03:06:19.273701 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:19.273635    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7zls\" (UniqueName: \"kubernetes.io/projected/7e3c34d5-a322-42bc-97fa-2f8a3477a198-kube-api-access-x7zls\") pod \"7e3c34d5-a322-42bc-97fa-2f8a3477a198\" (UID: \"7e3c34d5-a322-42bc-97fa-2f8a3477a198\") "
Sep 14 03:06:19.273980 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:19.273954    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e3c34d5-a322-42bc-97fa-2f8a3477a198-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7e3c34d5-a322-42bc-97fa-2f8a3477a198" (UID: "7e3c34d5-a322-42bc-97fa-2f8a3477a198"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:06:19.275661 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:19.275629    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e3c34d5-a322-42bc-97fa-2f8a3477a198-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7e3c34d5-a322-42bc-97fa-2f8a3477a198" (UID: "7e3c34d5-a322-42bc-97fa-2f8a3477a198"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:06:19.275786 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:19.275706    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e3c34d5-a322-42bc-97fa-2f8a3477a198-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7e3c34d5-a322-42bc-97fa-2f8a3477a198" (UID: "7e3c34d5-a322-42bc-97fa-2f8a3477a198"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:06:19.275786 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:19.275749    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e3c34d5-a322-42bc-97fa-2f8a3477a198-kube-api-access-x7zls" (OuterVolumeSpecName: "kube-api-access-x7zls") pod "7e3c34d5-a322-42bc-97fa-2f8a3477a198" (UID: "7e3c34d5-a322-42bc-97fa-2f8a3477a198"). InnerVolumeSpecName "kube-api-access-x7zls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:06:19.375426 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:19.375391    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e3c34d5-a322-42bc-97fa-2f8a3477a198-maas-api-tls\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:06:19.375426 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:19.375428    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e3c34d5-a322-42bc-97fa-2f8a3477a198-openshift-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:06:19.375624 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:19.375444    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e3c34d5-a322-42bc-97fa-2f8a3477a198-metrics-certs\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:06:19.375624 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:19.375459    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7zls\" (UniqueName: \"kubernetes.io/projected/7e3c34d5-a322-42bc-97fa-2f8a3477a198-kube-api-access-x7zls\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:06:19.613124 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:19.613090    2577 generic.go:356] "Generic (PLEG): container finished" podID="7e3c34d5-a322-42bc-97fa-2f8a3477a198" containerID="9e09c942ea4548dfb04956420dec529f54b8e958ae95f7c6309e90f2550ac394" exitCode=0
Sep 14 03:06:19.613336 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:19.613211    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp"
Sep 14 03:06:19.617876 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:19.617850    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp" event={"ID":"7e3c34d5-a322-42bc-97fa-2f8a3477a198","Type":"ContainerDied","Data":"9e09c942ea4548dfb04956420dec529f54b8e958ae95f7c6309e90f2550ac394"}
Sep 14 03:06:19.618007 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:19.617890    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp" event={"ID":"7e3c34d5-a322-42bc-97fa-2f8a3477a198","Type":"ContainerDied","Data":"6a66f2e9c259e445c8f20b737aac57238f85108cdc3961b62bef7ebdb109f169"}
Sep 14 03:06:19.618007 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:19.617913    2577 scope.go:117] "RemoveContainer" containerID="9e09c942ea4548dfb04956420dec529f54b8e958ae95f7c6309e90f2550ac394"
Sep 14 03:06:19.629951 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:19.629931    2577 scope.go:117] "RemoveContainer" containerID="9e09c942ea4548dfb04956420dec529f54b8e958ae95f7c6309e90f2550ac394"
Sep 14 03:06:19.630260 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:06:19.630238    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e09c942ea4548dfb04956420dec529f54b8e958ae95f7c6309e90f2550ac394\": container with ID starting with 9e09c942ea4548dfb04956420dec529f54b8e958ae95f7c6309e90f2550ac394 not found: ID does not exist" containerID="9e09c942ea4548dfb04956420dec529f54b8e958ae95f7c6309e90f2550ac394"
Sep 14 03:06:19.630376 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:19.630295    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e09c942ea4548dfb04956420dec529f54b8e958ae95f7c6309e90f2550ac394"} err="failed to get container status \"9e09c942ea4548dfb04956420dec529f54b8e958ae95f7c6309e90f2550ac394\": rpc error: code = NotFound desc = could not find container \"9e09c942ea4548dfb04956420dec529f54b8e958ae95f7c6309e90f2550ac394\": container with ID starting with 9e09c942ea4548dfb04956420dec529f54b8e958ae95f7c6309e90f2550ac394 not found: ID does not exist"
Sep 14 03:06:19.641306 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:19.641285    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp"]
Sep 14 03:06:19.645697 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:19.645675    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-07ec96b6-65f5d7844f-cpllp"]
Sep 14 03:06:21.619648 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:21.619606    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e3c34d5-a322-42bc-97fa-2f8a3477a198" path="/var/lib/kubelet/pods/7e3c34d5-a322-42bc-97fa-2f8a3477a198/volumes"
Sep 14 03:06:27.100538 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:27.100501    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv"]
Sep 14 03:06:27.101143 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:27.101122    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e3c34d5-a322-42bc-97fa-2f8a3477a198" containerName="maas-api"
Sep 14 03:06:27.101240 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:27.101145    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e3c34d5-a322-42bc-97fa-2f8a3477a198" containerName="maas-api"
Sep 14 03:06:27.101326 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:27.101247    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e3c34d5-a322-42bc-97fa-2f8a3477a198" containerName="maas-api"
Sep 14 03:06:27.105770 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:27.105744    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv"
Sep 14 03:06:27.108469 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:27.108450    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-89bb5623-serving-cert\""
Sep 14 03:06:27.115845 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:27.115821    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv"]
Sep 14 03:06:27.247382 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:27.247350    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7s89h\" (UniqueName: \"kubernetes.io/projected/241784fb-b4ea-40c5-ac4e-c51fe227e27c-kube-api-access-7s89h\") pod \"maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv\" (UID: \"241784fb-b4ea-40c5-ac4e-c51fe227e27c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv"
Sep 14 03:06:27.247521 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:27.247392    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/241784fb-b4ea-40c5-ac4e-c51fe227e27c-metrics-certs\") pod \"maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv\" (UID: \"241784fb-b4ea-40c5-ac4e-c51fe227e27c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv"
Sep 14 03:06:27.247521 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:27.247420    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/241784fb-b4ea-40c5-ac4e-c51fe227e27c-openshift-service-ca\") pod \"maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv\" (UID: \"241784fb-b4ea-40c5-ac4e-c51fe227e27c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv"
Sep 14 03:06:27.247521 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:27.247485    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/241784fb-b4ea-40c5-ac4e-c51fe227e27c-maas-api-tls\") pod \"maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv\" (UID: \"241784fb-b4ea-40c5-ac4e-c51fe227e27c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv"
Sep 14 03:06:27.348122 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:27.348092    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/241784fb-b4ea-40c5-ac4e-c51fe227e27c-metrics-certs\") pod \"maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv\" (UID: \"241784fb-b4ea-40c5-ac4e-c51fe227e27c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv"
Sep 14 03:06:27.348223 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:27.348133    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/241784fb-b4ea-40c5-ac4e-c51fe227e27c-openshift-service-ca\") pod \"maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv\" (UID: \"241784fb-b4ea-40c5-ac4e-c51fe227e27c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv"
Sep 14 03:06:27.348223 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:27.348154    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/241784fb-b4ea-40c5-ac4e-c51fe227e27c-maas-api-tls\") pod \"maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv\" (UID: \"241784fb-b4ea-40c5-ac4e-c51fe227e27c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv"
Sep 14 03:06:27.348333 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:27.348223    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7s89h\" (UniqueName: \"kubernetes.io/projected/241784fb-b4ea-40c5-ac4e-c51fe227e27c-kube-api-access-7s89h\") pod \"maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv\" (UID: \"241784fb-b4ea-40c5-ac4e-c51fe227e27c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv"
Sep 14 03:06:27.348830 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:27.348808    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/241784fb-b4ea-40c5-ac4e-c51fe227e27c-openshift-service-ca\") pod \"maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv\" (UID: \"241784fb-b4ea-40c5-ac4e-c51fe227e27c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv"
Sep 14 03:06:27.350626 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:27.350580    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/241784fb-b4ea-40c5-ac4e-c51fe227e27c-metrics-certs\") pod \"maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv\" (UID: \"241784fb-b4ea-40c5-ac4e-c51fe227e27c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv"
Sep 14 03:06:27.350777 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:27.350756    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/241784fb-b4ea-40c5-ac4e-c51fe227e27c-maas-api-tls\") pod \"maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv\" (UID: \"241784fb-b4ea-40c5-ac4e-c51fe227e27c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv"
Sep 14 03:06:27.363769 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:27.363751    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7s89h\" (UniqueName: \"kubernetes.io/projected/241784fb-b4ea-40c5-ac4e-c51fe227e27c-kube-api-access-7s89h\") pod \"maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv\" (UID: \"241784fb-b4ea-40c5-ac4e-c51fe227e27c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv"
Sep 14 03:06:27.416958 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:27.416936    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv"
Sep 14 03:06:27.697053 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:27.697025    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv"]
Sep 14 03:06:27.698366 ip-10-0-141-180 kubenswrapper[2577]: W0914 03:06:27.698339    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod241784fb_b4ea_40c5_ac4e_c51fe227e27c.slice/crio-ebf6d1579c9c9bd6055ebb55dd549128c92e54920297b40b47873ff1cfdcb5a5 WatchSource:0}: Error finding container ebf6d1579c9c9bd6055ebb55dd549128c92e54920297b40b47873ff1cfdcb5a5: Status 404 returned error can't find the container with id ebf6d1579c9c9bd6055ebb55dd549128c92e54920297b40b47873ff1cfdcb5a5
Sep 14 03:06:27.699620 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:27.699602    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 03:06:28.665464 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:28.665429    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv" event={"ID":"241784fb-b4ea-40c5-ac4e-c51fe227e27c","Type":"ContainerStarted","Data":"5bb94e36a556aca03fe5a88fe2f0f93fb64d99db8c83191e19cd8301b2a5e29e"}
Sep 14 03:06:28.665863 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:28.665476    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv" event={"ID":"241784fb-b4ea-40c5-ac4e-c51fe227e27c","Type":"ContainerStarted","Data":"ebf6d1579c9c9bd6055ebb55dd549128c92e54920297b40b47873ff1cfdcb5a5"}
Sep 14 03:06:28.665863 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:28.665494    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv"
Sep 14 03:06:28.684645 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:28.684600    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv" podStartSLOduration=1.335162606 podStartE2EDuration="1.684584729s" totalImagesPullingTime="349.422123ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:06:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:06:28.681161867 +0000 UTC m=+937.659139899" watchObservedRunningTime="2026-09-14 03:06:28.684584729 +0000 UTC m=+937.662562757"
Sep 14 03:06:34.675967 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:34.675930    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv"
Sep 14 03:06:49.622581 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:49.622548    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-558d9b8847-hbpwz" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:06:57.493399 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:57.493361    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv"]
Sep 14 03:06:57.493866 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:57.493580    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv" podUID="241784fb-b4ea-40c5-ac4e-c51fe227e27c" containerName="maas-api" containerID="cri-o://5bb94e36a556aca03fe5a88fe2f0f93fb64d99db8c83191e19cd8301b2a5e29e" gracePeriod=30
Sep 14 03:06:57.756846 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:57.756786    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv"
Sep 14 03:06:57.787076 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:57.787046    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7s89h\" (UniqueName: \"kubernetes.io/projected/241784fb-b4ea-40c5-ac4e-c51fe227e27c-kube-api-access-7s89h\") pod \"241784fb-b4ea-40c5-ac4e-c51fe227e27c\" (UID: \"241784fb-b4ea-40c5-ac4e-c51fe227e27c\") "
Sep 14 03:06:57.790751 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:57.787101    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/241784fb-b4ea-40c5-ac4e-c51fe227e27c-metrics-certs\") pod \"241784fb-b4ea-40c5-ac4e-c51fe227e27c\" (UID: \"241784fb-b4ea-40c5-ac4e-c51fe227e27c\") "
Sep 14 03:06:57.790751 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:57.787192    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/241784fb-b4ea-40c5-ac4e-c51fe227e27c-maas-api-tls\") pod \"241784fb-b4ea-40c5-ac4e-c51fe227e27c\" (UID: \"241784fb-b4ea-40c5-ac4e-c51fe227e27c\") "
Sep 14 03:06:57.790751 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:57.787222    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/241784fb-b4ea-40c5-ac4e-c51fe227e27c-openshift-service-ca\") pod \"241784fb-b4ea-40c5-ac4e-c51fe227e27c\" (UID: \"241784fb-b4ea-40c5-ac4e-c51fe227e27c\") "
Sep 14 03:06:57.790751 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:57.787900    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/241784fb-b4ea-40c5-ac4e-c51fe227e27c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "241784fb-b4ea-40c5-ac4e-c51fe227e27c" (UID: "241784fb-b4ea-40c5-ac4e-c51fe227e27c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:06:57.792829 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:57.792800    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/241784fb-b4ea-40c5-ac4e-c51fe227e27c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "241784fb-b4ea-40c5-ac4e-c51fe227e27c" (UID: "241784fb-b4ea-40c5-ac4e-c51fe227e27c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:06:57.792924 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:57.792896    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/241784fb-b4ea-40c5-ac4e-c51fe227e27c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "241784fb-b4ea-40c5-ac4e-c51fe227e27c" (UID: "241784fb-b4ea-40c5-ac4e-c51fe227e27c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:06:57.793244 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:57.793219    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/241784fb-b4ea-40c5-ac4e-c51fe227e27c-kube-api-access-7s89h" (OuterVolumeSpecName: "kube-api-access-7s89h") pod "241784fb-b4ea-40c5-ac4e-c51fe227e27c" (UID: "241784fb-b4ea-40c5-ac4e-c51fe227e27c"). InnerVolumeSpecName "kube-api-access-7s89h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:06:57.834591 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:57.834566    2577 generic.go:356] "Generic (PLEG): container finished" podID="241784fb-b4ea-40c5-ac4e-c51fe227e27c" containerID="5bb94e36a556aca03fe5a88fe2f0f93fb64d99db8c83191e19cd8301b2a5e29e" exitCode=0
Sep 14 03:06:57.834718 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:57.834608    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv"
Sep 14 03:06:57.834718 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:57.834650    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv" event={"ID":"241784fb-b4ea-40c5-ac4e-c51fe227e27c","Type":"ContainerDied","Data":"5bb94e36a556aca03fe5a88fe2f0f93fb64d99db8c83191e19cd8301b2a5e29e"}
Sep 14 03:06:57.834718 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:57.834710    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv" event={"ID":"241784fb-b4ea-40c5-ac4e-c51fe227e27c","Type":"ContainerDied","Data":"ebf6d1579c9c9bd6055ebb55dd549128c92e54920297b40b47873ff1cfdcb5a5"}
Sep 14 03:06:57.834850 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:57.834736    2577 scope.go:117] "RemoveContainer" containerID="5bb94e36a556aca03fe5a88fe2f0f93fb64d99db8c83191e19cd8301b2a5e29e"
Sep 14 03:06:57.845173 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:57.845153    2577 scope.go:117] "RemoveContainer" containerID="5bb94e36a556aca03fe5a88fe2f0f93fb64d99db8c83191e19cd8301b2a5e29e"
Sep 14 03:06:57.845515 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:06:57.845494    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bb94e36a556aca03fe5a88fe2f0f93fb64d99db8c83191e19cd8301b2a5e29e\": container with ID starting with 5bb94e36a556aca03fe5a88fe2f0f93fb64d99db8c83191e19cd8301b2a5e29e not found: ID does not exist" containerID="5bb94e36a556aca03fe5a88fe2f0f93fb64d99db8c83191e19cd8301b2a5e29e"
Sep 14 03:06:57.845608 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:57.845520    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bb94e36a556aca03fe5a88fe2f0f93fb64d99db8c83191e19cd8301b2a5e29e"} err="failed to get container status \"5bb94e36a556aca03fe5a88fe2f0f93fb64d99db8c83191e19cd8301b2a5e29e\": rpc error: code = NotFound desc = could not find container \"5bb94e36a556aca03fe5a88fe2f0f93fb64d99db8c83191e19cd8301b2a5e29e\": container with ID starting with 5bb94e36a556aca03fe5a88fe2f0f93fb64d99db8c83191e19cd8301b2a5e29e not found: ID does not exist"
Sep 14 03:06:57.857255 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:57.857230    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv"]
Sep 14 03:06:57.861528 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:57.861504    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-89bb5623-7bbb55c46c-ftgvv"]
Sep 14 03:06:57.888480 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:57.888459    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7s89h\" (UniqueName: \"kubernetes.io/projected/241784fb-b4ea-40c5-ac4e-c51fe227e27c-kube-api-access-7s89h\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:06:57.888480 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:57.888482    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/241784fb-b4ea-40c5-ac4e-c51fe227e27c-metrics-certs\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:06:57.888604 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:57.888491    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/241784fb-b4ea-40c5-ac4e-c51fe227e27c-maas-api-tls\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:06:57.888604 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:57.888499    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/241784fb-b4ea-40c5-ac4e-c51fe227e27c-openshift-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:06:58.612408 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:58.612375    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-74969cc5f8-nzb2n" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:06:59.618530 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:06:59.618503    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="241784fb-b4ea-40c5-ac4e-c51fe227e27c" path="/var/lib/kubelet/pods/241784fb-b4ea-40c5-ac4e-c51fe227e27c/volumes"
Sep 14 03:07:03.619847 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:03.619812    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-853daece-84cf5d9447-5dbww"]
Sep 14 03:07:03.620413 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:03.620390    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="241784fb-b4ea-40c5-ac4e-c51fe227e27c" containerName="maas-api"
Sep 14 03:07:03.620413 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:03.620414    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="241784fb-b4ea-40c5-ac4e-c51fe227e27c" containerName="maas-api"
Sep 14 03:07:03.620578 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:03.620555    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="241784fb-b4ea-40c5-ac4e-c51fe227e27c" containerName="maas-api"
Sep 14 03:07:03.625095 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:03.625072    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-853daece-84cf5d9447-5dbww"
Sep 14 03:07:03.628618 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:03.628592    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-853daece-serving-cert\""
Sep 14 03:07:03.638972 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:03.638949    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-853daece-84cf5d9447-5dbww"]
Sep 14 03:07:03.745583 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:03.745558    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5eb424a9-40e6-428a-a5bc-6baccd1881a8-maas-api-tls\") pod \"maas-api-e2e-derive-853daece-84cf5d9447-5dbww\" (UID: \"5eb424a9-40e6-428a-a5bc-6baccd1881a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-853daece-84cf5d9447-5dbww"
Sep 14 03:07:03.745715 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:03.745592    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5eb424a9-40e6-428a-a5bc-6baccd1881a8-metrics-certs\") pod \"maas-api-e2e-derive-853daece-84cf5d9447-5dbww\" (UID: \"5eb424a9-40e6-428a-a5bc-6baccd1881a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-853daece-84cf5d9447-5dbww"
Sep 14 03:07:03.745715 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:03.745652    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2dwh\" (UniqueName: \"kubernetes.io/projected/5eb424a9-40e6-428a-a5bc-6baccd1881a8-kube-api-access-l2dwh\") pod \"maas-api-e2e-derive-853daece-84cf5d9447-5dbww\" (UID: \"5eb424a9-40e6-428a-a5bc-6baccd1881a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-853daece-84cf5d9447-5dbww"
Sep 14 03:07:03.745845 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:03.745780    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5eb424a9-40e6-428a-a5bc-6baccd1881a8-openshift-service-ca\") pod \"maas-api-e2e-derive-853daece-84cf5d9447-5dbww\" (UID: \"5eb424a9-40e6-428a-a5bc-6baccd1881a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-853daece-84cf5d9447-5dbww"
Sep 14 03:07:03.846747 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:03.846721    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5eb424a9-40e6-428a-a5bc-6baccd1881a8-maas-api-tls\") pod \"maas-api-e2e-derive-853daece-84cf5d9447-5dbww\" (UID: \"5eb424a9-40e6-428a-a5bc-6baccd1881a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-853daece-84cf5d9447-5dbww"
Sep 14 03:07:03.846863 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:03.846751    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5eb424a9-40e6-428a-a5bc-6baccd1881a8-metrics-certs\") pod \"maas-api-e2e-derive-853daece-84cf5d9447-5dbww\" (UID: \"5eb424a9-40e6-428a-a5bc-6baccd1881a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-853daece-84cf5d9447-5dbww"
Sep 14 03:07:03.846863 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:03.846776    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2dwh\" (UniqueName: \"kubernetes.io/projected/5eb424a9-40e6-428a-a5bc-6baccd1881a8-kube-api-access-l2dwh\") pod \"maas-api-e2e-derive-853daece-84cf5d9447-5dbww\" (UID: \"5eb424a9-40e6-428a-a5bc-6baccd1881a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-853daece-84cf5d9447-5dbww"
Sep 14 03:07:03.846863 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:03.846838    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5eb424a9-40e6-428a-a5bc-6baccd1881a8-openshift-service-ca\") pod \"maas-api-e2e-derive-853daece-84cf5d9447-5dbww\" (UID: \"5eb424a9-40e6-428a-a5bc-6baccd1881a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-853daece-84cf5d9447-5dbww"
Sep 14 03:07:03.847536 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:03.847515    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5eb424a9-40e6-428a-a5bc-6baccd1881a8-openshift-service-ca\") pod \"maas-api-e2e-derive-853daece-84cf5d9447-5dbww\" (UID: \"5eb424a9-40e6-428a-a5bc-6baccd1881a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-853daece-84cf5d9447-5dbww"
Sep 14 03:07:03.849305 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:03.849248    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5eb424a9-40e6-428a-a5bc-6baccd1881a8-metrics-certs\") pod \"maas-api-e2e-derive-853daece-84cf5d9447-5dbww\" (UID: \"5eb424a9-40e6-428a-a5bc-6baccd1881a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-853daece-84cf5d9447-5dbww"
Sep 14 03:07:03.849413 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:03.849311    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5eb424a9-40e6-428a-a5bc-6baccd1881a8-maas-api-tls\") pod \"maas-api-e2e-derive-853daece-84cf5d9447-5dbww\" (UID: \"5eb424a9-40e6-428a-a5bc-6baccd1881a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-853daece-84cf5d9447-5dbww"
Sep 14 03:07:03.855682 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:03.855661    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2dwh\" (UniqueName: \"kubernetes.io/projected/5eb424a9-40e6-428a-a5bc-6baccd1881a8-kube-api-access-l2dwh\") pod \"maas-api-e2e-derive-853daece-84cf5d9447-5dbww\" (UID: \"5eb424a9-40e6-428a-a5bc-6baccd1881a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-853daece-84cf5d9447-5dbww"
Sep 14 03:07:03.938056 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:03.937990    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-853daece-84cf5d9447-5dbww"
Sep 14 03:07:04.227714 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:04.227599    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-853daece-84cf5d9447-5dbww"]
Sep 14 03:07:04.231135 ip-10-0-141-180 kubenswrapper[2577]: W0914 03:07:04.231109    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5eb424a9_40e6_428a_a5bc_6baccd1881a8.slice/crio-a69c3718acce3d64d68e79d119affe416cca215c3592342e6f5cf00fc204aaea WatchSource:0}: Error finding container a69c3718acce3d64d68e79d119affe416cca215c3592342e6f5cf00fc204aaea: Status 404 returned error can't find the container with id a69c3718acce3d64d68e79d119affe416cca215c3592342e6f5cf00fc204aaea
Sep 14 03:07:04.878122 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:04.878093    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-853daece-84cf5d9447-5dbww" event={"ID":"5eb424a9-40e6-428a-a5bc-6baccd1881a8","Type":"ContainerStarted","Data":"a69c3718acce3d64d68e79d119affe416cca215c3592342e6f5cf00fc204aaea"}
Sep 14 03:07:05.884922 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:05.884888    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-853daece-84cf5d9447-5dbww" event={"ID":"5eb424a9-40e6-428a-a5bc-6baccd1881a8","Type":"ContainerStarted","Data":"0df47f4e3004f03449dbd512b91d5f9267877654dd9867313bddf0b812f8d047"}
Sep 14 03:07:05.885332 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:05.884948    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-853daece-84cf5d9447-5dbww"
Sep 14 03:07:05.905533 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:05.905488    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-853daece-84cf5d9447-5dbww" podStartSLOduration=2.276972259 podStartE2EDuration="2.905474357s" totalImagesPullingTime="628.502098ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:07:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:07:05.903518482 +0000 UTC m=+974.881496513" watchObservedRunningTime="2026-09-14 03:07:05.905474357 +0000 UTC m=+974.883452385"
Sep 14 03:07:11.895732 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:11.895703    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-853daece-84cf5d9447-5dbww"
Sep 14 03:07:25.410218 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:25.410142    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e467de-7746bc5466-262fs"]
Sep 14 03:07:25.410751 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:25.410506    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e467de-7746bc5466-262fs" podUID="15a7d6f1-7858-4371-800f-2d7da9308450" containerName="maas-api" containerID="cri-o://07545cb69585f315eecca9474d95cd8538aa0fb4bab1b055e066d5b125e3b8b9" gracePeriod=30
Sep 14 03:07:25.533818 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:25.533785    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-8mn6v"]
Sep 14 03:07:25.534015 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:25.533986    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-8mn6v" podUID="aa380a57-8e96-4c29-bc4a-d317d335297b" containerName="payload-pre-processing" containerID="cri-o://d5026068dc54c823ec1bd8eaeeaa7f11d9d99b1bd0c83e839bf2fccdd1e1133c" gracePeriod=30
Sep 14 03:07:25.796233 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:25.796199    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-8mn6v"
Sep 14 03:07:25.873150 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:25.873120    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e467de-7746bc5466-262fs"
Sep 14 03:07:25.937842 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:25.937810    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tt22w\" (UniqueName: \"kubernetes.io/projected/aa380a57-8e96-4c29-bc4a-d317d335297b-kube-api-access-tt22w\") pod \"aa380a57-8e96-4c29-bc4a-d317d335297b\" (UID: \"aa380a57-8e96-4c29-bc4a-d317d335297b\") "
Sep 14 03:07:25.937987 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:25.937913    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa380a57-8e96-4c29-bc4a-d317d335297b-plugins-config-volume\") pod \"aa380a57-8e96-4c29-bc4a-d317d335297b\" (UID: \"aa380a57-8e96-4c29-bc4a-d317d335297b\") "
Sep 14 03:07:25.938374 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:25.938337    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa380a57-8e96-4c29-bc4a-d317d335297b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "aa380a57-8e96-4c29-bc4a-d317d335297b" (UID: "aa380a57-8e96-4c29-bc4a-d317d335297b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:07:25.940045 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:25.940017    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa380a57-8e96-4c29-bc4a-d317d335297b-kube-api-access-tt22w" (OuterVolumeSpecName: "kube-api-access-tt22w") pod "aa380a57-8e96-4c29-bc4a-d317d335297b" (UID: "aa380a57-8e96-4c29-bc4a-d317d335297b"). InnerVolumeSpecName "kube-api-access-tt22w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:07:26.006344 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:26.006316    2577 generic.go:356] "Generic (PLEG): container finished" podID="aa380a57-8e96-4c29-bc4a-d317d335297b" containerID="d5026068dc54c823ec1bd8eaeeaa7f11d9d99b1bd0c83e839bf2fccdd1e1133c" exitCode=0
Sep 14 03:07:26.006458 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:26.006347    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-8mn6v" event={"ID":"aa380a57-8e96-4c29-bc4a-d317d335297b","Type":"ContainerDied","Data":"d5026068dc54c823ec1bd8eaeeaa7f11d9d99b1bd0c83e839bf2fccdd1e1133c"}
Sep 14 03:07:26.006458 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:26.006370    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-8mn6v"
Sep 14 03:07:26.006458 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:26.006405    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-8mn6v" event={"ID":"aa380a57-8e96-4c29-bc4a-d317d335297b","Type":"ContainerDied","Data":"5ceb631dab5ac3bbd597f0ee367f2366fb0a8f97c60eee4ffe768fc0a39620ef"}
Sep 14 03:07:26.006458 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:26.006422    2577 scope.go:117] "RemoveContainer" containerID="d5026068dc54c823ec1bd8eaeeaa7f11d9d99b1bd0c83e839bf2fccdd1e1133c"
Sep 14 03:07:26.008508 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:26.008483    2577 generic.go:356] "Generic (PLEG): container finished" podID="15a7d6f1-7858-4371-800f-2d7da9308450" containerID="07545cb69585f315eecca9474d95cd8538aa0fb4bab1b055e066d5b125e3b8b9" exitCode=0
Sep 14 03:07:26.008664 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:26.008526    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e467de-7746bc5466-262fs"
Sep 14 03:07:26.008664 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:26.008553    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e467de-7746bc5466-262fs" event={"ID":"15a7d6f1-7858-4371-800f-2d7da9308450","Type":"ContainerDied","Data":"07545cb69585f315eecca9474d95cd8538aa0fb4bab1b055e066d5b125e3b8b9"}
Sep 14 03:07:26.008664 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:26.008599    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e467de-7746bc5466-262fs" event={"ID":"15a7d6f1-7858-4371-800f-2d7da9308450","Type":"ContainerDied","Data":"bfc86a3c8b1598ad409295ed7e76626f27707c2c17045010e522743d73dda4c3"}
Sep 14 03:07:26.017105 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:26.017089    2577 scope.go:117] "RemoveContainer" containerID="d5026068dc54c823ec1bd8eaeeaa7f11d9d99b1bd0c83e839bf2fccdd1e1133c"
Sep 14 03:07:26.017400 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:07:26.017379    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5026068dc54c823ec1bd8eaeeaa7f11d9d99b1bd0c83e839bf2fccdd1e1133c\": container with ID starting with d5026068dc54c823ec1bd8eaeeaa7f11d9d99b1bd0c83e839bf2fccdd1e1133c not found: ID does not exist" containerID="d5026068dc54c823ec1bd8eaeeaa7f11d9d99b1bd0c83e839bf2fccdd1e1133c"
Sep 14 03:07:26.017454 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:26.017407    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5026068dc54c823ec1bd8eaeeaa7f11d9d99b1bd0c83e839bf2fccdd1e1133c"} err="failed to get container status \"d5026068dc54c823ec1bd8eaeeaa7f11d9d99b1bd0c83e839bf2fccdd1e1133c\": rpc error: code = NotFound desc = could not find container \"d5026068dc54c823ec1bd8eaeeaa7f11d9d99b1bd0c83e839bf2fccdd1e1133c\": container with ID starting with d5026068dc54c823ec1bd8eaeeaa7f11d9d99b1bd0c83e839bf2fccdd1e1133c not found: ID does not exist"
Sep 14 03:07:26.017454 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:26.017423    2577 scope.go:117] "RemoveContainer" containerID="07545cb69585f315eecca9474d95cd8538aa0fb4bab1b055e066d5b125e3b8b9"
Sep 14 03:07:26.026619 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:26.026602    2577 scope.go:117] "RemoveContainer" containerID="07545cb69585f315eecca9474d95cd8538aa0fb4bab1b055e066d5b125e3b8b9"
Sep 14 03:07:26.026907 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:07:26.026885    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07545cb69585f315eecca9474d95cd8538aa0fb4bab1b055e066d5b125e3b8b9\": container with ID starting with 07545cb69585f315eecca9474d95cd8538aa0fb4bab1b055e066d5b125e3b8b9 not found: ID does not exist" containerID="07545cb69585f315eecca9474d95cd8538aa0fb4bab1b055e066d5b125e3b8b9"
Sep 14 03:07:26.026987 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:26.026917    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07545cb69585f315eecca9474d95cd8538aa0fb4bab1b055e066d5b125e3b8b9"} err="failed to get container status \"07545cb69585f315eecca9474d95cd8538aa0fb4bab1b055e066d5b125e3b8b9\": rpc error: code = NotFound desc = could not find container \"07545cb69585f315eecca9474d95cd8538aa0fb4bab1b055e066d5b125e3b8b9\": container with ID starting with 07545cb69585f315eecca9474d95cd8538aa0fb4bab1b055e066d5b125e3b8b9 not found: ID does not exist"
Sep 14 03:07:26.031392 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:26.031371    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-8mn6v"]
Sep 14 03:07:26.037956 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:26.037929    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-8mn6v"]
Sep 14 03:07:26.038571 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:26.038546    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15a7d6f1-7858-4371-800f-2d7da9308450-openshift-service-ca\") pod \"15a7d6f1-7858-4371-800f-2d7da9308450\" (UID: \"15a7d6f1-7858-4371-800f-2d7da9308450\") "
Sep 14 03:07:26.038677 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:26.038605    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4l9g\" (UniqueName: \"kubernetes.io/projected/15a7d6f1-7858-4371-800f-2d7da9308450-kube-api-access-b4l9g\") pod \"15a7d6f1-7858-4371-800f-2d7da9308450\" (UID: \"15a7d6f1-7858-4371-800f-2d7da9308450\") "
Sep 14 03:07:26.038677 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:26.038630    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15a7d6f1-7858-4371-800f-2d7da9308450-maas-api-tls\") pod \"15a7d6f1-7858-4371-800f-2d7da9308450\" (UID: \"15a7d6f1-7858-4371-800f-2d7da9308450\") "
Sep 14 03:07:26.038677 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:26.038655    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15a7d6f1-7858-4371-800f-2d7da9308450-metrics-certs\") pod \"15a7d6f1-7858-4371-800f-2d7da9308450\" (UID: \"15a7d6f1-7858-4371-800f-2d7da9308450\") "
Sep 14 03:07:26.038914 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:26.038889    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15a7d6f1-7858-4371-800f-2d7da9308450-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "15a7d6f1-7858-4371-800f-2d7da9308450" (UID: "15a7d6f1-7858-4371-800f-2d7da9308450"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:07:26.039049 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:26.039032    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa380a57-8e96-4c29-bc4a-d317d335297b-plugins-config-volume\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:07:26.039115 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:26.039059    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tt22w\" (UniqueName: \"kubernetes.io/projected/aa380a57-8e96-4c29-bc4a-d317d335297b-kube-api-access-tt22w\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:07:26.039115 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:26.039076    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15a7d6f1-7858-4371-800f-2d7da9308450-openshift-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:07:26.040785 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:26.040759    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15a7d6f1-7858-4371-800f-2d7da9308450-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "15a7d6f1-7858-4371-800f-2d7da9308450" (UID: "15a7d6f1-7858-4371-800f-2d7da9308450"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:07:26.040898 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:26.040759    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15a7d6f1-7858-4371-800f-2d7da9308450-kube-api-access-b4l9g" (OuterVolumeSpecName: "kube-api-access-b4l9g") pod "15a7d6f1-7858-4371-800f-2d7da9308450" (UID: "15a7d6f1-7858-4371-800f-2d7da9308450"). InnerVolumeSpecName "kube-api-access-b4l9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:07:26.040973 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:26.040955    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15a7d6f1-7858-4371-800f-2d7da9308450-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "15a7d6f1-7858-4371-800f-2d7da9308450" (UID: "15a7d6f1-7858-4371-800f-2d7da9308450"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:07:26.140459 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:26.140429    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4l9g\" (UniqueName: \"kubernetes.io/projected/15a7d6f1-7858-4371-800f-2d7da9308450-kube-api-access-b4l9g\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:07:26.140459 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:26.140453    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15a7d6f1-7858-4371-800f-2d7da9308450-maas-api-tls\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:07:26.140459 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:26.140462    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15a7d6f1-7858-4371-800f-2d7da9308450-metrics-certs\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:07:26.342467 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:26.342442    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e467de-7746bc5466-262fs"]
Sep 14 03:07:26.360675 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:26.360651    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e467de-7746bc5466-262fs"]
Sep 14 03:07:27.618686 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:27.618646    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15a7d6f1-7858-4371-800f-2d7da9308450" path="/var/lib/kubelet/pods/15a7d6f1-7858-4371-800f-2d7da9308450/volumes"
Sep 14 03:07:27.619087 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:27.619026    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa380a57-8e96-4c29-bc4a-d317d335297b" path="/var/lib/kubelet/pods/aa380a57-8e96-4c29-bc4a-d317d335297b/volumes"
Sep 14 03:07:32.009523 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:32.009485    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-853daece-84cf5d9447-5dbww"]
Sep 14 03:07:32.009966 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:32.009744    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-853daece-84cf5d9447-5dbww" podUID="5eb424a9-40e6-428a-a5bc-6baccd1881a8" containerName="maas-api" containerID="cri-o://0df47f4e3004f03449dbd512b91d5f9267877654dd9867313bddf0b812f8d047" gracePeriod=30
Sep 14 03:07:32.265523 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:32.265452    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-853daece-84cf5d9447-5dbww"
Sep 14 03:07:32.291715 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:32.291683    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5eb424a9-40e6-428a-a5bc-6baccd1881a8-metrics-certs\") pod \"5eb424a9-40e6-428a-a5bc-6baccd1881a8\" (UID: \"5eb424a9-40e6-428a-a5bc-6baccd1881a8\") "
Sep 14 03:07:32.291844 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:32.291756    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5eb424a9-40e6-428a-a5bc-6baccd1881a8-maas-api-tls\") pod \"5eb424a9-40e6-428a-a5bc-6baccd1881a8\" (UID: \"5eb424a9-40e6-428a-a5bc-6baccd1881a8\") "
Sep 14 03:07:32.291844 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:32.291788    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l2dwh\" (UniqueName: \"kubernetes.io/projected/5eb424a9-40e6-428a-a5bc-6baccd1881a8-kube-api-access-l2dwh\") pod \"5eb424a9-40e6-428a-a5bc-6baccd1881a8\" (UID: \"5eb424a9-40e6-428a-a5bc-6baccd1881a8\") "
Sep 14 03:07:32.291844 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:32.291828    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5eb424a9-40e6-428a-a5bc-6baccd1881a8-openshift-service-ca\") pod \"5eb424a9-40e6-428a-a5bc-6baccd1881a8\" (UID: \"5eb424a9-40e6-428a-a5bc-6baccd1881a8\") "
Sep 14 03:07:32.292530 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:32.292499    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5eb424a9-40e6-428a-a5bc-6baccd1881a8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5eb424a9-40e6-428a-a5bc-6baccd1881a8" (UID: "5eb424a9-40e6-428a-a5bc-6baccd1881a8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:07:32.294518 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:32.294478    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5eb424a9-40e6-428a-a5bc-6baccd1881a8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5eb424a9-40e6-428a-a5bc-6baccd1881a8" (UID: "5eb424a9-40e6-428a-a5bc-6baccd1881a8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:07:32.294713 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:32.294685    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5eb424a9-40e6-428a-a5bc-6baccd1881a8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5eb424a9-40e6-428a-a5bc-6baccd1881a8" (UID: "5eb424a9-40e6-428a-a5bc-6baccd1881a8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:07:32.294890 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:32.294804    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5eb424a9-40e6-428a-a5bc-6baccd1881a8-kube-api-access-l2dwh" (OuterVolumeSpecName: "kube-api-access-l2dwh") pod "5eb424a9-40e6-428a-a5bc-6baccd1881a8" (UID: "5eb424a9-40e6-428a-a5bc-6baccd1881a8"). InnerVolumeSpecName "kube-api-access-l2dwh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:07:32.393345 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:32.393314    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5eb424a9-40e6-428a-a5bc-6baccd1881a8-maas-api-tls\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:07:32.393345 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:32.393341    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2dwh\" (UniqueName: \"kubernetes.io/projected/5eb424a9-40e6-428a-a5bc-6baccd1881a8-kube-api-access-l2dwh\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:07:32.393345 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:32.393351    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5eb424a9-40e6-428a-a5bc-6baccd1881a8-openshift-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:07:32.393533 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:32.393360    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5eb424a9-40e6-428a-a5bc-6baccd1881a8-metrics-certs\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:07:33.057607 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:33.057577    2577 generic.go:356] "Generic (PLEG): container finished" podID="5eb424a9-40e6-428a-a5bc-6baccd1881a8" containerID="0df47f4e3004f03449dbd512b91d5f9267877654dd9867313bddf0b812f8d047" exitCode=0
Sep 14 03:07:33.057607 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:33.057612    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-853daece-84cf5d9447-5dbww" event={"ID":"5eb424a9-40e6-428a-a5bc-6baccd1881a8","Type":"ContainerDied","Data":"0df47f4e3004f03449dbd512b91d5f9267877654dd9867313bddf0b812f8d047"}
Sep 14 03:07:33.058127 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:33.057624    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-853daece-84cf5d9447-5dbww"
Sep 14 03:07:33.058127 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:33.057654    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-853daece-84cf5d9447-5dbww" event={"ID":"5eb424a9-40e6-428a-a5bc-6baccd1881a8","Type":"ContainerDied","Data":"a69c3718acce3d64d68e79d119affe416cca215c3592342e6f5cf00fc204aaea"}
Sep 14 03:07:33.058127 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:33.057675    2577 scope.go:117] "RemoveContainer" containerID="0df47f4e3004f03449dbd512b91d5f9267877654dd9867313bddf0b812f8d047"
Sep 14 03:07:33.067521 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:33.067503    2577 scope.go:117] "RemoveContainer" containerID="0df47f4e3004f03449dbd512b91d5f9267877654dd9867313bddf0b812f8d047"
Sep 14 03:07:33.067819 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:07:33.067799    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0df47f4e3004f03449dbd512b91d5f9267877654dd9867313bddf0b812f8d047\": container with ID starting with 0df47f4e3004f03449dbd512b91d5f9267877654dd9867313bddf0b812f8d047 not found: ID does not exist" containerID="0df47f4e3004f03449dbd512b91d5f9267877654dd9867313bddf0b812f8d047"
Sep 14 03:07:33.067890 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:33.067832    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0df47f4e3004f03449dbd512b91d5f9267877654dd9867313bddf0b812f8d047"} err="failed to get container status \"0df47f4e3004f03449dbd512b91d5f9267877654dd9867313bddf0b812f8d047\": rpc error: code = NotFound desc = could not find container \"0df47f4e3004f03449dbd512b91d5f9267877654dd9867313bddf0b812f8d047\": container with ID starting with 0df47f4e3004f03449dbd512b91d5f9267877654dd9867313bddf0b812f8d047 not found: ID does not exist"
Sep 14 03:07:33.090831 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:33.090765    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-853daece-84cf5d9447-5dbww"]
Sep 14 03:07:33.093327 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:33.093304    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-853daece-84cf5d9447-5dbww"]
Sep 14 03:07:33.617782 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:33.617749    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5eb424a9-40e6-428a-a5bc-6baccd1881a8" path="/var/lib/kubelet/pods/5eb424a9-40e6-428a-a5bc-6baccd1881a8/volumes"
Sep 14 03:07:45.606493 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:45.606457    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl"]
Sep 14 03:07:45.606974 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:45.606702    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl" podUID="4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9" containerName="maas-api" containerID="cri-o://a5fcc6ecce7213e034fc95d0a221eb5f22227ddc2275ce463207c8e0de10a6cf" gracePeriod=30
Sep 14 03:07:45.854650 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:45.854621    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl"
Sep 14 03:07:45.907502 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:45.907440    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dsqqz\" (UniqueName: \"kubernetes.io/projected/4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9-kube-api-access-dsqqz\") pod \"4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9\" (UID: \"4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9\") "
Sep 14 03:07:45.907611 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:45.907504    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9-metrics-certs\") pod \"4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9\" (UID: \"4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9\") "
Sep 14 03:07:45.907611 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:45.907523    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9-openshift-service-ca\") pod \"4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9\" (UID: \"4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9\") "
Sep 14 03:07:45.907611 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:45.907551    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9-maas-api-tls\") pod \"4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9\" (UID: \"4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9\") "
Sep 14 03:07:45.907926 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:45.907891    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9" (UID: "4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:07:45.909741 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:45.909704    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9" (UID: "4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:07:45.909843 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:45.909808    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9" (UID: "4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:07:45.910021 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:45.909998    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9-kube-api-access-dsqqz" (OuterVolumeSpecName: "kube-api-access-dsqqz") pod "4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9" (UID: "4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9"). InnerVolumeSpecName "kube-api-access-dsqqz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:07:46.008471 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:46.008436    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dsqqz\" (UniqueName: \"kubernetes.io/projected/4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9-kube-api-access-dsqqz\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:07:46.008696 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:46.008479    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9-metrics-certs\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:07:46.008696 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:46.008496    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9-openshift-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:07:46.008696 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:46.008510    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9-maas-api-tls\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:07:46.135846 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:46.135811    2577 generic.go:356] "Generic (PLEG): container finished" podID="4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9" containerID="a5fcc6ecce7213e034fc95d0a221eb5f22227ddc2275ce463207c8e0de10a6cf" exitCode=0
Sep 14 03:07:46.136017 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:46.135854    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl"
Sep 14 03:07:46.136017 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:46.135897    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl" event={"ID":"4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9","Type":"ContainerDied","Data":"a5fcc6ecce7213e034fc95d0a221eb5f22227ddc2275ce463207c8e0de10a6cf"}
Sep 14 03:07:46.136017 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:46.135952    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl" event={"ID":"4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9","Type":"ContainerDied","Data":"61ca711c5e167d191243ba415fb9e3177e2086ebe6be1fabe47a86dd1858d6e4"}
Sep 14 03:07:46.136017 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:46.135975    2577 scope.go:117] "RemoveContainer" containerID="a5fcc6ecce7213e034fc95d0a221eb5f22227ddc2275ce463207c8e0de10a6cf"
Sep 14 03:07:46.145962 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:46.145946    2577 scope.go:117] "RemoveContainer" containerID="a5fcc6ecce7213e034fc95d0a221eb5f22227ddc2275ce463207c8e0de10a6cf"
Sep 14 03:07:46.146235 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:07:46.146216    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5fcc6ecce7213e034fc95d0a221eb5f22227ddc2275ce463207c8e0de10a6cf\": container with ID starting with a5fcc6ecce7213e034fc95d0a221eb5f22227ddc2275ce463207c8e0de10a6cf not found: ID does not exist" containerID="a5fcc6ecce7213e034fc95d0a221eb5f22227ddc2275ce463207c8e0de10a6cf"
Sep 14 03:07:46.146310 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:46.146244    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5fcc6ecce7213e034fc95d0a221eb5f22227ddc2275ce463207c8e0de10a6cf"} err="failed to get container status \"a5fcc6ecce7213e034fc95d0a221eb5f22227ddc2275ce463207c8e0de10a6cf\": rpc error: code = NotFound desc = could not find container \"a5fcc6ecce7213e034fc95d0a221eb5f22227ddc2275ce463207c8e0de10a6cf\": container with ID starting with a5fcc6ecce7213e034fc95d0a221eb5f22227ddc2275ce463207c8e0de10a6cf not found: ID does not exist"
Sep 14 03:07:46.159080 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:46.159027    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl"]
Sep 14 03:07:46.162896 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:46.162876    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-bc1117-6f9cc6d655-8czgl"]
Sep 14 03:07:47.617847 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:47.617810    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9" path="/var/lib/kubelet/pods/4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9/volumes"
Sep 14 03:07:57.833832 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:57.833793    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl"]
Sep 14 03:07:57.834452 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:57.834370    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5eb424a9-40e6-428a-a5bc-6baccd1881a8" containerName="maas-api"
Sep 14 03:07:57.834452 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:57.834387    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5eb424a9-40e6-428a-a5bc-6baccd1881a8" containerName="maas-api"
Sep 14 03:07:57.834452 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:57.834397    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9" containerName="maas-api"
Sep 14 03:07:57.834452 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:57.834403    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9" containerName="maas-api"
Sep 14 03:07:57.834452 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:57.834420    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa380a57-8e96-4c29-bc4a-d317d335297b" containerName="payload-pre-processing"
Sep 14 03:07:57.834452 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:57.834425    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa380a57-8e96-4c29-bc4a-d317d335297b" containerName="payload-pre-processing"
Sep 14 03:07:57.834452 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:57.834437    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15a7d6f1-7858-4371-800f-2d7da9308450" containerName="maas-api"
Sep 14 03:07:57.834452 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:57.834441    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="15a7d6f1-7858-4371-800f-2d7da9308450" containerName="maas-api"
Sep 14 03:07:57.834900 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:57.834505    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa380a57-8e96-4c29-bc4a-d317d335297b" containerName="payload-pre-processing"
Sep 14 03:07:57.834900 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:57.834513    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cdebe1c-131d-4e07-b6cb-d4df32d4c7b9" containerName="maas-api"
Sep 14 03:07:57.834900 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:57.834519    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5eb424a9-40e6-428a-a5bc-6baccd1881a8" containerName="maas-api"
Sep 14 03:07:57.834900 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:57.834524    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="15a7d6f1-7858-4371-800f-2d7da9308450" containerName="maas-api"
Sep 14 03:07:57.839058 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:57.839036    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl"
Sep 14 03:07:57.841603 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:57.841579    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-0c919e-serving-cert\""
Sep 14 03:07:57.850608 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:57.850578    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl"]
Sep 14 03:07:57.905272 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:57.905249    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f053a8e7-a7fb-49da-acf4-c0c643c8ee34-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl\" (UID: \"f053a8e7-a7fb-49da-acf4-c0c643c8ee34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl"
Sep 14 03:07:57.905396 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:57.905371    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwhfq\" (UniqueName: \"kubernetes.io/projected/f053a8e7-a7fb-49da-acf4-c0c643c8ee34-kube-api-access-cwhfq\") pod \"maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl\" (UID: \"f053a8e7-a7fb-49da-acf4-c0c643c8ee34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl"
Sep 14 03:07:57.905466 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:57.905456    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f053a8e7-a7fb-49da-acf4-c0c643c8ee34-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl\" (UID: \"f053a8e7-a7fb-49da-acf4-c0c643c8ee34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl"
Sep 14 03:07:57.905510 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:57.905489    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f053a8e7-a7fb-49da-acf4-c0c643c8ee34-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl\" (UID: \"f053a8e7-a7fb-49da-acf4-c0c643c8ee34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl"
Sep 14 03:07:58.006860 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:58.006830    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwhfq\" (UniqueName: \"kubernetes.io/projected/f053a8e7-a7fb-49da-acf4-c0c643c8ee34-kube-api-access-cwhfq\") pod \"maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl\" (UID: \"f053a8e7-a7fb-49da-acf4-c0c643c8ee34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl"
Sep 14 03:07:58.006979 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:58.006884    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f053a8e7-a7fb-49da-acf4-c0c643c8ee34-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl\" (UID: \"f053a8e7-a7fb-49da-acf4-c0c643c8ee34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl"
Sep 14 03:07:58.006979 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:58.006909    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f053a8e7-a7fb-49da-acf4-c0c643c8ee34-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl\" (UID: \"f053a8e7-a7fb-49da-acf4-c0c643c8ee34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl"
Sep 14 03:07:58.006979 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:58.006933    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f053a8e7-a7fb-49da-acf4-c0c643c8ee34-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl\" (UID: \"f053a8e7-a7fb-49da-acf4-c0c643c8ee34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl"
Sep 14 03:07:58.007092 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:07:58.007048    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0c919e-serving-cert: secret "maas-api-e2e-shared-a-w2-0c919e-serving-cert" not found
Sep 14 03:07:58.007144 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:07:58.007131    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f053a8e7-a7fb-49da-acf4-c0c643c8ee34-maas-api-tls podName:f053a8e7-a7fb-49da-acf4-c0c643c8ee34 nodeName:}" failed. No retries permitted until 2026-09-14 03:07:58.507110862 +0000 UTC m=+1027.485088889 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f053a8e7-a7fb-49da-acf4-c0c643c8ee34-maas-api-tls") pod "maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl" (UID: "f053a8e7-a7fb-49da-acf4-c0c643c8ee34") : secret "maas-api-e2e-shared-a-w2-0c919e-serving-cert" not found
Sep 14 03:07:58.007537 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:58.007517    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f053a8e7-a7fb-49da-acf4-c0c643c8ee34-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl\" (UID: \"f053a8e7-a7fb-49da-acf4-c0c643c8ee34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl"
Sep 14 03:07:58.009485 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:58.009462    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f053a8e7-a7fb-49da-acf4-c0c643c8ee34-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl\" (UID: \"f053a8e7-a7fb-49da-acf4-c0c643c8ee34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl"
Sep 14 03:07:58.021177 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:58.021155    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwhfq\" (UniqueName: \"kubernetes.io/projected/f053a8e7-a7fb-49da-acf4-c0c643c8ee34-kube-api-access-cwhfq\") pod \"maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl\" (UID: \"f053a8e7-a7fb-49da-acf4-c0c643c8ee34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl"
Sep 14 03:07:58.512317 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:58.512284    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f053a8e7-a7fb-49da-acf4-c0c643c8ee34-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl\" (UID: \"f053a8e7-a7fb-49da-acf4-c0c643c8ee34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl"
Sep 14 03:07:58.514651 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:58.514628    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f053a8e7-a7fb-49da-acf4-c0c643c8ee34-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl\" (UID: \"f053a8e7-a7fb-49da-acf4-c0c643c8ee34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl"
Sep 14 03:07:58.752711 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:58.752676    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl"
Sep 14 03:07:59.024519 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:59.024476    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl"]
Sep 14 03:07:59.025892 ip-10-0-141-180 kubenswrapper[2577]: W0914 03:07:59.025858    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf053a8e7_a7fb_49da_acf4_c0c643c8ee34.slice/crio-06e5e31fb1ed35ab7686854702ff32e3b8386e975dee815c7d6ba6018f766070 WatchSource:0}: Error finding container 06e5e31fb1ed35ab7686854702ff32e3b8386e975dee815c7d6ba6018f766070: Status 404 returned error can't find the container with id 06e5e31fb1ed35ab7686854702ff32e3b8386e975dee815c7d6ba6018f766070
Sep 14 03:07:59.212344 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:07:59.212311    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl" event={"ID":"f053a8e7-a7fb-49da-acf4-c0c643c8ee34","Type":"ContainerStarted","Data":"06e5e31fb1ed35ab7686854702ff32e3b8386e975dee815c7d6ba6018f766070"}
Sep 14 03:08:00.220215 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:08:00.220155    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl" event={"ID":"f053a8e7-a7fb-49da-acf4-c0c643c8ee34","Type":"ContainerStarted","Data":"248b13798062fd443ead5c11b990c7d77e7e9e24d527792dfcf4b6ddf6997bc7"}
Sep 14 03:08:00.220657 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:08:00.220390    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl"
Sep 14 03:08:00.241470 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:08:00.241408    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl" podStartSLOduration=2.830324193 podStartE2EDuration="3.241392156s" totalImagesPullingTime="411.067963ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:07:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:08:00.238824902 +0000 UTC m=+1029.216802952" watchObservedRunningTime="2026-09-14 03:08:00.241392156 +0000 UTC m=+1029.219370184"
Sep 14 03:08:02.612124 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:08:02.612092    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-558d9b8847-hbpwz" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:08:04.612195 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:08:04.612160    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-74969cc5f8-nzb2n" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:08:06.231418 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:08:06.231389    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl"
Sep 14 03:08:26.498992 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:08:26.498952    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p"]
Sep 14 03:08:26.502821 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:08:26.502799    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p"
Sep 14 03:08:26.505442 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:08:26.505425    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-f83c87-serving-cert\""
Sep 14 03:08:26.513319 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:08:26.513294    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p"]
Sep 14 03:08:26.652212 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:08:26.652180    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17fa1b0c-3ada-4398-815f-b2f1670881ba-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p\" (UID: \"17fa1b0c-3ada-4398-815f-b2f1670881ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p"
Sep 14 03:08:26.652376 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:08:26.652248    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17fa1b0c-3ada-4398-815f-b2f1670881ba-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p\" (UID: \"17fa1b0c-3ada-4398-815f-b2f1670881ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p"
Sep 14 03:08:26.652376 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:08:26.652316    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17fa1b0c-3ada-4398-815f-b2f1670881ba-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p\" (UID: \"17fa1b0c-3ada-4398-815f-b2f1670881ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p"
Sep 14 03:08:26.652491 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:08:26.652377    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8bjf\" (UniqueName: \"kubernetes.io/projected/17fa1b0c-3ada-4398-815f-b2f1670881ba-kube-api-access-r8bjf\") pod \"maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p\" (UID: \"17fa1b0c-3ada-4398-815f-b2f1670881ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p"
Sep 14 03:08:26.753694 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:08:26.753635    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17fa1b0c-3ada-4398-815f-b2f1670881ba-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p\" (UID: \"17fa1b0c-3ada-4398-815f-b2f1670881ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p"
Sep 14 03:08:26.753694 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:08:26.753672    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17fa1b0c-3ada-4398-815f-b2f1670881ba-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p\" (UID: \"17fa1b0c-3ada-4398-815f-b2f1670881ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p"
Sep 14 03:08:26.753842 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:08:26.753706    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8bjf\" (UniqueName: \"kubernetes.io/projected/17fa1b0c-3ada-4398-815f-b2f1670881ba-kube-api-access-r8bjf\") pod \"maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p\" (UID: \"17fa1b0c-3ada-4398-815f-b2f1670881ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p"
Sep 14 03:08:26.753842 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:08:26.753742    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17fa1b0c-3ada-4398-815f-b2f1670881ba-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p\" (UID: \"17fa1b0c-3ada-4398-815f-b2f1670881ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p"
Sep 14 03:08:26.754222 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:08:26.754202    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17fa1b0c-3ada-4398-815f-b2f1670881ba-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p\" (UID: \"17fa1b0c-3ada-4398-815f-b2f1670881ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p"
Sep 14 03:08:26.756122 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:08:26.756104    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17fa1b0c-3ada-4398-815f-b2f1670881ba-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p\" (UID: \"17fa1b0c-3ada-4398-815f-b2f1670881ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p"
Sep 14 03:08:26.756189 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:08:26.756172    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17fa1b0c-3ada-4398-815f-b2f1670881ba-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p\" (UID: \"17fa1b0c-3ada-4398-815f-b2f1670881ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p"
Sep 14 03:08:26.763464 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:08:26.763443    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8bjf\" (UniqueName: \"kubernetes.io/projected/17fa1b0c-3ada-4398-815f-b2f1670881ba-kube-api-access-r8bjf\") pod \"maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p\" (UID: \"17fa1b0c-3ada-4398-815f-b2f1670881ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p"
Sep 14 03:08:26.814762 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:08:26.814738    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p"
Sep 14 03:08:27.099910 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:08:27.099880    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p"]
Sep 14 03:08:27.100667 ip-10-0-141-180 kubenswrapper[2577]: W0914 03:08:27.100644    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17fa1b0c_3ada_4398_815f_b2f1670881ba.slice/crio-31ac84be64110f68320778a0f720aa19ff2ae6bcfb7eae54452805c642d4c374 WatchSource:0}: Error finding container 31ac84be64110f68320778a0f720aa19ff2ae6bcfb7eae54452805c642d4c374: Status 404 returned error can't find the container with id 31ac84be64110f68320778a0f720aa19ff2ae6bcfb7eae54452805c642d4c374
Sep 14 03:08:27.380335 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:08:27.380242    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p" event={"ID":"17fa1b0c-3ada-4398-815f-b2f1670881ba","Type":"ContainerStarted","Data":"31ac84be64110f68320778a0f720aa19ff2ae6bcfb7eae54452805c642d4c374"}
Sep 14 03:08:28.387910 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:08:28.387874    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p" event={"ID":"17fa1b0c-3ada-4398-815f-b2f1670881ba","Type":"ContainerStarted","Data":"eb444d93360a2b35e13379990ace030e844af5e479f42290da400cc23e305c9d"}
Sep 14 03:08:28.388339 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:08:28.388002    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p"
Sep 14 03:08:28.406436 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:08:28.406394    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p" podStartSLOduration=2.079196975 podStartE2EDuration="2.406379718s" totalImagesPullingTime="327.182743ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:08:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:08:28.405591619 +0000 UTC m=+1057.383569651" watchObservedRunningTime="2026-09-14 03:08:28.406379718 +0000 UTC m=+1057.384357747"
Sep 14 03:08:34.399934 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:08:34.399893    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p"
Sep 14 03:09:10.612123 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:10.612085    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-558d9b8847-hbpwz" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:09:15.897143 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:15.897100    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp"]
Sep 14 03:09:15.900925 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:15.900903    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp"
Sep 14 03:09:15.905137 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:15.905116    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-dfbaeab8-serving-cert\""
Sep 14 03:09:15.919903 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:15.919872    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp"]
Sep 14 03:09:15.969623 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:15.969593    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c49dbd5-e035-45a0-af9b-4d378eb24f9d-openshift-service-ca\") pod \"maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp\" (UID: \"3c49dbd5-e035-45a0-af9b-4d378eb24f9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp"
Sep 14 03:09:15.969623 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:15.969622    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c49dbd5-e035-45a0-af9b-4d378eb24f9d-metrics-certs\") pod \"maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp\" (UID: \"3c49dbd5-e035-45a0-af9b-4d378eb24f9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp"
Sep 14 03:09:15.969759 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:15.969656    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c49dbd5-e035-45a0-af9b-4d378eb24f9d-maas-api-tls\") pod \"maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp\" (UID: \"3c49dbd5-e035-45a0-af9b-4d378eb24f9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp"
Sep 14 03:09:15.969759 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:15.969719    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7h7b\" (UniqueName: \"kubernetes.io/projected/3c49dbd5-e035-45a0-af9b-4d378eb24f9d-kube-api-access-f7h7b\") pod \"maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp\" (UID: \"3c49dbd5-e035-45a0-af9b-4d378eb24f9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp"
Sep 14 03:09:16.070502 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:16.070473    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c49dbd5-e035-45a0-af9b-4d378eb24f9d-maas-api-tls\") pod \"maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp\" (UID: \"3c49dbd5-e035-45a0-af9b-4d378eb24f9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp"
Sep 14 03:09:16.070641 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:16.070517    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7h7b\" (UniqueName: \"kubernetes.io/projected/3c49dbd5-e035-45a0-af9b-4d378eb24f9d-kube-api-access-f7h7b\") pod \"maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp\" (UID: \"3c49dbd5-e035-45a0-af9b-4d378eb24f9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp"
Sep 14 03:09:16.070641 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:16.070584    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c49dbd5-e035-45a0-af9b-4d378eb24f9d-openshift-service-ca\") pod \"maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp\" (UID: \"3c49dbd5-e035-45a0-af9b-4d378eb24f9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp"
Sep 14 03:09:16.070641 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:16.070598    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c49dbd5-e035-45a0-af9b-4d378eb24f9d-metrics-certs\") pod \"maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp\" (UID: \"3c49dbd5-e035-45a0-af9b-4d378eb24f9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp"
Sep 14 03:09:16.070641 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:09:16.070630    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-dfbaeab8-serving-cert: secret "maas-api-e2e-mt-dfbaeab8-serving-cert" not found
Sep 14 03:09:16.070871 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:09:16.070736    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c49dbd5-e035-45a0-af9b-4d378eb24f9d-maas-api-tls podName:3c49dbd5-e035-45a0-af9b-4d378eb24f9d nodeName:}" failed. No retries permitted until 2026-09-14 03:09:16.570718811 +0000 UTC m=+1105.548696843 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3c49dbd5-e035-45a0-af9b-4d378eb24f9d-maas-api-tls") pod "maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp" (UID: "3c49dbd5-e035-45a0-af9b-4d378eb24f9d") : secret "maas-api-e2e-mt-dfbaeab8-serving-cert" not found
Sep 14 03:09:16.071284 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:16.071248    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c49dbd5-e035-45a0-af9b-4d378eb24f9d-openshift-service-ca\") pod \"maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp\" (UID: \"3c49dbd5-e035-45a0-af9b-4d378eb24f9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp"
Sep 14 03:09:16.073006 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:16.072987    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c49dbd5-e035-45a0-af9b-4d378eb24f9d-metrics-certs\") pod \"maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp\" (UID: \"3c49dbd5-e035-45a0-af9b-4d378eb24f9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp"
Sep 14 03:09:16.082188 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:16.082167    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7h7b\" (UniqueName: \"kubernetes.io/projected/3c49dbd5-e035-45a0-af9b-4d378eb24f9d-kube-api-access-f7h7b\") pod \"maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp\" (UID: \"3c49dbd5-e035-45a0-af9b-4d378eb24f9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp"
Sep 14 03:09:16.574867 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:16.574833    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c49dbd5-e035-45a0-af9b-4d378eb24f9d-maas-api-tls\") pod \"maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp\" (UID: \"3c49dbd5-e035-45a0-af9b-4d378eb24f9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp"
Sep 14 03:09:16.577215 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:16.577188    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c49dbd5-e035-45a0-af9b-4d378eb24f9d-maas-api-tls\") pod \"maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp\" (UID: \"3c49dbd5-e035-45a0-af9b-4d378eb24f9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp"
Sep 14 03:09:16.811086 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:16.811054    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp"
Sep 14 03:09:17.094502 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:17.094474    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp"]
Sep 14 03:09:17.096602 ip-10-0-141-180 kubenswrapper[2577]: W0914 03:09:17.096562    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c49dbd5_e035_45a0_af9b_4d378eb24f9d.slice/crio-d35ebc76b1fbd396f76f95790812bdb491324691c3490e274f6825ae174a18e4 WatchSource:0}: Error finding container d35ebc76b1fbd396f76f95790812bdb491324691c3490e274f6825ae174a18e4: Status 404 returned error can't find the container with id d35ebc76b1fbd396f76f95790812bdb491324691c3490e274f6825ae174a18e4
Sep 14 03:09:17.673415 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:17.673380    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp" event={"ID":"3c49dbd5-e035-45a0-af9b-4d378eb24f9d","Type":"ContainerStarted","Data":"72a7970b86333e212d05fd3b4f9e01dfbbe06ba27d0cf1f1b9fb074fcf856f65"}
Sep 14 03:09:17.673558 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:17.673437    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp" event={"ID":"3c49dbd5-e035-45a0-af9b-4d378eb24f9d","Type":"ContainerStarted","Data":"d35ebc76b1fbd396f76f95790812bdb491324691c3490e274f6825ae174a18e4"}
Sep 14 03:09:17.673558 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:17.673526    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp"
Sep 14 03:09:17.712521 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:17.712471    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp" podStartSLOduration=2.265707803 podStartE2EDuration="2.712458494s" totalImagesPullingTime="446.750691ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:09:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:09:17.709872786 +0000 UTC m=+1106.687850814" watchObservedRunningTime="2026-09-14 03:09:17.712458494 +0000 UTC m=+1106.690436514"
Sep 14 03:09:23.691860 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:23.691829    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp"
Sep 14 03:09:29.612086 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:29.612052    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-74969cc5f8-nzb2n" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:09:57.895569 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:57.895535    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp"]
Sep 14 03:09:57.896480 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:57.895731    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp" podUID="3c49dbd5-e035-45a0-af9b-4d378eb24f9d" containerName="maas-api" containerID="cri-o://72a7970b86333e212d05fd3b4f9e01dfbbe06ba27d0cf1f1b9fb074fcf856f65" gracePeriod=30
Sep 14 03:09:58.156244 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:58.156190    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp"
Sep 14 03:09:58.309530 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:58.309500    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7h7b\" (UniqueName: \"kubernetes.io/projected/3c49dbd5-e035-45a0-af9b-4d378eb24f9d-kube-api-access-f7h7b\") pod \"3c49dbd5-e035-45a0-af9b-4d378eb24f9d\" (UID: \"3c49dbd5-e035-45a0-af9b-4d378eb24f9d\") "
Sep 14 03:09:58.309690 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:58.309548    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c49dbd5-e035-45a0-af9b-4d378eb24f9d-maas-api-tls\") pod \"3c49dbd5-e035-45a0-af9b-4d378eb24f9d\" (UID: \"3c49dbd5-e035-45a0-af9b-4d378eb24f9d\") "
Sep 14 03:09:58.309690 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:58.309582    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c49dbd5-e035-45a0-af9b-4d378eb24f9d-openshift-service-ca\") pod \"3c49dbd5-e035-45a0-af9b-4d378eb24f9d\" (UID: \"3c49dbd5-e035-45a0-af9b-4d378eb24f9d\") "
Sep 14 03:09:58.309690 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:58.309653    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c49dbd5-e035-45a0-af9b-4d378eb24f9d-metrics-certs\") pod \"3c49dbd5-e035-45a0-af9b-4d378eb24f9d\" (UID: \"3c49dbd5-e035-45a0-af9b-4d378eb24f9d\") "
Sep 14 03:09:58.310058 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:58.310020    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c49dbd5-e035-45a0-af9b-4d378eb24f9d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3c49dbd5-e035-45a0-af9b-4d378eb24f9d" (UID: "3c49dbd5-e035-45a0-af9b-4d378eb24f9d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:09:58.311763 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:58.311740    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c49dbd5-e035-45a0-af9b-4d378eb24f9d-kube-api-access-f7h7b" (OuterVolumeSpecName: "kube-api-access-f7h7b") pod "3c49dbd5-e035-45a0-af9b-4d378eb24f9d" (UID: "3c49dbd5-e035-45a0-af9b-4d378eb24f9d"). InnerVolumeSpecName "kube-api-access-f7h7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:09:58.312037 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:58.312011    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c49dbd5-e035-45a0-af9b-4d378eb24f9d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3c49dbd5-e035-45a0-af9b-4d378eb24f9d" (UID: "3c49dbd5-e035-45a0-af9b-4d378eb24f9d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:09:58.312183 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:58.312097    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c49dbd5-e035-45a0-af9b-4d378eb24f9d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3c49dbd5-e035-45a0-af9b-4d378eb24f9d" (UID: "3c49dbd5-e035-45a0-af9b-4d378eb24f9d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:09:58.411417 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:58.411350    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7h7b\" (UniqueName: \"kubernetes.io/projected/3c49dbd5-e035-45a0-af9b-4d378eb24f9d-kube-api-access-f7h7b\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:09:58.411417 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:58.411374    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c49dbd5-e035-45a0-af9b-4d378eb24f9d-maas-api-tls\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:09:58.411417 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:58.411385    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c49dbd5-e035-45a0-af9b-4d378eb24f9d-openshift-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:09:58.411417 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:58.411396    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c49dbd5-e035-45a0-af9b-4d378eb24f9d-metrics-certs\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:09:58.911943 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:58.911915    2577 generic.go:356] "Generic (PLEG): container finished" podID="3c49dbd5-e035-45a0-af9b-4d378eb24f9d" containerID="72a7970b86333e212d05fd3b4f9e01dfbbe06ba27d0cf1f1b9fb074fcf856f65" exitCode=0
Sep 14 03:09:58.912423 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:58.911956    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp"
Sep 14 03:09:58.912423 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:58.911990    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp" event={"ID":"3c49dbd5-e035-45a0-af9b-4d378eb24f9d","Type":"ContainerDied","Data":"72a7970b86333e212d05fd3b4f9e01dfbbe06ba27d0cf1f1b9fb074fcf856f65"}
Sep 14 03:09:58.912423 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:58.912045    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp" event={"ID":"3c49dbd5-e035-45a0-af9b-4d378eb24f9d","Type":"ContainerDied","Data":"d35ebc76b1fbd396f76f95790812bdb491324691c3490e274f6825ae174a18e4"}
Sep 14 03:09:58.912423 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:58.912068    2577 scope.go:117] "RemoveContainer" containerID="72a7970b86333e212d05fd3b4f9e01dfbbe06ba27d0cf1f1b9fb074fcf856f65"
Sep 14 03:09:58.921594 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:58.921576    2577 scope.go:117] "RemoveContainer" containerID="72a7970b86333e212d05fd3b4f9e01dfbbe06ba27d0cf1f1b9fb074fcf856f65"
Sep 14 03:09:58.921870 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:09:58.921853    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72a7970b86333e212d05fd3b4f9e01dfbbe06ba27d0cf1f1b9fb074fcf856f65\": container with ID starting with 72a7970b86333e212d05fd3b4f9e01dfbbe06ba27d0cf1f1b9fb074fcf856f65 not found: ID does not exist" containerID="72a7970b86333e212d05fd3b4f9e01dfbbe06ba27d0cf1f1b9fb074fcf856f65"
Sep 14 03:09:58.921936 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:58.921877    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72a7970b86333e212d05fd3b4f9e01dfbbe06ba27d0cf1f1b9fb074fcf856f65"} err="failed to get container status \"72a7970b86333e212d05fd3b4f9e01dfbbe06ba27d0cf1f1b9fb074fcf856f65\": rpc error: code = NotFound desc = could not find container \"72a7970b86333e212d05fd3b4f9e01dfbbe06ba27d0cf1f1b9fb074fcf856f65\": container with ID starting with 72a7970b86333e212d05fd3b4f9e01dfbbe06ba27d0cf1f1b9fb074fcf856f65 not found: ID does not exist"
Sep 14 03:09:58.934717 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:58.934692    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp"]
Sep 14 03:09:58.938182 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:58.938160    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dfbaeab8-b7c995bf8-l72pp"]
Sep 14 03:09:59.618583 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:09:59.618547    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c49dbd5-e035-45a0-af9b-4d378eb24f9d" path="/var/lib/kubelet/pods/3c49dbd5-e035-45a0-af9b-4d378eb24f9d/volumes"
Sep 14 03:10:25.794853 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:25.794774    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv"]
Sep 14 03:10:25.795223 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:25.795203    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c49dbd5-e035-45a0-af9b-4d378eb24f9d" containerName="maas-api"
Sep 14 03:10:25.795223 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:25.795215    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c49dbd5-e035-45a0-af9b-4d378eb24f9d" containerName="maas-api"
Sep 14 03:10:25.795346 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:25.795308    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c49dbd5-e035-45a0-af9b-4d378eb24f9d" containerName="maas-api"
Sep 14 03:10:25.798486 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:25.798466    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv"
Sep 14 03:10:25.801142 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:25.801122    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w1-6e87d0-serving-cert\""
Sep 14 03:10:25.813346 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:25.813309    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv"]
Sep 14 03:10:25.843599 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:25.843570    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19b1dd9e-60df-4ebc-99eb-2a75967a7e5d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv\" (UID: \"19b1dd9e-60df-4ebc-99eb-2a75967a7e5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv"
Sep 14 03:10:25.843730 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:25.843614    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kvlg\" (UniqueName: \"kubernetes.io/projected/19b1dd9e-60df-4ebc-99eb-2a75967a7e5d-kube-api-access-2kvlg\") pod \"maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv\" (UID: \"19b1dd9e-60df-4ebc-99eb-2a75967a7e5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv"
Sep 14 03:10:25.843730 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:25.843647    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19b1dd9e-60df-4ebc-99eb-2a75967a7e5d-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv\" (UID: \"19b1dd9e-60df-4ebc-99eb-2a75967a7e5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv"
Sep 14 03:10:25.843730 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:25.843676    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19b1dd9e-60df-4ebc-99eb-2a75967a7e5d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv\" (UID: \"19b1dd9e-60df-4ebc-99eb-2a75967a7e5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv"
Sep 14 03:10:25.945009 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:25.944976    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19b1dd9e-60df-4ebc-99eb-2a75967a7e5d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv\" (UID: \"19b1dd9e-60df-4ebc-99eb-2a75967a7e5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv"
Sep 14 03:10:25.945124 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:25.945093    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19b1dd9e-60df-4ebc-99eb-2a75967a7e5d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv\" (UID: \"19b1dd9e-60df-4ebc-99eb-2a75967a7e5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv"
Sep 14 03:10:25.945167 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:25.945122    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kvlg\" (UniqueName: \"kubernetes.io/projected/19b1dd9e-60df-4ebc-99eb-2a75967a7e5d-kube-api-access-2kvlg\") pod \"maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv\" (UID: \"19b1dd9e-60df-4ebc-99eb-2a75967a7e5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv"
Sep 14 03:10:25.945167 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:25.945162    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19b1dd9e-60df-4ebc-99eb-2a75967a7e5d-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv\" (UID: \"19b1dd9e-60df-4ebc-99eb-2a75967a7e5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv"
Sep 14 03:10:25.945263 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:10:25.945243    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-6e87d0-serving-cert: secret "maas-api-e2e-shared-a-w1-6e87d0-serving-cert" not found
Sep 14 03:10:25.945339 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:10:25.945332    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19b1dd9e-60df-4ebc-99eb-2a75967a7e5d-maas-api-tls podName:19b1dd9e-60df-4ebc-99eb-2a75967a7e5d nodeName:}" failed. No retries permitted until 2026-09-14 03:10:26.445313852 +0000 UTC m=+1175.423291859 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/19b1dd9e-60df-4ebc-99eb-2a75967a7e5d-maas-api-tls") pod "maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv" (UID: "19b1dd9e-60df-4ebc-99eb-2a75967a7e5d") : secret "maas-api-e2e-shared-a-w1-6e87d0-serving-cert" not found
Sep 14 03:10:25.945784 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:25.945760    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19b1dd9e-60df-4ebc-99eb-2a75967a7e5d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv\" (UID: \"19b1dd9e-60df-4ebc-99eb-2a75967a7e5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv"
Sep 14 03:10:25.947658 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:25.947634    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19b1dd9e-60df-4ebc-99eb-2a75967a7e5d-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv\" (UID: \"19b1dd9e-60df-4ebc-99eb-2a75967a7e5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv"
Sep 14 03:10:25.954072 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:25.954052    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kvlg\" (UniqueName: \"kubernetes.io/projected/19b1dd9e-60df-4ebc-99eb-2a75967a7e5d-kube-api-access-2kvlg\") pod \"maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv\" (UID: \"19b1dd9e-60df-4ebc-99eb-2a75967a7e5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv"
Sep 14 03:10:26.448609 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:26.448575    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19b1dd9e-60df-4ebc-99eb-2a75967a7e5d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv\" (UID: \"19b1dd9e-60df-4ebc-99eb-2a75967a7e5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv"
Sep 14 03:10:26.448777 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:10:26.448728    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-6e87d0-serving-cert: secret "maas-api-e2e-shared-a-w1-6e87d0-serving-cert" not found
Sep 14 03:10:26.448832 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:10:26.448795    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19b1dd9e-60df-4ebc-99eb-2a75967a7e5d-maas-api-tls podName:19b1dd9e-60df-4ebc-99eb-2a75967a7e5d nodeName:}" failed. No retries permitted until 2026-09-14 03:10:27.448779846 +0000 UTC m=+1176.426757858 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/19b1dd9e-60df-4ebc-99eb-2a75967a7e5d-maas-api-tls") pod "maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv" (UID: "19b1dd9e-60df-4ebc-99eb-2a75967a7e5d") : secret "maas-api-e2e-shared-a-w1-6e87d0-serving-cert" not found
Sep 14 03:10:27.457431 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:27.457399    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19b1dd9e-60df-4ebc-99eb-2a75967a7e5d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv\" (UID: \"19b1dd9e-60df-4ebc-99eb-2a75967a7e5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv"
Sep 14 03:10:27.459750 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:27.459730    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19b1dd9e-60df-4ebc-99eb-2a75967a7e5d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv\" (UID: \"19b1dd9e-60df-4ebc-99eb-2a75967a7e5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv"
Sep 14 03:10:27.609937 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:27.609904    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv"
Sep 14 03:10:27.884481 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:27.884455    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv"]
Sep 14 03:10:27.886198 ip-10-0-141-180 kubenswrapper[2577]: W0914 03:10:27.886170    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19b1dd9e_60df_4ebc_99eb_2a75967a7e5d.slice/crio-7bb564cfc9f1413187984cbb6647fd4d77795d5bd70fb3674d53f86f3ee1a9e2 WatchSource:0}: Error finding container 7bb564cfc9f1413187984cbb6647fd4d77795d5bd70fb3674d53f86f3ee1a9e2: Status 404 returned error can't find the container with id 7bb564cfc9f1413187984cbb6647fd4d77795d5bd70fb3674d53f86f3ee1a9e2
Sep 14 03:10:28.073720 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:28.073684    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv" event={"ID":"19b1dd9e-60df-4ebc-99eb-2a75967a7e5d","Type":"ContainerStarted","Data":"7bb564cfc9f1413187984cbb6647fd4d77795d5bd70fb3674d53f86f3ee1a9e2"}
Sep 14 03:10:29.081117 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:29.081078    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv" event={"ID":"19b1dd9e-60df-4ebc-99eb-2a75967a7e5d","Type":"ContainerStarted","Data":"734c60f7197339cb5561937b0517e88df43ca7b445e96649f69472fa82b40e67"}
Sep 14 03:10:29.081655 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:29.081147    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv"
Sep 14 03:10:29.104635 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:29.104586    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv" podStartSLOduration=3.750655728 podStartE2EDuration="4.104570143s" totalImagesPullingTime="353.914415ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:10:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:10:29.102034025 +0000 UTC m=+1178.080012055" watchObservedRunningTime="2026-09-14 03:10:29.104570143 +0000 UTC m=+1178.082548222"
Sep 14 03:10:35.091947 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:35.091919    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv"
Sep 14 03:10:38.612582 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:38.612535    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-558d9b8847-hbpwz" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:10:38.613036 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:38.612723    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-74969cc5f8-nzb2n" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:10:55.584671 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:55.584637    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm"]
Sep 14 03:10:55.589571 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:55.589548    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm"
Sep 14 03:10:55.591920 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:55.591898    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w1-9f77a9-serving-cert\""
Sep 14 03:10:55.600745 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:55.600717    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm"]
Sep 14 03:10:55.686016 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:55.685991    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0dfa7ddb-19df-488e-82cc-8766d4a284e6-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm\" (UID: \"0dfa7ddb-19df-488e-82cc-8766d4a284e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm"
Sep 14 03:10:55.686149 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:55.686038    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dfa7ddb-19df-488e-82cc-8766d4a284e6-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm\" (UID: \"0dfa7ddb-19df-488e-82cc-8766d4a284e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm"
Sep 14 03:10:55.686149 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:55.686066    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4f9jw\" (UniqueName: \"kubernetes.io/projected/0dfa7ddb-19df-488e-82cc-8766d4a284e6-kube-api-access-4f9jw\") pod \"maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm\" (UID: \"0dfa7ddb-19df-488e-82cc-8766d4a284e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm"
Sep 14 03:10:55.686259 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:55.686141    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dfa7ddb-19df-488e-82cc-8766d4a284e6-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm\" (UID: \"0dfa7ddb-19df-488e-82cc-8766d4a284e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm"
Sep 14 03:10:55.786908 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:55.786880    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0dfa7ddb-19df-488e-82cc-8766d4a284e6-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm\" (UID: \"0dfa7ddb-19df-488e-82cc-8766d4a284e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm"
Sep 14 03:10:55.787039 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:55.786947    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dfa7ddb-19df-488e-82cc-8766d4a284e6-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm\" (UID: \"0dfa7ddb-19df-488e-82cc-8766d4a284e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm"
Sep 14 03:10:55.787039 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:55.786993    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4f9jw\" (UniqueName: \"kubernetes.io/projected/0dfa7ddb-19df-488e-82cc-8766d4a284e6-kube-api-access-4f9jw\") pod \"maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm\" (UID: \"0dfa7ddb-19df-488e-82cc-8766d4a284e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm"
Sep 14 03:10:55.787039 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:55.787028    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dfa7ddb-19df-488e-82cc-8766d4a284e6-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm\" (UID: \"0dfa7ddb-19df-488e-82cc-8766d4a284e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm"
Sep 14 03:10:55.787991 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:55.787966    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dfa7ddb-19df-488e-82cc-8766d4a284e6-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm\" (UID: \"0dfa7ddb-19df-488e-82cc-8766d4a284e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm"
Sep 14 03:10:55.789581 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:55.789556    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dfa7ddb-19df-488e-82cc-8766d4a284e6-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm\" (UID: \"0dfa7ddb-19df-488e-82cc-8766d4a284e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm"
Sep 14 03:10:55.789665 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:55.789621    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0dfa7ddb-19df-488e-82cc-8766d4a284e6-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm\" (UID: \"0dfa7ddb-19df-488e-82cc-8766d4a284e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm"
Sep 14 03:10:55.798111 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:55.798085    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4f9jw\" (UniqueName: \"kubernetes.io/projected/0dfa7ddb-19df-488e-82cc-8766d4a284e6-kube-api-access-4f9jw\") pod \"maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm\" (UID: \"0dfa7ddb-19df-488e-82cc-8766d4a284e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm"
Sep 14 03:10:55.900428 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:55.900377    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm"
Sep 14 03:10:56.177124 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:56.177094    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm"]
Sep 14 03:10:56.177940 ip-10-0-141-180 kubenswrapper[2577]: W0914 03:10:56.177911    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dfa7ddb_19df_488e_82cc_8766d4a284e6.slice/crio-47615fa523c439d0f9f5c5c359b34ac91627d1984300e46b3e5fe240a33f0679 WatchSource:0}: Error finding container 47615fa523c439d0f9f5c5c359b34ac91627d1984300e46b3e5fe240a33f0679: Status 404 returned error can't find the container with id 47615fa523c439d0f9f5c5c359b34ac91627d1984300e46b3e5fe240a33f0679
Sep 14 03:10:56.233804 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:56.233777    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-grmh9"]
Sep 14 03:10:56.239008 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:56.238986    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-grmh9"
Sep 14 03:10:56.241648 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:56.241619    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w1-9f77a9\""
Sep 14 03:10:56.241735 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:56.241645    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w1-9f77a9-dockercfg-bhhxm\""
Sep 14 03:10:56.243354 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:56.243330    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm" event={"ID":"0dfa7ddb-19df-488e-82cc-8766d4a284e6","Type":"ContainerStarted","Data":"47615fa523c439d0f9f5c5c359b34ac91627d1984300e46b3e5fe240a33f0679"}
Sep 14 03:10:56.248080 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:56.248058    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-grmh9"]
Sep 14 03:10:56.395587 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:56.395555    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90137a4b-ceb7-476a-ad4c-0e5f35035fac-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-grmh9\" (UID: \"90137a4b-ceb7-476a-ad4c-0e5f35035fac\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-grmh9"
Sep 14 03:10:56.395763 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:56.395712    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrz9x\" (UniqueName: \"kubernetes.io/projected/90137a4b-ceb7-476a-ad4c-0e5f35035fac-kube-api-access-jrz9x\") pod \"payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-grmh9\" (UID: \"90137a4b-ceb7-476a-ad4c-0e5f35035fac\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-grmh9"
Sep 14 03:10:56.497172 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:56.497136    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrz9x\" (UniqueName: \"kubernetes.io/projected/90137a4b-ceb7-476a-ad4c-0e5f35035fac-kube-api-access-jrz9x\") pod \"payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-grmh9\" (UID: \"90137a4b-ceb7-476a-ad4c-0e5f35035fac\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-grmh9"
Sep 14 03:10:56.497360 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:56.497234    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90137a4b-ceb7-476a-ad4c-0e5f35035fac-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-grmh9\" (UID: \"90137a4b-ceb7-476a-ad4c-0e5f35035fac\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-grmh9"
Sep 14 03:10:56.497889 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:56.497867    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90137a4b-ceb7-476a-ad4c-0e5f35035fac-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-grmh9\" (UID: \"90137a4b-ceb7-476a-ad4c-0e5f35035fac\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-grmh9"
Sep 14 03:10:56.506166 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:56.506148    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrz9x\" (UniqueName: \"kubernetes.io/projected/90137a4b-ceb7-476a-ad4c-0e5f35035fac-kube-api-access-jrz9x\") pod \"payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-grmh9\" (UID: \"90137a4b-ceb7-476a-ad4c-0e5f35035fac\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-grmh9"
Sep 14 03:10:56.551974 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:56.551954    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-grmh9"
Sep 14 03:10:56.831974 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:56.831947    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-grmh9"]
Sep 14 03:10:56.833155 ip-10-0-141-180 kubenswrapper[2577]: W0914 03:10:56.833125    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90137a4b_ceb7_476a_ad4c_0e5f35035fac.slice/crio-497ea1d097bc8613e8ffbc26a662be25dd5687fd7cc8291db2f369572e9cc2c2 WatchSource:0}: Error finding container 497ea1d097bc8613e8ffbc26a662be25dd5687fd7cc8291db2f369572e9cc2c2: Status 404 returned error can't find the container with id 497ea1d097bc8613e8ffbc26a662be25dd5687fd7cc8291db2f369572e9cc2c2
Sep 14 03:10:57.251115 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:57.251077    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-grmh9" event={"ID":"90137a4b-ceb7-476a-ad4c-0e5f35035fac","Type":"ContainerStarted","Data":"0231cd77b3612e2941a7e07658f4ecbc696df362340cbf08781c543a9a452607"}
Sep 14 03:10:57.251315 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:57.251145    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-grmh9"
Sep 14 03:10:57.251315 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:57.251174    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-grmh9" event={"ID":"90137a4b-ceb7-476a-ad4c-0e5f35035fac","Type":"ContainerStarted","Data":"497ea1d097bc8613e8ffbc26a662be25dd5687fd7cc8291db2f369572e9cc2c2"}
Sep 14 03:10:57.252837 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:57.252804    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm" event={"ID":"0dfa7ddb-19df-488e-82cc-8766d4a284e6","Type":"ContainerStarted","Data":"de9f9688521f40a8bbb6565ccbd7d276ac07f662d4d5089618eda1db55de763e"}
Sep 14 03:10:57.252967 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:57.252885    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm"
Sep 14 03:10:57.268900 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:57.268862    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-grmh9" podStartSLOduration=1.2688489330000001 podStartE2EDuration="1.268848933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:10:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:10:57.267172293 +0000 UTC m=+1206.245150323" watchObservedRunningTime="2026-09-14 03:10:57.268848933 +0000 UTC m=+1206.246826960"
Sep 14 03:10:57.286498 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:10:57.286460    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm" podStartSLOduration=1.961700055 podStartE2EDuration="2.286447773s" totalImagesPullingTime="324.747718ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:10:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:10:57.284286328 +0000 UTC m=+1206.262264354" watchObservedRunningTime="2026-09-14 03:10:57.286447773 +0000 UTC m=+1206.264425802"
Sep 14 03:11:03.264881 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:11:03.264849    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm"
Sep 14 03:11:08.262063 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:11:08.262031    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-grmh9"
Sep 14 03:11:35.183549 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:11:35.183510    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll"]
Sep 14 03:11:35.187217 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:11:35.187201    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll"
Sep 14 03:11:35.190475 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:11:35.190453    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-d6ac6dc8-serving-cert\""
Sep 14 03:11:35.201674 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:11:35.201647    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll"]
Sep 14 03:11:35.325939 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:11:35.325903    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8xwl\" (UniqueName: \"kubernetes.io/projected/3351bf60-fffb-4a15-b29f-c8e6670fbe8e-kube-api-access-n8xwl\") pod \"maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll\" (UID: \"3351bf60-fffb-4a15-b29f-c8e6670fbe8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll"
Sep 14 03:11:35.326234 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:11:35.326202    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3351bf60-fffb-4a15-b29f-c8e6670fbe8e-metrics-certs\") pod \"maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll\" (UID: \"3351bf60-fffb-4a15-b29f-c8e6670fbe8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll"
Sep 14 03:11:35.326367 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:11:35.326252    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3351bf60-fffb-4a15-b29f-c8e6670fbe8e-maas-api-tls\") pod \"maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll\" (UID: \"3351bf60-fffb-4a15-b29f-c8e6670fbe8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll"
Sep 14 03:11:35.326425 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:11:35.326366    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3351bf60-fffb-4a15-b29f-c8e6670fbe8e-openshift-service-ca\") pod \"maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll\" (UID: \"3351bf60-fffb-4a15-b29f-c8e6670fbe8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll"
Sep 14 03:11:35.427724 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:11:35.427696    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3351bf60-fffb-4a15-b29f-c8e6670fbe8e-metrics-certs\") pod \"maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll\" (UID: \"3351bf60-fffb-4a15-b29f-c8e6670fbe8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll"
Sep 14 03:11:35.427825 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:11:35.427731    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3351bf60-fffb-4a15-b29f-c8e6670fbe8e-maas-api-tls\") pod \"maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll\" (UID: \"3351bf60-fffb-4a15-b29f-c8e6670fbe8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll"
Sep 14 03:11:35.427888 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:11:35.427868    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3351bf60-fffb-4a15-b29f-c8e6670fbe8e-openshift-service-ca\") pod \"maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll\" (UID: \"3351bf60-fffb-4a15-b29f-c8e6670fbe8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll"
Sep 14 03:11:35.427988 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:11:35.427973    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8xwl\" (UniqueName: \"kubernetes.io/projected/3351bf60-fffb-4a15-b29f-c8e6670fbe8e-kube-api-access-n8xwl\") pod \"maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll\" (UID: \"3351bf60-fffb-4a15-b29f-c8e6670fbe8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll"
Sep 14 03:11:35.428552 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:11:35.428529    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3351bf60-fffb-4a15-b29f-c8e6670fbe8e-openshift-service-ca\") pod \"maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll\" (UID: \"3351bf60-fffb-4a15-b29f-c8e6670fbe8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll"
Sep 14 03:11:35.430180 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:11:35.430157    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3351bf60-fffb-4a15-b29f-c8e6670fbe8e-metrics-certs\") pod \"maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll\" (UID: \"3351bf60-fffb-4a15-b29f-c8e6670fbe8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll"
Sep 14 03:11:35.430258 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:11:35.430204    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3351bf60-fffb-4a15-b29f-c8e6670fbe8e-maas-api-tls\") pod \"maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll\" (UID: \"3351bf60-fffb-4a15-b29f-c8e6670fbe8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll"
Sep 14 03:11:35.438004 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:11:35.437950    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8xwl\" (UniqueName: \"kubernetes.io/projected/3351bf60-fffb-4a15-b29f-c8e6670fbe8e-kube-api-access-n8xwl\") pod \"maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll\" (UID: \"3351bf60-fffb-4a15-b29f-c8e6670fbe8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll"
Sep 14 03:11:35.498386 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:11:35.498363    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll"
Sep 14 03:11:35.774999 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:11:35.774954    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll"]
Sep 14 03:11:35.776327 ip-10-0-141-180 kubenswrapper[2577]: W0914 03:11:35.776297    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3351bf60_fffb_4a15_b29f_c8e6670fbe8e.slice/crio-e462dd86bfe88e01de1a39bef92dca1dfbd22ba6830ae30bbc49e670ac41f382 WatchSource:0}: Error finding container e462dd86bfe88e01de1a39bef92dca1dfbd22ba6830ae30bbc49e670ac41f382: Status 404 returned error can't find the container with id e462dd86bfe88e01de1a39bef92dca1dfbd22ba6830ae30bbc49e670ac41f382
Sep 14 03:11:35.777772 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:11:35.777751    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 03:11:36.485071 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:11:36.485037    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll" event={"ID":"3351bf60-fffb-4a15-b29f-c8e6670fbe8e","Type":"ContainerStarted","Data":"8f2b5835e87cb39c0e43d27ad83d011b8ec75fd4eb57dcbbe82b3b9686441d36"}
Sep 14 03:11:36.485469 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:11:36.485088    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll" event={"ID":"3351bf60-fffb-4a15-b29f-c8e6670fbe8e","Type":"ContainerStarted","Data":"e462dd86bfe88e01de1a39bef92dca1dfbd22ba6830ae30bbc49e670ac41f382"}
Sep 14 03:11:36.485469 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:11:36.485129    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll"
Sep 14 03:11:36.503528 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:11:36.503480    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll" podStartSLOduration=1.172648614 podStartE2EDuration="1.503461121s" totalImagesPullingTime="330.812507ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:11:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:11:36.501749802 +0000 UTC m=+1245.479727832" watchObservedRunningTime="2026-09-14 03:11:36.503461121 +0000 UTC m=+1245.481439153"
Sep 14 03:11:42.497417 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:11:42.497380    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll"
Sep 14 03:11:42.612113 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:11:42.612079    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-558d9b8847-hbpwz" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:12:06.612188 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:12:06.612149    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-74969cc5f8-nzb2n" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:12:58.612461 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:12:58.612419    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-558d9b8847-hbpwz" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:13:07.617975 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:13:07.617941    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-74969cc5f8-nzb2n" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:13:21.944559 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:13:21.944479    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll"]
Sep 14 03:13:21.945015 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:13:21.944713    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll" podUID="3351bf60-fffb-4a15-b29f-c8e6670fbe8e" containerName="maas-api" containerID="cri-o://8f2b5835e87cb39c0e43d27ad83d011b8ec75fd4eb57dcbbe82b3b9686441d36" gracePeriod=30
Sep 14 03:13:22.126675 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:13:22.126441    2577 generic.go:356] "Generic (PLEG): container finished" podID="3351bf60-fffb-4a15-b29f-c8e6670fbe8e" containerID="8f2b5835e87cb39c0e43d27ad83d011b8ec75fd4eb57dcbbe82b3b9686441d36" exitCode=0
Sep 14 03:13:22.126675 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:13:22.126596    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll" event={"ID":"3351bf60-fffb-4a15-b29f-c8e6670fbe8e","Type":"ContainerDied","Data":"8f2b5835e87cb39c0e43d27ad83d011b8ec75fd4eb57dcbbe82b3b9686441d36"}
Sep 14 03:13:22.203305 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:13:22.203223    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll"
Sep 14 03:13:22.329590 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:13:22.329550    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n8xwl\" (UniqueName: \"kubernetes.io/projected/3351bf60-fffb-4a15-b29f-c8e6670fbe8e-kube-api-access-n8xwl\") pod \"3351bf60-fffb-4a15-b29f-c8e6670fbe8e\" (UID: \"3351bf60-fffb-4a15-b29f-c8e6670fbe8e\") "
Sep 14 03:13:22.329776 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:13:22.329628    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3351bf60-fffb-4a15-b29f-c8e6670fbe8e-metrics-certs\") pod \"3351bf60-fffb-4a15-b29f-c8e6670fbe8e\" (UID: \"3351bf60-fffb-4a15-b29f-c8e6670fbe8e\") "
Sep 14 03:13:22.329776 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:13:22.329660    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3351bf60-fffb-4a15-b29f-c8e6670fbe8e-maas-api-tls\") pod \"3351bf60-fffb-4a15-b29f-c8e6670fbe8e\" (UID: \"3351bf60-fffb-4a15-b29f-c8e6670fbe8e\") "
Sep 14 03:13:22.329776 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:13:22.329704    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3351bf60-fffb-4a15-b29f-c8e6670fbe8e-openshift-service-ca\") pod \"3351bf60-fffb-4a15-b29f-c8e6670fbe8e\" (UID: \"3351bf60-fffb-4a15-b29f-c8e6670fbe8e\") "
Sep 14 03:13:22.330393 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:13:22.330340    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3351bf60-fffb-4a15-b29f-c8e6670fbe8e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3351bf60-fffb-4a15-b29f-c8e6670fbe8e" (UID: "3351bf60-fffb-4a15-b29f-c8e6670fbe8e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:13:22.332641 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:13:22.332605    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3351bf60-fffb-4a15-b29f-c8e6670fbe8e-kube-api-access-n8xwl" (OuterVolumeSpecName: "kube-api-access-n8xwl") pod "3351bf60-fffb-4a15-b29f-c8e6670fbe8e" (UID: "3351bf60-fffb-4a15-b29f-c8e6670fbe8e"). InnerVolumeSpecName "kube-api-access-n8xwl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:13:22.332773 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:13:22.332702    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3351bf60-fffb-4a15-b29f-c8e6670fbe8e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3351bf60-fffb-4a15-b29f-c8e6670fbe8e" (UID: "3351bf60-fffb-4a15-b29f-c8e6670fbe8e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:13:22.332877 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:13:22.332851    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3351bf60-fffb-4a15-b29f-c8e6670fbe8e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3351bf60-fffb-4a15-b29f-c8e6670fbe8e" (UID: "3351bf60-fffb-4a15-b29f-c8e6670fbe8e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:13:22.431225 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:13:22.431193    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8xwl\" (UniqueName: \"kubernetes.io/projected/3351bf60-fffb-4a15-b29f-c8e6670fbe8e-kube-api-access-n8xwl\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:13:22.431225 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:13:22.431224    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3351bf60-fffb-4a15-b29f-c8e6670fbe8e-metrics-certs\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:13:22.431413 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:13:22.431233    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3351bf60-fffb-4a15-b29f-c8e6670fbe8e-maas-api-tls\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:13:22.431413 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:13:22.431241    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3351bf60-fffb-4a15-b29f-c8e6670fbe8e-openshift-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:13:23.133689 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:13:23.133661    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll"
Sep 14 03:13:23.134171 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:13:23.133648    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll" event={"ID":"3351bf60-fffb-4a15-b29f-c8e6670fbe8e","Type":"ContainerDied","Data":"e462dd86bfe88e01de1a39bef92dca1dfbd22ba6830ae30bbc49e670ac41f382"}
Sep 14 03:13:23.134171 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:13:23.133796    2577 scope.go:117] "RemoveContainer" containerID="8f2b5835e87cb39c0e43d27ad83d011b8ec75fd4eb57dcbbe82b3b9686441d36"
Sep 14 03:13:23.156668 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:13:23.156646    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll"]
Sep 14 03:13:23.164510 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:13:23.164490    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d6ac6dc8-fff6797bb-v6xll"]
Sep 14 03:13:23.617969 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:13:23.617936    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3351bf60-fffb-4a15-b29f-c8e6670fbe8e" path="/var/lib/kubelet/pods/3351bf60-fffb-4a15-b29f-c8e6670fbe8e/volumes"
Sep 14 03:14:17.611452 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:14:17.611418    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-74969cc5f8-nzb2n" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:14:22.611737 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:14:22.611704    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-558d9b8847-hbpwz" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:15:09.588039 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:09.587994    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7"]
Sep 14 03:15:09.588779 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:09.588754    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3351bf60-fffb-4a15-b29f-c8e6670fbe8e" containerName="maas-api"
Sep 14 03:15:09.588967 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:09.588946    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3351bf60-fffb-4a15-b29f-c8e6670fbe8e" containerName="maas-api"
Sep 14 03:15:09.589105 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:09.589090    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3351bf60-fffb-4a15-b29f-c8e6670fbe8e" containerName="maas-api"
Sep 14 03:15:09.592602 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:09.592581    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7"
Sep 14 03:15:09.595136 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:09.595110    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-2611af-serving-cert\""
Sep 14 03:15:09.604759 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:09.604737    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7"]
Sep 14 03:15:09.700793 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:09.700761    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58434515-4c3b-477b-87db-1c8a24698cab-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7\" (UID: \"58434515-4c3b-477b-87db-1c8a24698cab\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7"
Sep 14 03:15:09.700957 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:09.700869    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58434515-4c3b-477b-87db-1c8a24698cab-maas-api-tls\") pod \"maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7\" (UID: \"58434515-4c3b-477b-87db-1c8a24698cab\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7"
Sep 14 03:15:09.700957 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:09.700929    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58434515-4c3b-477b-87db-1c8a24698cab-metrics-certs\") pod \"maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7\" (UID: \"58434515-4c3b-477b-87db-1c8a24698cab\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7"
Sep 14 03:15:09.701081 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:09.700962    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhglm\" (UniqueName: \"kubernetes.io/projected/58434515-4c3b-477b-87db-1c8a24698cab-kube-api-access-mhglm\") pod \"maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7\" (UID: \"58434515-4c3b-477b-87db-1c8a24698cab\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7"
Sep 14 03:15:09.801735 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:09.801704    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58434515-4c3b-477b-87db-1c8a24698cab-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7\" (UID: \"58434515-4c3b-477b-87db-1c8a24698cab\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7"
Sep 14 03:15:09.801880 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:09.801757    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58434515-4c3b-477b-87db-1c8a24698cab-maas-api-tls\") pod \"maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7\" (UID: \"58434515-4c3b-477b-87db-1c8a24698cab\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7"
Sep 14 03:15:09.801880 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:09.801795    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58434515-4c3b-477b-87db-1c8a24698cab-metrics-certs\") pod \"maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7\" (UID: \"58434515-4c3b-477b-87db-1c8a24698cab\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7"
Sep 14 03:15:09.801880 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:09.801822    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhglm\" (UniqueName: \"kubernetes.io/projected/58434515-4c3b-477b-87db-1c8a24698cab-kube-api-access-mhglm\") pod \"maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7\" (UID: \"58434515-4c3b-477b-87db-1c8a24698cab\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7"
Sep 14 03:15:09.802356 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:09.802332    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58434515-4c3b-477b-87db-1c8a24698cab-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7\" (UID: \"58434515-4c3b-477b-87db-1c8a24698cab\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7"
Sep 14 03:15:09.804255 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:09.804237    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58434515-4c3b-477b-87db-1c8a24698cab-maas-api-tls\") pod \"maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7\" (UID: \"58434515-4c3b-477b-87db-1c8a24698cab\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7"
Sep 14 03:15:09.804503 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:09.804480    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58434515-4c3b-477b-87db-1c8a24698cab-metrics-certs\") pod \"maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7\" (UID: \"58434515-4c3b-477b-87db-1c8a24698cab\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7"
Sep 14 03:15:09.810065 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:09.810043    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhglm\" (UniqueName: \"kubernetes.io/projected/58434515-4c3b-477b-87db-1c8a24698cab-kube-api-access-mhglm\") pod \"maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7\" (UID: \"58434515-4c3b-477b-87db-1c8a24698cab\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7"
Sep 14 03:15:09.905783 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:09.905728    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7"
Sep 14 03:15:10.196819 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:10.196761    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7"]
Sep 14 03:15:10.197976 ip-10-0-141-180 kubenswrapper[2577]: W0914 03:15:10.197940    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58434515_4c3b_477b_87db_1c8a24698cab.slice/crio-155891aba109eac963f6348ce00023b1e0356b5b525a1b37fdf19081a942d13c WatchSource:0}: Error finding container 155891aba109eac963f6348ce00023b1e0356b5b525a1b37fdf19081a942d13c: Status 404 returned error can't find the container with id 155891aba109eac963f6348ce00023b1e0356b5b525a1b37fdf19081a942d13c
Sep 14 03:15:10.757373 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:10.757333    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7" event={"ID":"58434515-4c3b-477b-87db-1c8a24698cab","Type":"ContainerStarted","Data":"155891aba109eac963f6348ce00023b1e0356b5b525a1b37fdf19081a942d13c"}
Sep 14 03:15:11.764811 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:11.764776    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7" event={"ID":"58434515-4c3b-477b-87db-1c8a24698cab","Type":"ContainerStarted","Data":"6f00ca08f75fd6de7e43466616c0f25fc980aeb064b11ab4ad678c0ee04787d8"}
Sep 14 03:15:11.765177 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:11.764835    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7"
Sep 14 03:15:11.801573 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:11.801529    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7" podStartSLOduration=2.240095646 podStartE2EDuration="2.80151465s" totalImagesPullingTime="561.419004ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:15:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:15:11.799062838 +0000 UTC m=+1460.777040866" watchObservedRunningTime="2026-09-14 03:15:11.80151465 +0000 UTC m=+1460.779492678"
Sep 14 03:15:17.304142 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:17.304103    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-40c586-56dfb44f96-26bgt"]
Sep 14 03:15:17.318104 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:17.317992    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-40c586-56dfb44f96-26bgt"
Sep 14 03:15:17.322020 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:17.321803    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-40c586-serving-cert\""
Sep 14 03:15:17.322320 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:17.322286    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-40c586-56dfb44f96-26bgt"]
Sep 14 03:15:17.370176 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:17.370148    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c2f2433-1a57-44dd-9404-3d677ad1e9dd-metrics-certs\") pod \"maas-api-e2e-api-a-40c586-56dfb44f96-26bgt\" (UID: \"2c2f2433-1a57-44dd-9404-3d677ad1e9dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-40c586-56dfb44f96-26bgt"
Sep 14 03:15:17.370307 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:17.370196    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c2f2433-1a57-44dd-9404-3d677ad1e9dd-maas-api-tls\") pod \"maas-api-e2e-api-a-40c586-56dfb44f96-26bgt\" (UID: \"2c2f2433-1a57-44dd-9404-3d677ad1e9dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-40c586-56dfb44f96-26bgt"
Sep 14 03:15:17.370307 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:17.370290    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5vsj\" (UniqueName: \"kubernetes.io/projected/2c2f2433-1a57-44dd-9404-3d677ad1e9dd-kube-api-access-r5vsj\") pod \"maas-api-e2e-api-a-40c586-56dfb44f96-26bgt\" (UID: \"2c2f2433-1a57-44dd-9404-3d677ad1e9dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-40c586-56dfb44f96-26bgt"
Sep 14 03:15:17.370385 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:17.370314    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c2f2433-1a57-44dd-9404-3d677ad1e9dd-openshift-service-ca\") pod \"maas-api-e2e-api-a-40c586-56dfb44f96-26bgt\" (UID: \"2c2f2433-1a57-44dd-9404-3d677ad1e9dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-40c586-56dfb44f96-26bgt"
Sep 14 03:15:17.471240 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:17.471205    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c2f2433-1a57-44dd-9404-3d677ad1e9dd-openshift-service-ca\") pod \"maas-api-e2e-api-a-40c586-56dfb44f96-26bgt\" (UID: \"2c2f2433-1a57-44dd-9404-3d677ad1e9dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-40c586-56dfb44f96-26bgt"
Sep 14 03:15:17.471390 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:17.471317    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c2f2433-1a57-44dd-9404-3d677ad1e9dd-metrics-certs\") pod \"maas-api-e2e-api-a-40c586-56dfb44f96-26bgt\" (UID: \"2c2f2433-1a57-44dd-9404-3d677ad1e9dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-40c586-56dfb44f96-26bgt"
Sep 14 03:15:17.471390 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:17.471369    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c2f2433-1a57-44dd-9404-3d677ad1e9dd-maas-api-tls\") pod \"maas-api-e2e-api-a-40c586-56dfb44f96-26bgt\" (UID: \"2c2f2433-1a57-44dd-9404-3d677ad1e9dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-40c586-56dfb44f96-26bgt"
Sep 14 03:15:17.471474 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:17.471436    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5vsj\" (UniqueName: \"kubernetes.io/projected/2c2f2433-1a57-44dd-9404-3d677ad1e9dd-kube-api-access-r5vsj\") pod \"maas-api-e2e-api-a-40c586-56dfb44f96-26bgt\" (UID: \"2c2f2433-1a57-44dd-9404-3d677ad1e9dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-40c586-56dfb44f96-26bgt"
Sep 14 03:15:17.471538 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:15:17.471515    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-40c586-serving-cert: secret "maas-api-e2e-api-a-40c586-serving-cert" not found
Sep 14 03:15:17.471597 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:15:17.471587    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c2f2433-1a57-44dd-9404-3d677ad1e9dd-maas-api-tls podName:2c2f2433-1a57-44dd-9404-3d677ad1e9dd nodeName:}" failed. No retries permitted until 2026-09-14 03:15:17.971568997 +0000 UTC m=+1466.949547028 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2c2f2433-1a57-44dd-9404-3d677ad1e9dd-maas-api-tls") pod "maas-api-e2e-api-a-40c586-56dfb44f96-26bgt" (UID: "2c2f2433-1a57-44dd-9404-3d677ad1e9dd") : secret "maas-api-e2e-api-a-40c586-serving-cert" not found
Sep 14 03:15:17.471999 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:17.471980    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c2f2433-1a57-44dd-9404-3d677ad1e9dd-openshift-service-ca\") pod \"maas-api-e2e-api-a-40c586-56dfb44f96-26bgt\" (UID: \"2c2f2433-1a57-44dd-9404-3d677ad1e9dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-40c586-56dfb44f96-26bgt"
Sep 14 03:15:17.473661 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:17.473644    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c2f2433-1a57-44dd-9404-3d677ad1e9dd-metrics-certs\") pod \"maas-api-e2e-api-a-40c586-56dfb44f96-26bgt\" (UID: \"2c2f2433-1a57-44dd-9404-3d677ad1e9dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-40c586-56dfb44f96-26bgt"
Sep 14 03:15:17.483027 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:17.483005    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5vsj\" (UniqueName: \"kubernetes.io/projected/2c2f2433-1a57-44dd-9404-3d677ad1e9dd-kube-api-access-r5vsj\") pod \"maas-api-e2e-api-a-40c586-56dfb44f96-26bgt\" (UID: \"2c2f2433-1a57-44dd-9404-3d677ad1e9dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-40c586-56dfb44f96-26bgt"
Sep 14 03:15:17.775785 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:17.775756    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7"
Sep 14 03:15:17.977884 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:17.977852    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c2f2433-1a57-44dd-9404-3d677ad1e9dd-maas-api-tls\") pod \"maas-api-e2e-api-a-40c586-56dfb44f96-26bgt\" (UID: \"2c2f2433-1a57-44dd-9404-3d677ad1e9dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-40c586-56dfb44f96-26bgt"
Sep 14 03:15:17.978044 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:15:17.978007    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-40c586-serving-cert: secret "maas-api-e2e-api-a-40c586-serving-cert" not found
Sep 14 03:15:17.978095 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:15:17.978085    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c2f2433-1a57-44dd-9404-3d677ad1e9dd-maas-api-tls podName:2c2f2433-1a57-44dd-9404-3d677ad1e9dd nodeName:}" failed. No retries permitted until 2026-09-14 03:15:18.978069322 +0000 UTC m=+1467.956047330 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2c2f2433-1a57-44dd-9404-3d677ad1e9dd-maas-api-tls") pod "maas-api-e2e-api-a-40c586-56dfb44f96-26bgt" (UID: "2c2f2433-1a57-44dd-9404-3d677ad1e9dd") : secret "maas-api-e2e-api-a-40c586-serving-cert" not found
Sep 14 03:15:18.612171 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:18.612133    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-74969cc5f8-nzb2n" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:15:18.986551 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:18.986515    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c2f2433-1a57-44dd-9404-3d677ad1e9dd-maas-api-tls\") pod \"maas-api-e2e-api-a-40c586-56dfb44f96-26bgt\" (UID: \"2c2f2433-1a57-44dd-9404-3d677ad1e9dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-40c586-56dfb44f96-26bgt"
Sep 14 03:15:18.988641 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:18.988614    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c2f2433-1a57-44dd-9404-3d677ad1e9dd-maas-api-tls\") pod \"maas-api-e2e-api-a-40c586-56dfb44f96-26bgt\" (UID: \"2c2f2433-1a57-44dd-9404-3d677ad1e9dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-40c586-56dfb44f96-26bgt"
Sep 14 03:15:19.133486 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:19.133458    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-40c586-56dfb44f96-26bgt"
Sep 14 03:15:19.411463 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:19.411438    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-40c586-56dfb44f96-26bgt"]
Sep 14 03:15:19.413966 ip-10-0-141-180 kubenswrapper[2577]: W0914 03:15:19.413939    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c2f2433_1a57_44dd_9404_3d677ad1e9dd.slice/crio-65b1e62c944b34222e270c58e183256246a347c4624f3ed6ab5f5b1096b0667b WatchSource:0}: Error finding container 65b1e62c944b34222e270c58e183256246a347c4624f3ed6ab5f5b1096b0667b: Status 404 returned error can't find the container with id 65b1e62c944b34222e270c58e183256246a347c4624f3ed6ab5f5b1096b0667b
Sep 14 03:15:19.813696 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:19.813661    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-40c586-56dfb44f96-26bgt" event={"ID":"2c2f2433-1a57-44dd-9404-3d677ad1e9dd","Type":"ContainerStarted","Data":"674efc5507db242838d127226bbbab30162d500d1b38385dac4ac3ecf3299371"}
Sep 14 03:15:19.814046 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:19.813720    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-40c586-56dfb44f96-26bgt" event={"ID":"2c2f2433-1a57-44dd-9404-3d677ad1e9dd","Type":"ContainerStarted","Data":"65b1e62c944b34222e270c58e183256246a347c4624f3ed6ab5f5b1096b0667b"}
Sep 14 03:15:19.814046 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:19.813762    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-40c586-56dfb44f96-26bgt"
Sep 14 03:15:19.834112 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:19.834070    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-40c586-56dfb44f96-26bgt" podStartSLOduration=2.516648075 podStartE2EDuration="2.834054553s" totalImagesPullingTime="317.406478ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:15:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:15:19.831188913 +0000 UTC m=+1468.809166943" watchObservedRunningTime="2026-09-14 03:15:19.834054553 +0000 UTC m=+1468.812032581"
Sep 14 03:15:24.611715 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:24.611678    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-558d9b8847-hbpwz" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:15:25.825889 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:25.825858    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-40c586-56dfb44f96-26bgt"
Sep 14 03:15:44.205436 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:44.205396    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4"]
Sep 14 03:15:44.209632 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:44.209596    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4"
Sep 14 03:15:44.213537 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:44.213519    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-bdf5b3-serving-cert\""
Sep 14 03:15:44.222037 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:44.222015    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4"]
Sep 14 03:15:44.306232 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:44.306210    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02dd5974-c5ea-446d-a79b-258fbdabfb1a-metrics-certs\") pod \"maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4\" (UID: \"02dd5974-c5ea-446d-a79b-258fbdabfb1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4"
Sep 14 03:15:44.306351 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:44.306244    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29xp2\" (UniqueName: \"kubernetes.io/projected/02dd5974-c5ea-446d-a79b-258fbdabfb1a-kube-api-access-29xp2\") pod \"maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4\" (UID: \"02dd5974-c5ea-446d-a79b-258fbdabfb1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4"
Sep 14 03:15:44.306351 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:44.306288    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02dd5974-c5ea-446d-a79b-258fbdabfb1a-maas-api-tls\") pod \"maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4\" (UID: \"02dd5974-c5ea-446d-a79b-258fbdabfb1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4"
Sep 14 03:15:44.306451 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:44.306345    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02dd5974-c5ea-446d-a79b-258fbdabfb1a-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4\" (UID: \"02dd5974-c5ea-446d-a79b-258fbdabfb1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4"
Sep 14 03:15:44.407056 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:44.407022    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02dd5974-c5ea-446d-a79b-258fbdabfb1a-metrics-certs\") pod \"maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4\" (UID: \"02dd5974-c5ea-446d-a79b-258fbdabfb1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4"
Sep 14 03:15:44.407190 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:44.407061    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29xp2\" (UniqueName: \"kubernetes.io/projected/02dd5974-c5ea-446d-a79b-258fbdabfb1a-kube-api-access-29xp2\") pod \"maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4\" (UID: \"02dd5974-c5ea-446d-a79b-258fbdabfb1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4"
Sep 14 03:15:44.407190 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:44.407088    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02dd5974-c5ea-446d-a79b-258fbdabfb1a-maas-api-tls\") pod \"maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4\" (UID: \"02dd5974-c5ea-446d-a79b-258fbdabfb1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4"
Sep 14 03:15:44.407190 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:44.407113    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02dd5974-c5ea-446d-a79b-258fbdabfb1a-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4\" (UID: \"02dd5974-c5ea-446d-a79b-258fbdabfb1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4"
Sep 14 03:15:44.407404 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:15:44.407236    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-bdf5b3-serving-cert: secret "maas-api-e2e-ipp-b-bdf5b3-serving-cert" not found
Sep 14 03:15:44.407404 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:15:44.407335    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02dd5974-c5ea-446d-a79b-258fbdabfb1a-maas-api-tls podName:02dd5974-c5ea-446d-a79b-258fbdabfb1a nodeName:}" failed. No retries permitted until 2026-09-14 03:15:44.907313489 +0000 UTC m=+1493.885291515 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/02dd5974-c5ea-446d-a79b-258fbdabfb1a-maas-api-tls") pod "maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4" (UID: "02dd5974-c5ea-446d-a79b-258fbdabfb1a") : secret "maas-api-e2e-ipp-b-bdf5b3-serving-cert" not found
Sep 14 03:15:44.407749 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:44.407732    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02dd5974-c5ea-446d-a79b-258fbdabfb1a-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4\" (UID: \"02dd5974-c5ea-446d-a79b-258fbdabfb1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4"
Sep 14 03:15:44.409580 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:44.409558    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02dd5974-c5ea-446d-a79b-258fbdabfb1a-metrics-certs\") pod \"maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4\" (UID: \"02dd5974-c5ea-446d-a79b-258fbdabfb1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4"
Sep 14 03:15:44.416835 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:44.416811    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29xp2\" (UniqueName: \"kubernetes.io/projected/02dd5974-c5ea-446d-a79b-258fbdabfb1a-kube-api-access-29xp2\") pod \"maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4\" (UID: \"02dd5974-c5ea-446d-a79b-258fbdabfb1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4"
Sep 14 03:15:44.912153 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:44.912116    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02dd5974-c5ea-446d-a79b-258fbdabfb1a-maas-api-tls\") pod \"maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4\" (UID: \"02dd5974-c5ea-446d-a79b-258fbdabfb1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4"
Sep 14 03:15:44.914604 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:44.914575    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02dd5974-c5ea-446d-a79b-258fbdabfb1a-maas-api-tls\") pod \"maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4\" (UID: \"02dd5974-c5ea-446d-a79b-258fbdabfb1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4"
Sep 14 03:15:45.120830 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:45.120798    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4"
Sep 14 03:15:45.400229 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:45.400205    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4"]
Sep 14 03:15:45.402106 ip-10-0-141-180 kubenswrapper[2577]: W0914 03:15:45.402076    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02dd5974_c5ea_446d_a79b_258fbdabfb1a.slice/crio-30d10e0d553dfa209a3dfcc8ce3e600a3046bb7451bafc7eda13373028b45cfa WatchSource:0}: Error finding container 30d10e0d553dfa209a3dfcc8ce3e600a3046bb7451bafc7eda13373028b45cfa: Status 404 returned error can't find the container with id 30d10e0d553dfa209a3dfcc8ce3e600a3046bb7451bafc7eda13373028b45cfa
Sep 14 03:15:45.990754 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:45.990661    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4" event={"ID":"02dd5974-c5ea-446d-a79b-258fbdabfb1a","Type":"ContainerStarted","Data":"d3ce54f30133ba9cc7a3b63590b9be3439a1f5e41d287b698fad5df2fbf9a863"}
Sep 14 03:15:45.990754 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:45.990724    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4" event={"ID":"02dd5974-c5ea-446d-a79b-258fbdabfb1a","Type":"ContainerStarted","Data":"30d10e0d553dfa209a3dfcc8ce3e600a3046bb7451bafc7eda13373028b45cfa"}
Sep 14 03:15:45.990927 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:45.990764    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4"
Sep 14 03:15:46.029676 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:46.029629    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4" podStartSLOduration=1.6961709740000002 podStartE2EDuration="2.029614242s" totalImagesPullingTime="333.443268ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:15:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:15:46.027066376 +0000 UTC m=+1495.005044404" watchObservedRunningTime="2026-09-14 03:15:46.029614242 +0000 UTC m=+1495.007592271"
Sep 14 03:15:47.323919 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:47.323881    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl"]
Sep 14 03:15:47.327776 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:47.327756    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl"
Sep 14 03:15:47.330411 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:47.330393    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-13d31c-serving-cert\""
Sep 14 03:15:47.345478 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:47.345455    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl"]
Sep 14 03:15:47.434741 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:47.434689    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p8gk\" (UniqueName: \"kubernetes.io/projected/6d05fc80-bd0b-47bf-b255-e4a195c60955-kube-api-access-8p8gk\") pod \"maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl\" (UID: \"6d05fc80-bd0b-47bf-b255-e4a195c60955\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl"
Sep 14 03:15:47.434848 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:47.434768    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d05fc80-bd0b-47bf-b255-e4a195c60955-maas-api-tls\") pod \"maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl\" (UID: \"6d05fc80-bd0b-47bf-b255-e4a195c60955\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl"
Sep 14 03:15:47.434911 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:47.434881    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d05fc80-bd0b-47bf-b255-e4a195c60955-metrics-certs\") pod \"maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl\" (UID: \"6d05fc80-bd0b-47bf-b255-e4a195c60955\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl"
Sep 14 03:15:47.434955 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:47.434910    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d05fc80-bd0b-47bf-b255-e4a195c60955-openshift-service-ca\") pod \"maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl\" (UID: \"6d05fc80-bd0b-47bf-b255-e4a195c60955\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl"
Sep 14 03:15:47.535635 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:47.535611    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d05fc80-bd0b-47bf-b255-e4a195c60955-openshift-service-ca\") pod \"maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl\" (UID: \"6d05fc80-bd0b-47bf-b255-e4a195c60955\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl"
Sep 14 03:15:47.535764 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:47.535679    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p8gk\" (UniqueName: \"kubernetes.io/projected/6d05fc80-bd0b-47bf-b255-e4a195c60955-kube-api-access-8p8gk\") pod \"maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl\" (UID: \"6d05fc80-bd0b-47bf-b255-e4a195c60955\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl"
Sep 14 03:15:47.535764 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:47.535702    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d05fc80-bd0b-47bf-b255-e4a195c60955-maas-api-tls\") pod \"maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl\" (UID: \"6d05fc80-bd0b-47bf-b255-e4a195c60955\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl"
Sep 14 03:15:47.535764 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:47.535739    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d05fc80-bd0b-47bf-b255-e4a195c60955-metrics-certs\") pod \"maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl\" (UID: \"6d05fc80-bd0b-47bf-b255-e4a195c60955\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl"
Sep 14 03:15:47.536459 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:47.536435    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d05fc80-bd0b-47bf-b255-e4a195c60955-openshift-service-ca\") pod \"maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl\" (UID: \"6d05fc80-bd0b-47bf-b255-e4a195c60955\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl"
Sep 14 03:15:47.538229 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:47.538205    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d05fc80-bd0b-47bf-b255-e4a195c60955-metrics-certs\") pod \"maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl\" (UID: \"6d05fc80-bd0b-47bf-b255-e4a195c60955\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl"
Sep 14 03:15:47.538338 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:47.538232    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d05fc80-bd0b-47bf-b255-e4a195c60955-maas-api-tls\") pod \"maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl\" (UID: \"6d05fc80-bd0b-47bf-b255-e4a195c60955\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl"
Sep 14 03:15:47.544487 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:47.544460    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p8gk\" (UniqueName: \"kubernetes.io/projected/6d05fc80-bd0b-47bf-b255-e4a195c60955-kube-api-access-8p8gk\") pod \"maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl\" (UID: \"6d05fc80-bd0b-47bf-b255-e4a195c60955\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl"
Sep 14 03:15:47.637760 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:47.637699    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl"
Sep 14 03:15:47.924477 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:47.924444    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl"]
Sep 14 03:15:47.924875 ip-10-0-141-180 kubenswrapper[2577]: W0914 03:15:47.924850    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d05fc80_bd0b_47bf_b255_e4a195c60955.slice/crio-37c23340064569e8626090cf37106bc19f64d1a89b14bbecae679cf26315ed21 WatchSource:0}: Error finding container 37c23340064569e8626090cf37106bc19f64d1a89b14bbecae679cf26315ed21: Status 404 returned error can't find the container with id 37c23340064569e8626090cf37106bc19f64d1a89b14bbecae679cf26315ed21
Sep 14 03:15:48.002361 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:48.002311    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl" event={"ID":"6d05fc80-bd0b-47bf-b255-e4a195c60955","Type":"ContainerStarted","Data":"37c23340064569e8626090cf37106bc19f64d1a89b14bbecae679cf26315ed21"}
Sep 14 03:15:49.010421 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:49.010385    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl" event={"ID":"6d05fc80-bd0b-47bf-b255-e4a195c60955","Type":"ContainerStarted","Data":"4659e85b1a885fa17c0dc1a5e03bbde4684d02f60b4f64c53f91ca09ba5f480d"}
Sep 14 03:15:49.010843 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:49.010442    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl"
Sep 14 03:15:49.040701 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:49.040661    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl" podStartSLOduration=1.622248667 podStartE2EDuration="2.040645818s" totalImagesPullingTime="418.397151ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:15:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:15:49.038035169 +0000 UTC m=+1498.016013199" watchObservedRunningTime="2026-09-14 03:15:49.040645818 +0000 UTC m=+1498.018623900"
Sep 14 03:15:52.001776 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:52.001745    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4"
Sep 14 03:15:55.021644 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:15:55.021611    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl"
Sep 14 03:16:15.733915 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:15.733835    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-74969cc5f8-nzb2n_02709d9b-915f-4d2b-8bd2-0b73f296f99b/payload-processing/0.log"
Sep 14 03:16:15.860134 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:15.860102    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-74969cc5f8-nzb2n_02709d9b-915f-4d2b-8bd2-0b73f296f99b/payload-processing/0.log"
Sep 14 03:16:22.611781 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:22.611750    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-74969cc5f8-nzb2n" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:16:31.333063 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:31.333026    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-40c586-56dfb44f96-26bgt"]
Sep 14 03:16:31.335685 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:31.333284    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-40c586-56dfb44f96-26bgt" podUID="2c2f2433-1a57-44dd-9404-3d677ad1e9dd" containerName="maas-api" containerID="cri-o://674efc5507db242838d127226bbbab30162d500d1b38385dac4ac3ecf3299371" gracePeriod=30
Sep 14 03:16:31.596780 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:31.596726    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-40c586-56dfb44f96-26bgt"
Sep 14 03:16:31.626408 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:31.626379    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c2f2433-1a57-44dd-9404-3d677ad1e9dd-metrics-certs\") pod \"2c2f2433-1a57-44dd-9404-3d677ad1e9dd\" (UID: \"2c2f2433-1a57-44dd-9404-3d677ad1e9dd\") "
Sep 14 03:16:31.626726 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:31.626703    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5vsj\" (UniqueName: \"kubernetes.io/projected/2c2f2433-1a57-44dd-9404-3d677ad1e9dd-kube-api-access-r5vsj\") pod \"2c2f2433-1a57-44dd-9404-3d677ad1e9dd\" (UID: \"2c2f2433-1a57-44dd-9404-3d677ad1e9dd\") "
Sep 14 03:16:31.626877 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:31.626859    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c2f2433-1a57-44dd-9404-3d677ad1e9dd-openshift-service-ca\") pod \"2c2f2433-1a57-44dd-9404-3d677ad1e9dd\" (UID: \"2c2f2433-1a57-44dd-9404-3d677ad1e9dd\") "
Sep 14 03:16:31.627021 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:31.627007    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c2f2433-1a57-44dd-9404-3d677ad1e9dd-maas-api-tls\") pod \"2c2f2433-1a57-44dd-9404-3d677ad1e9dd\" (UID: \"2c2f2433-1a57-44dd-9404-3d677ad1e9dd\") "
Sep 14 03:16:31.628583 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:31.628556    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c2f2433-1a57-44dd-9404-3d677ad1e9dd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2c2f2433-1a57-44dd-9404-3d677ad1e9dd" (UID: "2c2f2433-1a57-44dd-9404-3d677ad1e9dd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:16:31.629872 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:31.629440    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c2f2433-1a57-44dd-9404-3d677ad1e9dd-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2c2f2433-1a57-44dd-9404-3d677ad1e9dd" (UID: "2c2f2433-1a57-44dd-9404-3d677ad1e9dd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:16:31.630580 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:31.630554    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c2f2433-1a57-44dd-9404-3d677ad1e9dd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2c2f2433-1a57-44dd-9404-3d677ad1e9dd" (UID: "2c2f2433-1a57-44dd-9404-3d677ad1e9dd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:16:31.631590 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:31.631552    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c2f2433-1a57-44dd-9404-3d677ad1e9dd-kube-api-access-r5vsj" (OuterVolumeSpecName: "kube-api-access-r5vsj") pod "2c2f2433-1a57-44dd-9404-3d677ad1e9dd" (UID: "2c2f2433-1a57-44dd-9404-3d677ad1e9dd"). InnerVolumeSpecName "kube-api-access-r5vsj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:16:31.729081 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:31.729050    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5vsj\" (UniqueName: \"kubernetes.io/projected/2c2f2433-1a57-44dd-9404-3d677ad1e9dd-kube-api-access-r5vsj\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:16:31.729081 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:31.729083    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c2f2433-1a57-44dd-9404-3d677ad1e9dd-openshift-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:16:31.729295 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:31.729098    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c2f2433-1a57-44dd-9404-3d677ad1e9dd-maas-api-tls\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:16:31.729295 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:31.729111    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c2f2433-1a57-44dd-9404-3d677ad1e9dd-metrics-certs\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:16:32.282952 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:32.282922    2577 generic.go:356] "Generic (PLEG): container finished" podID="2c2f2433-1a57-44dd-9404-3d677ad1e9dd" containerID="674efc5507db242838d127226bbbab30162d500d1b38385dac4ac3ecf3299371" exitCode=0
Sep 14 03:16:32.283133 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:32.282957    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-40c586-56dfb44f96-26bgt" event={"ID":"2c2f2433-1a57-44dd-9404-3d677ad1e9dd","Type":"ContainerDied","Data":"674efc5507db242838d127226bbbab30162d500d1b38385dac4ac3ecf3299371"}
Sep 14 03:16:32.283133 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:32.282982    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-40c586-56dfb44f96-26bgt"
Sep 14 03:16:32.283133 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:32.282996    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-40c586-56dfb44f96-26bgt" event={"ID":"2c2f2433-1a57-44dd-9404-3d677ad1e9dd","Type":"ContainerDied","Data":"65b1e62c944b34222e270c58e183256246a347c4624f3ed6ab5f5b1096b0667b"}
Sep 14 03:16:32.283133 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:32.283017    2577 scope.go:117] "RemoveContainer" containerID="674efc5507db242838d127226bbbab30162d500d1b38385dac4ac3ecf3299371"
Sep 14 03:16:32.292434 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:32.292414    2577 scope.go:117] "RemoveContainer" containerID="674efc5507db242838d127226bbbab30162d500d1b38385dac4ac3ecf3299371"
Sep 14 03:16:32.292727 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:16:32.292706    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"674efc5507db242838d127226bbbab30162d500d1b38385dac4ac3ecf3299371\": container with ID starting with 674efc5507db242838d127226bbbab30162d500d1b38385dac4ac3ecf3299371 not found: ID does not exist" containerID="674efc5507db242838d127226bbbab30162d500d1b38385dac4ac3ecf3299371"
Sep 14 03:16:32.292802 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:32.292743    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"674efc5507db242838d127226bbbab30162d500d1b38385dac4ac3ecf3299371"} err="failed to get container status \"674efc5507db242838d127226bbbab30162d500d1b38385dac4ac3ecf3299371\": rpc error: code = NotFound desc = could not find container \"674efc5507db242838d127226bbbab30162d500d1b38385dac4ac3ecf3299371\": container with ID starting with 674efc5507db242838d127226bbbab30162d500d1b38385dac4ac3ecf3299371 not found: ID does not exist"
Sep 14 03:16:32.306136 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:32.306105    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-40c586-56dfb44f96-26bgt"]
Sep 14 03:16:32.318815 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:32.318794    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-40c586-56dfb44f96-26bgt"]
Sep 14 03:16:33.618925 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:33.618892    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c2f2433-1a57-44dd-9404-3d677ad1e9dd" path="/var/lib/kubelet/pods/2c2f2433-1a57-44dd-9404-3d677ad1e9dd/volumes"
Sep 14 03:16:47.611523 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:47.611484    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-558d9b8847-hbpwz" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:16:54.908311 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:54.908256    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl"]
Sep 14 03:16:54.909097 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:54.909057    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl" podUID="6d05fc80-bd0b-47bf-b255-e4a195c60955" containerName="maas-api" containerID="cri-o://4659e85b1a885fa17c0dc1a5e03bbde4684d02f60b4f64c53f91ca09ba5f480d" gracePeriod=30
Sep 14 03:16:55.017221 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:55.017186    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl" podUID="6d05fc80-bd0b-47bf-b255-e4a195c60955" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.78:8443/health\": dial tcp 10.133.0.78:8443: connect: connection refused"
Sep 14 03:16:55.168948 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:55.168887    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl"
Sep 14 03:16:55.318173 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:55.318146    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d05fc80-bd0b-47bf-b255-e4a195c60955-metrics-certs\") pod \"6d05fc80-bd0b-47bf-b255-e4a195c60955\" (UID: \"6d05fc80-bd0b-47bf-b255-e4a195c60955\") "
Sep 14 03:16:55.318173 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:55.318178    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d05fc80-bd0b-47bf-b255-e4a195c60955-openshift-service-ca\") pod \"6d05fc80-bd0b-47bf-b255-e4a195c60955\" (UID: \"6d05fc80-bd0b-47bf-b255-e4a195c60955\") "
Sep 14 03:16:55.318441 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:55.318245    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8p8gk\" (UniqueName: \"kubernetes.io/projected/6d05fc80-bd0b-47bf-b255-e4a195c60955-kube-api-access-8p8gk\") pod \"6d05fc80-bd0b-47bf-b255-e4a195c60955\" (UID: \"6d05fc80-bd0b-47bf-b255-e4a195c60955\") "
Sep 14 03:16:55.318441 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:55.318281    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d05fc80-bd0b-47bf-b255-e4a195c60955-maas-api-tls\") pod \"6d05fc80-bd0b-47bf-b255-e4a195c60955\" (UID: \"6d05fc80-bd0b-47bf-b255-e4a195c60955\") "
Sep 14 03:16:55.318614 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:55.318591    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d05fc80-bd0b-47bf-b255-e4a195c60955-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6d05fc80-bd0b-47bf-b255-e4a195c60955" (UID: "6d05fc80-bd0b-47bf-b255-e4a195c60955"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:16:55.320402 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:55.320373    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d05fc80-bd0b-47bf-b255-e4a195c60955-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6d05fc80-bd0b-47bf-b255-e4a195c60955" (UID: "6d05fc80-bd0b-47bf-b255-e4a195c60955"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:16:55.320508 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:55.320442    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d05fc80-bd0b-47bf-b255-e4a195c60955-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6d05fc80-bd0b-47bf-b255-e4a195c60955" (UID: "6d05fc80-bd0b-47bf-b255-e4a195c60955"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:16:55.320508 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:55.320453    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d05fc80-bd0b-47bf-b255-e4a195c60955-kube-api-access-8p8gk" (OuterVolumeSpecName: "kube-api-access-8p8gk") pod "6d05fc80-bd0b-47bf-b255-e4a195c60955" (UID: "6d05fc80-bd0b-47bf-b255-e4a195c60955"). InnerVolumeSpecName "kube-api-access-8p8gk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:16:55.419417 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:55.419316    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8p8gk\" (UniqueName: \"kubernetes.io/projected/6d05fc80-bd0b-47bf-b255-e4a195c60955-kube-api-access-8p8gk\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:16:55.419417 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:55.419342    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d05fc80-bd0b-47bf-b255-e4a195c60955-maas-api-tls\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:16:55.419417 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:55.419354    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d05fc80-bd0b-47bf-b255-e4a195c60955-metrics-certs\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:16:55.419417 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:55.419365    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d05fc80-bd0b-47bf-b255-e4a195c60955-openshift-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:16:55.423138 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:55.423114    2577 generic.go:356] "Generic (PLEG): container finished" podID="6d05fc80-bd0b-47bf-b255-e4a195c60955" containerID="4659e85b1a885fa17c0dc1a5e03bbde4684d02f60b4f64c53f91ca09ba5f480d" exitCode=0
Sep 14 03:16:55.423262 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:55.423160    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl"
Sep 14 03:16:55.423262 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:55.423200    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl" event={"ID":"6d05fc80-bd0b-47bf-b255-e4a195c60955","Type":"ContainerDied","Data":"4659e85b1a885fa17c0dc1a5e03bbde4684d02f60b4f64c53f91ca09ba5f480d"}
Sep 14 03:16:55.423262 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:55.423249    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl" event={"ID":"6d05fc80-bd0b-47bf-b255-e4a195c60955","Type":"ContainerDied","Data":"37c23340064569e8626090cf37106bc19f64d1a89b14bbecae679cf26315ed21"}
Sep 14 03:16:55.423456 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:55.423290    2577 scope.go:117] "RemoveContainer" containerID="4659e85b1a885fa17c0dc1a5e03bbde4684d02f60b4f64c53f91ca09ba5f480d"
Sep 14 03:16:55.436232 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:55.436209    2577 scope.go:117] "RemoveContainer" containerID="4659e85b1a885fa17c0dc1a5e03bbde4684d02f60b4f64c53f91ca09ba5f480d"
Sep 14 03:16:55.436610 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:16:55.436584    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4659e85b1a885fa17c0dc1a5e03bbde4684d02f60b4f64c53f91ca09ba5f480d\": container with ID starting with 4659e85b1a885fa17c0dc1a5e03bbde4684d02f60b4f64c53f91ca09ba5f480d not found: ID does not exist" containerID="4659e85b1a885fa17c0dc1a5e03bbde4684d02f60b4f64c53f91ca09ba5f480d"
Sep 14 03:16:55.436684 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:55.436623    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4659e85b1a885fa17c0dc1a5e03bbde4684d02f60b4f64c53f91ca09ba5f480d"} err="failed to get container status \"4659e85b1a885fa17c0dc1a5e03bbde4684d02f60b4f64c53f91ca09ba5f480d\": rpc error: code = NotFound desc = could not find container \"4659e85b1a885fa17c0dc1a5e03bbde4684d02f60b4f64c53f91ca09ba5f480d\": container with ID starting with 4659e85b1a885fa17c0dc1a5e03bbde4684d02f60b4f64c53f91ca09ba5f480d not found: ID does not exist"
Sep 14 03:16:55.448309 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:55.448262    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl"]
Sep 14 03:16:55.453439 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:55.453414    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-13d31c-595bf7dd84-w7zdl"]
Sep 14 03:16:55.618369 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:16:55.618338    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d05fc80-bd0b-47bf-b255-e4a195c60955" path="/var/lib/kubelet/pods/6d05fc80-bd0b-47bf-b255-e4a195c60955/volumes"
Sep 14 03:17:06.323778 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:06.323749    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-74969cc5f8-nzb2n_02709d9b-915f-4d2b-8bd2-0b73f296f99b/payload-processing/0.log"
Sep 14 03:17:06.454678 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:06.454642    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-74969cc5f8-nzb2n_02709d9b-915f-4d2b-8bd2-0b73f296f99b/payload-processing/0.log"
Sep 14 03:17:15.588155 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:15.588123    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf"]
Sep 14 03:17:15.589092 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:15.589069    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c2f2433-1a57-44dd-9404-3d677ad1e9dd" containerName="maas-api"
Sep 14 03:17:15.589233 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:15.589216    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c2f2433-1a57-44dd-9404-3d677ad1e9dd" containerName="maas-api"
Sep 14 03:17:15.589332 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:15.589290    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d05fc80-bd0b-47bf-b255-e4a195c60955" containerName="maas-api"
Sep 14 03:17:15.589332 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:15.589308    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d05fc80-bd0b-47bf-b255-e4a195c60955" containerName="maas-api"
Sep 14 03:17:15.589474 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:15.589460    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c2f2433-1a57-44dd-9404-3d677ad1e9dd" containerName="maas-api"
Sep 14 03:17:15.589530 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:15.589475    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d05fc80-bd0b-47bf-b255-e4a195c60955" containerName="maas-api"
Sep 14 03:17:15.594307 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:15.594261    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf"
Sep 14 03:17:15.599208 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:15.599183    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-a35f4b-serving-cert\""
Sep 14 03:17:15.602635 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:15.602610    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcmm9\" (UniqueName: \"kubernetes.io/projected/d47dba28-60db-42c4-a91e-453a31679587-kube-api-access-gcmm9\") pod \"maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf\" (UID: \"d47dba28-60db-42c4-a91e-453a31679587\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf"
Sep 14 03:17:15.602771 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:15.602691    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d47dba28-60db-42c4-a91e-453a31679587-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf\" (UID: \"d47dba28-60db-42c4-a91e-453a31679587\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf"
Sep 14 03:17:15.602771 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:15.602762    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d47dba28-60db-42c4-a91e-453a31679587-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf\" (UID: \"d47dba28-60db-42c4-a91e-453a31679587\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf"
Sep 14 03:17:15.602898 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:15.602867    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d47dba28-60db-42c4-a91e-453a31679587-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf\" (UID: \"d47dba28-60db-42c4-a91e-453a31679587\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf"
Sep 14 03:17:15.606924 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:15.606899    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf"]
Sep 14 03:17:15.703613 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:15.703581    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d47dba28-60db-42c4-a91e-453a31679587-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf\" (UID: \"d47dba28-60db-42c4-a91e-453a31679587\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf"
Sep 14 03:17:15.703783 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:15.703669    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d47dba28-60db-42c4-a91e-453a31679587-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf\" (UID: \"d47dba28-60db-42c4-a91e-453a31679587\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf"
Sep 14 03:17:15.703783 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:15.703720    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcmm9\" (UniqueName: \"kubernetes.io/projected/d47dba28-60db-42c4-a91e-453a31679587-kube-api-access-gcmm9\") pod \"maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf\" (UID: \"d47dba28-60db-42c4-a91e-453a31679587\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf"
Sep 14 03:17:15.703783 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:15.703772    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d47dba28-60db-42c4-a91e-453a31679587-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf\" (UID: \"d47dba28-60db-42c4-a91e-453a31679587\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf"
Sep 14 03:17:15.704000 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:17:15.703980    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a35f4b-serving-cert: secret "maas-api-e2e-ipp-cleanup-a35f4b-serving-cert" not found
Sep 14 03:17:15.704095 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:17:15.704076    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d47dba28-60db-42c4-a91e-453a31679587-maas-api-tls podName:d47dba28-60db-42c4-a91e-453a31679587 nodeName:}" failed. No retries permitted until 2026-09-14 03:17:16.204055818 +0000 UTC m=+1585.182033839 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d47dba28-60db-42c4-a91e-453a31679587-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf" (UID: "d47dba28-60db-42c4-a91e-453a31679587") : secret "maas-api-e2e-ipp-cleanup-a35f4b-serving-cert" not found
Sep 14 03:17:15.704640 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:15.704613    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d47dba28-60db-42c4-a91e-453a31679587-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf\" (UID: \"d47dba28-60db-42c4-a91e-453a31679587\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf"
Sep 14 03:17:15.706656 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:15.706624    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d47dba28-60db-42c4-a91e-453a31679587-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf\" (UID: \"d47dba28-60db-42c4-a91e-453a31679587\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf"
Sep 14 03:17:15.715459 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:15.715438    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcmm9\" (UniqueName: \"kubernetes.io/projected/d47dba28-60db-42c4-a91e-453a31679587-kube-api-access-gcmm9\") pod \"maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf\" (UID: \"d47dba28-60db-42c4-a91e-453a31679587\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf"
Sep 14 03:17:16.208524 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:16.208488    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d47dba28-60db-42c4-a91e-453a31679587-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf\" (UID: \"d47dba28-60db-42c4-a91e-453a31679587\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf"
Sep 14 03:17:16.210802 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:16.210771    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d47dba28-60db-42c4-a91e-453a31679587-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf\" (UID: \"d47dba28-60db-42c4-a91e-453a31679587\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf"
Sep 14 03:17:16.429145 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:16.429108    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv"]
Sep 14 03:17:16.429443 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:16.429396    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv" podUID="19b1dd9e-60df-4ebc-99eb-2a75967a7e5d" containerName="maas-api" containerID="cri-o://734c60f7197339cb5561937b0517e88df43ca7b445e96649f69472fa82b40e67" gracePeriod=30
Sep 14 03:17:16.505759 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:16.505603    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf"
Sep 14 03:17:16.700704 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:16.700679    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv"
Sep 14 03:17:16.714454 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:16.714428    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19b1dd9e-60df-4ebc-99eb-2a75967a7e5d-openshift-service-ca\") pod \"19b1dd9e-60df-4ebc-99eb-2a75967a7e5d\" (UID: \"19b1dd9e-60df-4ebc-99eb-2a75967a7e5d\") "
Sep 14 03:17:16.714586 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:16.714497    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19b1dd9e-60df-4ebc-99eb-2a75967a7e5d-metrics-certs\") pod \"19b1dd9e-60df-4ebc-99eb-2a75967a7e5d\" (UID: \"19b1dd9e-60df-4ebc-99eb-2a75967a7e5d\") "
Sep 14 03:17:16.714586 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:16.714541    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19b1dd9e-60df-4ebc-99eb-2a75967a7e5d-maas-api-tls\") pod \"19b1dd9e-60df-4ebc-99eb-2a75967a7e5d\" (UID: \"19b1dd9e-60df-4ebc-99eb-2a75967a7e5d\") "
Sep 14 03:17:16.714718 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:16.714595    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2kvlg\" (UniqueName: \"kubernetes.io/projected/19b1dd9e-60df-4ebc-99eb-2a75967a7e5d-kube-api-access-2kvlg\") pod \"19b1dd9e-60df-4ebc-99eb-2a75967a7e5d\" (UID: \"19b1dd9e-60df-4ebc-99eb-2a75967a7e5d\") "
Sep 14 03:17:16.715053 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:16.714987    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19b1dd9e-60df-4ebc-99eb-2a75967a7e5d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "19b1dd9e-60df-4ebc-99eb-2a75967a7e5d" (UID: "19b1dd9e-60df-4ebc-99eb-2a75967a7e5d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:17:16.716808 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:16.716782    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19b1dd9e-60df-4ebc-99eb-2a75967a7e5d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "19b1dd9e-60df-4ebc-99eb-2a75967a7e5d" (UID: "19b1dd9e-60df-4ebc-99eb-2a75967a7e5d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:17:16.717089 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:16.717059    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19b1dd9e-60df-4ebc-99eb-2a75967a7e5d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "19b1dd9e-60df-4ebc-99eb-2a75967a7e5d" (UID: "19b1dd9e-60df-4ebc-99eb-2a75967a7e5d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:17:16.717160 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:16.717112    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19b1dd9e-60df-4ebc-99eb-2a75967a7e5d-kube-api-access-2kvlg" (OuterVolumeSpecName: "kube-api-access-2kvlg") pod "19b1dd9e-60df-4ebc-99eb-2a75967a7e5d" (UID: "19b1dd9e-60df-4ebc-99eb-2a75967a7e5d"). InnerVolumeSpecName "kube-api-access-2kvlg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:17:16.802476 ip-10-0-141-180 kubenswrapper[2577]: W0914 03:17:16.802409    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd47dba28_60db_42c4_a91e_453a31679587.slice/crio-2a4aa5efe3faefd1a3e3ab8e4279b11b061c05681d977878be4509b003d4cd21 WatchSource:0}: Error finding container 2a4aa5efe3faefd1a3e3ab8e4279b11b061c05681d977878be4509b003d4cd21: Status 404 returned error can't find the container with id 2a4aa5efe3faefd1a3e3ab8e4279b11b061c05681d977878be4509b003d4cd21
Sep 14 03:17:16.802623 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:16.802517    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf"]
Sep 14 03:17:16.804355 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:16.804338    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 03:17:16.815835 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:16.815815    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2kvlg\" (UniqueName: \"kubernetes.io/projected/19b1dd9e-60df-4ebc-99eb-2a75967a7e5d-kube-api-access-2kvlg\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:17:16.815936 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:16.815836    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19b1dd9e-60df-4ebc-99eb-2a75967a7e5d-openshift-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:17:16.815936 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:16.815846    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19b1dd9e-60df-4ebc-99eb-2a75967a7e5d-metrics-certs\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:17:16.815936 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:16.815855    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19b1dd9e-60df-4ebc-99eb-2a75967a7e5d-maas-api-tls\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:17:17.553221 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:17.553180    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf" event={"ID":"d47dba28-60db-42c4-a91e-453a31679587","Type":"ContainerStarted","Data":"40228d8d421134711f28e72405fadade8579aa438a21516fec550be872492e14"}
Sep 14 03:17:17.553416 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:17.553244    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf" event={"ID":"d47dba28-60db-42c4-a91e-453a31679587","Type":"ContainerStarted","Data":"2a4aa5efe3faefd1a3e3ab8e4279b11b061c05681d977878be4509b003d4cd21"}
Sep 14 03:17:17.553416 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:17.553380    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf"
Sep 14 03:17:17.554551 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:17.554528    2577 generic.go:356] "Generic (PLEG): container finished" podID="19b1dd9e-60df-4ebc-99eb-2a75967a7e5d" containerID="734c60f7197339cb5561937b0517e88df43ca7b445e96649f69472fa82b40e67" exitCode=0
Sep 14 03:17:17.554677 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:17.554562    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv" event={"ID":"19b1dd9e-60df-4ebc-99eb-2a75967a7e5d","Type":"ContainerDied","Data":"734c60f7197339cb5561937b0517e88df43ca7b445e96649f69472fa82b40e67"}
Sep 14 03:17:17.554677 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:17.554608    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv" event={"ID":"19b1dd9e-60df-4ebc-99eb-2a75967a7e5d","Type":"ContainerDied","Data":"7bb564cfc9f1413187984cbb6647fd4d77795d5bd70fb3674d53f86f3ee1a9e2"}
Sep 14 03:17:17.554677 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:17.554629    2577 scope.go:117] "RemoveContainer" containerID="734c60f7197339cb5561937b0517e88df43ca7b445e96649f69472fa82b40e67"
Sep 14 03:17:17.554677 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:17.554568    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv"
Sep 14 03:17:17.565795 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:17.565686    2577 scope.go:117] "RemoveContainer" containerID="734c60f7197339cb5561937b0517e88df43ca7b445e96649f69472fa82b40e67"
Sep 14 03:17:17.566063 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:17:17.566033    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"734c60f7197339cb5561937b0517e88df43ca7b445e96649f69472fa82b40e67\": container with ID starting with 734c60f7197339cb5561937b0517e88df43ca7b445e96649f69472fa82b40e67 not found: ID does not exist" containerID="734c60f7197339cb5561937b0517e88df43ca7b445e96649f69472fa82b40e67"
Sep 14 03:17:17.566157 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:17.566074    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"734c60f7197339cb5561937b0517e88df43ca7b445e96649f69472fa82b40e67"} err="failed to get container status \"734c60f7197339cb5561937b0517e88df43ca7b445e96649f69472fa82b40e67\": rpc error: code = NotFound desc = could not find container \"734c60f7197339cb5561937b0517e88df43ca7b445e96649f69472fa82b40e67\": container with ID starting with 734c60f7197339cb5561937b0517e88df43ca7b445e96649f69472fa82b40e67 not found: ID does not exist"
Sep 14 03:17:17.577135 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:17.577092    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf" podStartSLOduration=2.247689476 podStartE2EDuration="2.57707746s" totalImagesPullingTime="329.387984ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:17:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:17:17.573750172 +0000 UTC m=+1586.551728201" watchObservedRunningTime="2026-09-14 03:17:17.57707746 +0000 UTC m=+1586.555055490"
Sep 14 03:17:17.594216 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:17.594192    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv"]
Sep 14 03:17:17.596824 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:17.596801    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-6e87d0-8589ccfc54-mpgdv"]
Sep 14 03:17:17.618894 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:17.618864    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19b1dd9e-60df-4ebc-99eb-2a75967a7e5d" path="/var/lib/kubelet/pods/19b1dd9e-60df-4ebc-99eb-2a75967a7e5d/volumes"
Sep 14 03:17:23.567839 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:23.567812    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf"
Sep 14 03:17:25.611666 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:25.611631    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-74969cc5f8-nzb2n" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:17:37.332778 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:37.332742    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm"]
Sep 14 03:17:37.333359 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:37.332960    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm" podUID="0dfa7ddb-19df-488e-82cc-8766d4a284e6" containerName="maas-api" containerID="cri-o://de9f9688521f40a8bbb6565ccbd7d276ac07f662d4d5089618eda1db55de763e" gracePeriod=30
Sep 14 03:17:37.398468 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:37.398435    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-grmh9"]
Sep 14 03:17:37.398710 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:37.398675    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-grmh9" podUID="90137a4b-ceb7-476a-ad4c-0e5f35035fac" containerName="payload-processing" containerID="cri-o://0231cd77b3612e2941a7e07658f4ecbc696df362340cbf08781c543a9a452607" gracePeriod=30
Sep 14 03:17:37.667066 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:37.667036    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm"
Sep 14 03:17:37.688647 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:37.688620    2577 generic.go:356] "Generic (PLEG): container finished" podID="90137a4b-ceb7-476a-ad4c-0e5f35035fac" containerID="0231cd77b3612e2941a7e07658f4ecbc696df362340cbf08781c543a9a452607" exitCode=0
Sep 14 03:17:37.688764 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:37.688685    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-grmh9" event={"ID":"90137a4b-ceb7-476a-ad4c-0e5f35035fac","Type":"ContainerDied","Data":"0231cd77b3612e2941a7e07658f4ecbc696df362340cbf08781c543a9a452607"}
Sep 14 03:17:37.688764 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:37.688732    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-grmh9" event={"ID":"90137a4b-ceb7-476a-ad4c-0e5f35035fac","Type":"ContainerDied","Data":"497ea1d097bc8613e8ffbc26a662be25dd5687fd7cc8291db2f369572e9cc2c2"}
Sep 14 03:17:37.688764 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:37.688754    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="497ea1d097bc8613e8ffbc26a662be25dd5687fd7cc8291db2f369572e9cc2c2"
Sep 14 03:17:37.690454 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:37.690382    2577 generic.go:356] "Generic (PLEG): container finished" podID="0dfa7ddb-19df-488e-82cc-8766d4a284e6" containerID="de9f9688521f40a8bbb6565ccbd7d276ac07f662d4d5089618eda1db55de763e" exitCode=0
Sep 14 03:17:37.690454 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:37.690418    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm" event={"ID":"0dfa7ddb-19df-488e-82cc-8766d4a284e6","Type":"ContainerDied","Data":"de9f9688521f40a8bbb6565ccbd7d276ac07f662d4d5089618eda1db55de763e"}
Sep 14 03:17:37.690454 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:37.690436    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm"
Sep 14 03:17:37.690683 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:37.690473    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm" event={"ID":"0dfa7ddb-19df-488e-82cc-8766d4a284e6","Type":"ContainerDied","Data":"47615fa523c439d0f9f5c5c359b34ac91627d1984300e46b3e5fe240a33f0679"}
Sep 14 03:17:37.690683 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:37.690496    2577 scope.go:117] "RemoveContainer" containerID="de9f9688521f40a8bbb6565ccbd7d276ac07f662d4d5089618eda1db55de763e"
Sep 14 03:17:37.694628 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:37.694576    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-grmh9"
Sep 14 03:17:37.701250 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:37.701230    2577 scope.go:117] "RemoveContainer" containerID="de9f9688521f40a8bbb6565ccbd7d276ac07f662d4d5089618eda1db55de763e"
Sep 14 03:17:37.701740 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:17:37.701711    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de9f9688521f40a8bbb6565ccbd7d276ac07f662d4d5089618eda1db55de763e\": container with ID starting with de9f9688521f40a8bbb6565ccbd7d276ac07f662d4d5089618eda1db55de763e not found: ID does not exist" containerID="de9f9688521f40a8bbb6565ccbd7d276ac07f662d4d5089618eda1db55de763e"
Sep 14 03:17:37.701840 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:37.701750    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de9f9688521f40a8bbb6565ccbd7d276ac07f662d4d5089618eda1db55de763e"} err="failed to get container status \"de9f9688521f40a8bbb6565ccbd7d276ac07f662d4d5089618eda1db55de763e\": rpc error: code = NotFound desc = could not find container \"de9f9688521f40a8bbb6565ccbd7d276ac07f662d4d5089618eda1db55de763e\": container with ID starting with de9f9688521f40a8bbb6565ccbd7d276ac07f662d4d5089618eda1db55de763e not found: ID does not exist"
Sep 14 03:17:37.807361 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:37.807331    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jrz9x\" (UniqueName: \"kubernetes.io/projected/90137a4b-ceb7-476a-ad4c-0e5f35035fac-kube-api-access-jrz9x\") pod \"90137a4b-ceb7-476a-ad4c-0e5f35035fac\" (UID: \"90137a4b-ceb7-476a-ad4c-0e5f35035fac\") "
Sep 14 03:17:37.807530 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:37.807396    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4f9jw\" (UniqueName: \"kubernetes.io/projected/0dfa7ddb-19df-488e-82cc-8766d4a284e6-kube-api-access-4f9jw\") pod \"0dfa7ddb-19df-488e-82cc-8766d4a284e6\" (UID: \"0dfa7ddb-19df-488e-82cc-8766d4a284e6\") "
Sep 14 03:17:37.807530 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:37.807433    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dfa7ddb-19df-488e-82cc-8766d4a284e6-metrics-certs\") pod \"0dfa7ddb-19df-488e-82cc-8766d4a284e6\" (UID: \"0dfa7ddb-19df-488e-82cc-8766d4a284e6\") "
Sep 14 03:17:37.807530 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:37.807505    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0dfa7ddb-19df-488e-82cc-8766d4a284e6-maas-api-tls\") pod \"0dfa7ddb-19df-488e-82cc-8766d4a284e6\" (UID: \"0dfa7ddb-19df-488e-82cc-8766d4a284e6\") "
Sep 14 03:17:37.807700 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:37.807554    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90137a4b-ceb7-476a-ad4c-0e5f35035fac-plugins-config-volume\") pod \"90137a4b-ceb7-476a-ad4c-0e5f35035fac\" (UID: \"90137a4b-ceb7-476a-ad4c-0e5f35035fac\") "
Sep 14 03:17:37.807700 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:37.807578    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dfa7ddb-19df-488e-82cc-8766d4a284e6-openshift-service-ca\") pod \"0dfa7ddb-19df-488e-82cc-8766d4a284e6\" (UID: \"0dfa7ddb-19df-488e-82cc-8766d4a284e6\") "
Sep 14 03:17:37.807931 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:37.807900    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90137a4b-ceb7-476a-ad4c-0e5f35035fac-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "90137a4b-ceb7-476a-ad4c-0e5f35035fac" (UID: "90137a4b-ceb7-476a-ad4c-0e5f35035fac"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:17:37.808012 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:37.807916    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0dfa7ddb-19df-488e-82cc-8766d4a284e6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0dfa7ddb-19df-488e-82cc-8766d4a284e6" (UID: "0dfa7ddb-19df-488e-82cc-8766d4a284e6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:17:37.809682 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:37.809655    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90137a4b-ceb7-476a-ad4c-0e5f35035fac-kube-api-access-jrz9x" (OuterVolumeSpecName: "kube-api-access-jrz9x") pod "90137a4b-ceb7-476a-ad4c-0e5f35035fac" (UID: "90137a4b-ceb7-476a-ad4c-0e5f35035fac"). InnerVolumeSpecName "kube-api-access-jrz9x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:17:37.809774 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:37.809705    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0dfa7ddb-19df-488e-82cc-8766d4a284e6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0dfa7ddb-19df-488e-82cc-8766d4a284e6" (UID: "0dfa7ddb-19df-488e-82cc-8766d4a284e6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:17:37.809895 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:37.809873    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0dfa7ddb-19df-488e-82cc-8766d4a284e6-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0dfa7ddb-19df-488e-82cc-8766d4a284e6" (UID: "0dfa7ddb-19df-488e-82cc-8766d4a284e6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:17:37.809995 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:37.809927    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0dfa7ddb-19df-488e-82cc-8766d4a284e6-kube-api-access-4f9jw" (OuterVolumeSpecName: "kube-api-access-4f9jw") pod "0dfa7ddb-19df-488e-82cc-8766d4a284e6" (UID: "0dfa7ddb-19df-488e-82cc-8766d4a284e6"). InnerVolumeSpecName "kube-api-access-4f9jw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:17:37.909021 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:37.908954    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrz9x\" (UniqueName: \"kubernetes.io/projected/90137a4b-ceb7-476a-ad4c-0e5f35035fac-kube-api-access-jrz9x\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:17:37.909021 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:37.908979    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4f9jw\" (UniqueName: \"kubernetes.io/projected/0dfa7ddb-19df-488e-82cc-8766d4a284e6-kube-api-access-4f9jw\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:17:37.909021 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:37.908990    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dfa7ddb-19df-488e-82cc-8766d4a284e6-metrics-certs\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:17:37.909021 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:37.909000    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0dfa7ddb-19df-488e-82cc-8766d4a284e6-maas-api-tls\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:17:37.909021 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:37.909008    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90137a4b-ceb7-476a-ad4c-0e5f35035fac-plugins-config-volume\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:17:37.909021 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:37.909016    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dfa7ddb-19df-488e-82cc-8766d4a284e6-openshift-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:17:38.014123 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:38.014091    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm"]
Sep 14 03:17:38.018847 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:38.018824    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-9f77a9-84d8df6d5b-w6hlm"]
Sep 14 03:17:38.697605 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:38.697535    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-grmh9"
Sep 14 03:17:38.721135 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:38.721105    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-grmh9"]
Sep 14 03:17:38.727861 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:38.727836    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-grmh9"]
Sep 14 03:17:39.621709 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:39.621671    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0dfa7ddb-19df-488e-82cc-8766d4a284e6" path="/var/lib/kubelet/pods/0dfa7ddb-19df-488e-82cc-8766d4a284e6/volumes"
Sep 14 03:17:39.622256 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:39.622233    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90137a4b-ceb7-476a-ad4c-0e5f35035fac" path="/var/lib/kubelet/pods/90137a4b-ceb7-476a-ad4c-0e5f35035fac/volumes"
Sep 14 03:17:42.370939 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:42.370901    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf"]
Sep 14 03:17:42.371387 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:42.371124    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf" podUID="d47dba28-60db-42c4-a91e-453a31679587" containerName="maas-api" containerID="cri-o://40228d8d421134711f28e72405fadade8579aa438a21516fec550be872492e14" gracePeriod=30
Sep 14 03:17:42.636894 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:42.636818    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf"
Sep 14 03:17:42.721165 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:42.721130    2577 generic.go:356] "Generic (PLEG): container finished" podID="d47dba28-60db-42c4-a91e-453a31679587" containerID="40228d8d421134711f28e72405fadade8579aa438a21516fec550be872492e14" exitCode=0
Sep 14 03:17:42.721165 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:42.721166    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf"
Sep 14 03:17:42.721389 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:42.721208    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf" event={"ID":"d47dba28-60db-42c4-a91e-453a31679587","Type":"ContainerDied","Data":"40228d8d421134711f28e72405fadade8579aa438a21516fec550be872492e14"}
Sep 14 03:17:42.721389 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:42.721256    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf" event={"ID":"d47dba28-60db-42c4-a91e-453a31679587","Type":"ContainerDied","Data":"2a4aa5efe3faefd1a3e3ab8e4279b11b061c05681d977878be4509b003d4cd21"}
Sep 14 03:17:42.721389 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:42.721295    2577 scope.go:117] "RemoveContainer" containerID="40228d8d421134711f28e72405fadade8579aa438a21516fec550be872492e14"
Sep 14 03:17:42.733165 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:42.733134    2577 scope.go:117] "RemoveContainer" containerID="40228d8d421134711f28e72405fadade8579aa438a21516fec550be872492e14"
Sep 14 03:17:42.733566 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:17:42.733537    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40228d8d421134711f28e72405fadade8579aa438a21516fec550be872492e14\": container with ID starting with 40228d8d421134711f28e72405fadade8579aa438a21516fec550be872492e14 not found: ID does not exist" containerID="40228d8d421134711f28e72405fadade8579aa438a21516fec550be872492e14"
Sep 14 03:17:42.733697 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:42.733577    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40228d8d421134711f28e72405fadade8579aa438a21516fec550be872492e14"} err="failed to get container status \"40228d8d421134711f28e72405fadade8579aa438a21516fec550be872492e14\": rpc error: code = NotFound desc = could not find container \"40228d8d421134711f28e72405fadade8579aa438a21516fec550be872492e14\": container with ID starting with 40228d8d421134711f28e72405fadade8579aa438a21516fec550be872492e14 not found: ID does not exist"
Sep 14 03:17:42.758009 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:42.757988    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d47dba28-60db-42c4-a91e-453a31679587-maas-api-tls\") pod \"d47dba28-60db-42c4-a91e-453a31679587\" (UID: \"d47dba28-60db-42c4-a91e-453a31679587\") "
Sep 14 03:17:42.758112 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:42.758029    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gcmm9\" (UniqueName: \"kubernetes.io/projected/d47dba28-60db-42c4-a91e-453a31679587-kube-api-access-gcmm9\") pod \"d47dba28-60db-42c4-a91e-453a31679587\" (UID: \"d47dba28-60db-42c4-a91e-453a31679587\") "
Sep 14 03:17:42.758112 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:42.758060    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d47dba28-60db-42c4-a91e-453a31679587-metrics-certs\") pod \"d47dba28-60db-42c4-a91e-453a31679587\" (UID: \"d47dba28-60db-42c4-a91e-453a31679587\") "
Sep 14 03:17:42.758112 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:42.758103    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d47dba28-60db-42c4-a91e-453a31679587-openshift-service-ca\") pod \"d47dba28-60db-42c4-a91e-453a31679587\" (UID: \"d47dba28-60db-42c4-a91e-453a31679587\") "
Sep 14 03:17:42.758552 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:42.758527    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d47dba28-60db-42c4-a91e-453a31679587-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d47dba28-60db-42c4-a91e-453a31679587" (UID: "d47dba28-60db-42c4-a91e-453a31679587"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:17:42.760095 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:42.760068    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d47dba28-60db-42c4-a91e-453a31679587-kube-api-access-gcmm9" (OuterVolumeSpecName: "kube-api-access-gcmm9") pod "d47dba28-60db-42c4-a91e-453a31679587" (UID: "d47dba28-60db-42c4-a91e-453a31679587"). InnerVolumeSpecName "kube-api-access-gcmm9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:17:42.760553 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:42.760530    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d47dba28-60db-42c4-a91e-453a31679587-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d47dba28-60db-42c4-a91e-453a31679587" (UID: "d47dba28-60db-42c4-a91e-453a31679587"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:17:42.760660 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:42.760548    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d47dba28-60db-42c4-a91e-453a31679587-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d47dba28-60db-42c4-a91e-453a31679587" (UID: "d47dba28-60db-42c4-a91e-453a31679587"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:17:42.859629 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:42.859597    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d47dba28-60db-42c4-a91e-453a31679587-maas-api-tls\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:17:42.859629 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:42.859625    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcmm9\" (UniqueName: \"kubernetes.io/projected/d47dba28-60db-42c4-a91e-453a31679587-kube-api-access-gcmm9\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:17:42.859629 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:42.859635    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d47dba28-60db-42c4-a91e-453a31679587-metrics-certs\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:17:42.859826 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:42.859643    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d47dba28-60db-42c4-a91e-453a31679587-openshift-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:17:43.050305 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:43.050278    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf"]
Sep 14 03:17:43.061625 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:43.061601    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a35f4b-6455fd5797-tsbsf"]
Sep 14 03:17:43.617771 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:43.617728    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d47dba28-60db-42c4-a91e-453a31679587" path="/var/lib/kubelet/pods/d47dba28-60db-42c4-a91e-453a31679587/volumes"
Sep 14 03:17:48.611988 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:48.611950    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-558d9b8847-hbpwz" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:17:51.728664 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:17:51.728625    2577 scope.go:117] "RemoveContainer" containerID="0231cd77b3612e2941a7e07658f4ecbc696df362340cbf08781c543a9a452607"
Sep 14 03:18:08.905168 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:18:08.905132    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7"]
Sep 14 03:18:08.905580 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:18:08.905482    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7" podUID="58434515-4c3b-477b-87db-1c8a24698cab" containerName="maas-api" containerID="cri-o://6f00ca08f75fd6de7e43466616c0f25fc980aeb064b11ab4ad678c0ee04787d8" gracePeriod=30
Sep 14 03:18:09.163031 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:18:09.162973    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7"
Sep 14 03:18:09.286127 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:18:09.286094    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mhglm\" (UniqueName: \"kubernetes.io/projected/58434515-4c3b-477b-87db-1c8a24698cab-kube-api-access-mhglm\") pod \"58434515-4c3b-477b-87db-1c8a24698cab\" (UID: \"58434515-4c3b-477b-87db-1c8a24698cab\") "
Sep 14 03:18:09.286305 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:18:09.286181    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58434515-4c3b-477b-87db-1c8a24698cab-metrics-certs\") pod \"58434515-4c3b-477b-87db-1c8a24698cab\" (UID: \"58434515-4c3b-477b-87db-1c8a24698cab\") "
Sep 14 03:18:09.286305 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:18:09.286218    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58434515-4c3b-477b-87db-1c8a24698cab-maas-api-tls\") pod \"58434515-4c3b-477b-87db-1c8a24698cab\" (UID: \"58434515-4c3b-477b-87db-1c8a24698cab\") "
Sep 14 03:18:09.286438 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:18:09.286403    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58434515-4c3b-477b-87db-1c8a24698cab-openshift-service-ca\") pod \"58434515-4c3b-477b-87db-1c8a24698cab\" (UID: \"58434515-4c3b-477b-87db-1c8a24698cab\") "
Sep 14 03:18:09.286775 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:18:09.286739    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58434515-4c3b-477b-87db-1c8a24698cab-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "58434515-4c3b-477b-87db-1c8a24698cab" (UID: "58434515-4c3b-477b-87db-1c8a24698cab"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:18:09.288700 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:18:09.288671    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58434515-4c3b-477b-87db-1c8a24698cab-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "58434515-4c3b-477b-87db-1c8a24698cab" (UID: "58434515-4c3b-477b-87db-1c8a24698cab"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:18:09.288860 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:18:09.288837    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58434515-4c3b-477b-87db-1c8a24698cab-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "58434515-4c3b-477b-87db-1c8a24698cab" (UID: "58434515-4c3b-477b-87db-1c8a24698cab"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:18:09.288860 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:18:09.288852    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58434515-4c3b-477b-87db-1c8a24698cab-kube-api-access-mhglm" (OuterVolumeSpecName: "kube-api-access-mhglm") pod "58434515-4c3b-477b-87db-1c8a24698cab" (UID: "58434515-4c3b-477b-87db-1c8a24698cab"). InnerVolumeSpecName "kube-api-access-mhglm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:18:09.387648 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:18:09.387614    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhglm\" (UniqueName: \"kubernetes.io/projected/58434515-4c3b-477b-87db-1c8a24698cab-kube-api-access-mhglm\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:18:09.387648 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:18:09.387642    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58434515-4c3b-477b-87db-1c8a24698cab-metrics-certs\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:18:09.387648 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:18:09.387652    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58434515-4c3b-477b-87db-1c8a24698cab-maas-api-tls\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:18:09.387865 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:18:09.387661    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58434515-4c3b-477b-87db-1c8a24698cab-openshift-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:18:09.878233 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:18:09.878145    2577 generic.go:356] "Generic (PLEG): container finished" podID="58434515-4c3b-477b-87db-1c8a24698cab" containerID="6f00ca08f75fd6de7e43466616c0f25fc980aeb064b11ab4ad678c0ee04787d8" exitCode=0
Sep 14 03:18:09.878233 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:18:09.878187    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7"
Sep 14 03:18:09.878461 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:18:09.878231    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7" event={"ID":"58434515-4c3b-477b-87db-1c8a24698cab","Type":"ContainerDied","Data":"6f00ca08f75fd6de7e43466616c0f25fc980aeb064b11ab4ad678c0ee04787d8"}
Sep 14 03:18:09.878461 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:18:09.878300    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7" event={"ID":"58434515-4c3b-477b-87db-1c8a24698cab","Type":"ContainerDied","Data":"155891aba109eac963f6348ce00023b1e0356b5b525a1b37fdf19081a942d13c"}
Sep 14 03:18:09.878461 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:18:09.878325    2577 scope.go:117] "RemoveContainer" containerID="6f00ca08f75fd6de7e43466616c0f25fc980aeb064b11ab4ad678c0ee04787d8"
Sep 14 03:18:09.887452 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:18:09.887419    2577 scope.go:117] "RemoveContainer" containerID="6f00ca08f75fd6de7e43466616c0f25fc980aeb064b11ab4ad678c0ee04787d8"
Sep 14 03:18:09.887687 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:18:09.887666    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f00ca08f75fd6de7e43466616c0f25fc980aeb064b11ab4ad678c0ee04787d8\": container with ID starting with 6f00ca08f75fd6de7e43466616c0f25fc980aeb064b11ab4ad678c0ee04787d8 not found: ID does not exist" containerID="6f00ca08f75fd6de7e43466616c0f25fc980aeb064b11ab4ad678c0ee04787d8"
Sep 14 03:18:09.887742 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:18:09.887695    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f00ca08f75fd6de7e43466616c0f25fc980aeb064b11ab4ad678c0ee04787d8"} err="failed to get container status \"6f00ca08f75fd6de7e43466616c0f25fc980aeb064b11ab4ad678c0ee04787d8\": rpc error: code = NotFound desc = could not find container \"6f00ca08f75fd6de7e43466616c0f25fc980aeb064b11ab4ad678c0ee04787d8\": container with ID starting with 6f00ca08f75fd6de7e43466616c0f25fc980aeb064b11ab4ad678c0ee04787d8 not found: ID does not exist"
Sep 14 03:18:09.899652 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:18:09.899632    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7"]
Sep 14 03:18:09.905738 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:18:09.905717    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-2611af-7fcd95dd46-5zpj7"]
Sep 14 03:18:11.617599 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:18:11.617568    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58434515-4c3b-477b-87db-1c8a24698cab" path="/var/lib/kubelet/pods/58434515-4c3b-477b-87db-1c8a24698cab/volumes"
Sep 14 03:18:44.611942 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:18:44.611907    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-74969cc5f8-nzb2n" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:19:15.264342 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:15.263975    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4"]
Sep 14 03:19:15.264342 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:15.264205    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4" podUID="02dd5974-c5ea-446d-a79b-258fbdabfb1a" containerName="maas-api" containerID="cri-o://d3ce54f30133ba9cc7a3b63590b9be3439a1f5e41d287b698fad5df2fbf9a863" gracePeriod=30
Sep 14 03:19:15.513225 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:15.513200    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4"
Sep 14 03:19:15.611818 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:15.611735    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-558d9b8847-hbpwz" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:19:15.628172 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:15.628134    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02dd5974-c5ea-446d-a79b-258fbdabfb1a-openshift-service-ca\") pod \"02dd5974-c5ea-446d-a79b-258fbdabfb1a\" (UID: \"02dd5974-c5ea-446d-a79b-258fbdabfb1a\") "
Sep 14 03:19:15.628347 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:15.628261    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02dd5974-c5ea-446d-a79b-258fbdabfb1a-maas-api-tls\") pod \"02dd5974-c5ea-446d-a79b-258fbdabfb1a\" (UID: \"02dd5974-c5ea-446d-a79b-258fbdabfb1a\") "
Sep 14 03:19:15.628347 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:15.628329    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-29xp2\" (UniqueName: \"kubernetes.io/projected/02dd5974-c5ea-446d-a79b-258fbdabfb1a-kube-api-access-29xp2\") pod \"02dd5974-c5ea-446d-a79b-258fbdabfb1a\" (UID: \"02dd5974-c5ea-446d-a79b-258fbdabfb1a\") "
Sep 14 03:19:15.628593 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:15.628546    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02dd5974-c5ea-446d-a79b-258fbdabfb1a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "02dd5974-c5ea-446d-a79b-258fbdabfb1a" (UID: "02dd5974-c5ea-446d-a79b-258fbdabfb1a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:19:15.628735 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:15.628647    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02dd5974-c5ea-446d-a79b-258fbdabfb1a-metrics-certs\") pod \"02dd5974-c5ea-446d-a79b-258fbdabfb1a\" (UID: \"02dd5974-c5ea-446d-a79b-258fbdabfb1a\") "
Sep 14 03:19:15.628980 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:15.628961    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02dd5974-c5ea-446d-a79b-258fbdabfb1a-openshift-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:19:15.630626 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:15.630599    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02dd5974-c5ea-446d-a79b-258fbdabfb1a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "02dd5974-c5ea-446d-a79b-258fbdabfb1a" (UID: "02dd5974-c5ea-446d-a79b-258fbdabfb1a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:19:15.630838 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:15.630811    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02dd5974-c5ea-446d-a79b-258fbdabfb1a-kube-api-access-29xp2" (OuterVolumeSpecName: "kube-api-access-29xp2") pod "02dd5974-c5ea-446d-a79b-258fbdabfb1a" (UID: "02dd5974-c5ea-446d-a79b-258fbdabfb1a"). InnerVolumeSpecName "kube-api-access-29xp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:19:15.631069 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:15.631044    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02dd5974-c5ea-446d-a79b-258fbdabfb1a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "02dd5974-c5ea-446d-a79b-258fbdabfb1a" (UID: "02dd5974-c5ea-446d-a79b-258fbdabfb1a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:19:15.730549 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:15.730514    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02dd5974-c5ea-446d-a79b-258fbdabfb1a-maas-api-tls\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:19:15.730707 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:15.730552    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29xp2\" (UniqueName: \"kubernetes.io/projected/02dd5974-c5ea-446d-a79b-258fbdabfb1a-kube-api-access-29xp2\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:19:15.730707 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:15.730570    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02dd5974-c5ea-446d-a79b-258fbdabfb1a-metrics-certs\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:19:16.295897 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:16.295862    2577 generic.go:356] "Generic (PLEG): container finished" podID="02dd5974-c5ea-446d-a79b-258fbdabfb1a" containerID="d3ce54f30133ba9cc7a3b63590b9be3439a1f5e41d287b698fad5df2fbf9a863" exitCode=0
Sep 14 03:19:16.296355 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:16.295914    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4"
Sep 14 03:19:16.296355 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:16.295951    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4" event={"ID":"02dd5974-c5ea-446d-a79b-258fbdabfb1a","Type":"ContainerDied","Data":"d3ce54f30133ba9cc7a3b63590b9be3439a1f5e41d287b698fad5df2fbf9a863"}
Sep 14 03:19:16.296355 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:16.296009    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4" event={"ID":"02dd5974-c5ea-446d-a79b-258fbdabfb1a","Type":"ContainerDied","Data":"30d10e0d553dfa209a3dfcc8ce3e600a3046bb7451bafc7eda13373028b45cfa"}
Sep 14 03:19:16.296355 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:16.296031    2577 scope.go:117] "RemoveContainer" containerID="d3ce54f30133ba9cc7a3b63590b9be3439a1f5e41d287b698fad5df2fbf9a863"
Sep 14 03:19:16.306512 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:16.306495    2577 scope.go:117] "RemoveContainer" containerID="d3ce54f30133ba9cc7a3b63590b9be3439a1f5e41d287b698fad5df2fbf9a863"
Sep 14 03:19:16.306774 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:19:16.306754    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3ce54f30133ba9cc7a3b63590b9be3439a1f5e41d287b698fad5df2fbf9a863\": container with ID starting with d3ce54f30133ba9cc7a3b63590b9be3439a1f5e41d287b698fad5df2fbf9a863 not found: ID does not exist" containerID="d3ce54f30133ba9cc7a3b63590b9be3439a1f5e41d287b698fad5df2fbf9a863"
Sep 14 03:19:16.306831 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:16.306784    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3ce54f30133ba9cc7a3b63590b9be3439a1f5e41d287b698fad5df2fbf9a863"} err="failed to get container status \"d3ce54f30133ba9cc7a3b63590b9be3439a1f5e41d287b698fad5df2fbf9a863\": rpc error: code = NotFound desc = could not find container \"d3ce54f30133ba9cc7a3b63590b9be3439a1f5e41d287b698fad5df2fbf9a863\": container with ID starting with d3ce54f30133ba9cc7a3b63590b9be3439a1f5e41d287b698fad5df2fbf9a863 not found: ID does not exist"
Sep 14 03:19:16.319178 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:16.319154    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4"]
Sep 14 03:19:16.323313 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:16.323290    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-bdf5b3-5b58cb4d64-zmkf4"]
Sep 14 03:19:17.617560 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:17.617525    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02dd5974-c5ea-446d-a79b-258fbdabfb1a" path="/var/lib/kubelet/pods/02dd5974-c5ea-446d-a79b-258fbdabfb1a/volumes"
Sep 14 03:19:33.469048 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:33.469014    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl"]
Sep 14 03:19:33.469542 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:33.469316    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl" podUID="f053a8e7-a7fb-49da-acf4-c0c643c8ee34" containerName="maas-api" containerID="cri-o://248b13798062fd443ead5c11b990c7d77e7e9e24d527792dfcf4b6ddf6997bc7" gracePeriod=30
Sep 14 03:19:33.728214 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:33.728153    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl"
Sep 14 03:19:33.886606 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:33.886571    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f053a8e7-a7fb-49da-acf4-c0c643c8ee34-metrics-certs\") pod \"f053a8e7-a7fb-49da-acf4-c0c643c8ee34\" (UID: \"f053a8e7-a7fb-49da-acf4-c0c643c8ee34\") "
Sep 14 03:19:33.886818 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:33.886614    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f053a8e7-a7fb-49da-acf4-c0c643c8ee34-openshift-service-ca\") pod \"f053a8e7-a7fb-49da-acf4-c0c643c8ee34\" (UID: \"f053a8e7-a7fb-49da-acf4-c0c643c8ee34\") "
Sep 14 03:19:33.886818 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:33.886643    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cwhfq\" (UniqueName: \"kubernetes.io/projected/f053a8e7-a7fb-49da-acf4-c0c643c8ee34-kube-api-access-cwhfq\") pod \"f053a8e7-a7fb-49da-acf4-c0c643c8ee34\" (UID: \"f053a8e7-a7fb-49da-acf4-c0c643c8ee34\") "
Sep 14 03:19:33.886953 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:33.886834    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f053a8e7-a7fb-49da-acf4-c0c643c8ee34-maas-api-tls\") pod \"f053a8e7-a7fb-49da-acf4-c0c643c8ee34\" (UID: \"f053a8e7-a7fb-49da-acf4-c0c643c8ee34\") "
Sep 14 03:19:33.887016 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:33.886987    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f053a8e7-a7fb-49da-acf4-c0c643c8ee34-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f053a8e7-a7fb-49da-acf4-c0c643c8ee34" (UID: "f053a8e7-a7fb-49da-acf4-c0c643c8ee34"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:19:33.887223 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:33.887189    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f053a8e7-a7fb-49da-acf4-c0c643c8ee34-openshift-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:19:33.889538 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:33.889473    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f053a8e7-a7fb-49da-acf4-c0c643c8ee34-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f053a8e7-a7fb-49da-acf4-c0c643c8ee34" (UID: "f053a8e7-a7fb-49da-acf4-c0c643c8ee34"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:19:33.890133 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:33.890092    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f053a8e7-a7fb-49da-acf4-c0c643c8ee34-kube-api-access-cwhfq" (OuterVolumeSpecName: "kube-api-access-cwhfq") pod "f053a8e7-a7fb-49da-acf4-c0c643c8ee34" (UID: "f053a8e7-a7fb-49da-acf4-c0c643c8ee34"). InnerVolumeSpecName "kube-api-access-cwhfq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:19:33.891049 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:33.891021    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f053a8e7-a7fb-49da-acf4-c0c643c8ee34-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f053a8e7-a7fb-49da-acf4-c0c643c8ee34" (UID: "f053a8e7-a7fb-49da-acf4-c0c643c8ee34"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:19:33.988656 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:33.988631    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f053a8e7-a7fb-49da-acf4-c0c643c8ee34-maas-api-tls\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:19:33.988656 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:33.988655    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f053a8e7-a7fb-49da-acf4-c0c643c8ee34-metrics-certs\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:19:33.988800 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:33.988665    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwhfq\" (UniqueName: \"kubernetes.io/projected/f053a8e7-a7fb-49da-acf4-c0c643c8ee34-kube-api-access-cwhfq\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:19:34.416663 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:34.416575    2577 generic.go:356] "Generic (PLEG): container finished" podID="f053a8e7-a7fb-49da-acf4-c0c643c8ee34" containerID="248b13798062fd443ead5c11b990c7d77e7e9e24d527792dfcf4b6ddf6997bc7" exitCode=0
Sep 14 03:19:34.416663 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:34.416622    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl"
Sep 14 03:19:34.416838 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:34.416668    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl" event={"ID":"f053a8e7-a7fb-49da-acf4-c0c643c8ee34","Type":"ContainerDied","Data":"248b13798062fd443ead5c11b990c7d77e7e9e24d527792dfcf4b6ddf6997bc7"}
Sep 14 03:19:34.416838 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:34.416724    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl" event={"ID":"f053a8e7-a7fb-49da-acf4-c0c643c8ee34","Type":"ContainerDied","Data":"06e5e31fb1ed35ab7686854702ff32e3b8386e975dee815c7d6ba6018f766070"}
Sep 14 03:19:34.416838 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:34.416747    2577 scope.go:117] "RemoveContainer" containerID="248b13798062fd443ead5c11b990c7d77e7e9e24d527792dfcf4b6ddf6997bc7"
Sep 14 03:19:34.426851 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:34.426835    2577 scope.go:117] "RemoveContainer" containerID="248b13798062fd443ead5c11b990c7d77e7e9e24d527792dfcf4b6ddf6997bc7"
Sep 14 03:19:34.427160 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:19:34.427141    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"248b13798062fd443ead5c11b990c7d77e7e9e24d527792dfcf4b6ddf6997bc7\": container with ID starting with 248b13798062fd443ead5c11b990c7d77e7e9e24d527792dfcf4b6ddf6997bc7 not found: ID does not exist" containerID="248b13798062fd443ead5c11b990c7d77e7e9e24d527792dfcf4b6ddf6997bc7"
Sep 14 03:19:34.427219 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:34.427168    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"248b13798062fd443ead5c11b990c7d77e7e9e24d527792dfcf4b6ddf6997bc7"} err="failed to get container status \"248b13798062fd443ead5c11b990c7d77e7e9e24d527792dfcf4b6ddf6997bc7\": rpc error: code = NotFound desc = could not find container \"248b13798062fd443ead5c11b990c7d77e7e9e24d527792dfcf4b6ddf6997bc7\": container with ID starting with 248b13798062fd443ead5c11b990c7d77e7e9e24d527792dfcf4b6ddf6997bc7 not found: ID does not exist"
Sep 14 03:19:34.458166 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:34.458146    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl"]
Sep 14 03:19:34.462472 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:34.462452    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0c919e-fbd7545d8-954nl"]
Sep 14 03:19:35.618040 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:35.618006    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f053a8e7-a7fb-49da-acf4-c0c643c8ee34" path="/var/lib/kubelet/pods/f053a8e7-a7fb-49da-acf4-c0c643c8ee34/volumes"
Sep 14 03:19:51.620220 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:51.620188    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p"]
Sep 14 03:19:51.620848 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:51.620372    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p" podUID="17fa1b0c-3ada-4398-815f-b2f1670881ba" containerName="maas-api" containerID="cri-o://eb444d93360a2b35e13379990ace030e844af5e479f42290da400cc23e305c9d" gracePeriod=30
Sep 14 03:19:51.871490 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:51.871431    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p"
Sep 14 03:19:51.936361 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:51.936335    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17fa1b0c-3ada-4398-815f-b2f1670881ba-openshift-service-ca\") pod \"17fa1b0c-3ada-4398-815f-b2f1670881ba\" (UID: \"17fa1b0c-3ada-4398-815f-b2f1670881ba\") "
Sep 14 03:19:51.936504 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:51.936392    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17fa1b0c-3ada-4398-815f-b2f1670881ba-metrics-certs\") pod \"17fa1b0c-3ada-4398-815f-b2f1670881ba\" (UID: \"17fa1b0c-3ada-4398-815f-b2f1670881ba\") "
Sep 14 03:19:51.936504 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:51.936417    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17fa1b0c-3ada-4398-815f-b2f1670881ba-maas-api-tls\") pod \"17fa1b0c-3ada-4398-815f-b2f1670881ba\" (UID: \"17fa1b0c-3ada-4398-815f-b2f1670881ba\") "
Sep 14 03:19:51.936504 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:51.936474    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8bjf\" (UniqueName: \"kubernetes.io/projected/17fa1b0c-3ada-4398-815f-b2f1670881ba-kube-api-access-r8bjf\") pod \"17fa1b0c-3ada-4398-815f-b2f1670881ba\" (UID: \"17fa1b0c-3ada-4398-815f-b2f1670881ba\") "
Sep 14 03:19:51.936681 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:51.936640    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17fa1b0c-3ada-4398-815f-b2f1670881ba-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "17fa1b0c-3ada-4398-815f-b2f1670881ba" (UID: "17fa1b0c-3ada-4398-815f-b2f1670881ba"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:19:51.936774 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:51.936753    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17fa1b0c-3ada-4398-815f-b2f1670881ba-openshift-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:19:51.938453 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:51.938425    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17fa1b0c-3ada-4398-815f-b2f1670881ba-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "17fa1b0c-3ada-4398-815f-b2f1670881ba" (UID: "17fa1b0c-3ada-4398-815f-b2f1670881ba"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:19:51.938453 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:51.938442    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17fa1b0c-3ada-4398-815f-b2f1670881ba-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "17fa1b0c-3ada-4398-815f-b2f1670881ba" (UID: "17fa1b0c-3ada-4398-815f-b2f1670881ba"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:19:51.938618 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:51.938575    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17fa1b0c-3ada-4398-815f-b2f1670881ba-kube-api-access-r8bjf" (OuterVolumeSpecName: "kube-api-access-r8bjf") pod "17fa1b0c-3ada-4398-815f-b2f1670881ba" (UID: "17fa1b0c-3ada-4398-815f-b2f1670881ba"). InnerVolumeSpecName "kube-api-access-r8bjf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:19:52.037252 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:52.037219    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8bjf\" (UniqueName: \"kubernetes.io/projected/17fa1b0c-3ada-4398-815f-b2f1670881ba-kube-api-access-r8bjf\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:19:52.037252 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:52.037250    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17fa1b0c-3ada-4398-815f-b2f1670881ba-metrics-certs\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:19:52.037252 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:52.037260    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17fa1b0c-3ada-4398-815f-b2f1670881ba-maas-api-tls\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:19:52.534783 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:52.534748    2577 generic.go:356] "Generic (PLEG): container finished" podID="17fa1b0c-3ada-4398-815f-b2f1670881ba" containerID="eb444d93360a2b35e13379990ace030e844af5e479f42290da400cc23e305c9d" exitCode=0
Sep 14 03:19:52.534953 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:52.534799    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p"
Sep 14 03:19:52.534953 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:52.534827    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p" event={"ID":"17fa1b0c-3ada-4398-815f-b2f1670881ba","Type":"ContainerDied","Data":"eb444d93360a2b35e13379990ace030e844af5e479f42290da400cc23e305c9d"}
Sep 14 03:19:52.534953 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:52.534878    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p" event={"ID":"17fa1b0c-3ada-4398-815f-b2f1670881ba","Type":"ContainerDied","Data":"31ac84be64110f68320778a0f720aa19ff2ae6bcfb7eae54452805c642d4c374"}
Sep 14 03:19:52.534953 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:52.534912    2577 scope.go:117] "RemoveContainer" containerID="eb444d93360a2b35e13379990ace030e844af5e479f42290da400cc23e305c9d"
Sep 14 03:19:52.544620 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:52.544588    2577 scope.go:117] "RemoveContainer" containerID="eb444d93360a2b35e13379990ace030e844af5e479f42290da400cc23e305c9d"
Sep 14 03:19:52.544888 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:19:52.544866    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb444d93360a2b35e13379990ace030e844af5e479f42290da400cc23e305c9d\": container with ID starting with eb444d93360a2b35e13379990ace030e844af5e479f42290da400cc23e305c9d not found: ID does not exist" containerID="eb444d93360a2b35e13379990ace030e844af5e479f42290da400cc23e305c9d"
Sep 14 03:19:52.544962 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:52.544895    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb444d93360a2b35e13379990ace030e844af5e479f42290da400cc23e305c9d"} err="failed to get container status \"eb444d93360a2b35e13379990ace030e844af5e479f42290da400cc23e305c9d\": rpc error: code = NotFound desc = could not find container \"eb444d93360a2b35e13379990ace030e844af5e479f42290da400cc23e305c9d\": container with ID starting with eb444d93360a2b35e13379990ace030e844af5e479f42290da400cc23e305c9d not found: ID does not exist"
Sep 14 03:19:52.562206 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:52.562184    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p"]
Sep 14 03:19:52.570299 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:52.570277    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f83c87-988b8c9c5-fjl9p"]
Sep 14 03:19:53.617831 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:19:53.617790    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17fa1b0c-3ada-4398-815f-b2f1670881ba" path="/var/lib/kubelet/pods/17fa1b0c-3ada-4398-815f-b2f1670881ba/volumes"
Sep 14 03:20:01.008833 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:20:01.008796    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-64b88c4d64-xrsdd"]
Sep 14 03:20:01.009379 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:20:01.008997    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-64b88c4d64-xrsdd" podUID="afeb4abf-2d9b-40d1-b896-c44607e40c34" containerName="manager" containerID="cri-o://b2f9aeef6a8ee51a53048ee3a294fb1cfc32fcb2a116a78665cca3de126e1947" gracePeriod=30
Sep 14 03:20:01.247613 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:20:01.247589    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64b88c4d64-xrsdd"
Sep 14 03:20:01.308278 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:20:01.308186    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afeb4abf-2d9b-40d1-b896-c44607e40c34-metrics-certs\") pod \"afeb4abf-2d9b-40d1-b896-c44607e40c34\" (UID: \"afeb4abf-2d9b-40d1-b896-c44607e40c34\") "
Sep 14 03:20:01.308278 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:20:01.308236    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdslx\" (UniqueName: \"kubernetes.io/projected/afeb4abf-2d9b-40d1-b896-c44607e40c34-kube-api-access-gdslx\") pod \"afeb4abf-2d9b-40d1-b896-c44607e40c34\" (UID: \"afeb4abf-2d9b-40d1-b896-c44607e40c34\") "
Sep 14 03:20:01.308503 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:20:01.308345    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/afeb4abf-2d9b-40d1-b896-c44607e40c34-webhook-cert\") pod \"afeb4abf-2d9b-40d1-b896-c44607e40c34\" (UID: \"afeb4abf-2d9b-40d1-b896-c44607e40c34\") "
Sep 14 03:20:01.310381 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:20:01.310351    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afeb4abf-2d9b-40d1-b896-c44607e40c34-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "afeb4abf-2d9b-40d1-b896-c44607e40c34" (UID: "afeb4abf-2d9b-40d1-b896-c44607e40c34"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:20:01.310381 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:20:01.310365    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afeb4abf-2d9b-40d1-b896-c44607e40c34-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "afeb4abf-2d9b-40d1-b896-c44607e40c34" (UID: "afeb4abf-2d9b-40d1-b896-c44607e40c34"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:20:01.310619 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:20:01.310603    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afeb4abf-2d9b-40d1-b896-c44607e40c34-kube-api-access-gdslx" (OuterVolumeSpecName: "kube-api-access-gdslx") pod "afeb4abf-2d9b-40d1-b896-c44607e40c34" (UID: "afeb4abf-2d9b-40d1-b896-c44607e40c34"). InnerVolumeSpecName "kube-api-access-gdslx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:20:01.409812 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:20:01.409786    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afeb4abf-2d9b-40d1-b896-c44607e40c34-metrics-certs\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:20:01.409812 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:20:01.409811    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdslx\" (UniqueName: \"kubernetes.io/projected/afeb4abf-2d9b-40d1-b896-c44607e40c34-kube-api-access-gdslx\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:20:01.409944 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:20:01.409820    2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/afeb4abf-2d9b-40d1-b896-c44607e40c34-webhook-cert\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:20:01.591912 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:20:01.591836    2577 generic.go:356] "Generic (PLEG): container finished" podID="afeb4abf-2d9b-40d1-b896-c44607e40c34" containerID="b2f9aeef6a8ee51a53048ee3a294fb1cfc32fcb2a116a78665cca3de126e1947" exitCode=0
Sep 14 03:20:01.591912 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:20:01.591893    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64b88c4d64-xrsdd"
Sep 14 03:20:01.592086 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:20:01.591922    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64b88c4d64-xrsdd" event={"ID":"afeb4abf-2d9b-40d1-b896-c44607e40c34","Type":"ContainerDied","Data":"b2f9aeef6a8ee51a53048ee3a294fb1cfc32fcb2a116a78665cca3de126e1947"}
Sep 14 03:20:01.592086 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:20:01.591994    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64b88c4d64-xrsdd" event={"ID":"afeb4abf-2d9b-40d1-b896-c44607e40c34","Type":"ContainerDied","Data":"19e1691be1cae313d91a0a9cddb2ab5aa4099506437e3cc97d0ca664235ec800"}
Sep 14 03:20:01.592086 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:20:01.592020    2577 scope.go:117] "RemoveContainer" containerID="b2f9aeef6a8ee51a53048ee3a294fb1cfc32fcb2a116a78665cca3de126e1947"
Sep 14 03:20:01.602894 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:20:01.602872    2577 scope.go:117] "RemoveContainer" containerID="b2f9aeef6a8ee51a53048ee3a294fb1cfc32fcb2a116a78665cca3de126e1947"
Sep 14 03:20:01.603199 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:20:01.603178    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2f9aeef6a8ee51a53048ee3a294fb1cfc32fcb2a116a78665cca3de126e1947\": container with ID starting with b2f9aeef6a8ee51a53048ee3a294fb1cfc32fcb2a116a78665cca3de126e1947 not found: ID does not exist" containerID="b2f9aeef6a8ee51a53048ee3a294fb1cfc32fcb2a116a78665cca3de126e1947"
Sep 14 03:20:01.603250 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:20:01.603210    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2f9aeef6a8ee51a53048ee3a294fb1cfc32fcb2a116a78665cca3de126e1947"} err="failed to get container status \"b2f9aeef6a8ee51a53048ee3a294fb1cfc32fcb2a116a78665cca3de126e1947\": rpc error: code = NotFound desc = could not find container \"b2f9aeef6a8ee51a53048ee3a294fb1cfc32fcb2a116a78665cca3de126e1947\": container with ID starting with b2f9aeef6a8ee51a53048ee3a294fb1cfc32fcb2a116a78665cca3de126e1947 not found: ID does not exist"
Sep 14 03:20:01.619806 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:20:01.619778    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-64b88c4d64-xrsdd"]
Sep 14 03:20:01.624007 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:20:01.623983    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-64b88c4d64-xrsdd"]
Sep 14 03:20:03.617854 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:20:03.617825    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afeb4abf-2d9b-40d1-b896-c44607e40c34" path="/var/lib/kubelet/pods/afeb4abf-2d9b-40d1-b896-c44607e40c34/volumes"
Sep 14 03:20:12.611364 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:20:12.611329    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-74969cc5f8-nzb2n" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:20:37.611886 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:20:37.611839    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-558d9b8847-hbpwz" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:21:02.448574 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.448536    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-64b88c4d64-h5zlg"]
Sep 14 03:21:02.449054 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.448921    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0dfa7ddb-19df-488e-82cc-8766d4a284e6" containerName="maas-api"
Sep 14 03:21:02.449054 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.448933    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0dfa7ddb-19df-488e-82cc-8766d4a284e6" containerName="maas-api"
Sep 14 03:21:02.449054 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.448941    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02dd5974-c5ea-446d-a79b-258fbdabfb1a" containerName="maas-api"
Sep 14 03:21:02.449054 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.448947    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="02dd5974-c5ea-446d-a79b-258fbdabfb1a" containerName="maas-api"
Sep 14 03:21:02.449054 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.448960    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17fa1b0c-3ada-4398-815f-b2f1670881ba" containerName="maas-api"
Sep 14 03:21:02.449054 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.448966    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="17fa1b0c-3ada-4398-815f-b2f1670881ba" containerName="maas-api"
Sep 14 03:21:02.449054 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.448976    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90137a4b-ceb7-476a-ad4c-0e5f35035fac" containerName="payload-processing"
Sep 14 03:21:02.449054 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.448982    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="90137a4b-ceb7-476a-ad4c-0e5f35035fac" containerName="payload-processing"
Sep 14 03:21:02.449054 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.448991    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19b1dd9e-60df-4ebc-99eb-2a75967a7e5d" containerName="maas-api"
Sep 14 03:21:02.449054 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.448998    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="19b1dd9e-60df-4ebc-99eb-2a75967a7e5d" containerName="maas-api"
Sep 14 03:21:02.449054 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.449007    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afeb4abf-2d9b-40d1-b896-c44607e40c34" containerName="manager"
Sep 14 03:21:02.449054 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.449012    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="afeb4abf-2d9b-40d1-b896-c44607e40c34" containerName="manager"
Sep 14 03:21:02.449054 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.449019    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58434515-4c3b-477b-87db-1c8a24698cab" containerName="maas-api"
Sep 14 03:21:02.449054 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.449024    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="58434515-4c3b-477b-87db-1c8a24698cab" containerName="maas-api"
Sep 14 03:21:02.449054 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.449037    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f053a8e7-a7fb-49da-acf4-c0c643c8ee34" containerName="maas-api"
Sep 14 03:21:02.449054 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.449042    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f053a8e7-a7fb-49da-acf4-c0c643c8ee34" containerName="maas-api"
Sep 14 03:21:02.449054 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.449049    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d47dba28-60db-42c4-a91e-453a31679587" containerName="maas-api"
Sep 14 03:21:02.449054 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.449056    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d47dba28-60db-42c4-a91e-453a31679587" containerName="maas-api"
Sep 14 03:21:02.449716 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.449111    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="02dd5974-c5ea-446d-a79b-258fbdabfb1a" containerName="maas-api"
Sep 14 03:21:02.449716 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.449118    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d47dba28-60db-42c4-a91e-453a31679587" containerName="maas-api"
Sep 14 03:21:02.449716 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.449124    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f053a8e7-a7fb-49da-acf4-c0c643c8ee34" containerName="maas-api"
Sep 14 03:21:02.449716 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.449131    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="90137a4b-ceb7-476a-ad4c-0e5f35035fac" containerName="payload-processing"
Sep 14 03:21:02.449716 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.449138    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="19b1dd9e-60df-4ebc-99eb-2a75967a7e5d" containerName="maas-api"
Sep 14 03:21:02.449716 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.449144    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0dfa7ddb-19df-488e-82cc-8766d4a284e6" containerName="maas-api"
Sep 14 03:21:02.449716 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.449150    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="afeb4abf-2d9b-40d1-b896-c44607e40c34" containerName="manager"
Sep 14 03:21:02.449716 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.449156    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="58434515-4c3b-477b-87db-1c8a24698cab" containerName="maas-api"
Sep 14 03:21:02.449716 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.449163    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="17fa1b0c-3ada-4398-815f-b2f1670881ba" containerName="maas-api"
Sep 14 03:21:02.452399 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.452384    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64b88c4d64-h5zlg"
Sep 14 03:21:02.456242 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.456220    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 14 03:21:02.456401 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.456258    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-ktqbd\""
Sep 14 03:21:02.456401 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.456222    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 14 03:21:02.456401 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.456222    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 14 03:21:02.464639 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.464615    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64b88c4d64-h5zlg"]
Sep 14 03:21:02.508829 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.508798    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/869e431d-2326-47b3-a5a3-14e52a4de7e9-metrics-certs\") pod \"maas-controller-64b88c4d64-h5zlg\" (UID: \"869e431d-2326-47b3-a5a3-14e52a4de7e9\") " pod="opendatahub/maas-controller-64b88c4d64-h5zlg"
Sep 14 03:21:02.508967 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.508850    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/869e431d-2326-47b3-a5a3-14e52a4de7e9-webhook-cert\") pod \"maas-controller-64b88c4d64-h5zlg\" (UID: \"869e431d-2326-47b3-a5a3-14e52a4de7e9\") " pod="opendatahub/maas-controller-64b88c4d64-h5zlg"
Sep 14 03:21:02.508967 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.508871    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49h5r\" (UniqueName: \"kubernetes.io/projected/869e431d-2326-47b3-a5a3-14e52a4de7e9-kube-api-access-49h5r\") pod \"maas-controller-64b88c4d64-h5zlg\" (UID: \"869e431d-2326-47b3-a5a3-14e52a4de7e9\") " pod="opendatahub/maas-controller-64b88c4d64-h5zlg"
Sep 14 03:21:02.609722 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.609691    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/869e431d-2326-47b3-a5a3-14e52a4de7e9-webhook-cert\") pod \"maas-controller-64b88c4d64-h5zlg\" (UID: \"869e431d-2326-47b3-a5a3-14e52a4de7e9\") " pod="opendatahub/maas-controller-64b88c4d64-h5zlg"
Sep 14 03:21:02.609848 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.609728    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49h5r\" (UniqueName: \"kubernetes.io/projected/869e431d-2326-47b3-a5a3-14e52a4de7e9-kube-api-access-49h5r\") pod \"maas-controller-64b88c4d64-h5zlg\" (UID: \"869e431d-2326-47b3-a5a3-14e52a4de7e9\") " pod="opendatahub/maas-controller-64b88c4d64-h5zlg"
Sep 14 03:21:02.609888 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.609849    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/869e431d-2326-47b3-a5a3-14e52a4de7e9-metrics-certs\") pod \"maas-controller-64b88c4d64-h5zlg\" (UID: \"869e431d-2326-47b3-a5a3-14e52a4de7e9\") " pod="opendatahub/maas-controller-64b88c4d64-h5zlg"
Sep 14 03:21:02.612324 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.612298    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/869e431d-2326-47b3-a5a3-14e52a4de7e9-webhook-cert\") pod \"maas-controller-64b88c4d64-h5zlg\" (UID: \"869e431d-2326-47b3-a5a3-14e52a4de7e9\") " pod="opendatahub/maas-controller-64b88c4d64-h5zlg"
Sep 14 03:21:02.612443 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.612424    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/869e431d-2326-47b3-a5a3-14e52a4de7e9-metrics-certs\") pod \"maas-controller-64b88c4d64-h5zlg\" (UID: \"869e431d-2326-47b3-a5a3-14e52a4de7e9\") " pod="opendatahub/maas-controller-64b88c4d64-h5zlg"
Sep 14 03:21:02.623805 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.623785    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49h5r\" (UniqueName: \"kubernetes.io/projected/869e431d-2326-47b3-a5a3-14e52a4de7e9-kube-api-access-49h5r\") pod \"maas-controller-64b88c4d64-h5zlg\" (UID: \"869e431d-2326-47b3-a5a3-14e52a4de7e9\") " pod="opendatahub/maas-controller-64b88c4d64-h5zlg"
Sep 14 03:21:02.763069 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:02.763035    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64b88c4d64-h5zlg"
Sep 14 03:21:03.039529 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:03.039450    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64b88c4d64-h5zlg"]
Sep 14 03:21:03.041866 ip-10-0-141-180 kubenswrapper[2577]: W0914 03:21:03.041833    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod869e431d_2326_47b3_a5a3_14e52a4de7e9.slice/crio-b8ed6430edd363e297132e616cbea5a031344b73658935faeb037bf5c867df0d WatchSource:0}: Error finding container b8ed6430edd363e297132e616cbea5a031344b73658935faeb037bf5c867df0d: Status 404 returned error can't find the container with id b8ed6430edd363e297132e616cbea5a031344b73658935faeb037bf5c867df0d
Sep 14 03:21:03.932981 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:03.932887    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64b88c4d64-h5zlg" event={"ID":"869e431d-2326-47b3-a5a3-14e52a4de7e9","Type":"ContainerStarted","Data":"fda3e911edae14c80f9f8326b1f0a03360d6d1182a1bb498dc74313a5521b99b"}
Sep 14 03:21:03.932981 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:03.932941    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64b88c4d64-h5zlg" event={"ID":"869e431d-2326-47b3-a5a3-14e52a4de7e9","Type":"ContainerStarted","Data":"b8ed6430edd363e297132e616cbea5a031344b73658935faeb037bf5c867df0d"}
Sep 14 03:21:03.932981 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:03.932962    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-64b88c4d64-h5zlg"
Sep 14 03:21:03.957190 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:03.957146    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-64b88c4d64-h5zlg" podStartSLOduration=1.324323402 podStartE2EDuration="1.957131248s" totalImagesPullingTime="632.807846ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:21:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:21:03.95212386 +0000 UTC m=+1812.930101890" watchObservedRunningTime="2026-09-14 03:21:03.957131248 +0000 UTC m=+1812.935109277"
Sep 14 03:21:14.941029 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:14.941000    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-64b88c4d64-h5zlg"
Sep 14 03:21:16.611688 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:16.611657    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-74969cc5f8-nzb2n" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:21:24.262160 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:24.262103    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-64b88c4d64-h5zlg"]
Sep 14 03:21:24.262685 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:24.262338    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-64b88c4d64-h5zlg" podUID="869e431d-2326-47b3-a5a3-14e52a4de7e9" containerName="manager" containerID="cri-o://fda3e911edae14c80f9f8326b1f0a03360d6d1182a1bb498dc74313a5521b99b" gracePeriod=30
Sep 14 03:21:24.508821 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:24.508796    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64b88c4d64-h5zlg"
Sep 14 03:21:24.588336 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:24.588250    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/869e431d-2326-47b3-a5a3-14e52a4de7e9-metrics-certs\") pod \"869e431d-2326-47b3-a5a3-14e52a4de7e9\" (UID: \"869e431d-2326-47b3-a5a3-14e52a4de7e9\") "
Sep 14 03:21:24.588486 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:24.588362    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/869e431d-2326-47b3-a5a3-14e52a4de7e9-webhook-cert\") pod \"869e431d-2326-47b3-a5a3-14e52a4de7e9\" (UID: \"869e431d-2326-47b3-a5a3-14e52a4de7e9\") "
Sep 14 03:21:24.588486 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:24.588428    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49h5r\" (UniqueName: \"kubernetes.io/projected/869e431d-2326-47b3-a5a3-14e52a4de7e9-kube-api-access-49h5r\") pod \"869e431d-2326-47b3-a5a3-14e52a4de7e9\" (UID: \"869e431d-2326-47b3-a5a3-14e52a4de7e9\") "
Sep 14 03:21:24.590394 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:24.590360    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/869e431d-2326-47b3-a5a3-14e52a4de7e9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "869e431d-2326-47b3-a5a3-14e52a4de7e9" (UID: "869e431d-2326-47b3-a5a3-14e52a4de7e9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:21:24.590505 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:24.590389    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/869e431d-2326-47b3-a5a3-14e52a4de7e9-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "869e431d-2326-47b3-a5a3-14e52a4de7e9" (UID: "869e431d-2326-47b3-a5a3-14e52a4de7e9"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:21:24.590505 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:24.590454    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/869e431d-2326-47b3-a5a3-14e52a4de7e9-kube-api-access-49h5r" (OuterVolumeSpecName: "kube-api-access-49h5r") pod "869e431d-2326-47b3-a5a3-14e52a4de7e9" (UID: "869e431d-2326-47b3-a5a3-14e52a4de7e9"). InnerVolumeSpecName "kube-api-access-49h5r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:21:24.689774 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:24.689741    2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/869e431d-2326-47b3-a5a3-14e52a4de7e9-webhook-cert\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:21:24.689774 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:24.689773    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49h5r\" (UniqueName: \"kubernetes.io/projected/869e431d-2326-47b3-a5a3-14e52a4de7e9-kube-api-access-49h5r\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:21:24.689985 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:24.689790    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/869e431d-2326-47b3-a5a3-14e52a4de7e9-metrics-certs\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:21:25.056872 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:25.056831    2577 generic.go:356] "Generic (PLEG): container finished" podID="869e431d-2326-47b3-a5a3-14e52a4de7e9" containerID="fda3e911edae14c80f9f8326b1f0a03360d6d1182a1bb498dc74313a5521b99b" exitCode=0
Sep 14 03:21:25.057033 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:25.056880    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64b88c4d64-h5zlg"
Sep 14 03:21:25.057033 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:25.056917    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64b88c4d64-h5zlg" event={"ID":"869e431d-2326-47b3-a5a3-14e52a4de7e9","Type":"ContainerDied","Data":"fda3e911edae14c80f9f8326b1f0a03360d6d1182a1bb498dc74313a5521b99b"}
Sep 14 03:21:25.057033 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:25.056971    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64b88c4d64-h5zlg" event={"ID":"869e431d-2326-47b3-a5a3-14e52a4de7e9","Type":"ContainerDied","Data":"b8ed6430edd363e297132e616cbea5a031344b73658935faeb037bf5c867df0d"}
Sep 14 03:21:25.057033 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:25.056998    2577 scope.go:117] "RemoveContainer" containerID="fda3e911edae14c80f9f8326b1f0a03360d6d1182a1bb498dc74313a5521b99b"
Sep 14 03:21:25.067039 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:25.067019    2577 scope.go:117] "RemoveContainer" containerID="fda3e911edae14c80f9f8326b1f0a03360d6d1182a1bb498dc74313a5521b99b"
Sep 14 03:21:25.067329 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:21:25.067299    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fda3e911edae14c80f9f8326b1f0a03360d6d1182a1bb498dc74313a5521b99b\": container with ID starting with fda3e911edae14c80f9f8326b1f0a03360d6d1182a1bb498dc74313a5521b99b not found: ID does not exist" containerID="fda3e911edae14c80f9f8326b1f0a03360d6d1182a1bb498dc74313a5521b99b"
Sep 14 03:21:25.067398 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:25.067337    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fda3e911edae14c80f9f8326b1f0a03360d6d1182a1bb498dc74313a5521b99b"} err="failed to get container status \"fda3e911edae14c80f9f8326b1f0a03360d6d1182a1bb498dc74313a5521b99b\": rpc error: code = NotFound desc = could not find container \"fda3e911edae14c80f9f8326b1f0a03360d6d1182a1bb498dc74313a5521b99b\": container with ID starting with fda3e911edae14c80f9f8326b1f0a03360d6d1182a1bb498dc74313a5521b99b not found: ID does not exist"
Sep 14 03:21:25.084559 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:25.084535    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-64b88c4d64-h5zlg"]
Sep 14 03:21:25.090321 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:25.090302    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-64b88c4d64-h5zlg"]
Sep 14 03:21:25.621456 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:25.621411    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="869e431d-2326-47b3-a5a3-14e52a4de7e9" path="/var/lib/kubelet/pods/869e431d-2326-47b3-a5a3-14e52a4de7e9/volumes"
Sep 14 03:21:25.851569 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:25.851535    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-64b88c4d64-f9j6t"]
Sep 14 03:21:25.851974 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:25.851956    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="869e431d-2326-47b3-a5a3-14e52a4de7e9" containerName="manager"
Sep 14 03:21:25.852017 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:25.851977    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="869e431d-2326-47b3-a5a3-14e52a4de7e9" containerName="manager"
Sep 14 03:21:25.852069 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:25.852047    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="869e431d-2326-47b3-a5a3-14e52a4de7e9" containerName="manager"
Sep 14 03:21:25.856530 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:25.856514    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64b88c4d64-f9j6t"
Sep 14 03:21:25.860589 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:25.860563    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 14 03:21:25.860734 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:25.860655    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 14 03:21:25.860734 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:25.860694    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 14 03:21:25.860734 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:25.860694    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-ktqbd\""
Sep 14 03:21:25.866640 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:25.866618    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64b88c4d64-f9j6t"]
Sep 14 03:21:26.001996 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:26.001974    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd7fdffb-de2a-43e2-8b00-7c3f8d293d67-metrics-certs\") pod \"maas-controller-64b88c4d64-f9j6t\" (UID: \"bd7fdffb-de2a-43e2-8b00-7c3f8d293d67\") " pod="opendatahub/maas-controller-64b88c4d64-f9j6t"
Sep 14 03:21:26.002091 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:26.002013    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bd7fdffb-de2a-43e2-8b00-7c3f8d293d67-webhook-cert\") pod \"maas-controller-64b88c4d64-f9j6t\" (UID: \"bd7fdffb-de2a-43e2-8b00-7c3f8d293d67\") " pod="opendatahub/maas-controller-64b88c4d64-f9j6t"
Sep 14 03:21:26.002091 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:26.002040    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgbwq\" (UniqueName: \"kubernetes.io/projected/bd7fdffb-de2a-43e2-8b00-7c3f8d293d67-kube-api-access-bgbwq\") pod \"maas-controller-64b88c4d64-f9j6t\" (UID: \"bd7fdffb-de2a-43e2-8b00-7c3f8d293d67\") " pod="opendatahub/maas-controller-64b88c4d64-f9j6t"
Sep 14 03:21:26.103228 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:26.103206    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd7fdffb-de2a-43e2-8b00-7c3f8d293d67-metrics-certs\") pod \"maas-controller-64b88c4d64-f9j6t\" (UID: \"bd7fdffb-de2a-43e2-8b00-7c3f8d293d67\") " pod="opendatahub/maas-controller-64b88c4d64-f9j6t"
Sep 14 03:21:26.103372 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:26.103261    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bd7fdffb-de2a-43e2-8b00-7c3f8d293d67-webhook-cert\") pod \"maas-controller-64b88c4d64-f9j6t\" (UID: \"bd7fdffb-de2a-43e2-8b00-7c3f8d293d67\") " pod="opendatahub/maas-controller-64b88c4d64-f9j6t"
Sep 14 03:21:26.103372 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:26.103324    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgbwq\" (UniqueName: \"kubernetes.io/projected/bd7fdffb-de2a-43e2-8b00-7c3f8d293d67-kube-api-access-bgbwq\") pod \"maas-controller-64b88c4d64-f9j6t\" (UID: \"bd7fdffb-de2a-43e2-8b00-7c3f8d293d67\") " pod="opendatahub/maas-controller-64b88c4d64-f9j6t"
Sep 14 03:21:26.105718 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:26.105694    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd7fdffb-de2a-43e2-8b00-7c3f8d293d67-metrics-certs\") pod \"maas-controller-64b88c4d64-f9j6t\" (UID: \"bd7fdffb-de2a-43e2-8b00-7c3f8d293d67\") " pod="opendatahub/maas-controller-64b88c4d64-f9j6t"
Sep 14 03:21:26.105812 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:26.105742    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bd7fdffb-de2a-43e2-8b00-7c3f8d293d67-webhook-cert\") pod \"maas-controller-64b88c4d64-f9j6t\" (UID: \"bd7fdffb-de2a-43e2-8b00-7c3f8d293d67\") " pod="opendatahub/maas-controller-64b88c4d64-f9j6t"
Sep 14 03:21:26.112961 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:26.112929    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgbwq\" (UniqueName: \"kubernetes.io/projected/bd7fdffb-de2a-43e2-8b00-7c3f8d293d67-kube-api-access-bgbwq\") pod \"maas-controller-64b88c4d64-f9j6t\" (UID: \"bd7fdffb-de2a-43e2-8b00-7c3f8d293d67\") " pod="opendatahub/maas-controller-64b88c4d64-f9j6t"
Sep 14 03:21:26.168569 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:26.168546    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64b88c4d64-f9j6t"
Sep 14 03:21:26.446315 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:26.446285    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64b88c4d64-f9j6t"]
Sep 14 03:21:26.447454 ip-10-0-141-180 kubenswrapper[2577]: W0914 03:21:26.447429    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd7fdffb_de2a_43e2_8b00_7c3f8d293d67.slice/crio-71d516dc12311aab2488c7e82d8ccd2a06c96c59045658483e5616884c9292ac WatchSource:0}: Error finding container 71d516dc12311aab2488c7e82d8ccd2a06c96c59045658483e5616884c9292ac: Status 404 returned error can't find the container with id 71d516dc12311aab2488c7e82d8ccd2a06c96c59045658483e5616884c9292ac
Sep 14 03:21:27.070835 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:27.070800    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64b88c4d64-f9j6t" event={"ID":"bd7fdffb-de2a-43e2-8b00-7c3f8d293d67","Type":"ContainerStarted","Data":"8c1dae1a96279275ac34cdc1b2d5972ce02246e67c33da825634cf6c116091d5"}
Sep 14 03:21:27.071311 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:27.070850    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64b88c4d64-f9j6t" event={"ID":"bd7fdffb-de2a-43e2-8b00-7c3f8d293d67","Type":"ContainerStarted","Data":"71d516dc12311aab2488c7e82d8ccd2a06c96c59045658483e5616884c9292ac"}
Sep 14 03:21:27.071311 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:27.070934    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-64b88c4d64-f9j6t"
Sep 14 03:21:38.079261 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:38.079219    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-64b88c4d64-f9j6t"
Sep 14 03:21:38.100154 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:38.100105    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-64b88c4d64-f9j6t" podStartSLOduration=12.70869372 podStartE2EDuration="13.100089272s" totalImagesPullingTime="391.395552ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:21:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:21:27.091486435 +0000 UTC m=+1836.069464465" watchObservedRunningTime="2026-09-14 03:21:38.100089272 +0000 UTC m=+1847.078067301"
Sep 14 03:21:39.612411 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:21:39.612370    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-558d9b8847-hbpwz" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:22:25.612072 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:22:25.611992    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-74969cc5f8-nzb2n" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:22:40.612370 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:22:40.612325    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-558d9b8847-hbpwz" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:23:29.611636 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:23:29.611597    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-74969cc5f8-nzb2n" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:23:51.615072 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:23:51.614828    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-558d9b8847-hbpwz" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:24:45.612182 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:24:45.612144    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-74969cc5f8-nzb2n" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:24:55.612298 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:24:55.612250    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-558d9b8847-hbpwz" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:25:17.510506 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:17.510423    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-86d9ddc875-52f96"]
Sep 14 03:25:17.514384 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:17.514356    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-52f96"
Sep 14 03:25:17.527248 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:17.527222    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-86d9ddc875-52f96"]
Sep 14 03:25:17.573730 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:17.573701    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79cedd48-b701-42e9-b17a-7458c057af3f-openshift-service-ca\") pod \"maas-api-86d9ddc875-52f96\" (UID: \"79cedd48-b701-42e9-b17a-7458c057af3f\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-52f96"
Sep 14 03:25:17.573895 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:17.573740    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79cedd48-b701-42e9-b17a-7458c057af3f-metrics-certs\") pod \"maas-api-86d9ddc875-52f96\" (UID: \"79cedd48-b701-42e9-b17a-7458c057af3f\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-52f96"
Sep 14 03:25:17.573895 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:17.573763    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2qnn\" (UniqueName: \"kubernetes.io/projected/79cedd48-b701-42e9-b17a-7458c057af3f-kube-api-access-z2qnn\") pod \"maas-api-86d9ddc875-52f96\" (UID: \"79cedd48-b701-42e9-b17a-7458c057af3f\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-52f96"
Sep 14 03:25:17.573895 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:17.573860    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79cedd48-b701-42e9-b17a-7458c057af3f-maas-api-tls\") pod \"maas-api-86d9ddc875-52f96\" (UID: \"79cedd48-b701-42e9-b17a-7458c057af3f\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-52f96"
Sep 14 03:25:17.674924 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:17.674892    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79cedd48-b701-42e9-b17a-7458c057af3f-maas-api-tls\") pod \"maas-api-86d9ddc875-52f96\" (UID: \"79cedd48-b701-42e9-b17a-7458c057af3f\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-52f96"
Sep 14 03:25:17.675094 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:17.674983    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79cedd48-b701-42e9-b17a-7458c057af3f-openshift-service-ca\") pod \"maas-api-86d9ddc875-52f96\" (UID: \"79cedd48-b701-42e9-b17a-7458c057af3f\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-52f96"
Sep 14 03:25:17.675094 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:17.675010    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79cedd48-b701-42e9-b17a-7458c057af3f-metrics-certs\") pod \"maas-api-86d9ddc875-52f96\" (UID: \"79cedd48-b701-42e9-b17a-7458c057af3f\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-52f96"
Sep 14 03:25:17.675094 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:17.675033    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2qnn\" (UniqueName: \"kubernetes.io/projected/79cedd48-b701-42e9-b17a-7458c057af3f-kube-api-access-z2qnn\") pod \"maas-api-86d9ddc875-52f96\" (UID: \"79cedd48-b701-42e9-b17a-7458c057af3f\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-52f96"
Sep 14 03:25:17.676087 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:17.676063    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79cedd48-b701-42e9-b17a-7458c057af3f-openshift-service-ca\") pod \"maas-api-86d9ddc875-52f96\" (UID: \"79cedd48-b701-42e9-b17a-7458c057af3f\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-52f96"
Sep 14 03:25:17.677677 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:17.677650    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79cedd48-b701-42e9-b17a-7458c057af3f-metrics-certs\") pod \"maas-api-86d9ddc875-52f96\" (UID: \"79cedd48-b701-42e9-b17a-7458c057af3f\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-52f96"
Sep 14 03:25:17.677800 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:17.677786    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79cedd48-b701-42e9-b17a-7458c057af3f-maas-api-tls\") pod \"maas-api-86d9ddc875-52f96\" (UID: \"79cedd48-b701-42e9-b17a-7458c057af3f\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-52f96"
Sep 14 03:25:17.684307 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:17.684261    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2qnn\" (UniqueName: \"kubernetes.io/projected/79cedd48-b701-42e9-b17a-7458c057af3f-kube-api-access-z2qnn\") pod \"maas-api-86d9ddc875-52f96\" (UID: \"79cedd48-b701-42e9-b17a-7458c057af3f\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-52f96"
Sep 14 03:25:17.826024 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:17.825945    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-52f96"
Sep 14 03:25:18.103735 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:18.103705    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-86d9ddc875-52f96"]
Sep 14 03:25:18.104634 ip-10-0-141-180 kubenswrapper[2577]: W0914 03:25:18.104607    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79cedd48_b701_42e9_b17a_7458c057af3f.slice/crio-f6f7d65282e06fa7401ba1c7821ec049a59fe14e91ac224506a513dbe6ab127d WatchSource:0}: Error finding container f6f7d65282e06fa7401ba1c7821ec049a59fe14e91ac224506a513dbe6ab127d: Status 404 returned error can't find the container with id f6f7d65282e06fa7401ba1c7821ec049a59fe14e91ac224506a513dbe6ab127d
Sep 14 03:25:18.105977 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:18.105956    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 03:25:18.387887 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:18.387805    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-52f96" event={"ID":"79cedd48-b701-42e9-b17a-7458c057af3f","Type":"ContainerStarted","Data":"f6f7d65282e06fa7401ba1c7821ec049a59fe14e91ac224506a513dbe6ab127d"}
Sep 14 03:25:19.402017 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:19.401973    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-52f96" event={"ID":"79cedd48-b701-42e9-b17a-7458c057af3f","Type":"ContainerStarted","Data":"5ee3b98ef1a7ff86d822e8b58ca2cad7f38cb7119f4d7e638d1fe0bfe75563e6"}
Sep 14 03:25:19.402428 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:19.402075    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-52f96"
Sep 14 03:25:19.421927 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:19.421880    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-52f96" podStartSLOduration=2.037259326 podStartE2EDuration="2.421866172s" totalImagesPullingTime="384.606846ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:25:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:25:19.420147443 +0000 UTC m=+2068.398125473" watchObservedRunningTime="2026-09-14 03:25:19.421866172 +0000 UTC m=+2068.399844200"
Sep 14 03:25:25.414980 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:25.414943    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-52f96"
Sep 14 03:25:34.949030 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:34.948993    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-w6xsp"]
Sep 14 03:25:34.949485 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:34.949190    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-w6xsp" podUID="31d8f6c7-b07f-4544-bbf5-067a7297b6cc" containerName="maas-api" containerID="cri-o://7f2a280b5481c9572fb65eb1d726096b4679e57972d2d2f0b9a53c03fa3aa9e9" gracePeriod=30
Sep 14 03:25:36.194583 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:36.194560    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-w6xsp"
Sep 14 03:25:36.334015 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:36.333942    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7rn4q\" (UniqueName: \"kubernetes.io/projected/31d8f6c7-b07f-4544-bbf5-067a7297b6cc-kube-api-access-7rn4q\") pod \"31d8f6c7-b07f-4544-bbf5-067a7297b6cc\" (UID: \"31d8f6c7-b07f-4544-bbf5-067a7297b6cc\") "
Sep 14 03:25:36.334015 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:36.333991    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31d8f6c7-b07f-4544-bbf5-067a7297b6cc-metrics-certs\") pod \"31d8f6c7-b07f-4544-bbf5-067a7297b6cc\" (UID: \"31d8f6c7-b07f-4544-bbf5-067a7297b6cc\") "
Sep 14 03:25:36.334015 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:36.334013    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31d8f6c7-b07f-4544-bbf5-067a7297b6cc-maas-api-tls\") pod \"31d8f6c7-b07f-4544-bbf5-067a7297b6cc\" (UID: \"31d8f6c7-b07f-4544-bbf5-067a7297b6cc\") "
Sep 14 03:25:36.334288 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:36.334115    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31d8f6c7-b07f-4544-bbf5-067a7297b6cc-openshift-service-ca\") pod \"31d8f6c7-b07f-4544-bbf5-067a7297b6cc\" (UID: \"31d8f6c7-b07f-4544-bbf5-067a7297b6cc\") "
Sep 14 03:25:36.334507 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:36.334482    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31d8f6c7-b07f-4544-bbf5-067a7297b6cc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "31d8f6c7-b07f-4544-bbf5-067a7297b6cc" (UID: "31d8f6c7-b07f-4544-bbf5-067a7297b6cc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:25:36.336035 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:36.336008    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31d8f6c7-b07f-4544-bbf5-067a7297b6cc-kube-api-access-7rn4q" (OuterVolumeSpecName: "kube-api-access-7rn4q") pod "31d8f6c7-b07f-4544-bbf5-067a7297b6cc" (UID: "31d8f6c7-b07f-4544-bbf5-067a7297b6cc"). InnerVolumeSpecName "kube-api-access-7rn4q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:25:36.336128 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:36.336044    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31d8f6c7-b07f-4544-bbf5-067a7297b6cc-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "31d8f6c7-b07f-4544-bbf5-067a7297b6cc" (UID: "31d8f6c7-b07f-4544-bbf5-067a7297b6cc"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:25:36.336128 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:36.336057    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31d8f6c7-b07f-4544-bbf5-067a7297b6cc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "31d8f6c7-b07f-4544-bbf5-067a7297b6cc" (UID: "31d8f6c7-b07f-4544-bbf5-067a7297b6cc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:25:36.434826 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:36.434797    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7rn4q\" (UniqueName: \"kubernetes.io/projected/31d8f6c7-b07f-4544-bbf5-067a7297b6cc-kube-api-access-7rn4q\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:25:36.434826 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:36.434824    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31d8f6c7-b07f-4544-bbf5-067a7297b6cc-metrics-certs\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:25:36.434826 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:36.434834    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31d8f6c7-b07f-4544-bbf5-067a7297b6cc-maas-api-tls\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:25:36.435003 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:36.434845    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31d8f6c7-b07f-4544-bbf5-067a7297b6cc-openshift-service-ca\") on node \"ip-10-0-141-180.ec2.internal\" DevicePath \"\""
Sep 14 03:25:36.504929 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:36.504898    2577 generic.go:356] "Generic (PLEG): container finished" podID="31d8f6c7-b07f-4544-bbf5-067a7297b6cc" containerID="7f2a280b5481c9572fb65eb1d726096b4679e57972d2d2f0b9a53c03fa3aa9e9" exitCode=0
Sep 14 03:25:36.505032 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:36.504951    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-w6xsp"
Sep 14 03:25:36.505081 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:36.504951    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-w6xsp" event={"ID":"31d8f6c7-b07f-4544-bbf5-067a7297b6cc","Type":"ContainerDied","Data":"7f2a280b5481c9572fb65eb1d726096b4679e57972d2d2f0b9a53c03fa3aa9e9"}
Sep 14 03:25:36.505142 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:36.505119    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59f6566bbc-w6xsp" event={"ID":"31d8f6c7-b07f-4544-bbf5-067a7297b6cc","Type":"ContainerDied","Data":"9c373cca057b25b96790808cb119f420149fea390b7a1b7e8516fdf1b091d956"}
Sep 14 03:25:36.505200 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:36.505162    2577 scope.go:117] "RemoveContainer" containerID="7f2a280b5481c9572fb65eb1d726096b4679e57972d2d2f0b9a53c03fa3aa9e9"
Sep 14 03:25:36.514943 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:36.514928    2577 scope.go:117] "RemoveContainer" containerID="7f2a280b5481c9572fb65eb1d726096b4679e57972d2d2f0b9a53c03fa3aa9e9"
Sep 14 03:25:36.515165 ip-10-0-141-180 kubenswrapper[2577]: E0914 03:25:36.515143    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f2a280b5481c9572fb65eb1d726096b4679e57972d2d2f0b9a53c03fa3aa9e9\": container with ID starting with 7f2a280b5481c9572fb65eb1d726096b4679e57972d2d2f0b9a53c03fa3aa9e9 not found: ID does not exist" containerID="7f2a280b5481c9572fb65eb1d726096b4679e57972d2d2f0b9a53c03fa3aa9e9"
Sep 14 03:25:36.515229 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:36.515176    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f2a280b5481c9572fb65eb1d726096b4679e57972d2d2f0b9a53c03fa3aa9e9"} err="failed to get container status \"7f2a280b5481c9572fb65eb1d726096b4679e57972d2d2f0b9a53c03fa3aa9e9\": rpc error: code = NotFound desc = could not find container \"7f2a280b5481c9572fb65eb1d726096b4679e57972d2d2f0b9a53c03fa3aa9e9\": container with ID starting with 7f2a280b5481c9572fb65eb1d726096b4679e57972d2d2f0b9a53c03fa3aa9e9 not found: ID does not exist"
Sep 14 03:25:36.527899 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:36.527875    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-w6xsp"]
Sep 14 03:25:36.534604 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:36.534582    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-59f6566bbc-w6xsp"]
Sep 14 03:25:37.618029 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:37.617995    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31d8f6c7-b07f-4544-bbf5-067a7297b6cc" path="/var/lib/kubelet/pods/31d8f6c7-b07f-4544-bbf5-067a7297b6cc/volumes"
Sep 14 03:25:50.651548 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:50.651504    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-hckjv_003c37b0-6bdd-4b28-9a26-fa2a4ed813ec/global-pull-secret-syncer/0.log"
Sep 14 03:25:50.865672 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:50.865649    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-r8rwc_9dcf46a1-8ecf-4333-8c37-13c9e717bde5/konnectivity-agent/0.log"
Sep 14 03:25:50.933962 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:50.933909    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-180.ec2.internal_2336dac84cecf30d916b12e72e4b2ecc/haproxy/0.log"
Sep 14 03:25:52.611500 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:52.611467    2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-74969cc5f8-nzb2n" secret="" err="secret \"payload-processing-dockercfg-zhr5x\" not found"
Sep 14 03:25:55.559723 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:55.559692    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-km4tk_31e6517f-03ef-4862-bde0-71d4293f96c0/manager/0.log"
Sep 14 03:25:55.737599 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:55.737568    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-7p4ps_ed8c4796-f1d6-43f3-b956-4f0a43fcaec9/manager/0.log"
Sep 14 03:25:57.523692 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:57.523654    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5d85778b9d-d8jr5_e5dc70e7-0fd4-457b-8e8b-8ceb4149c3d3/monitoring-plugin/0.log"
Sep 14 03:25:57.649370 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:57.649340    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qvfz4_cf1cfe36-99ab-461f-8845-2c4c8f3383ac/node-exporter/0.log"
Sep 14 03:25:57.673283 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:57.673248    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qvfz4_cf1cfe36-99ab-461f-8845-2c4c8f3383ac/kube-rbac-proxy/0.log"
Sep 14 03:25:57.702178 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:57.702153    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qvfz4_cf1cfe36-99ab-461f-8845-2c4c8f3383ac/init-textfile/0.log"
Sep 14 03:25:57.850338 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:57.850315    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-4zqc5_973e81ff-b6f7-4d26-bae3-c9f91ca1aaed/kube-rbac-proxy-main/0.log"
Sep 14 03:25:57.879167 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:57.879147    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-4zqc5_973e81ff-b6f7-4d26-bae3-c9f91ca1aaed/kube-rbac-proxy-self/0.log"
Sep 14 03:25:57.906868 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:57.906843    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-4zqc5_973e81ff-b6f7-4d26-bae3-c9f91ca1aaed/openshift-state-metrics/0.log"
Sep 14 03:25:58.180489 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:58.180430    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-nkzdx_139bb2e4-4c56-4bdb-ba66-995245cc7a2c/prometheus-operator/0.log"
Sep 14 03:25:58.213748 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:58.213729    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-nkzdx_139bb2e4-4c56-4bdb-ba66-995245cc7a2c/kube-rbac-proxy/0.log"
Sep 14 03:25:58.254546 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:58.254525    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-t2l4g_c6505647-069b-4004-bab3-64af301005f1/prometheus-operator-admission-webhook/0.log"
Sep 14 03:25:58.397438 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:58.397418    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5798cfc64f-fd7cc_59a4ce0f-f4fe-4c46-a137-55f612b3240b/thanos-query/0.log"
Sep 14 03:25:58.432339 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:58.432290    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5798cfc64f-fd7cc_59a4ce0f-f4fe-4c46-a137-55f612b3240b/kube-rbac-proxy-web/0.log"
Sep 14 03:25:58.484452 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:58.484424    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5798cfc64f-fd7cc_59a4ce0f-f4fe-4c46-a137-55f612b3240b/kube-rbac-proxy/0.log"
Sep 14 03:25:58.519929 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:58.519911    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5798cfc64f-fd7cc_59a4ce0f-f4fe-4c46-a137-55f612b3240b/prom-label-proxy/0.log"
Sep 14 03:25:58.544000 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:58.543979    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5798cfc64f-fd7cc_59a4ce0f-f4fe-4c46-a137-55f612b3240b/kube-rbac-proxy-rules/0.log"
Sep 14 03:25:58.570001 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:58.569976    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5798cfc64f-fd7cc_59a4ce0f-f4fe-4c46-a137-55f612b3240b/kube-rbac-proxy-metrics/0.log"
Sep 14 03:25:59.935174 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:25:59.935139    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-6vkfm_1b15bf18-f4ef-4d01-9916-780ac2d581ab/networking-console-plugin/0.log"
Sep 14 03:26:00.890023 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:00.889981    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9xgtv/perf-node-gather-daemonset-g8s92"]
Sep 14 03:26:00.890532 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:00.890510    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31d8f6c7-b07f-4544-bbf5-067a7297b6cc" containerName="maas-api"
Sep 14 03:26:00.890532 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:00.890533    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="31d8f6c7-b07f-4544-bbf5-067a7297b6cc" containerName="maas-api"
Sep 14 03:26:00.890719 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:00.890653    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="31d8f6c7-b07f-4544-bbf5-067a7297b6cc" containerName="maas-api"
Sep 14 03:26:00.893902 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:00.893881    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-g8s92"
Sep 14 03:26:00.896263 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:00.896238    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9xgtv\"/\"default-dockercfg-dfvsv\""
Sep 14 03:26:00.897392 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:00.897369    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9xgtv\"/\"kube-root-ca.crt\""
Sep 14 03:26:00.897518 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:00.897369    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9xgtv\"/\"openshift-service-ca.crt\""
Sep 14 03:26:00.902858 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:00.902833    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9xgtv/perf-node-gather-daemonset-g8s92"]
Sep 14 03:26:00.930369 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:00.930344    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6gsd\" (UniqueName: \"kubernetes.io/projected/5ba14147-39a2-4d0c-80f1-02903bf00351-kube-api-access-q6gsd\") pod \"perf-node-gather-daemonset-g8s92\" (UID: \"5ba14147-39a2-4d0c-80f1-02903bf00351\") " pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-g8s92"
Sep 14 03:26:00.930469 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:00.930378    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5ba14147-39a2-4d0c-80f1-02903bf00351-podres\") pod \"perf-node-gather-daemonset-g8s92\" (UID: \"5ba14147-39a2-4d0c-80f1-02903bf00351\") " pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-g8s92"
Sep 14 03:26:00.930469 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:00.930410    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5ba14147-39a2-4d0c-80f1-02903bf00351-proc\") pod \"perf-node-gather-daemonset-g8s92\" (UID: \"5ba14147-39a2-4d0c-80f1-02903bf00351\") " pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-g8s92"
Sep 14 03:26:00.930469 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:00.930439    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ba14147-39a2-4d0c-80f1-02903bf00351-sys\") pod \"perf-node-gather-daemonset-g8s92\" (UID: \"5ba14147-39a2-4d0c-80f1-02903bf00351\") " pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-g8s92"
Sep 14 03:26:00.930582 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:00.930517    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5ba14147-39a2-4d0c-80f1-02903bf00351-lib-modules\") pod \"perf-node-gather-daemonset-g8s92\" (UID: \"5ba14147-39a2-4d0c-80f1-02903bf00351\") " pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-g8s92"
Sep 14 03:26:01.031718 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:01.031687    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6gsd\" (UniqueName: \"kubernetes.io/projected/5ba14147-39a2-4d0c-80f1-02903bf00351-kube-api-access-q6gsd\") pod \"perf-node-gather-daemonset-g8s92\" (UID: \"5ba14147-39a2-4d0c-80f1-02903bf00351\") " pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-g8s92"
Sep 14 03:26:01.032023 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:01.031723    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5ba14147-39a2-4d0c-80f1-02903bf00351-podres\") pod \"perf-node-gather-daemonset-g8s92\" (UID: \"5ba14147-39a2-4d0c-80f1-02903bf00351\") " pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-g8s92"
Sep 14 03:26:01.032023 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:01.031746    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5ba14147-39a2-4d0c-80f1-02903bf00351-proc\") pod \"perf-node-gather-daemonset-g8s92\" (UID: \"5ba14147-39a2-4d0c-80f1-02903bf00351\") " pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-g8s92"
Sep 14 03:26:01.032023 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:01.031804    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5ba14147-39a2-4d0c-80f1-02903bf00351-proc\") pod \"perf-node-gather-daemonset-g8s92\" (UID: \"5ba14147-39a2-4d0c-80f1-02903bf00351\") " pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-g8s92"
Sep 14 03:26:01.032023 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:01.031875    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ba14147-39a2-4d0c-80f1-02903bf00351-sys\") pod \"perf-node-gather-daemonset-g8s92\" (UID: \"5ba14147-39a2-4d0c-80f1-02903bf00351\") " pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-g8s92"
Sep 14 03:26:01.032023 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:01.031914    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ba14147-39a2-4d0c-80f1-02903bf00351-sys\") pod \"perf-node-gather-daemonset-g8s92\" (UID: \"5ba14147-39a2-4d0c-80f1-02903bf00351\") " pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-g8s92"
Sep 14 03:26:01.032023 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:01.031892    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5ba14147-39a2-4d0c-80f1-02903bf00351-podres\") pod \"perf-node-gather-daemonset-g8s92\" (UID: \"5ba14147-39a2-4d0c-80f1-02903bf00351\") " pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-g8s92"
Sep 14 03:26:01.032023 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:01.031987    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5ba14147-39a2-4d0c-80f1-02903bf00351-lib-modules\") pod \"perf-node-gather-daemonset-g8s92\" (UID: \"5ba14147-39a2-4d0c-80f1-02903bf00351\") " pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-g8s92"
Sep 14 03:26:01.032300 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:01.032119    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5ba14147-39a2-4d0c-80f1-02903bf00351-lib-modules\") pod \"perf-node-gather-daemonset-g8s92\" (UID: \"5ba14147-39a2-4d0c-80f1-02903bf00351\") " pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-g8s92"
Sep 14 03:26:01.040380 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:01.040351    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6gsd\" (UniqueName: \"kubernetes.io/projected/5ba14147-39a2-4d0c-80f1-02903bf00351-kube-api-access-q6gsd\") pod \"perf-node-gather-daemonset-g8s92\" (UID: \"5ba14147-39a2-4d0c-80f1-02903bf00351\") " pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-g8s92"
Sep 14 03:26:01.206224 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:01.206162    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-g8s92"
Sep 14 03:26:01.233579 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:01.233553    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-57cf94574b-b8kq5_d58f253a-1db9-42a9-ab18-786f2024a97f/console/0.log"
Sep 14 03:26:01.479181 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:01.479153    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9xgtv/perf-node-gather-daemonset-g8s92"]
Sep 14 03:26:01.481664 ip-10-0-141-180 kubenswrapper[2577]: W0914 03:26:01.481637    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5ba14147_39a2_4d0c_80f1_02903bf00351.slice/crio-871ab6a3d9bf28832fe85ecb51e2ec12c55f34b8aa07ce19f6b127b3c9d307fd WatchSource:0}: Error finding container 871ab6a3d9bf28832fe85ecb51e2ec12c55f34b8aa07ce19f6b127b3c9d307fd: Status 404 returned error can't find the container with id 871ab6a3d9bf28832fe85ecb51e2ec12c55f34b8aa07ce19f6b127b3c9d307fd
Sep 14 03:26:01.651905 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:01.651879    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-g8s92" event={"ID":"5ba14147-39a2-4d0c-80f1-02903bf00351","Type":"ContainerStarted","Data":"5af583dbd0c95b38ceedb57e6e2a8924f4582198b6b8cb5adbc5337b0f297362"}
Sep 14 03:26:01.652016 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:01.651932    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-g8s92"
Sep 14 03:26:01.652016 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:01.651956    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-g8s92" event={"ID":"5ba14147-39a2-4d0c-80f1-02903bf00351","Type":"ContainerStarted","Data":"871ab6a3d9bf28832fe85ecb51e2ec12c55f34b8aa07ce19f6b127b3c9d307fd"}
Sep 14 03:26:01.674144 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:01.674090    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-g8s92" podStartSLOduration=1.674074683 podStartE2EDuration="1.674074683s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 03:26:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:26:01.672726606 +0000 UTC m=+2110.650704639" watchObservedRunningTime="2026-09-14 03:26:01.674074683 +0000 UTC m=+2110.652052711"
Sep 14 03:26:03.227115 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:03.227084    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-v5tc4_df25d2a1-9825-4117-a68b-bdb22001d3d6/dns/0.log"
Sep 14 03:26:03.249826 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:03.249793    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-v5tc4_df25d2a1-9825-4117-a68b-bdb22001d3d6/kube-rbac-proxy/0.log"
Sep 14 03:26:03.334488 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:03.334462    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-szm7r_79878e2e-9231-4688-b888-dc8db6184780/dns-node-resolver/0.log"
Sep 14 03:26:04.010760 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:04.010727    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-57d69d9c4c-rvt8r_673eb2ed-da11-47d3-b059-228075d65381/registry/0.log"
Sep 14 03:26:04.104041 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:04.104009    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-w8fds_af1c701f-81cb-47aa-85a1-439d4f260cea/node-ca/0.log"
Sep 14 03:26:05.644243 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:05.644211    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-fnd68_7dd41e91-99ac-4015-9f0f-c9423b47fe3f/discovery/0.log"
Sep 14 03:26:05.691310 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:05.691286    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66f955bb57-d45jp_1b30e897-08f8-4bcd-a435-380fcd3ce905/kube-auth-proxy/0.log"
Sep 14 03:26:05.748986 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:05.748939    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-558d9b8847-hbpwz_9d22b886-4625-41cd-b7ed-bf3d0a25020e/payload-pre-processing/0.log"
Sep 14 03:26:05.774372 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:05.774354    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-74969cc5f8-nzb2n_02709d9b-915f-4d2b-8bd2-0b73f296f99b/payload-processing/0.log"
Sep 14 03:26:05.806067 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:05.806050    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6d9b48664b-648rd_713ae3f6-786b-4911-92ae-854cf6c825b1/router/0.log"
Sep 14 03:26:06.714368 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:06.714341    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lvxg5_b0761e95-5448-4b3d-a607-5eebfc1afc7d/serve-healthcheck-canary/0.log"
Sep 14 03:26:07.234457 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:07.234432    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bqblc_7484daa2-029b-4488-b2aa-1123a49b3fd8/kube-rbac-proxy/0.log"
Sep 14 03:26:07.260088 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:07.260067    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bqblc_7484daa2-029b-4488-b2aa-1123a49b3fd8/exporter/0.log"
Sep 14 03:26:07.292774 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:07.292739    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bqblc_7484daa2-029b-4488-b2aa-1123a49b3fd8/extractor/0.log"
Sep 14 03:26:07.669102 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:07.669025    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9xgtv/perf-node-gather-daemonset-g8s92"
Sep 14 03:26:10.246587 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:10.246560    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-nnzpk_824cabb4-f00b-405c-a42f-8c8d1a7d19bd/manager/0.log"
Sep 14 03:26:10.315032 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:10.315009    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-64b88c4d64-f9j6t_bd7fdffb-de2a-43e2-8b00-7c3f8d293d67/manager/0.log"
Sep 14 03:26:10.340581 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:10.340561    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-ghwhx_398f036d-5636-4f09-b52c-c21ecbaf33f5/server/0.log"
Sep 14 03:26:10.465776 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:10.465748    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-6c2d2_4afcf0b4-40de-4320-8239-222e77a6ebdc/manager/0.log"
Sep 14 03:26:10.491561 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:10.491542    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f647699c6-58rls_f9f740b2-f51d-42b7-9867-a864ac7e05ee/manager/0.log"
Sep 14 03:26:10.511848 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:10.511831    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f647699c6-58rls_f9f740b2-f51d-42b7-9867-a864ac7e05ee/copy-manifests/0.log"
Sep 14 03:26:11.608117 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:11.608087    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-fnd68_7dd41e91-99ac-4015-9f0f-c9423b47fe3f/discovery/0.log"
Sep 14 03:26:11.639260 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:11.639225    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66f955bb57-d45jp_1b30e897-08f8-4bcd-a435-380fcd3ce905/kube-auth-proxy/0.log"
Sep 14 03:26:11.681099 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:11.681076    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-558d9b8847-hbpwz_9d22b886-4625-41cd-b7ed-bf3d0a25020e/payload-pre-processing/0.log"
Sep 14 03:26:11.697505 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:11.697485    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-74969cc5f8-nzb2n_02709d9b-915f-4d2b-8bd2-0b73f296f99b/payload-processing/0.log"
Sep 14 03:26:11.720959 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:11.720939    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6d9b48664b-648rd_713ae3f6-786b-4911-92ae-854cf6c825b1/router/0.log"
Sep 14 03:26:13.769622 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:13.769591    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-fnd68_7dd41e91-99ac-4015-9f0f-c9423b47fe3f/discovery/0.log"
Sep 14 03:26:13.805495 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:13.805468    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66f955bb57-d45jp_1b30e897-08f8-4bcd-a435-380fcd3ce905/kube-auth-proxy/0.log"
Sep 14 03:26:13.844539 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:13.844519    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-558d9b8847-hbpwz_9d22b886-4625-41cd-b7ed-bf3d0a25020e/payload-pre-processing/0.log"
Sep 14 03:26:13.858966 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:13.858937    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-74969cc5f8-nzb2n_02709d9b-915f-4d2b-8bd2-0b73f296f99b/payload-processing/0.log"
Sep 14 03:26:13.884535 ip-10-0-141-180 kubenswrapper[2577]: I0914 03:26:13.884504    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6d9b48664b-648rd_713ae3f6-786b-4911-92ae-854cf6c825b1/router/0.log"