Sep 15 20:04:52.382467 ip-10-0-140-72 systemd[1]: Starting Kubernetes Kubelet...
Sep 15 20:04:52.762283 ip-10-0-140-72 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 15 20:04:52.762283 ip-10-0-140-72 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 15 20:04:52.762283 ip-10-0-140-72 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 15 20:04:52.762283 ip-10-0-140-72 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 15 20:04:52.762283 ip-10-0-140-72 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 15 20:04:52.762988 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.762904    2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 15 20:04:52.766477 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766461    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 20:04:52.766477 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766476    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 20:04:52.766544 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766482    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 20:04:52.766544 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766487    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 20:04:52.766544 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766490    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 20:04:52.766544 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766493    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 20:04:52.766544 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766496    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 20:04:52.766544 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766499    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 20:04:52.766544 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766503    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 20:04:52.766544 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766505    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 20:04:52.766544 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766508    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 20:04:52.766544 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766511    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 20:04:52.766544 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766513    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 20:04:52.766544 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766516    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 20:04:52.766544 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766519    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 20:04:52.766544 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766522    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 20:04:52.766544 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766524    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 20:04:52.766544 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766527    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 20:04:52.766544 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766530    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 20:04:52.766544 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766532    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 20:04:52.766544 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766535    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 20:04:52.766995 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766537    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 20:04:52.766995 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766540    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 20:04:52.766995 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766542    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 20:04:52.766995 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766545    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 20:04:52.766995 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766548    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 20:04:52.766995 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766551    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 20:04:52.766995 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766553    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 20:04:52.766995 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766556    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 20:04:52.766995 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766559    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 20:04:52.766995 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766562    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 20:04:52.766995 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766564    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 20:04:52.766995 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766567    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 20:04:52.766995 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766570    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 20:04:52.766995 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766572    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 20:04:52.766995 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766575    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 20:04:52.766995 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766577    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 20:04:52.766995 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766580    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 20:04:52.766995 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766582    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 20:04:52.766995 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766585    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 20:04:52.766995 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766588    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 20:04:52.767507 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766590    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 20:04:52.767507 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766592    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 20:04:52.767507 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766595    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 20:04:52.767507 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766598    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 20:04:52.767507 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766600    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 20:04:52.767507 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766603    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 20:04:52.767507 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766605    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 20:04:52.767507 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766609    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 20:04:52.767507 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766612    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 20:04:52.767507 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766615    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 20:04:52.767507 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766618    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 20:04:52.767507 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766620    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 20:04:52.767507 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766623    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 20:04:52.767507 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766626    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 20:04:52.767507 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766628    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 20:04:52.767507 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766631    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 20:04:52.767507 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766635    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 20:04:52.767507 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766639    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 20:04:52.767507 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766641    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 20:04:52.767963 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766645    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 20:04:52.767963 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766648    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 20:04:52.767963 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766651    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 20:04:52.767963 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766654    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 20:04:52.767963 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766656    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 20:04:52.767963 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766659    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 20:04:52.767963 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766662    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 20:04:52.767963 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766664    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 20:04:52.767963 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766667    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 20:04:52.767963 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766669    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 20:04:52.767963 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766672    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 20:04:52.767963 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766675    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 20:04:52.767963 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766678    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 20:04:52.767963 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766680    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 20:04:52.767963 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766683    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 20:04:52.767963 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766685    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 20:04:52.767963 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766687    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 20:04:52.767963 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766691    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 20:04:52.767963 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766693    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 20:04:52.767963 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766696    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 20:04:52.768471 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766699    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 20:04:52.768471 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766702    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 20:04:52.768471 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766704    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 20:04:52.768471 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766707    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 20:04:52.768471 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766709    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 20:04:52.768471 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766712    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 20:04:52.768471 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766715    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 20:04:52.768471 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766718    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 20:04:52.768471 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766721    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 20:04:52.768471 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766723    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 20:04:52.768471 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766726    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 20:04:52.768471 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766729    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 20:04:52.768471 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766731    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 20:04:52.768471 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766734    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 20:04:52.768471 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766736    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 20:04:52.768471 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766738    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 20:04:52.768471 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766741    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 20:04:52.768471 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766743    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 20:04:52.768471 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.766746    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 20:04:52.768471 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767449    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 20:04:52.768960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767456    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 20:04:52.768960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767461    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 20:04:52.768960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767465    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 20:04:52.768960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767469    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 20:04:52.768960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767472    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 20:04:52.768960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767476    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 20:04:52.768960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767478    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 20:04:52.768960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767481    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 20:04:52.768960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767484    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 20:04:52.768960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767487    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 20:04:52.768960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767489    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 20:04:52.768960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767492    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 20:04:52.768960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767494    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 20:04:52.768960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767497    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 20:04:52.768960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767501    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 20:04:52.768960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767504    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 20:04:52.768960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767506    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 20:04:52.768960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767511    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 20:04:52.768960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767514    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 20:04:52.769478 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767516    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 20:04:52.769478 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767519    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 20:04:52.769478 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767522    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 20:04:52.769478 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767525    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 20:04:52.769478 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767529    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 20:04:52.769478 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767533    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 20:04:52.769478 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767535    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 20:04:52.769478 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767538    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 20:04:52.769478 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767541    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 20:04:52.769478 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767544    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 20:04:52.769478 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767547    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 20:04:52.769478 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767550    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 20:04:52.769478 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767553    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 20:04:52.769478 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767557    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 20:04:52.769478 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767559    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 20:04:52.769478 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767562    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 20:04:52.769478 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767565    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 20:04:52.769478 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767567    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 20:04:52.769478 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767570    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 20:04:52.769956 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767573    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 20:04:52.769956 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767576    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 20:04:52.769956 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767578    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 20:04:52.769956 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767581    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 20:04:52.769956 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767584    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 20:04:52.769956 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767586    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 20:04:52.769956 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767589    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 20:04:52.769956 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767591    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 20:04:52.769956 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767594    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 20:04:52.769956 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767597    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 20:04:52.769956 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767599    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 20:04:52.769956 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767603    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 20:04:52.769956 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767606    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 20:04:52.769956 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767609    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 20:04:52.769956 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767611    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 20:04:52.769956 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767614    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 20:04:52.769956 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767616    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 20:04:52.769956 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767619    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 20:04:52.769956 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767621    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 20:04:52.769956 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767624    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 20:04:52.770467 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767626    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 20:04:52.770467 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767629    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 20:04:52.770467 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767631    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 20:04:52.770467 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767634    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 20:04:52.770467 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767636    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 20:04:52.770467 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767639    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 20:04:52.770467 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767641    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 20:04:52.770467 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767644    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 20:04:52.770467 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767646    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 20:04:52.770467 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767649    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 20:04:52.770467 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767652    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 20:04:52.770467 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767655    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 20:04:52.770467 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767657    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 20:04:52.770467 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767660    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 20:04:52.770467 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767662    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 20:04:52.770467 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767664    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 20:04:52.770467 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767667    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 20:04:52.770467 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767669    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 20:04:52.770467 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767672    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 20:04:52.770467 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767674    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 20:04:52.770960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767676    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 20:04:52.770960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767679    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 20:04:52.770960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767682    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 20:04:52.770960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767685    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 20:04:52.770960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767687    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 20:04:52.770960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767690    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 20:04:52.770960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767692    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 20:04:52.770960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767695    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 20:04:52.770960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767697    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 20:04:52.770960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767700    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 20:04:52.770960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767702    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 20:04:52.770960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767705    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 20:04:52.770960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767707    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 20:04:52.770960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767710    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 20:04:52.770960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767712    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 20:04:52.770960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767715    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 20:04:52.770960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767717    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 20:04:52.770960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767720    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 20:04:52.770960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767722    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 20:04:52.770960 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.767725    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 20:04:52.771463 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768782    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 20:04:52.771463 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768789    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 20:04:52.771463 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768793    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 20:04:52.771463 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768796    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 20:04:52.771463 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768799    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 20:04:52.771463 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768801    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 20:04:52.771463 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768804    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 20:04:52.771463 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768807    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 20:04:52.771463 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768809    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 20:04:52.771463 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768812    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 20:04:52.771463 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768815    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 20:04:52.771463 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768817    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 20:04:52.771463 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768820    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 20:04:52.771463 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768823    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 20:04:52.771463 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768826    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 20:04:52.771463 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768830    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 20:04:52.771463 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768832    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 20:04:52.771463 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768836    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 20:04:52.771463 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768840    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 20:04:52.771930 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768843    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 20:04:52.771930 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768846    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 20:04:52.771930 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768848    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 20:04:52.771930 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768852    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 20:04:52.771930 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768854    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 20:04:52.771930 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768857    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 20:04:52.771930 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768860    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 20:04:52.771930 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768863    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 20:04:52.771930 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768865    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 20:04:52.771930 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768868    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 20:04:52.771930 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768871    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 20:04:52.771930 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768873    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 20:04:52.771930 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768876    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 20:04:52.771930 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768878    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 20:04:52.771930 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768881    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 20:04:52.771930 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768883    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 20:04:52.771930 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768886    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 20:04:52.771930 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768888    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 20:04:52.771930 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768899    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 20:04:52.771930 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768902    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 20:04:52.772422 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768905    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 20:04:52.772422 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768908    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 20:04:52.772422 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768911    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 20:04:52.772422 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768913    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 20:04:52.772422 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768916    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 20:04:52.772422 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768918    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 20:04:52.772422 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768921    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 20:04:52.772422 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768927    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 20:04:52.772422 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768931    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 20:04:52.772422 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768934    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 20:04:52.772422 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768937    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 20:04:52.772422 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768940    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 20:04:52.772422 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768942    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 20:04:52.772422 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768945    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 20:04:52.772422 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768947    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 20:04:52.772422 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768950    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 20:04:52.772422 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768952    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 20:04:52.772422 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768955    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 20:04:52.772422 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768957    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 20:04:52.772422 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768960    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 20:04:52.772920 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768963    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 20:04:52.772920 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768965    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 20:04:52.772920 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768968    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 20:04:52.772920 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768971    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 20:04:52.772920 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768974    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 20:04:52.772920 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768977    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 20:04:52.772920 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768979    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 20:04:52.772920 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768981    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 20:04:52.772920 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768984    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 20:04:52.772920 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768986    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 20:04:52.772920 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768989    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 20:04:52.772920 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768992    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 20:04:52.772920 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768995    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 20:04:52.772920 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.768998    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 20:04:52.772920 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769000    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 20:04:52.772920 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769003    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 20:04:52.772920 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769005    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 20:04:52.772920 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769008    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 20:04:52.772920 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769010    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 20:04:52.772920 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769014    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 20:04:52.772920 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769017    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 20:04:52.773462 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769019    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 20:04:52.773462 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769022    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 20:04:52.773462 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769024    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 20:04:52.773462 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769026    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 20:04:52.773462 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769030    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 20:04:52.773462 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769033    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 20:04:52.773462 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769036    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 20:04:52.773462 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769038    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 20:04:52.773462 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769041    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 20:04:52.773462 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769044    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 20:04:52.773462 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769046    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 20:04:52.773462 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769048    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 20:04:52.773462 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769052    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 20:04:52.773462 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769055    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 20:04:52.773462 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769057    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 20:04:52.773462 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769059    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 20:04:52.773462 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769062    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 20:04:52.773462 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769064    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 20:04:52.773462 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769067    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 20:04:52.773920 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769130    2577 flags.go:64] FLAG: --address="0.0.0.0"
Sep 15 20:04:52.773920 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769174    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 15 20:04:52.773920 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769181    2577 flags.go:64] FLAG: --anonymous-auth="true"
Sep 15 20:04:52.773920 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769185    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 15 20:04:52.773920 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769190    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 15 20:04:52.773920 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769194    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 15 20:04:52.773920 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769198    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 15 20:04:52.773920 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769203    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 15 20:04:52.773920 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769207    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 15 20:04:52.773920 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769210    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 15 20:04:52.773920 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769213    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 15 20:04:52.773920 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769217    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 15 20:04:52.773920 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769220    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 15 20:04:52.773920 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769224    2577 flags.go:64] FLAG: --cgroup-root=""
Sep 15 20:04:52.773920 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769227    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 15 20:04:52.773920 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769230    2577 flags.go:64] FLAG: --client-ca-file=""
Sep 15 20:04:52.773920 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769233    2577 flags.go:64] FLAG: --cloud-provider="external"
Sep 15 20:04:52.773920 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769236    2577 flags.go:64] FLAG: --cluster-dns="[]"
Sep 15 20:04:52.773920 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769241    2577 flags.go:64] FLAG: --cluster-domain=""
Sep 15 20:04:52.773920 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769244    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 15 20:04:52.773920 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769247    2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 15 20:04:52.773920 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769251    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 15 20:04:52.773920 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769254    2577 flags.go:64] FLAG: --container-log-max-files="5"
Sep 15 20:04:52.773920 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769258    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 15 20:04:52.774572 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769261    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 15 20:04:52.774572 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769264    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 15 20:04:52.774572 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769267    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 15 20:04:52.774572 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769270    2577 flags.go:64] FLAG: --contention-profiling="false"
Sep 15 20:04:52.774572 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769273    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 15 20:04:52.774572 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769277    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 15 20:04:52.774572 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769280    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 15 20:04:52.774572 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769283    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 15 20:04:52.774572 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769292    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 15 20:04:52.774572 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769295    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 15 20:04:52.774572 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769298    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 15 20:04:52.774572 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769301    2577 flags.go:64] FLAG: --enable-load-reader="false"
Sep 15 20:04:52.774572 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769304    2577 flags.go:64] FLAG: --enable-server="true"
Sep 15 20:04:52.774572 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769307    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 15 20:04:52.774572 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769312    2577 flags.go:64] FLAG: --event-burst="100"
Sep 15 20:04:52.774572 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769315    2577 flags.go:64] FLAG: --event-qps="50"
Sep 15 20:04:52.774572 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769318    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 15 20:04:52.774572 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769321    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 15 20:04:52.774572 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769324    2577 flags.go:64] FLAG: --eviction-hard=""
Sep 15 20:04:52.774572 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769328    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 15 20:04:52.774572 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769331    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 15 20:04:52.774572 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769334    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 15 20:04:52.774572 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769338    2577 flags.go:64] FLAG: --eviction-soft=""
Sep 15 20:04:52.774572 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769341    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 15 20:04:52.774572 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769344    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 15 20:04:52.775217 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769347    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 15 20:04:52.775217 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769350    2577 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 15 20:04:52.775217 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769353    2577 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 15 20:04:52.775217 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769356    2577 flags.go:64] FLAG: --fail-swap-on="true"
Sep 15 20:04:52.775217 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769359    2577 flags.go:64] FLAG: --feature-gates=""
Sep 15 20:04:52.775217 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769363    2577 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 15 20:04:52.775217 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769366    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 15 20:04:52.775217 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769369    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 15 20:04:52.775217 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769372    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 15 20:04:52.775217 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769375    2577 flags.go:64] FLAG: --healthz-port="10248"
Sep 15 20:04:52.775217 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769378    2577 flags.go:64] FLAG: --help="false"
Sep 15 20:04:52.775217 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769381    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-140-72.ec2.internal"
Sep 15 20:04:52.775217 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769384    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 15 20:04:52.775217 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769387    2577 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 15 20:04:52.775217 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769390    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 15 20:04:52.775217 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769393    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 15 20:04:52.775217 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769397    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 15 20:04:52.775217 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769399    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 15 20:04:52.775217 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769402    2577 flags.go:64] FLAG: --image-service-endpoint=""
Sep 15 20:04:52.775217 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769407    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 15 20:04:52.775217 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769410    2577 flags.go:64] FLAG: --kube-api-burst="100"
Sep 15 20:04:52.775217 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769413    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 15 20:04:52.775217 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769416    2577 flags.go:64] FLAG: --kube-api-qps="50"
Sep 15 20:04:52.775217 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769419    2577 flags.go:64] FLAG: --kube-reserved=""
Sep 15 20:04:52.775816 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769422    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 15 20:04:52.775816 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769426    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 15 20:04:52.775816 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769429    2577 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 15 20:04:52.775816 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769432    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 15 20:04:52.775816 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769435    2577 flags.go:64] FLAG: --lock-file=""
Sep 15 20:04:52.775816 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769438    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 15 20:04:52.775816 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769441    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 15 20:04:52.775816 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769444    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 15 20:04:52.775816 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769449    2577 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 15 20:04:52.775816 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769452    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 15 20:04:52.775816 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769455    2577 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 15 20:04:52.775816 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769458    2577 flags.go:64] FLAG: --logging-format="text"
Sep 15 20:04:52.775816 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769461    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 15 20:04:52.775816 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769464    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 15 20:04:52.775816 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769467    2577 flags.go:64] FLAG: --manifest-url=""
Sep 15 20:04:52.775816 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769470    2577 flags.go:64] FLAG: --manifest-url-header=""
Sep 15 20:04:52.775816 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769474    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 15 20:04:52.775816 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769477    2577 flags.go:64] FLAG: --max-open-files="1000000"
Sep 15 20:04:52.775816 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769481    2577 flags.go:64] FLAG: --max-pods="110"
Sep 15 20:04:52.775816 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769484    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 15 20:04:52.775816 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769487    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 15 20:04:52.775816 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769490    2577 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 15 20:04:52.775816 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769493    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 15 20:04:52.775816 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769497    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 15 20:04:52.775816 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769500    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 15 20:04:52.776457 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769503    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 15 20:04:52.776457 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769510    2577 flags.go:64] FLAG: --node-status-max-images="50"
Sep 15 20:04:52.776457 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769515    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 15 20:04:52.776457 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769518    2577 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 15 20:04:52.776457 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769521    2577 flags.go:64] FLAG: --pod-cidr=""
Sep 15 20:04:52.776457 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769524    2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 15 20:04:52.776457 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769530    2577 flags.go:64] FLAG: --pod-manifest-path=""
Sep 15 20:04:52.776457 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769533    2577 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 15 20:04:52.776457 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769536    2577 flags.go:64] FLAG: --pods-per-core="0"
Sep 15 20:04:52.776457 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769539    2577 flags.go:64] FLAG: --port="10250"
Sep 15 20:04:52.776457 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769544    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 15 20:04:52.776457 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769547    2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-07db14071518ef79f"
Sep 15 20:04:52.776457 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769551    2577 flags.go:64] FLAG: --qos-reserved=""
Sep 15 20:04:52.776457 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769554    2577 flags.go:64] FLAG: --read-only-port="10255"
Sep 15 20:04:52.776457 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769557    2577 flags.go:64] FLAG: --register-node="true"
Sep 15 20:04:52.776457 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769560    2577 flags.go:64] FLAG: --register-with-taints=""
Sep 15 20:04:52.776457 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769568    2577 flags.go:64] FLAG: --registry-burst="10"
Sep 15 20:04:52.776457 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769571    2577 flags.go:64] FLAG: --registry-qps="5"
Sep 15 20:04:52.776457 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769574    2577 flags.go:64] FLAG: --reserved-cpus=""
Sep 15 20:04:52.776457 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769577    2577 flags.go:64] FLAG: --reserved-memory=""
Sep 15 20:04:52.776457 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769581    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 15 20:04:52.776457 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769584    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 15 20:04:52.776457 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769587    2577 flags.go:64] FLAG: --rotate-certificates="false"
Sep 15 20:04:52.776457 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769590    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 15 20:04:52.776457 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769593    2577 flags.go:64] FLAG: --runonce="false"
Sep 15 20:04:52.777121 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769596    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 15 20:04:52.777121 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769599    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 15 20:04:52.777121 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769602    2577 flags.go:64] FLAG: --seccomp-default="false"
Sep 15 20:04:52.777121 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769605    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 15 20:04:52.777121 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769608    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 15 20:04:52.777121 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769611    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 15 20:04:52.777121 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769614    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 15 20:04:52.777121 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769617    2577 flags.go:64] FLAG: --storage-driver-password="root"
Sep 15 20:04:52.777121 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769620    2577 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 15 20:04:52.777121 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769624    2577 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 15 20:04:52.777121 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769627    2577 flags.go:64] FLAG: --storage-driver-user="root"
Sep 15 20:04:52.777121 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769629    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 15 20:04:52.777121 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769633    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 15 20:04:52.777121 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769635    2577 flags.go:64] FLAG: --system-cgroups=""
Sep 15 20:04:52.777121 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769638    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 15 20:04:52.777121 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769644    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 15 20:04:52.777121 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769647    2577 flags.go:64] FLAG: --tls-cert-file=""
Sep 15 20:04:52.777121 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769651    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 15 20:04:52.777121 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769656    2577 flags.go:64] FLAG: --tls-min-version=""
Sep 15 20:04:52.777121 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769658    2577 flags.go:64] FLAG: --tls-private-key-file=""
Sep 15 20:04:52.777121 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769661    2577 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 15 20:04:52.777121 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769664    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 15 20:04:52.777121 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769668    2577 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 15 20:04:52.777121 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769671    2577 flags.go:64] FLAG: --v="2"
Sep 15 20:04:52.777121 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769674    2577 flags.go:64] FLAG: --version="false"
Sep 15 20:04:52.777849 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769678    2577 flags.go:64] FLAG: --vmodule=""
Sep 15 20:04:52.777849 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769682    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 15 20:04:52.777849 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.769685    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 15 20:04:52.777849 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769800    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 20:04:52.777849 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769804    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 20:04:52.777849 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769807    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 20:04:52.777849 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769810    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 20:04:52.777849 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769813    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 20:04:52.777849 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769816    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 20:04:52.777849 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769818    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 20:04:52.777849 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769821    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 20:04:52.777849 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769824    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 20:04:52.777849 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769826    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 20:04:52.777849 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769829    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 20:04:52.777849 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769832    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 20:04:52.777849 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769834    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 20:04:52.777849 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769839    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 20:04:52.777849 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769841    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 20:04:52.777849 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769844    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 20:04:52.777849 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769846    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 20:04:52.778366 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769849    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 20:04:52.778366 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769851    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 20:04:52.778366 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769854    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 20:04:52.778366 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769857    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 20:04:52.778366 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769868    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 20:04:52.778366 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769871    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 20:04:52.778366 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769874    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 20:04:52.778366 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769876    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 20:04:52.778366 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769879    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 20:04:52.778366 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769881    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 20:04:52.778366 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769885    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 20:04:52.778366 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769887    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 20:04:52.778366 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769890    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 20:04:52.778366 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769894    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 20:04:52.778366 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769898    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 20:04:52.778366 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769901    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 20:04:52.778366 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769903    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 20:04:52.778366 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769906    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 20:04:52.778366 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769908    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 20:04:52.778952 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769913    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 20:04:52.778952 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769916    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 20:04:52.778952 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769920    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 20:04:52.778952 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769922    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 20:04:52.778952 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769925    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 20:04:52.778952 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769928    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 20:04:52.778952 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769930    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 20:04:52.778952 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769933    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 20:04:52.778952 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769936    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 20:04:52.778952 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769940    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 20:04:52.778952 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769943    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 20:04:52.778952 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769946    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 20:04:52.778952 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769949    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 20:04:52.778952 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769951    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 20:04:52.778952 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769954    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 20:04:52.778952 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769957    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 20:04:52.778952 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769960    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 20:04:52.778952 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769964    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 20:04:52.778952 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769967    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 20:04:52.778952 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769969    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 20:04:52.779574 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769972    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 20:04:52.779574 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769975    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 20:04:52.779574 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769979    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 20:04:52.779574 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769982    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 20:04:52.779574 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769986    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 20:04:52.779574 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769989    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 20:04:52.779574 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769991    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 20:04:52.779574 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769994    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 20:04:52.779574 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.769997    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 20:04:52.779574 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.770000    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 20:04:52.779574 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.770002    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 20:04:52.779574 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.770005    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 20:04:52.779574 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.770007    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 20:04:52.779574 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.770010    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 20:04:52.779574 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.770012    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 20:04:52.779574 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.770015    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 20:04:52.779574 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.770017    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 20:04:52.779574 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.770020    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 20:04:52.779574 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.770023    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 20:04:52.780275 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.770026    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 20:04:52.780275 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.770028    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 20:04:52.780275 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.770032    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 20:04:52.780275 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.770035    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 20:04:52.780275 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.770037    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 20:04:52.780275 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.770040    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 20:04:52.780275 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.770042    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 20:04:52.780275 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.770045    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 20:04:52.780275 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.770047    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 20:04:52.780275 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.770050    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 20:04:52.780275 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.770054    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 20:04:52.780275 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.770056    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 20:04:52.780275 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.770059    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 20:04:52.780275 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.770061    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 20:04:52.780275 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.770064    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 20:04:52.780275 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.770066    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 20:04:52.780275 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.770069    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 20:04:52.780275 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.770072    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 20:04:52.780275 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.770075    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 20:04:52.780275 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.770077    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 20:04:52.781011 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.770080    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 20:04:52.781011 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.770082    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 20:04:52.781011 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.770085    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 20:04:52.781011 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.770087    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 20:04:52.781011 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.771228    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 15 20:04:52.781011 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.777821    2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 15 20:04:52.781011 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.777838    2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 15 20:04:52.781011 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.777852    2577 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 15 20:04:52.781011 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.777858    2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 15 20:04:52.781011 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.777930    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 20:04:52.781011 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.777935    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 20:04:52.781011 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.777938    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 20:04:52.781011 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.777941    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 20:04:52.781011 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.777944    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 20:04:52.781011 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.777947    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 20:04:52.781431 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.777949    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 20:04:52.781431 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.777952    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 20:04:52.781431 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.777954    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 20:04:52.781431 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.777957    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 20:04:52.781431 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.777960    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 20:04:52.781431 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.777962    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 20:04:52.781431 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.777965    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 20:04:52.781431 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.777968    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 20:04:52.781431 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.777970    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 20:04:52.781431 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.777973    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 20:04:52.781431 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.777976    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 20:04:52.781431 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.777978    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 20:04:52.781431 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.777981    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 20:04:52.781431 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.777983    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 20:04:52.781431 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.777986    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 20:04:52.781431 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.777991    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 20:04:52.781431 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.777994    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 20:04:52.781431 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.777997    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 20:04:52.781431 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.777999    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 20:04:52.781431 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778002    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 20:04:52.782202 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778005    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 20:04:52.782202 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778008    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 20:04:52.782202 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778011    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 20:04:52.782202 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778014    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 20:04:52.782202 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778017    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 20:04:52.782202 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778019    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 20:04:52.782202 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778022    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 20:04:52.782202 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778025    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 20:04:52.782202 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778029    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 20:04:52.782202 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778033    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 20:04:52.782202 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778036    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 20:04:52.782202 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778038    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 20:04:52.782202 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778041    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 20:04:52.782202 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778043    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 20:04:52.782202 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778046    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 20:04:52.782202 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778049    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 20:04:52.782202 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778053    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 20:04:52.782202 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778057    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 20:04:52.782898 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778060    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 20:04:52.782898 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778063    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 20:04:52.782898 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778065    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 20:04:52.782898 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778068    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 20:04:52.782898 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778070    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 20:04:52.782898 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778073    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 20:04:52.782898 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778075    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 20:04:52.782898 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778079    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 20:04:52.782898 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778083    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 20:04:52.782898 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778085    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 20:04:52.782898 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778089    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 20:04:52.782898 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778092    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 20:04:52.782898 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778095    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 20:04:52.782898 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778097    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 20:04:52.782898 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778100    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 20:04:52.782898 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778103    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 20:04:52.782898 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778105    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 20:04:52.782898 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778109    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 20:04:52.782898 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778112    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 20:04:52.782898 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778114    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 20:04:52.783645 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778117    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 20:04:52.783645 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778119    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 20:04:52.783645 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778122    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 20:04:52.783645 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778125    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 20:04:52.783645 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778127    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 20:04:52.783645 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778130    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 20:04:52.783645 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778132    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 20:04:52.783645 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778150    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 20:04:52.783645 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778154    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 20:04:52.783645 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778157    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 20:04:52.783645 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778159    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 20:04:52.783645 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778162    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 20:04:52.783645 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778165    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 20:04:52.783645 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778168    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 20:04:52.783645 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778170    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 20:04:52.783645 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778173    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 20:04:52.783645 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778175    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 20:04:52.783645 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778178    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 20:04:52.783645 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778180    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 20:04:52.783645 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778183    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 20:04:52.784134 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778185    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 20:04:52.784134 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778188    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 20:04:52.784134 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778190    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 20:04:52.784134 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778194    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 20:04:52.784134 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778196    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 20:04:52.784134 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778199    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 20:04:52.784134 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778202    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 20:04:52.784134 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778205    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 20:04:52.784134 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778207    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 20:04:52.784134 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778211    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 20:04:52.784134 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778213    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 20:04:52.784134 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778216    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 20:04:52.784134 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778219    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 20:04:52.784134 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778221    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 20:04:52.784134 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778224    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 20:04:52.784582 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.778229    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 15 20:04:52.784582 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778342    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 20:04:52.784582 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778347    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 20:04:52.784582 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778351    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 20:04:52.784582 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778355    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 20:04:52.784582 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778359    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 20:04:52.784582 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778362    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 20:04:52.784582 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778365    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 20:04:52.784582 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778368    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 20:04:52.784582 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778372    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 20:04:52.784582 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778374    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 20:04:52.784582 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778377    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 20:04:52.784582 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778380    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 20:04:52.784582 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778383    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 20:04:52.784949 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778386    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 20:04:52.784949 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778388    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 20:04:52.784949 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778391    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 20:04:52.784949 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778394    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 20:04:52.784949 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778396    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 20:04:52.784949 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778399    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 20:04:52.784949 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778401    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 20:04:52.784949 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778405    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 20:04:52.784949 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778407    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 20:04:52.784949 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778410    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 20:04:52.784949 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778412    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 20:04:52.784949 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778415    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 20:04:52.784949 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778418    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 20:04:52.784949 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778421    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 20:04:52.784949 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778423    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 20:04:52.784949 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778426    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 20:04:52.784949 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778429    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 20:04:52.784949 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778431    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 20:04:52.784949 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778433    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 20:04:52.784949 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778436    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 20:04:52.785459 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778438    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 20:04:52.785459 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778441    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 20:04:52.785459 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778443    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 20:04:52.785459 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778446    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 20:04:52.785459 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778448    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 20:04:52.785459 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778451    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 20:04:52.785459 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778453    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 20:04:52.785459 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778456    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 20:04:52.785459 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778458    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 20:04:52.785459 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778461    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 20:04:52.785459 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778463    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 20:04:52.785459 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778466    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 20:04:52.785459 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778469    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 20:04:52.785459 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778472    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 20:04:52.785459 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778474    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 20:04:52.785459 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778477    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 20:04:52.785459 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778479    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 20:04:52.785459 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778481    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 20:04:52.785459 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778484    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 20:04:52.785459 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778486    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 20:04:52.786013 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778490    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 20:04:52.786013 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778492    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 20:04:52.786013 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778495    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 20:04:52.786013 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778498    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 20:04:52.786013 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778501    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 20:04:52.786013 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778503    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 20:04:52.786013 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778506    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 20:04:52.786013 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778508    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 20:04:52.786013 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778511    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 20:04:52.786013 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778513    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 20:04:52.786013 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778516    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 20:04:52.786013 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778518    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 20:04:52.786013 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778521    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 20:04:52.786013 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778524    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 20:04:52.786013 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778526    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 20:04:52.786013 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778528    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 20:04:52.786013 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778531    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 20:04:52.786013 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778533    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 20:04:52.786013 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778536    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 20:04:52.786013 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778538    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 20:04:52.786553 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778541    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 20:04:52.786553 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778543    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 20:04:52.786553 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778545    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 20:04:52.786553 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778548    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 20:04:52.786553 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778551    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 20:04:52.786553 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778553    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 20:04:52.786553 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778556    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 20:04:52.786553 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778559    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 20:04:52.786553 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778561    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 20:04:52.786553 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778564    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 20:04:52.786553 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778566    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 20:04:52.786553 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778568    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 20:04:52.786553 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778571    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 20:04:52.786553 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778574    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 20:04:52.786553 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778577    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 20:04:52.786553 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778580    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 20:04:52.786553 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778582    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 20:04:52.786553 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778585    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 20:04:52.786553 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778588    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 20:04:52.786553 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778590    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 20:04:52.787116 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778593    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 20:04:52.787116 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778595    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 20:04:52.787116 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778599    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 20:04:52.787116 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778602    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 20:04:52.787116 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778605    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 20:04:52.787116 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:52.778608    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 20:04:52.787116 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.778612    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 15 20:04:52.787116 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.779624    2577 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 15 20:04:52.787116 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.783287    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 15 20:04:52.787116 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.784397    2577 server.go:1019] "Starting client certificate rotation"
Sep 15 20:04:52.787116 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.784501    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 15 20:04:52.787116 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.784531    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 15 20:04:52.787636 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.787569    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 15 20:04:52.792245 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.792223    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 15 20:04:52.803494 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.803475    2577 log.go:25] "Validated CRI v1 runtime API"
Sep 15 20:04:52.806691 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.806673    2577 log.go:25] "Validated CRI v1 image API"
Sep 15 20:04:52.808151 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.808120    2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 15 20:04:52.811003 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.810979    2577 fs.go:135] Filesystem UUIDs: map[0952b4dc-8403-496d-a732-796a6821b350:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 ca49202f-b2cc-4810-85b6-70dfeb42030e:/dev/nvme0n1p4]
Sep 15 20:04:52.811050 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.811003    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 15 20:04:52.814903 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.814886    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 15 20:04:52.815639 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.815495    2577 manager.go:216] Machine: {Timestamp:2026-09-15 20:04:52.814983592 +0000 UTC m=+0.324522162 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100078 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c86cccadc03615c4f735d357532e2 SystemUUID:ec2c86cc-cadc-0361-5c4f-735d357532e2 BootID:8f8eb5aa-6ad7-41c8-aa9e-cb9a2ffe70d5 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:d8:ab:30:99 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:d8:ab:30:99 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1e:96:95:d5:10:78 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 15 20:04:52.815639 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.815637    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 15 20:04:52.815766 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.815754    2577 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 15 20:04:52.817355 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.817321    2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 15 20:04:52.817524 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.817358    2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-72.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 15 20:04:52.818597 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.818584    2577 topology_manager.go:138] "Creating topology manager with none policy"
Sep 15 20:04:52.818663 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.818601    2577 container_manager_linux.go:309] "Creating device plugin manager"
Sep 15 20:04:52.818663 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.818614    2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 15 20:04:52.819183 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.819170    2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 15 20:04:52.819963 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.819951    2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 15 20:04:52.822967 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.822952    2577 state_mem.go:36] "Initialized new in-memory state store"
Sep 15 20:04:52.823108 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.823097    2577 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 15 20:04:52.823196 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.823153    2577 kubelet.go:486] "Attempting to sync node with API server"
Sep 15 20:04:52.823196 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.823168    2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 15 20:04:52.823196 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.823184    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 15 20:04:52.823310 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.823198    2577 kubelet.go:398] "Adding apiserver pod source"
Sep 15 20:04:52.823310 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.823210    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 15 20:04:52.823616 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.823598    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 15 20:04:52.823688 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.823626    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 15 20:04:52.827318 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.827235    2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 15 20:04:52.829533 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.829512    2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 15 20:04:52.829605 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.829547    2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 15 20:04:52.831710 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.831695    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 15 20:04:52.831771 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.831714    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 15 20:04:52.831771 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.831721    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 15 20:04:52.831771 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.831726    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 15 20:04:52.831771 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.831736    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 15 20:04:52.831771 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.831741    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 15 20:04:52.831771 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.831754    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 15 20:04:52.831771 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.831762    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 15 20:04:52.831771 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.831772    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 15 20:04:52.832002 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.831778    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 15 20:04:52.832002 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.831787    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 15 20:04:52.832002 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.831792    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 15 20:04:52.832283 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:52.832259    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-72.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 15 20:04:52.832348 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:52.832273    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 15 20:04:52.832710 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.832700    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 15 20:04:52.832745 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.832711    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 15 20:04:52.835876 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.835853    2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-72.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 15 20:04:52.836417 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.836405    2577 server.go:1268] "Started kubelet"
Sep 15 20:04:52.836535 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.836486    2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 15 20:04:52.836579 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.836560    2577 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 15 20:04:52.837096 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.837079    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nmksh"
Sep 15 20:04:52.837312 ip-10-0-140-72 systemd[1]: Started Kubernetes Kubelet.
Sep 15 20:04:52.837481 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.837361    2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 15 20:04:52.837616 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.837600    2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 15 20:04:52.839366 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.839349    2577 server.go:310] "Adding debug handlers to kubelet server"
Sep 15 20:04:52.842290 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.842277    2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 15 20:04:52.842360 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.842341    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 15 20:04:52.842539 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:52.842495    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-72.ec2.internal\" not found"
Sep 15 20:04:52.842886 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.842835    2577 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 15 20:04:52.842886 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.842851    2577 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 15 20:04:52.842996 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.842952    2577 reconstruct.go:97] "Volume reconstruction finished"
Sep 15 20:04:52.842996 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:52.842952    2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 15 20:04:52.842996 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.842960    2577 reconciler.go:29] "Reconciler: start to sync state"
Sep 15 20:04:52.843263 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.843245    2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 15 20:04:52.843495 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.843476    2577 factory.go:55] Registering systemd factory
Sep 15 20:04:52.843495 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.843495    2577 factory.go:223] Registration of the systemd container factory successfully
Sep 15 20:04:52.843712 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.843695    2577 factory.go:153] Registering CRI-O factory
Sep 15 20:04:52.843768 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.843719    2577 factory.go:223] Registration of the crio container factory successfully
Sep 15 20:04:52.843823 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.843773    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 15 20:04:52.843823 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.843794    2577 factory.go:103] Registering Raw factory
Sep 15 20:04:52.843823 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.843811    2577 manager.go:1196] Started watching for new ooms in manager
Sep 15 20:04:52.844223 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.844209    2577 manager.go:349] Starting recovery of all containers
Sep 15 20:04:52.844511 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.844490    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nmksh"
Sep 15 20:04:52.850401 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:52.850213    2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-72.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 15 20:04:52.850401 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:52.850292    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 15 20:04:52.851313 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:52.850273    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-72.ec2.internal.18d59737cb1e01fa  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-72.ec2.internal,UID:ip-10-0-140-72.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-72.ec2.internal,},FirstTimestamp:2026-09-15 20:04:52.836377082 +0000 UTC m=+0.345915651,LastTimestamp:2026-09-15 20:04:52.836377082 +0000 UTC m=+0.345915651,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-72.ec2.internal,}"
Sep 15 20:04:52.854319 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.854301    2577 manager.go:354] Recovery completed
Sep 15 20:04:52.859657 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.859643    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 20:04:52.863546 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.863528    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-72.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 20:04:52.863622 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.863562    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-72.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 20:04:52.863622 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.863578    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-72.ec2.internal" event="NodeHasSufficientPID"
Sep 15 20:04:52.864130 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.864115    2577 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 15 20:04:52.864195 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.864130    2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 15 20:04:52.864225 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.864132    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 20:04:52.864225 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.864211    2577 state_mem.go:36] "Initialized new in-memory state store"
Sep 15 20:04:52.865116 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.865100    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-72.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 20:04:52.865195 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.865126    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-72.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 20:04:52.865195 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.865160    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-72.ec2.internal" event="NodeHasSufficientPID"
Sep 15 20:04:52.865534 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.865519    2577 policy_none.go:49] "None policy: Start"
Sep 15 20:04:52.865534 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.865534    2577 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 15 20:04:52.865637 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.865545    2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 15 20:04:52.866442 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.866431    2577 policy_none.go:47] "Start"
Sep 15 20:04:52.907834 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.903734    2577 manager.go:337] "Starting Device Plugin manager"
Sep 15 20:04:52.907834 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:52.903880    2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 15 20:04:52.907834 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.903890    2577 server.go:87] "Starting device plugin registration server"
Sep 15 20:04:52.907834 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.904128    2577 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 15 20:04:52.907834 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.904157    2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 15 20:04:52.907834 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.904255    2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 15 20:04:52.907834 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.904328    2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 15 20:04:52.907834 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.904336    2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 15 20:04:52.907834 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:52.904796    2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 15 20:04:52.907834 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:52.904850    2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-72.ec2.internal\" not found"
Sep 15 20:04:52.933512 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.933486    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 15 20:04:52.934708 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.934687    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 15 20:04:52.934789 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.934748    2577 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 15 20:04:52.934789 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.934775    2577 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 15 20:04:52.934877 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:52.934864    2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 15 20:04:52.937046 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:52.937028    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 20:04:53.005178 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:53.005126    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 20:04:53.006206 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:53.006181    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-72.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 20:04:53.006338 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:53.006218    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-72.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 20:04:53.006338 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:53.006232    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-72.ec2.internal" event="NodeHasSufficientPID"
Sep 15 20:04:53.006338 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:53.006261    2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-72.ec2.internal"
Sep 15 20:04:53.012247 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:53.012231    2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-72.ec2.internal"
Sep 15 20:04:53.012355 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:53.012254    2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-72.ec2.internal\": node \"ip-10-0-140-72.ec2.internal\" not found"
Sep 15 20:04:53.031930 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:53.031907    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-72.ec2.internal\" not found"
Sep 15 20:04:53.035973 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:53.035947    2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-72.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-72.ec2.internal"]
Sep 15 20:04:53.036021 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:53.036017    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 20:04:53.037605 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:53.037589    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-72.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 20:04:53.037676 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:53.037618    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-72.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 20:04:53.037676 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:53.037633    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-72.ec2.internal" event="NodeHasSufficientPID"
Sep 15 20:04:53.039026 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:53.039014    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 20:04:53.039160 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:53.039130    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-72.ec2.internal"
Sep 15 20:04:53.039200 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:53.039183    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 20:04:53.039783 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:53.039768    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-72.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 20:04:53.039850 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:53.039768    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-72.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 20:04:53.039850 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:53.039799    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-72.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 20:04:53.039850 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:53.039810    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-72.ec2.internal" event="NodeHasSufficientPID"
Sep 15 20:04:53.039850 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:53.039817    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-72.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 20:04:53.039850 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:53.039834    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-72.ec2.internal" event="NodeHasSufficientPID"
Sep 15 20:04:53.040918 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:53.040902    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-72.ec2.internal"
Sep 15 20:04:53.040970 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:53.040930    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 20:04:53.041621 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:53.041607    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-72.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 20:04:53.041706 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:53.041638    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-72.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 20:04:53.041706 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:53.041652    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-72.ec2.internal" event="NodeHasSufficientPID"
Sep 15 20:04:53.068861 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:53.068839    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-72.ec2.internal\" not found" node="ip-10-0-140-72.ec2.internal"
Sep 15 20:04:53.073345 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:53.073326    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-72.ec2.internal\" not found" node="ip-10-0-140-72.ec2.internal"
Sep 15 20:04:53.132783 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:53.132760    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-72.ec2.internal\" not found"
Sep 15 20:04:53.145129 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:53.145106    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/eac46b2c823b79bb709b7641ecf92419-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-72.ec2.internal\" (UID: \"eac46b2c823b79bb709b7641ecf92419\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-72.ec2.internal"
Sep 15 20:04:53.145213 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:53.145150    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eac46b2c823b79bb709b7641ecf92419-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-72.ec2.internal\" (UID: \"eac46b2c823b79bb709b7641ecf92419\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-72.ec2.internal"
Sep 15 20:04:53.145213 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:53.145168    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6967c7c3d2c13b8fd404f2bb5001259f-config\") pod \"kube-apiserver-proxy-ip-10-0-140-72.ec2.internal\" (UID: \"6967c7c3d2c13b8fd404f2bb5001259f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-72.ec2.internal"
Sep 15 20:04:53.233599 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:53.233577    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-72.ec2.internal\" not found"
Sep 15 20:04:53.246005 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:53.245985    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/eac46b2c823b79bb709b7641ecf92419-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-72.ec2.internal\" (UID: \"eac46b2c823b79bb709b7641ecf92419\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-72.ec2.internal"
Sep 15 20:04:53.246061 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:53.246012    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eac46b2c823b79bb709b7641ecf92419-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-72.ec2.internal\" (UID: \"eac46b2c823b79bb709b7641ecf92419\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-72.ec2.internal"
Sep 15 20:04:53.246061 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:53.246028    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6967c7c3d2c13b8fd404f2bb5001259f-config\") pod \"kube-apiserver-proxy-ip-10-0-140-72.ec2.internal\" (UID: \"6967c7c3d2c13b8fd404f2bb5001259f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-72.ec2.internal"
Sep 15 20:04:53.246129 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:53.246065    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6967c7c3d2c13b8fd404f2bb5001259f-config\") pod \"kube-apiserver-proxy-ip-10-0-140-72.ec2.internal\" (UID: \"6967c7c3d2c13b8fd404f2bb5001259f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-72.ec2.internal"
Sep 15 20:04:53.246129 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:53.246069    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/eac46b2c823b79bb709b7641ecf92419-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-72.ec2.internal\" (UID: \"eac46b2c823b79bb709b7641ecf92419\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-72.ec2.internal"
Sep 15 20:04:53.246129 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:53.246096    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eac46b2c823b79bb709b7641ecf92419-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-72.ec2.internal\" (UID: \"eac46b2c823b79bb709b7641ecf92419\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-72.ec2.internal"
Sep 15 20:04:53.334502 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:53.334420    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-72.ec2.internal\" not found"
Sep 15 20:04:53.370927 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:53.370901    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-72.ec2.internal"
Sep 15 20:04:53.375381 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:53.375279    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-72.ec2.internal"
Sep 15 20:04:53.434857 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:53.434820    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-72.ec2.internal\" not found"
Sep 15 20:04:54.654050 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:54.654005    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-72.ec2.internal\" not found"
Sep 15 20:04:54.754632 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:54.754590    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-72.ec2.internal\" not found"
Sep 15 20:04:54.855229 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:54.855183    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-72.ec2.internal\" not found"
Sep 15 20:04:54.902777 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:54.902743    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 15 20:04:54.903392 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:54.902881    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 15 20:04:54.955291 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:54.955258    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-72.ec2.internal\" not found"
Sep 15 20:04:54.966946 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:54.966905    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-14 19:59:53 +0000 UTC" deadline="2028-03-14 18:28:06.791371339 +0000 UTC"
Sep 15 20:04:54.966946 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:54.966941    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13102h23m11.824434685s"
Sep 15 20:04:55.056145 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:55.056124    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-72.ec2.internal\" not found"
Sep 15 20:04:55.057191 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.057173    2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 20:04:55.060384 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:55.060353    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6967c7c3d2c13b8fd404f2bb5001259f.slice/crio-4f006af009faaab1d0b1e343b56d56d50b2328a6904eb5fd53ad93f5144cf723 WatchSource:0}: Error finding container 4f006af009faaab1d0b1e343b56d56d50b2328a6904eb5fd53ad93f5144cf723: Status 404 returned error can't find the container with id 4f006af009faaab1d0b1e343b56d56d50b2328a6904eb5fd53ad93f5144cf723
Sep 15 20:04:55.061427 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.061408    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-72.ec2.internal"
Sep 15 20:04:55.065618 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.065602    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 20:04:55.072364 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.072347    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 15 20:04:55.074056 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.074042    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-72.ec2.internal"
Sep 15 20:04:55.083653 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.083633    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 15 20:04:55.130595 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:55.130563    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeac46b2c823b79bb709b7641ecf92419.slice/crio-07a7785cc88a73f195c67f8dd6e81ff5b9c461264a1a7317fd17f0a67597de32 WatchSource:0}: Error finding container 07a7785cc88a73f195c67f8dd6e81ff5b9c461264a1a7317fd17f0a67597de32: Status 404 returned error can't find the container with id 07a7785cc88a73f195c67f8dd6e81ff5b9c461264a1a7317fd17f0a67597de32
Sep 15 20:04:55.324326 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.324243    2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 20:04:55.333172 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.333145    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 20:04:55.942053 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.942018    2577 apiserver.go:52] "Watching apiserver"
Sep 15 20:04:55.954113 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.954086    2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 15 20:04:55.955308 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.955275    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-z5wjn","openshift-image-registry/node-ca-vm6lc","openshift-multus/multus-additional-cni-plugins-kjp4l","openshift-multus/multus-brdnq","openshift-network-operator/iptables-alerter-fmkfr","kube-system/global-pull-secret-syncer-9dwjh","kube-system/kube-apiserver-proxy-ip-10-0-140-72.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqj9r","openshift-cluster-node-tuning-operator/tuned-k6cn5","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-72.ec2.internal","openshift-multus/network-metrics-daemon-2v4rt","openshift-network-diagnostics/network-check-target-jqbq7","openshift-ovn-kubernetes/ovnkube-node-8n6nm","kube-system/konnectivity-agent-qvw2x"]
Sep 15 20:04:55.957019 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.956999    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:55.958429 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.958407    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vm6lc"
Sep 15 20:04:55.960202 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.960037    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kjp4l"
Sep 15 20:04:55.960483 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.960462    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 15 20:04:55.960909 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.960890    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 15 20:04:55.961010 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.960916    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 15 20:04:55.961794 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.961246    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-lfxsw\""
Sep 15 20:04:55.961794 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.961480    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 15 20:04:55.961794 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.961649    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 15 20:04:55.962863 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.962098    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 15 20:04:55.962863 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.962154    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 15 20:04:55.962863 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.962439    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 15 20:04:55.962863 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.962488    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 15 20:04:55.962863 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.962697    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 15 20:04:55.962863 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.962784    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4tbcc\""
Sep 15 20:04:55.963266 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.963176    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-brdnq"
Sep 15 20:04:55.963433 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.963416    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 15 20:04:55.963524 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.963485    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 15 20:04:55.963524 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.963497    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 15 20:04:55.963619 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.963573    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 15 20:04:55.963619 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.963573    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-vws64\""
Sep 15 20:04:55.964081 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.963781    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 15 20:04:55.964381 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.964360    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fmkfr"
Sep 15 20:04:55.965382 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.965363    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-t4whs\""
Sep 15 20:04:55.965831 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.965814    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 15 20:04:55.966128 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.966110    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9dwjh"
Sep 15 20:04:55.966206 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:55.966179    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9dwjh" podUID="7e124866-fa5b-4ecb-bca9-075fa8bdb5a5"
Sep 15 20:04:55.966458 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.966441    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 15 20:04:55.967067 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.967050    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 15 20:04:55.967182 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.967096    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 15 20:04:55.967182 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.967170    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7sdhq\""
Sep 15 20:04:55.967547 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.967518    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-z5wjn"
Sep 15 20:04:55.969457 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.969438    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqj9r"
Sep 15 20:04:55.970039 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.970020    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-r4hcp\""
Sep 15 20:04:55.970117 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.970062    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 15 20:04:55.970236 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.970216    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 15 20:04:55.971044 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.971029    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:55.971592 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.971571    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 15 20:04:55.971680 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.971576    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 15 20:04:55.972175 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.972156    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-chq7b\""
Sep 15 20:04:55.972358 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.972345    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 15 20:04:55.972472 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.972456    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qvw2x"
Sep 15 20:04:55.972874 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.972856    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-host-var-lib-cni-multus\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:55.972964 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.972880    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-hostroot\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:55.972964 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.972898    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jzxf\" (UniqueName: \"kubernetes.io/projected/a8a416c3-51a5-4b55-9c73-9aaca4f93242-kube-api-access-4jzxf\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:55.972964 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.972922    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-host-kubelet\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:55.973066 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.972970    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xl7bs\" (UniqueName: \"kubernetes.io/projected/81e9d392-e3c5-48f6-b98a-4bcaddda3336-kube-api-access-xl7bs\") pod \"node-ca-vm6lc\" (UID: \"81e9d392-e3c5-48f6-b98a-4bcaddda3336\") " pod="openshift-image-registry/node-ca-vm6lc"
Sep 15 20:04:55.973066 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.973016    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fd4ce615-2127-43bd-b843-f1b5a4bfcdc7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kjp4l\" (UID: \"fd4ce615-2127-43bd-b843-f1b5a4bfcdc7\") " pod="openshift-multus/multus-additional-cni-plugins-kjp4l"
Sep 15 20:04:55.973066 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.973039    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-multus-socket-dir-parent\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:55.973182 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.973067    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-host-run-k8s-cni-cncf-io\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:55.973182 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.973087    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/87b7bd71-28d6-4214-af3c-44739f696128-ovnkube-config\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:55.973182 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.973108    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wf4b6\" (UniqueName: \"kubernetes.io/projected/87b7bd71-28d6-4214-af3c-44739f696128-kube-api-access-wf4b6\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:55.973182 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.973153    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fd4ce615-2127-43bd-b843-f1b5a4bfcdc7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kjp4l\" (UID: \"fd4ce615-2127-43bd-b843-f1b5a4bfcdc7\") " pod="openshift-multus/multus-additional-cni-plugins-kjp4l"
Sep 15 20:04:55.973368 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.973186    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-cnibin\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:55.973368 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.973215    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-host-var-lib-cni-bin\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:55.973368 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.973265    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a8a416c3-51a5-4b55-9c73-9aaca4f93242-multus-daemon-config\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:55.973368 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.973294    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-host-run-multus-certs\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:55.973368 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.973320    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-etc-kubernetes\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:55.973368 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.973363    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-host-cni-netd\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:55.973673 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.973385    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/81e9d392-e3c5-48f6-b98a-4bcaddda3336-serviceca\") pod \"node-ca-vm6lc\" (UID: \"81e9d392-e3c5-48f6-b98a-4bcaddda3336\") " pod="openshift-image-registry/node-ca-vm6lc"
Sep 15 20:04:55.973673 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.973419    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fd4ce615-2127-43bd-b843-f1b5a4bfcdc7-cnibin\") pod \"multus-additional-cni-plugins-kjp4l\" (UID: \"fd4ce615-2127-43bd-b843-f1b5a4bfcdc7\") " pod="openshift-multus/multus-additional-cni-plugins-kjp4l"
Sep 15 20:04:55.973673 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.973477    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-run-openvswitch\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:55.973673 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.973524    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-host-cni-bin\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:55.973673 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.973561    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hj74x\" (UniqueName: \"kubernetes.io/projected/fd4ce615-2127-43bd-b843-f1b5a4bfcdc7-kube-api-access-hj74x\") pod \"multus-additional-cni-plugins-kjp4l\" (UID: \"fd4ce615-2127-43bd-b843-f1b5a4bfcdc7\") " pod="openshift-multus/multus-additional-cni-plugins-kjp4l"
Sep 15 20:04:55.973673 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.973607    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-multus-conf-dir\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:55.973673 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.973651    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-host-slash\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:55.973673 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.973673    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-log-socket\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:55.973991 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.973688    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-system-cni-dir\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:55.973991 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.973708    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a8a416c3-51a5-4b55-9c73-9aaca4f93242-cni-binary-copy\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:55.973991 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.973716    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 15 20:04:55.973991 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.973738    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-host-run-netns\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:55.973991 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.973801    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 15 20:04:55.973991 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.973839    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-host-run-netns\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:55.973991 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.973843    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-95s7m\""
Sep 15 20:04:55.973991 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.973874    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-host-run-ovn-kubernetes\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:55.973991 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.973932    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd4ce615-2127-43bd-b843-f1b5a4bfcdc7-system-cni-dir\") pod \"multus-additional-cni-plugins-kjp4l\" (UID: \"fd4ce615-2127-43bd-b843-f1b5a4bfcdc7\") " pod="openshift-multus/multus-additional-cni-plugins-kjp4l"
Sep 15 20:04:55.973991 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.973955    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-systemd-units\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:55.973991 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.973972    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:55.974411 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.973996    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/87b7bd71-28d6-4214-af3c-44739f696128-ovnkube-script-lib\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:55.974411 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.974026    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-host-var-lib-kubelet\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:55.974411 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.974047    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/81e9d392-e3c5-48f6-b98a-4bcaddda3336-host\") pod \"node-ca-vm6lc\" (UID: \"81e9d392-e3c5-48f6-b98a-4bcaddda3336\") " pod="openshift-image-registry/node-ca-vm6lc"
Sep 15 20:04:55.974411 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.974075    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-node-log\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:55.974411 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.974105    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-run-systemd\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:55.974411 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.974132    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-var-lib-openvswitch\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:55.974411 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.974174    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-etc-openvswitch\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:55.974411 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.974189    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-run-ovn\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:55.974411 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.974210    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-multus-cni-dir\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:55.974411 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.974225    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/87b7bd71-28d6-4214-af3c-44739f696128-env-overrides\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:55.974411 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.974239    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/87b7bd71-28d6-4214-af3c-44739f696128-ovn-node-metrics-cert\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:55.974411 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.974287    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fd4ce615-2127-43bd-b843-f1b5a4bfcdc7-cni-binary-copy\") pod \"multus-additional-cni-plugins-kjp4l\" (UID: \"fd4ce615-2127-43bd-b843-f1b5a4bfcdc7\") " pod="openshift-multus/multus-additional-cni-plugins-kjp4l"
Sep 15 20:04:55.974411 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.974314    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fd4ce615-2127-43bd-b843-f1b5a4bfcdc7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kjp4l\" (UID: \"fd4ce615-2127-43bd-b843-f1b5a4bfcdc7\") " pod="openshift-multus/multus-additional-cni-plugins-kjp4l"
Sep 15 20:04:55.974999 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.974451    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v4rt"
Sep 15 20:04:55.974999 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:55.974528    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v4rt" podUID="5482d254-b248-4eda-a7d8-37bcb3289d69"
Sep 15 20:04:55.974999 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.974878    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 15 20:04:55.974999 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.974883    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9ssgz\""
Sep 15 20:04:55.974999 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.974910    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 15 20:04:55.976268 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.975694    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 15 20:04:55.978289 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:55.976966    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqbq7"
Sep 15 20:04:55.978289 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:55.977058    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqbq7" podUID="a7a98707-ce3f-4f6c-b723-a19113d90c41"
Sep 15 20:04:56.002693 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.002670    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-s75q9"
Sep 15 20:04:56.009158 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.009039    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-s75q9"
Sep 15 20:04:56.058533 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.058458    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-72.ec2.internal" event={"ID":"eac46b2c823b79bb709b7641ecf92419","Type":"ContainerStarted","Data":"07a7785cc88a73f195c67f8dd6e81ff5b9c461264a1a7317fd17f0a67597de32"}
Sep 15 20:04:56.060130 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.060061    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-72.ec2.internal" event={"ID":"6967c7c3d2c13b8fd404f2bb5001259f","Type":"ContainerStarted","Data":"4f006af009faaab1d0b1e343b56d56d50b2328a6904eb5fd53ad93f5144cf723"}
Sep 15 20:04:56.061774 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.061751    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 20:04:56.063024 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.063001    2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 15 20:04:56.074753 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.074709    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-host-slash\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.074871 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.074768    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-system-cni-dir\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:56.074871 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.074800    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e124866-fa5b-4ecb-bca9-075fa8bdb5a5-original-pull-secret\") pod \"global-pull-secret-syncer-9dwjh\" (UID: \"7e124866-fa5b-4ecb-bca9-075fa8bdb5a5\") " pod="kube-system/global-pull-secret-syncer-9dwjh"
Sep 15 20:04:56.074871 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.074803    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-host-slash\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.074871 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.074825    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bnj8\" (UniqueName: \"kubernetes.io/projected/a7a98707-ce3f-4f6c-b723-a19113d90c41-kube-api-access-4bnj8\") pod \"network-check-target-jqbq7\" (UID: \"a7a98707-ce3f-4f6c-b723-a19113d90c41\") " pod="openshift-network-diagnostics/network-check-target-jqbq7"
Sep 15 20:04:56.074871 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.074850    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c0566e5c-2129-4093-8fd4-d13f98007c00-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cqj9r\" (UID: \"c0566e5c-2129-4093-8fd4-d13f98007c00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqj9r"
Sep 15 20:04:56.075067 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.074872    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-etc-sysctl-d\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.075067 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.074897    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd4ce615-2127-43bd-b843-f1b5a4bfcdc7-system-cni-dir\") pod \"multus-additional-cni-plugins-kjp4l\" (UID: \"fd4ce615-2127-43bd-b843-f1b5a4bfcdc7\") " pod="openshift-multus/multus-additional-cni-plugins-kjp4l"
Sep 15 20:04:56.075067 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.074942    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-system-cni-dir\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:56.075067 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.074947    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c0566e5c-2129-4093-8fd4-d13f98007c00-registration-dir\") pod \"aws-ebs-csi-driver-node-cqj9r\" (UID: \"c0566e5c-2129-4093-8fd4-d13f98007c00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqj9r"
Sep 15 20:04:56.075067 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.074955    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd4ce615-2127-43bd-b843-f1b5a4bfcdc7-system-cni-dir\") pod \"multus-additional-cni-plugins-kjp4l\" (UID: \"fd4ce615-2127-43bd-b843-f1b5a4bfcdc7\") " pod="openshift-multus/multus-additional-cni-plugins-kjp4l"
Sep 15 20:04:56.075067 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.074994    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-etc-tuned\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.075067 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075040    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-systemd-units\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.075384 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075067    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.075384 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075095    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-host-var-lib-kubelet\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:56.075384 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075120    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4b044f60-fa8f-4ae9-9f9b-6722d67b4fb0-host-slash\") pod \"iptables-alerter-fmkfr\" (UID: \"4b044f60-fa8f-4ae9-9f9b-6722d67b4fb0\") " pod="openshift-network-operator/iptables-alerter-fmkfr"
Sep 15 20:04:56.075384 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075072    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-systemd-units\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.075384 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075154    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/81e9d392-e3c5-48f6-b98a-4bcaddda3336-host\") pod \"node-ca-vm6lc\" (UID: \"81e9d392-e3c5-48f6-b98a-4bcaddda3336\") " pod="openshift-image-registry/node-ca-vm6lc"
Sep 15 20:04:56.075384 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075178    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-node-log\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.075384 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075187    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.075384 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075202    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-run-systemd\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.075384 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075238    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-host-var-lib-kubelet\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:56.075384 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075240    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-var-lib-openvswitch\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.075384 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075249    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/81e9d392-e3c5-48f6-b98a-4bcaddda3336-host\") pod \"node-ca-vm6lc\" (UID: \"81e9d392-e3c5-48f6-b98a-4bcaddda3336\") " pod="openshift-image-registry/node-ca-vm6lc"
Sep 15 20:04:56.075384 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075264    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-run-systemd\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.075384 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075264    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-etc-openvswitch\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.075384 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075248    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-node-log\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.075384 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075301    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-etc-openvswitch\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.075384 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075300    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-multus-cni-dir\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:56.075384 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075321    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-var-lib-openvswitch\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.076173 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075341    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-etc-systemd\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.076173 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075360    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-multus-cni-dir\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:56.076173 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075381    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/87b7bd71-28d6-4214-af3c-44739f696128-ovn-node-metrics-cert\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.076173 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075413    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fd4ce615-2127-43bd-b843-f1b5a4bfcdc7-cni-binary-copy\") pod \"multus-additional-cni-plugins-kjp4l\" (UID: \"fd4ce615-2127-43bd-b843-f1b5a4bfcdc7\") " pod="openshift-multus/multus-additional-cni-plugins-kjp4l"
Sep 15 20:04:56.076173 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075435    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fd4ce615-2127-43bd-b843-f1b5a4bfcdc7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kjp4l\" (UID: \"fd4ce615-2127-43bd-b843-f1b5a4bfcdc7\") " pod="openshift-multus/multus-additional-cni-plugins-kjp4l"
Sep 15 20:04:56.076173 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075459    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-host-var-lib-cni-multus\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:56.076173 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075490    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4b044f60-fa8f-4ae9-9f9b-6722d67b4fb0-iptables-alerter-script\") pod \"iptables-alerter-fmkfr\" (UID: \"4b044f60-fa8f-4ae9-9f9b-6722d67b4fb0\") " pod="openshift-network-operator/iptables-alerter-fmkfr"
Sep 15 20:04:56.076173 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075533    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-run\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.076173 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075545    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-host-var-lib-cni-multus\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:56.076173 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075557    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-host-kubelet\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.076173 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075587    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-etc-sysconfig\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.076173 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075622    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-host-kubelet\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.076173 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075653    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wf4b6\" (UniqueName: \"kubernetes.io/projected/87b7bd71-28d6-4214-af3c-44739f696128-kube-api-access-wf4b6\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.076173 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075675    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fd4ce615-2127-43bd-b843-f1b5a4bfcdc7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kjp4l\" (UID: \"fd4ce615-2127-43bd-b843-f1b5a4bfcdc7\") " pod="openshift-multus/multus-additional-cni-plugins-kjp4l"
Sep 15 20:04:56.076173 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075688    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fd4ce615-2127-43bd-b843-f1b5a4bfcdc7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kjp4l\" (UID: \"fd4ce615-2127-43bd-b843-f1b5a4bfcdc7\") " pod="openshift-multus/multus-additional-cni-plugins-kjp4l"
Sep 15 20:04:56.076173 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075751    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-cnibin\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:56.076173 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075790    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-etc-kubernetes\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:56.076886 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075817    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7e124866-fa5b-4ecb-bca9-075fa8bdb5a5-dbus\") pod \"global-pull-secret-syncer-9dwjh\" (UID: \"7e124866-fa5b-4ecb-bca9-075fa8bdb5a5\") " pod="kube-system/global-pull-secret-syncer-9dwjh"
Sep 15 20:04:56.076886 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075844    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1547d350-30fe-4c0b-9448-1cbd9c4ae5b9-agent-certs\") pod \"konnectivity-agent-qvw2x\" (UID: \"1547d350-30fe-4c0b-9448-1cbd9c4ae5b9\") " pod="kube-system/konnectivity-agent-qvw2x"
Sep 15 20:04:56.076886 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075871    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-var-lib-kubelet\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.076886 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075896    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkp2c\" (UniqueName: \"kubernetes.io/projected/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-kube-api-access-rkp2c\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.076886 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075903    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-etc-kubernetes\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:56.076886 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.075943    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/81e9d392-e3c5-48f6-b98a-4bcaddda3336-serviceca\") pod \"node-ca-vm6lc\" (UID: \"81e9d392-e3c5-48f6-b98a-4bcaddda3336\") " pod="openshift-image-registry/node-ca-vm6lc"
Sep 15 20:04:56.076886 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076012    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fd4ce615-2127-43bd-b843-f1b5a4bfcdc7-cnibin\") pod \"multus-additional-cni-plugins-kjp4l\" (UID: \"fd4ce615-2127-43bd-b843-f1b5a4bfcdc7\") " pod="openshift-multus/multus-additional-cni-plugins-kjp4l"
Sep 15 20:04:56.076886 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076000    2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 15 20:04:56.076886 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076035    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fd4ce615-2127-43bd-b843-f1b5a4bfcdc7-cni-binary-copy\") pod \"multus-additional-cni-plugins-kjp4l\" (UID: \"fd4ce615-2127-43bd-b843-f1b5a4bfcdc7\") " pod="openshift-multus/multus-additional-cni-plugins-kjp4l"
Sep 15 20:04:56.076886 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076064    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-cnibin\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:56.076886 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076068    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fd4ce615-2127-43bd-b843-f1b5a4bfcdc7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kjp4l\" (UID: \"fd4ce615-2127-43bd-b843-f1b5a4bfcdc7\") " pod="openshift-multus/multus-additional-cni-plugins-kjp4l"
Sep 15 20:04:56.076886 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076092    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-run-openvswitch\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.076886 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076112    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fd4ce615-2127-43bd-b843-f1b5a4bfcdc7-cnibin\") pod \"multus-additional-cni-plugins-kjp4l\" (UID: \"fd4ce615-2127-43bd-b843-f1b5a4bfcdc7\") " pod="openshift-multus/multus-additional-cni-plugins-kjp4l"
Sep 15 20:04:56.076886 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076122    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hj74x\" (UniqueName: \"kubernetes.io/projected/fd4ce615-2127-43bd-b843-f1b5a4bfcdc7-kube-api-access-hj74x\") pod \"multus-additional-cni-plugins-kjp4l\" (UID: \"fd4ce615-2127-43bd-b843-f1b5a4bfcdc7\") " pod="openshift-multus/multus-additional-cni-plugins-kjp4l"
Sep 15 20:04:56.076886 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076162    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-run-openvswitch\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.076886 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076165    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b6f901c3-0f40-46dc-8406-eaee847b1c10-hosts-file\") pod \"node-resolver-z5wjn\" (UID: \"b6f901c3-0f40-46dc-8406-eaee847b1c10\") " pod="openshift-dns/node-resolver-z5wjn"
Sep 15 20:04:56.076886 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076211    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6f901c3-0f40-46dc-8406-eaee847b1c10-tmp-dir\") pod \"node-resolver-z5wjn\" (UID: \"b6f901c3-0f40-46dc-8406-eaee847b1c10\") " pod="openshift-dns/node-resolver-z5wjn"
Sep 15 20:04:56.077707 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076232    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1547d350-30fe-4c0b-9448-1cbd9c4ae5b9-konnectivity-ca\") pod \"konnectivity-agent-qvw2x\" (UID: \"1547d350-30fe-4c0b-9448-1cbd9c4ae5b9\") " pod="kube-system/konnectivity-agent-qvw2x"
Sep 15 20:04:56.077707 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076250    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-host\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.077707 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076272    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-log-socket\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.077707 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076299    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a8a416c3-51a5-4b55-9c73-9aaca4f93242-cni-binary-copy\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:56.077707 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076326    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-log-socket\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.077707 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076342    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/81e9d392-e3c5-48f6-b98a-4bcaddda3336-serviceca\") pod \"node-ca-vm6lc\" (UID: \"81e9d392-e3c5-48f6-b98a-4bcaddda3336\") " pod="openshift-image-registry/node-ca-vm6lc"
Sep 15 20:04:56.077707 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076347    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-host-run-netns\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:56.077707 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076386    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c0566e5c-2129-4093-8fd4-d13f98007c00-sys-fs\") pod \"aws-ebs-csi-driver-node-cqj9r\" (UID: \"c0566e5c-2129-4093-8fd4-d13f98007c00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqj9r"
Sep 15 20:04:56.077707 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076392    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-host-run-netns\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:56.077707 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076413    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-etc-kubernetes\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.077707 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076465    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-tmp\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.077707 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076490    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-host-run-netns\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.077707 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076511    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-host-run-ovn-kubernetes\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.077707 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076545    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-lib-modules\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.077707 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076564    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-host-run-netns\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.077707 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076574    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mr4mv\" (UniqueName: \"kubernetes.io/projected/5482d254-b248-4eda-a7d8-37bcb3289d69-kube-api-access-mr4mv\") pod \"network-metrics-daemon-2v4rt\" (UID: \"5482d254-b248-4eda-a7d8-37bcb3289d69\") " pod="openshift-multus/network-metrics-daemon-2v4rt"
Sep 15 20:04:56.077707 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076599    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-host-run-ovn-kubernetes\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.078518 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076634    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/87b7bd71-28d6-4214-af3c-44739f696128-ovnkube-script-lib\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.078518 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076661    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c0566e5c-2129-4093-8fd4-d13f98007c00-socket-dir\") pod \"aws-ebs-csi-driver-node-cqj9r\" (UID: \"c0566e5c-2129-4093-8fd4-d13f98007c00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqj9r"
Sep 15 20:04:56.078518 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076694    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-run-ovn\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.078518 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076745    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8688\" (UniqueName: \"kubernetes.io/projected/4b044f60-fa8f-4ae9-9f9b-6722d67b4fb0-kube-api-access-g8688\") pod \"iptables-alerter-fmkfr\" (UID: \"4b044f60-fa8f-4ae9-9f9b-6722d67b4fb0\") " pod="openshift-network-operator/iptables-alerter-fmkfr"
Sep 15 20:04:56.078518 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076780    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-run-ovn\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.078518 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076825    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wchgz\" (UniqueName: \"kubernetes.io/projected/c0566e5c-2129-4093-8fd4-d13f98007c00-kube-api-access-wchgz\") pod \"aws-ebs-csi-driver-node-cqj9r\" (UID: \"c0566e5c-2129-4093-8fd4-d13f98007c00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqj9r"
Sep 15 20:04:56.078518 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076841    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a8a416c3-51a5-4b55-9c73-9aaca4f93242-cni-binary-copy\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:56.078518 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076847    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-etc-sysctl-conf\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.078518 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076886    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/87b7bd71-28d6-4214-af3c-44739f696128-env-overrides\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.078518 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076925    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-hostroot\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:56.078518 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076966    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-hostroot\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:56.078518 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.076993    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jzxf\" (UniqueName: \"kubernetes.io/projected/a8a416c3-51a5-4b55-9c73-9aaca4f93242-kube-api-access-4jzxf\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:56.078518 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.077017    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7e124866-fa5b-4ecb-bca9-075fa8bdb5a5-kubelet-config\") pod \"global-pull-secret-syncer-9dwjh\" (UID: \"7e124866-fa5b-4ecb-bca9-075fa8bdb5a5\") " pod="kube-system/global-pull-secret-syncer-9dwjh"
Sep 15 20:04:56.078518 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.077035    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-sys\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.078518 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.077070    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5482d254-b248-4eda-a7d8-37bcb3289d69-metrics-certs\") pod \"network-metrics-daemon-2v4rt\" (UID: \"5482d254-b248-4eda-a7d8-37bcb3289d69\") " pod="openshift-multus/network-metrics-daemon-2v4rt"
Sep 15 20:04:56.078518 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.077094    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xl7bs\" (UniqueName: \"kubernetes.io/projected/81e9d392-e3c5-48f6-b98a-4bcaddda3336-kube-api-access-xl7bs\") pod \"node-ca-vm6lc\" (UID: \"81e9d392-e3c5-48f6-b98a-4bcaddda3336\") " pod="openshift-image-registry/node-ca-vm6lc"
Sep 15 20:04:56.078518 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.077114    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fd4ce615-2127-43bd-b843-f1b5a4bfcdc7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kjp4l\" (UID: \"fd4ce615-2127-43bd-b843-f1b5a4bfcdc7\") " pod="openshift-multus/multus-additional-cni-plugins-kjp4l"
Sep 15 20:04:56.079315 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.077136    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-multus-socket-dir-parent\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:56.079315 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.077171    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-host-run-k8s-cni-cncf-io\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:56.079315 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.077193    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c0566e5c-2129-4093-8fd4-d13f98007c00-device-dir\") pod \"aws-ebs-csi-driver-node-cqj9r\" (UID: \"c0566e5c-2129-4093-8fd4-d13f98007c00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqj9r"
Sep 15 20:04:56.079315 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.077213    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/87b7bd71-28d6-4214-af3c-44739f696128-ovnkube-config\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.079315 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.077257    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-host-var-lib-cni-bin\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:56.079315 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.077258    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/87b7bd71-28d6-4214-af3c-44739f696128-ovnkube-script-lib\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.079315 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.077571    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/87b7bd71-28d6-4214-af3c-44739f696128-env-overrides\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.079315 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.077839    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-host-run-k8s-cni-cncf-io\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:56.079315 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.078113    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-multus-socket-dir-parent\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:56.079315 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.078164    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a8a416c3-51a5-4b55-9c73-9aaca4f93242-multus-daemon-config\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:56.079315 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.078194    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-host-run-multus-certs\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:56.079315 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.078264    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fd4ce615-2127-43bd-b843-f1b5a4bfcdc7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kjp4l\" (UID: \"fd4ce615-2127-43bd-b843-f1b5a4bfcdc7\") " pod="openshift-multus/multus-additional-cni-plugins-kjp4l"
Sep 15 20:04:56.079315 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.078276    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-host-cni-netd\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.079315 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.078303    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-host-cni-bin\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.079315 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.078305    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-host-var-lib-cni-bin\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:56.079315 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.078433    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-multus-conf-dir\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:56.079315 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.078462    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrckq\" (UniqueName: \"kubernetes.io/projected/b6f901c3-0f40-46dc-8406-eaee847b1c10-kube-api-access-xrckq\") pod \"node-resolver-z5wjn\" (UID: \"b6f901c3-0f40-46dc-8406-eaee847b1c10\") " pod="openshift-dns/node-resolver-z5wjn"
Sep 15 20:04:56.080172 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.078494    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c0566e5c-2129-4093-8fd4-d13f98007c00-etc-selinux\") pod \"aws-ebs-csi-driver-node-cqj9r\" (UID: \"c0566e5c-2129-4093-8fd4-d13f98007c00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqj9r"
Sep 15 20:04:56.080172 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.078516    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-etc-modprobe-d\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.080172 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.078642    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/87b7bd71-28d6-4214-af3c-44739f696128-ovnkube-config\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.080172 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.078680    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-multus-conf-dir\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:56.080172 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.078704    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-host-cni-netd\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.080172 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.078747    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87b7bd71-28d6-4214-af3c-44739f696128-host-cni-bin\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.080172 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.078787    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a8a416c3-51a5-4b55-9c73-9aaca4f93242-host-run-multus-certs\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:56.080172 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.079012    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a8a416c3-51a5-4b55-9c73-9aaca4f93242-multus-daemon-config\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:56.080172 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.080080    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/87b7bd71-28d6-4214-af3c-44739f696128-ovn-node-metrics-cert\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.085311 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.085278    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wf4b6\" (UniqueName: \"kubernetes.io/projected/87b7bd71-28d6-4214-af3c-44739f696128-kube-api-access-wf4b6\") pod \"ovnkube-node-8n6nm\" (UID: \"87b7bd71-28d6-4214-af3c-44739f696128\") " pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.085519 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.085487    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hj74x\" (UniqueName: \"kubernetes.io/projected/fd4ce615-2127-43bd-b843-f1b5a4bfcdc7-kube-api-access-hj74x\") pod \"multus-additional-cni-plugins-kjp4l\" (UID: \"fd4ce615-2127-43bd-b843-f1b5a4bfcdc7\") " pod="openshift-multus/multus-additional-cni-plugins-kjp4l"
Sep 15 20:04:56.085624 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.085525    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jzxf\" (UniqueName: \"kubernetes.io/projected/a8a416c3-51a5-4b55-9c73-9aaca4f93242-kube-api-access-4jzxf\") pod \"multus-brdnq\" (UID: \"a8a416c3-51a5-4b55-9c73-9aaca4f93242\") " pod="openshift-multus/multus-brdnq"
Sep 15 20:04:56.086591 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.086570    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xl7bs\" (UniqueName: \"kubernetes.io/projected/81e9d392-e3c5-48f6-b98a-4bcaddda3336-kube-api-access-xl7bs\") pod \"node-ca-vm6lc\" (UID: \"81e9d392-e3c5-48f6-b98a-4bcaddda3336\") " pod="openshift-image-registry/node-ca-vm6lc"
Sep 15 20:04:56.179049 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.179007    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mr4mv\" (UniqueName: \"kubernetes.io/projected/5482d254-b248-4eda-a7d8-37bcb3289d69-kube-api-access-mr4mv\") pod \"network-metrics-daemon-2v4rt\" (UID: \"5482d254-b248-4eda-a7d8-37bcb3289d69\") " pod="openshift-multus/network-metrics-daemon-2v4rt"
Sep 15 20:04:56.179218 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.179060    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c0566e5c-2129-4093-8fd4-d13f98007c00-socket-dir\") pod \"aws-ebs-csi-driver-node-cqj9r\" (UID: \"c0566e5c-2129-4093-8fd4-d13f98007c00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqj9r"
Sep 15 20:04:56.179282 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.179226    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c0566e5c-2129-4093-8fd4-d13f98007c00-socket-dir\") pod \"aws-ebs-csi-driver-node-cqj9r\" (UID: \"c0566e5c-2129-4093-8fd4-d13f98007c00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqj9r"
Sep 15 20:04:56.179326 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.179289    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8688\" (UniqueName: \"kubernetes.io/projected/4b044f60-fa8f-4ae9-9f9b-6722d67b4fb0-kube-api-access-g8688\") pod \"iptables-alerter-fmkfr\" (UID: \"4b044f60-fa8f-4ae9-9f9b-6722d67b4fb0\") " pod="openshift-network-operator/iptables-alerter-fmkfr"
Sep 15 20:04:56.179326 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.179316    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wchgz\" (UniqueName: \"kubernetes.io/projected/c0566e5c-2129-4093-8fd4-d13f98007c00-kube-api-access-wchgz\") pod \"aws-ebs-csi-driver-node-cqj9r\" (UID: \"c0566e5c-2129-4093-8fd4-d13f98007c00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqj9r"
Sep 15 20:04:56.179421 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.179342    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-etc-sysctl-conf\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.179421 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.179369    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7e124866-fa5b-4ecb-bca9-075fa8bdb5a5-kubelet-config\") pod \"global-pull-secret-syncer-9dwjh\" (UID: \"7e124866-fa5b-4ecb-bca9-075fa8bdb5a5\") " pod="kube-system/global-pull-secret-syncer-9dwjh"
Sep 15 20:04:56.179421 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.179393    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-sys\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.179564 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.179421    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5482d254-b248-4eda-a7d8-37bcb3289d69-metrics-certs\") pod \"network-metrics-daemon-2v4rt\" (UID: \"5482d254-b248-4eda-a7d8-37bcb3289d69\") " pod="openshift-multus/network-metrics-daemon-2v4rt"
Sep 15 20:04:56.179564 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.179458    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c0566e5c-2129-4093-8fd4-d13f98007c00-device-dir\") pod \"aws-ebs-csi-driver-node-cqj9r\" (UID: \"c0566e5c-2129-4093-8fd4-d13f98007c00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqj9r"
Sep 15 20:04:56.179564 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.179499    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrckq\" (UniqueName: \"kubernetes.io/projected/b6f901c3-0f40-46dc-8406-eaee847b1c10-kube-api-access-xrckq\") pod \"node-resolver-z5wjn\" (UID: \"b6f901c3-0f40-46dc-8406-eaee847b1c10\") " pod="openshift-dns/node-resolver-z5wjn"
Sep 15 20:04:56.179564 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.179499    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7e124866-fa5b-4ecb-bca9-075fa8bdb5a5-kubelet-config\") pod \"global-pull-secret-syncer-9dwjh\" (UID: \"7e124866-fa5b-4ecb-bca9-075fa8bdb5a5\") " pod="kube-system/global-pull-secret-syncer-9dwjh"
Sep 15 20:04:56.179564 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.179523    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c0566e5c-2129-4093-8fd4-d13f98007c00-etc-selinux\") pod \"aws-ebs-csi-driver-node-cqj9r\" (UID: \"c0566e5c-2129-4093-8fd4-d13f98007c00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqj9r"
Sep 15 20:04:56.179564 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.179521    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-etc-sysctl-conf\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.179564 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.179542    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-sys\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.179564 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.179547    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-etc-modprobe-d\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.179971 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.179594    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c0566e5c-2129-4093-8fd4-d13f98007c00-device-dir\") pod \"aws-ebs-csi-driver-node-cqj9r\" (UID: \"c0566e5c-2129-4093-8fd4-d13f98007c00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqj9r"
Sep 15 20:04:56.179971 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.179593    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e124866-fa5b-4ecb-bca9-075fa8bdb5a5-original-pull-secret\") pod \"global-pull-secret-syncer-9dwjh\" (UID: \"7e124866-fa5b-4ecb-bca9-075fa8bdb5a5\") " pod="kube-system/global-pull-secret-syncer-9dwjh"
Sep 15 20:04:56.179971 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.179601    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c0566e5c-2129-4093-8fd4-d13f98007c00-etc-selinux\") pod \"aws-ebs-csi-driver-node-cqj9r\" (UID: \"c0566e5c-2129-4093-8fd4-d13f98007c00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqj9r"
Sep 15 20:04:56.179971 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:56.179622    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:04:56.179971 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:56.179650    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 20:04:56.179971 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.179624    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bnj8\" (UniqueName: \"kubernetes.io/projected/a7a98707-ce3f-4f6c-b723-a19113d90c41-kube-api-access-4bnj8\") pod \"network-check-target-jqbq7\" (UID: \"a7a98707-ce3f-4f6c-b723-a19113d90c41\") " pod="openshift-network-diagnostics/network-check-target-jqbq7"
Sep 15 20:04:56.179971 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:56.179703    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5482d254-b248-4eda-a7d8-37bcb3289d69-metrics-certs podName:5482d254-b248-4eda-a7d8-37bcb3289d69 nodeName:}" failed. No retries permitted until 2026-09-15 20:04:56.679678545 +0000 UTC m=+3.070631583 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5482d254-b248-4eda-a7d8-37bcb3289d69-metrics-certs") pod "network-metrics-daemon-2v4rt" (UID: "5482d254-b248-4eda-a7d8-37bcb3289d69") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:04:56.179971 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:56.179766    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e124866-fa5b-4ecb-bca9-075fa8bdb5a5-original-pull-secret podName:7e124866-fa5b-4ecb-bca9-075fa8bdb5a5 nodeName:}" failed. No retries permitted until 2026-09-15 20:04:56.679747913 +0000 UTC m=+3.070700897 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7e124866-fa5b-4ecb-bca9-075fa8bdb5a5-original-pull-secret") pod "global-pull-secret-syncer-9dwjh" (UID: "7e124866-fa5b-4ecb-bca9-075fa8bdb5a5") : object "kube-system"/"original-pull-secret" not registered
Sep 15 20:04:56.179971 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.179783    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c0566e5c-2129-4093-8fd4-d13f98007c00-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cqj9r\" (UID: \"c0566e5c-2129-4093-8fd4-d13f98007c00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqj9r"
Sep 15 20:04:56.179971 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.179796    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-etc-modprobe-d\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.179971 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.179808    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-etc-sysctl-d\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.179971 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.179835    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c0566e5c-2129-4093-8fd4-d13f98007c00-registration-dir\") pod \"aws-ebs-csi-driver-node-cqj9r\" (UID: \"c0566e5c-2129-4093-8fd4-d13f98007c00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqj9r"
Sep 15 20:04:56.179971 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.179845    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c0566e5c-2129-4093-8fd4-d13f98007c00-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cqj9r\" (UID: \"c0566e5c-2129-4093-8fd4-d13f98007c00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqj9r"
Sep 15 20:04:56.179971 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.179858    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-etc-tuned\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.179971 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.179883    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4b044f60-fa8f-4ae9-9f9b-6722d67b4fb0-host-slash\") pod \"iptables-alerter-fmkfr\" (UID: \"4b044f60-fa8f-4ae9-9f9b-6722d67b4fb0\") " pod="openshift-network-operator/iptables-alerter-fmkfr"
Sep 15 20:04:56.179971 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.179897    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c0566e5c-2129-4093-8fd4-d13f98007c00-registration-dir\") pod \"aws-ebs-csi-driver-node-cqj9r\" (UID: \"c0566e5c-2129-4093-8fd4-d13f98007c00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqj9r"
Sep 15 20:04:56.180676 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.179915    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-etc-sysctl-d\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.180676 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.179933    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-etc-systemd\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.180676 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.179964    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4b044f60-fa8f-4ae9-9f9b-6722d67b4fb0-host-slash\") pod \"iptables-alerter-fmkfr\" (UID: \"4b044f60-fa8f-4ae9-9f9b-6722d67b4fb0\") " pod="openshift-network-operator/iptables-alerter-fmkfr"
Sep 15 20:04:56.180676 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.179991    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-etc-systemd\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.180676 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.179985    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4b044f60-fa8f-4ae9-9f9b-6722d67b4fb0-iptables-alerter-script\") pod \"iptables-alerter-fmkfr\" (UID: \"4b044f60-fa8f-4ae9-9f9b-6722d67b4fb0\") " pod="openshift-network-operator/iptables-alerter-fmkfr"
Sep 15 20:04:56.180676 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.180035    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-run\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.180676 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.180072    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-etc-sysconfig\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.180676 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.180099    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7e124866-fa5b-4ecb-bca9-075fa8bdb5a5-dbus\") pod \"global-pull-secret-syncer-9dwjh\" (UID: \"7e124866-fa5b-4ecb-bca9-075fa8bdb5a5\") " pod="kube-system/global-pull-secret-syncer-9dwjh"
Sep 15 20:04:56.180676 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.180114    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-run\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.180676 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.180122    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1547d350-30fe-4c0b-9448-1cbd9c4ae5b9-agent-certs\") pod \"konnectivity-agent-qvw2x\" (UID: \"1547d350-30fe-4c0b-9448-1cbd9c4ae5b9\") " pod="kube-system/konnectivity-agent-qvw2x"
Sep 15 20:04:56.180676 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.180169    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-var-lib-kubelet\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.180676 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.180199    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7e124866-fa5b-4ecb-bca9-075fa8bdb5a5-dbus\") pod \"global-pull-secret-syncer-9dwjh\" (UID: \"7e124866-fa5b-4ecb-bca9-075fa8bdb5a5\") " pod="kube-system/global-pull-secret-syncer-9dwjh"
Sep 15 20:04:56.180676 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.180209    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkp2c\" (UniqueName: \"kubernetes.io/projected/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-kube-api-access-rkp2c\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.180676 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.180220    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-var-lib-kubelet\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.180676 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.180177    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-etc-sysconfig\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.180676 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.180245    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b6f901c3-0f40-46dc-8406-eaee847b1c10-hosts-file\") pod \"node-resolver-z5wjn\" (UID: \"b6f901c3-0f40-46dc-8406-eaee847b1c10\") " pod="openshift-dns/node-resolver-z5wjn"
Sep 15 20:04:56.180676 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.180262    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6f901c3-0f40-46dc-8406-eaee847b1c10-tmp-dir\") pod \"node-resolver-z5wjn\" (UID: \"b6f901c3-0f40-46dc-8406-eaee847b1c10\") " pod="openshift-dns/node-resolver-z5wjn"
Sep 15 20:04:56.180676 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.180285    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1547d350-30fe-4c0b-9448-1cbd9c4ae5b9-konnectivity-ca\") pod \"konnectivity-agent-qvw2x\" (UID: \"1547d350-30fe-4c0b-9448-1cbd9c4ae5b9\") " pod="kube-system/konnectivity-agent-qvw2x"
Sep 15 20:04:56.181550 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.180321    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-host\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.181550 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.180346    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c0566e5c-2129-4093-8fd4-d13f98007c00-sys-fs\") pod \"aws-ebs-csi-driver-node-cqj9r\" (UID: \"c0566e5c-2129-4093-8fd4-d13f98007c00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqj9r"
Sep 15 20:04:56.181550 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.180372    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-etc-kubernetes\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.181550 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.180396    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-tmp\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.181550 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.180420    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-lib-modules\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.181550 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.180459    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b6f901c3-0f40-46dc-8406-eaee847b1c10-hosts-file\") pod \"node-resolver-z5wjn\" (UID: \"b6f901c3-0f40-46dc-8406-eaee847b1c10\") " pod="openshift-dns/node-resolver-z5wjn"
Sep 15 20:04:56.181550 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.180514    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c0566e5c-2129-4093-8fd4-d13f98007c00-sys-fs\") pod \"aws-ebs-csi-driver-node-cqj9r\" (UID: \"c0566e5c-2129-4093-8fd4-d13f98007c00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqj9r"
Sep 15 20:04:56.181550 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.180524    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-lib-modules\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.181550 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.180574    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-etc-kubernetes\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.181550 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.180577    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4b044f60-fa8f-4ae9-9f9b-6722d67b4fb0-iptables-alerter-script\") pod \"iptables-alerter-fmkfr\" (UID: \"4b044f60-fa8f-4ae9-9f9b-6722d67b4fb0\") " pod="openshift-network-operator/iptables-alerter-fmkfr"
Sep 15 20:04:56.181550 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.180626    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6f901c3-0f40-46dc-8406-eaee847b1c10-tmp-dir\") pod \"node-resolver-z5wjn\" (UID: \"b6f901c3-0f40-46dc-8406-eaee847b1c10\") " pod="openshift-dns/node-resolver-z5wjn"
Sep 15 20:04:56.181550 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.180637    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-host\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.181550 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.181094    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1547d350-30fe-4c0b-9448-1cbd9c4ae5b9-konnectivity-ca\") pod \"konnectivity-agent-qvw2x\" (UID: \"1547d350-30fe-4c0b-9448-1cbd9c4ae5b9\") " pod="kube-system/konnectivity-agent-qvw2x"
Sep 15 20:04:56.182414 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.182394    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-etc-tuned\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.182743 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.182709    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1547d350-30fe-4c0b-9448-1cbd9c4ae5b9-agent-certs\") pod \"konnectivity-agent-qvw2x\" (UID: \"1547d350-30fe-4c0b-9448-1cbd9c4ae5b9\") " pod="kube-system/konnectivity-agent-qvw2x"
Sep 15 20:04:56.182833 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.182799    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-tmp\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.188561 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:56.188524    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 20:04:56.188561 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:56.188548    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 20:04:56.188561 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:56.188562    2577 projected.go:196] Error preparing data for projected volume kube-api-access-4bnj8 for pod openshift-network-diagnostics/network-check-target-jqbq7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:04:56.188818 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:56.188623    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7a98707-ce3f-4f6c-b723-a19113d90c41-kube-api-access-4bnj8 podName:a7a98707-ce3f-4f6c-b723-a19113d90c41 nodeName:}" failed. No retries permitted until 2026-09-15 20:04:56.688605348 +0000 UTC m=+3.079558347 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4bnj8" (UniqueName: "kubernetes.io/projected/a7a98707-ce3f-4f6c-b723-a19113d90c41-kube-api-access-4bnj8") pod "network-check-target-jqbq7" (UID: "a7a98707-ce3f-4f6c-b723-a19113d90c41") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:04:56.191252 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.191231    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mr4mv\" (UniqueName: \"kubernetes.io/projected/5482d254-b248-4eda-a7d8-37bcb3289d69-kube-api-access-mr4mv\") pod \"network-metrics-daemon-2v4rt\" (UID: \"5482d254-b248-4eda-a7d8-37bcb3289d69\") " pod="openshift-multus/network-metrics-daemon-2v4rt"
Sep 15 20:04:56.191681 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.191659    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrckq\" (UniqueName: \"kubernetes.io/projected/b6f901c3-0f40-46dc-8406-eaee847b1c10-kube-api-access-xrckq\") pod \"node-resolver-z5wjn\" (UID: \"b6f901c3-0f40-46dc-8406-eaee847b1c10\") " pod="openshift-dns/node-resolver-z5wjn"
Sep 15 20:04:56.191962 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.191936    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wchgz\" (UniqueName: \"kubernetes.io/projected/c0566e5c-2129-4093-8fd4-d13f98007c00-kube-api-access-wchgz\") pod \"aws-ebs-csi-driver-node-cqj9r\" (UID: \"c0566e5c-2129-4093-8fd4-d13f98007c00\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqj9r"
Sep 15 20:04:56.192230 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.192182    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8688\" (UniqueName: \"kubernetes.io/projected/4b044f60-fa8f-4ae9-9f9b-6722d67b4fb0-kube-api-access-g8688\") pod \"iptables-alerter-fmkfr\" (UID: \"4b044f60-fa8f-4ae9-9f9b-6722d67b4fb0\") " pod="openshift-network-operator/iptables-alerter-fmkfr"
Sep 15 20:04:56.193299 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.193280    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkp2c\" (UniqueName: \"kubernetes.io/projected/a1d3494a-ca54-44cd-bcc2-bb19321b6f11-kube-api-access-rkp2c\") pod \"tuned-k6cn5\" (UID: \"a1d3494a-ca54-44cd-bcc2-bb19321b6f11\") " pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.271298 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.271265    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:04:56.280141 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.280109    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vm6lc"
Sep 15 20:04:56.286899 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.286872    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kjp4l"
Sep 15 20:04:56.291682 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.291661    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-brdnq"
Sep 15 20:04:56.298165 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.298134    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fmkfr"
Sep 15 20:04:56.303804 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.303784    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-z5wjn"
Sep 15 20:04:56.309362 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.309345    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqj9r"
Sep 15 20:04:56.316881 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.316864    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-k6cn5"
Sep 15 20:04:56.323244 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.323225    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qvw2x"
Sep 15 20:04:56.684686 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.684654    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e124866-fa5b-4ecb-bca9-075fa8bdb5a5-original-pull-secret\") pod \"global-pull-secret-syncer-9dwjh\" (UID: \"7e124866-fa5b-4ecb-bca9-075fa8bdb5a5\") " pod="kube-system/global-pull-secret-syncer-9dwjh"
Sep 15 20:04:56.684862 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.684749    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5482d254-b248-4eda-a7d8-37bcb3289d69-metrics-certs\") pod \"network-metrics-daemon-2v4rt\" (UID: \"5482d254-b248-4eda-a7d8-37bcb3289d69\") " pod="openshift-multus/network-metrics-daemon-2v4rt"
Sep 15 20:04:56.684862 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:56.684820    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 20:04:56.684862 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:56.684844    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:04:56.684968 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:56.684886    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5482d254-b248-4eda-a7d8-37bcb3289d69-metrics-certs podName:5482d254-b248-4eda-a7d8-37bcb3289d69 nodeName:}" failed. No retries permitted until 2026-09-15 20:04:57.68486961 +0000 UTC m=+4.075822600 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5482d254-b248-4eda-a7d8-37bcb3289d69-metrics-certs") pod "network-metrics-daemon-2v4rt" (UID: "5482d254-b248-4eda-a7d8-37bcb3289d69") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:04:56.684968 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:56.684901    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e124866-fa5b-4ecb-bca9-075fa8bdb5a5-original-pull-secret podName:7e124866-fa5b-4ecb-bca9-075fa8bdb5a5 nodeName:}" failed. No retries permitted until 2026-09-15 20:04:57.684893459 +0000 UTC m=+4.075846434 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7e124866-fa5b-4ecb-bca9-075fa8bdb5a5-original-pull-secret") pod "global-pull-secret-syncer-9dwjh" (UID: "7e124866-fa5b-4ecb-bca9-075fa8bdb5a5") : object "kube-system"/"original-pull-secret" not registered
Sep 15 20:04:56.785966 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:56.785938    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bnj8\" (UniqueName: \"kubernetes.io/projected/a7a98707-ce3f-4f6c-b723-a19113d90c41-kube-api-access-4bnj8\") pod \"network-check-target-jqbq7\" (UID: \"a7a98707-ce3f-4f6c-b723-a19113d90c41\") " pod="openshift-network-diagnostics/network-check-target-jqbq7"
Sep 15 20:04:56.786131 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:56.786087    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 20:04:56.786131 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:56.786110    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 20:04:56.786131 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:56.786124    2577 projected.go:196] Error preparing data for projected volume kube-api-access-4bnj8 for pod openshift-network-diagnostics/network-check-target-jqbq7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:04:56.786262 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:56.786179    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7a98707-ce3f-4f6c-b723-a19113d90c41-kube-api-access-4bnj8 podName:a7a98707-ce3f-4f6c-b723-a19113d90c41 nodeName:}" failed. No retries permitted until 2026-09-15 20:04:57.786164532 +0000 UTC m=+4.177117511 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4bnj8" (UniqueName: "kubernetes.io/projected/a7a98707-ce3f-4f6c-b723-a19113d90c41-kube-api-access-4bnj8") pod "network-check-target-jqbq7" (UID: "a7a98707-ce3f-4f6c-b723-a19113d90c41") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:04:56.999779 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:56.999746    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b044f60_fa8f_4ae9_9f9b_6722d67b4fb0.slice/crio-eb1fc6d3db502adbda78824ce0343b4d60928d92b2184398d53d992f8018068d WatchSource:0}: Error finding container eb1fc6d3db502adbda78824ce0343b4d60928d92b2184398d53d992f8018068d: Status 404 returned error can't find the container with id eb1fc6d3db502adbda78824ce0343b4d60928d92b2184398d53d992f8018068d
Sep 15 20:04:57.000271 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:57.000243    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1547d350_30fe_4c0b_9448_1cbd9c4ae5b9.slice/crio-9e0e78f24a52134ffc9a0dbdc03d444d31570ff602172ef3f206aeb6a290e1fe WatchSource:0}: Error finding container 9e0e78f24a52134ffc9a0dbdc03d444d31570ff602172ef3f206aeb6a290e1fe: Status 404 returned error can't find the container with id 9e0e78f24a52134ffc9a0dbdc03d444d31570ff602172ef3f206aeb6a290e1fe
Sep 15 20:04:57.001344 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:57.001310    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87b7bd71_28d6_4214_af3c_44739f696128.slice/crio-d19f08e9c165f55a9c6785c939f8a30cc65556bf83c34b74fa82e2913df6ac62 WatchSource:0}: Error finding container d19f08e9c165f55a9c6785c939f8a30cc65556bf83c34b74fa82e2913df6ac62: Status 404 returned error can't find the container with id d19f08e9c165f55a9c6785c939f8a30cc65556bf83c34b74fa82e2913df6ac62
Sep 15 20:04:57.002736 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:57.002699    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1d3494a_ca54_44cd_bcc2_bb19321b6f11.slice/crio-bb22a60f8b41192d74e717ef6b1cf26abf118d5b1364b2aaceca042d4e705f0b WatchSource:0}: Error finding container bb22a60f8b41192d74e717ef6b1cf26abf118d5b1364b2aaceca042d4e705f0b: Status 404 returned error can't find the container with id bb22a60f8b41192d74e717ef6b1cf26abf118d5b1364b2aaceca042d4e705f0b
Sep 15 20:04:57.005373 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:57.005347    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8a416c3_51a5_4b55_9c73_9aaca4f93242.slice/crio-9d15c8dcfeb0c8eff9198ad0c146a5aba8e44c46979a9f77f90d5922775c3015 WatchSource:0}: Error finding container 9d15c8dcfeb0c8eff9198ad0c146a5aba8e44c46979a9f77f90d5922775c3015: Status 404 returned error can't find the container with id 9d15c8dcfeb0c8eff9198ad0c146a5aba8e44c46979a9f77f90d5922775c3015
Sep 15 20:04:57.007411 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:57.007388    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81e9d392_e3c5_48f6_b98a_4bcaddda3336.slice/crio-f61fefd0d2d73996e70bec003d7812cf5d8f633cc37481c8bb653ff107a0f5a2 WatchSource:0}: Error finding container f61fefd0d2d73996e70bec003d7812cf5d8f633cc37481c8bb653ff107a0f5a2: Status 404 returned error can't find the container with id f61fefd0d2d73996e70bec003d7812cf5d8f633cc37481c8bb653ff107a0f5a2
Sep 15 20:04:57.008455 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:57.008415    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0566e5c_2129_4093_8fd4_d13f98007c00.slice/crio-123fa83de5a2c640cba689e2e266dfa003157fa40e4f4bca4157b2317741c25a WatchSource:0}: Error finding container 123fa83de5a2c640cba689e2e266dfa003157fa40e4f4bca4157b2317741c25a: Status 404 returned error can't find the container with id 123fa83de5a2c640cba689e2e266dfa003157fa40e4f4bca4157b2317741c25a
Sep 15 20:04:57.009691 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:57.009655    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6f901c3_0f40_46dc_8406_eaee847b1c10.slice/crio-18276a3ead4cf29b398a9513e1712a7cd87edb0ece86b9426bbcee41abfd0f9c WatchSource:0}: Error finding container 18276a3ead4cf29b398a9513e1712a7cd87edb0ece86b9426bbcee41abfd0f9c: Status 404 returned error can't find the container with id 18276a3ead4cf29b398a9513e1712a7cd87edb0ece86b9426bbcee41abfd0f9c
Sep 15 20:04:57.011189 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:04:57.011000    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd4ce615_2127_43bd_b843_f1b5a4bfcdc7.slice/crio-fbb2d38605fec34b357cf3b3ef7d49345f32ba27f2ec44b71cba73a1e6952080 WatchSource:0}: Error finding container fbb2d38605fec34b357cf3b3ef7d49345f32ba27f2ec44b71cba73a1e6952080: Status 404 returned error can't find the container with id fbb2d38605fec34b357cf3b3ef7d49345f32ba27f2ec44b71cba73a1e6952080
Sep 15 20:04:57.011189 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:57.011047    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-14 19:59:56 +0000 UTC" deadline="2028-05-11 03:16:13.689389268 +0000 UTC"
Sep 15 20:04:57.011189 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:57.011070    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14479h11m16.678323098s"
Sep 15 20:04:57.053988 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:57.053965    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9dwjh"
Sep 15 20:04:57.054102 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:57.053974    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v4rt"
Sep 15 20:04:57.054158 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:57.054058    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9dwjh" podUID="7e124866-fa5b-4ecb-bca9-075fa8bdb5a5"
Sep 15 20:04:57.054201 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:57.054167    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v4rt" podUID="5482d254-b248-4eda-a7d8-37bcb3289d69"
Sep 15 20:04:57.063175 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:57.063037    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fmkfr" event={"ID":"4b044f60-fa8f-4ae9-9f9b-6722d67b4fb0","Type":"ContainerStarted","Data":"eb1fc6d3db502adbda78824ce0343b4d60928d92b2184398d53d992f8018068d"}
Sep 15 20:04:57.064213 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:57.064186    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-z5wjn" event={"ID":"b6f901c3-0f40-46dc-8406-eaee847b1c10","Type":"ContainerStarted","Data":"18276a3ead4cf29b398a9513e1712a7cd87edb0ece86b9426bbcee41abfd0f9c"}
Sep 15 20:04:57.065607 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:57.065573    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqj9r" event={"ID":"c0566e5c-2129-4093-8fd4-d13f98007c00","Type":"ContainerStarted","Data":"123fa83de5a2c640cba689e2e266dfa003157fa40e4f4bca4157b2317741c25a"}
Sep 15 20:04:57.066870 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:57.066851    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-brdnq" event={"ID":"a8a416c3-51a5-4b55-9c73-9aaca4f93242","Type":"ContainerStarted","Data":"9d15c8dcfeb0c8eff9198ad0c146a5aba8e44c46979a9f77f90d5922775c3015"}
Sep 15 20:04:57.068020 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:57.068001    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qvw2x" event={"ID":"1547d350-30fe-4c0b-9448-1cbd9c4ae5b9","Type":"ContainerStarted","Data":"9e0e78f24a52134ffc9a0dbdc03d444d31570ff602172ef3f206aeb6a290e1fe"}
Sep 15 20:04:57.069155 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:57.069132    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kjp4l" event={"ID":"fd4ce615-2127-43bd-b843-f1b5a4bfcdc7","Type":"ContainerStarted","Data":"fbb2d38605fec34b357cf3b3ef7d49345f32ba27f2ec44b71cba73a1e6952080"}
Sep 15 20:04:57.070366 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:57.070332    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vm6lc" event={"ID":"81e9d392-e3c5-48f6-b98a-4bcaddda3336","Type":"ContainerStarted","Data":"f61fefd0d2d73996e70bec003d7812cf5d8f633cc37481c8bb653ff107a0f5a2"}
Sep 15 20:04:57.071437 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:57.071419    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-k6cn5" event={"ID":"a1d3494a-ca54-44cd-bcc2-bb19321b6f11","Type":"ContainerStarted","Data":"bb22a60f8b41192d74e717ef6b1cf26abf118d5b1364b2aaceca042d4e705f0b"}
Sep 15 20:04:57.073005 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:57.072978    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm" event={"ID":"87b7bd71-28d6-4214-af3c-44739f696128","Type":"ContainerStarted","Data":"d19f08e9c165f55a9c6785c939f8a30cc65556bf83c34b74fa82e2913df6ac62"}
Sep 15 20:04:57.693153 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:57.693065    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e124866-fa5b-4ecb-bca9-075fa8bdb5a5-original-pull-secret\") pod \"global-pull-secret-syncer-9dwjh\" (UID: \"7e124866-fa5b-4ecb-bca9-075fa8bdb5a5\") " pod="kube-system/global-pull-secret-syncer-9dwjh"
Sep 15 20:04:57.693310 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:57.693173    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5482d254-b248-4eda-a7d8-37bcb3289d69-metrics-certs\") pod \"network-metrics-daemon-2v4rt\" (UID: \"5482d254-b248-4eda-a7d8-37bcb3289d69\") " pod="openshift-multus/network-metrics-daemon-2v4rt"
Sep 15 20:04:57.693310 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:57.693300    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:04:57.693423 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:57.693361    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5482d254-b248-4eda-a7d8-37bcb3289d69-metrics-certs podName:5482d254-b248-4eda-a7d8-37bcb3289d69 nodeName:}" failed. No retries permitted until 2026-09-15 20:04:59.693343546 +0000 UTC m=+6.084296526 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5482d254-b248-4eda-a7d8-37bcb3289d69-metrics-certs") pod "network-metrics-daemon-2v4rt" (UID: "5482d254-b248-4eda-a7d8-37bcb3289d69") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:04:57.693918 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:57.693774    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 20:04:57.693918 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:57.693823    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e124866-fa5b-4ecb-bca9-075fa8bdb5a5-original-pull-secret podName:7e124866-fa5b-4ecb-bca9-075fa8bdb5a5 nodeName:}" failed. No retries permitted until 2026-09-15 20:04:59.693809366 +0000 UTC m=+6.084762348 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7e124866-fa5b-4ecb-bca9-075fa8bdb5a5-original-pull-secret") pod "global-pull-secret-syncer-9dwjh" (UID: "7e124866-fa5b-4ecb-bca9-075fa8bdb5a5") : object "kube-system"/"original-pull-secret" not registered
Sep 15 20:04:57.793719 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:57.793680    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bnj8\" (UniqueName: \"kubernetes.io/projected/a7a98707-ce3f-4f6c-b723-a19113d90c41-kube-api-access-4bnj8\") pod \"network-check-target-jqbq7\" (UID: \"a7a98707-ce3f-4f6c-b723-a19113d90c41\") " pod="openshift-network-diagnostics/network-check-target-jqbq7"
Sep 15 20:04:57.793913 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:57.793892    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 20:04:57.793972 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:57.793915    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 20:04:57.793972 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:57.793930    2577 projected.go:196] Error preparing data for projected volume kube-api-access-4bnj8 for pod openshift-network-diagnostics/network-check-target-jqbq7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:04:57.794076 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:57.793990    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7a98707-ce3f-4f6c-b723-a19113d90c41-kube-api-access-4bnj8 podName:a7a98707-ce3f-4f6c-b723-a19113d90c41 nodeName:}" failed. No retries permitted until 2026-09-15 20:04:59.793971644 +0000 UTC m=+6.184924640 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4bnj8" (UniqueName: "kubernetes.io/projected/a7a98707-ce3f-4f6c-b723-a19113d90c41-kube-api-access-4bnj8") pod "network-check-target-jqbq7" (UID: "a7a98707-ce3f-4f6c-b723-a19113d90c41") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:04:58.011309 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:58.011212    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-14 19:59:56 +0000 UTC" deadline="2028-02-16 04:45:12.366771049 +0000 UTC"
Sep 15 20:04:58.011309 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:58.011278    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12440h40m14.355496738s"
Sep 15 20:04:58.056290 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:58.056260    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqbq7"
Sep 15 20:04:58.056468 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:58.056386    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqbq7" podUID="a7a98707-ce3f-4f6c-b723-a19113d90c41"
Sep 15 20:04:58.101442 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:58.101405    2577 generic.go:356] "Generic (PLEG): container finished" podID="eac46b2c823b79bb709b7641ecf92419" containerID="ecdb964755fc1ce8f088af6337dfe4fa20ed32b0b8ca83cbe99bb80c03145650" exitCode=0
Sep 15 20:04:58.101619 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:58.101510    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-72.ec2.internal" event={"ID":"eac46b2c823b79bb709b7641ecf92419","Type":"ContainerDied","Data":"ecdb964755fc1ce8f088af6337dfe4fa20ed32b0b8ca83cbe99bb80c03145650"}
Sep 15 20:04:58.118195 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:58.118160    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-72.ec2.internal" event={"ID":"6967c7c3d2c13b8fd404f2bb5001259f","Type":"ContainerStarted","Data":"a58842bcadf59c3d74f46b57bcda221e3a78676c4bfd3daaa358cd3e5bbe9f6a"}
Sep 15 20:04:59.054139 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:59.054105    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v4rt"
Sep 15 20:04:59.054808 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:59.054250    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v4rt" podUID="5482d254-b248-4eda-a7d8-37bcb3289d69"
Sep 15 20:04:59.054808 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:59.054629    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9dwjh"
Sep 15 20:04:59.054808 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:59.054718    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9dwjh" podUID="7e124866-fa5b-4ecb-bca9-075fa8bdb5a5"
Sep 15 20:04:59.147372 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:59.147259    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-72.ec2.internal" event={"ID":"eac46b2c823b79bb709b7641ecf92419","Type":"ContainerStarted","Data":"1099686b9108b9a64561b5ba3165b521b5c0de95e948e40e47e8a18f1cbfc5a4"}
Sep 15 20:04:59.166784 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:59.165524    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-72.ec2.internal" podStartSLOduration=4.165496914 podStartE2EDuration="4.165496914s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:04:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:04:58.137923697 +0000 UTC m=+4.528876695" watchObservedRunningTime="2026-09-15 20:04:59.165496914 +0000 UTC m=+5.556449911"
Sep 15 20:04:59.713947 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:59.713906    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5482d254-b248-4eda-a7d8-37bcb3289d69-metrics-certs\") pod \"network-metrics-daemon-2v4rt\" (UID: \"5482d254-b248-4eda-a7d8-37bcb3289d69\") " pod="openshift-multus/network-metrics-daemon-2v4rt"
Sep 15 20:04:59.714123 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:59.713974    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e124866-fa5b-4ecb-bca9-075fa8bdb5a5-original-pull-secret\") pod \"global-pull-secret-syncer-9dwjh\" (UID: \"7e124866-fa5b-4ecb-bca9-075fa8bdb5a5\") " pod="kube-system/global-pull-secret-syncer-9dwjh"
Sep 15 20:04:59.714204 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:59.714164    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 20:04:59.714260 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:59.714235    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e124866-fa5b-4ecb-bca9-075fa8bdb5a5-original-pull-secret podName:7e124866-fa5b-4ecb-bca9-075fa8bdb5a5 nodeName:}" failed. No retries permitted until 2026-09-15 20:05:03.714216106 +0000 UTC m=+10.105169123 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7e124866-fa5b-4ecb-bca9-075fa8bdb5a5-original-pull-secret") pod "global-pull-secret-syncer-9dwjh" (UID: "7e124866-fa5b-4ecb-bca9-075fa8bdb5a5") : object "kube-system"/"original-pull-secret" not registered
Sep 15 20:04:59.714260 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:59.714165    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:04:59.714371 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:59.714317    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5482d254-b248-4eda-a7d8-37bcb3289d69-metrics-certs podName:5482d254-b248-4eda-a7d8-37bcb3289d69 nodeName:}" failed. No retries permitted until 2026-09-15 20:05:03.71429862 +0000 UTC m=+10.105251599 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5482d254-b248-4eda-a7d8-37bcb3289d69-metrics-certs") pod "network-metrics-daemon-2v4rt" (UID: "5482d254-b248-4eda-a7d8-37bcb3289d69") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:04:59.814853 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:04:59.814818    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bnj8\" (UniqueName: \"kubernetes.io/projected/a7a98707-ce3f-4f6c-b723-a19113d90c41-kube-api-access-4bnj8\") pod \"network-check-target-jqbq7\" (UID: \"a7a98707-ce3f-4f6c-b723-a19113d90c41\") " pod="openshift-network-diagnostics/network-check-target-jqbq7"
Sep 15 20:04:59.815040 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:59.815022    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 20:04:59.815107 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:59.815042    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 20:04:59.815107 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:59.815057    2577 projected.go:196] Error preparing data for projected volume kube-api-access-4bnj8 for pod openshift-network-diagnostics/network-check-target-jqbq7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:04:59.815210 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:04:59.815117    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7a98707-ce3f-4f6c-b723-a19113d90c41-kube-api-access-4bnj8 podName:a7a98707-ce3f-4f6c-b723-a19113d90c41 nodeName:}" failed. No retries permitted until 2026-09-15 20:05:03.815098428 +0000 UTC m=+10.206051403 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4bnj8" (UniqueName: "kubernetes.io/projected/a7a98707-ce3f-4f6c-b723-a19113d90c41-kube-api-access-4bnj8") pod "network-check-target-jqbq7" (UID: "a7a98707-ce3f-4f6c-b723-a19113d90c41") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:05:00.056779 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:00.054742    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqbq7"
Sep 15 20:05:00.056779 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:00.054880    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqbq7" podUID="a7a98707-ce3f-4f6c-b723-a19113d90c41"
Sep 15 20:05:01.056215 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:01.055545    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9dwjh"
Sep 15 20:05:01.056215 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:01.055674    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9dwjh" podUID="7e124866-fa5b-4ecb-bca9-075fa8bdb5a5"
Sep 15 20:05:01.056215 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:01.056063    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v4rt"
Sep 15 20:05:01.056215 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:01.056168    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v4rt" podUID="5482d254-b248-4eda-a7d8-37bcb3289d69"
Sep 15 20:05:02.054519 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:02.054488    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqbq7"
Sep 15 20:05:02.054939 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:02.054602    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqbq7" podUID="a7a98707-ce3f-4f6c-b723-a19113d90c41"
Sep 15 20:05:03.054070 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:03.053988    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v4rt"
Sep 15 20:05:03.054235 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:03.054130    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v4rt" podUID="5482d254-b248-4eda-a7d8-37bcb3289d69"
Sep 15 20:05:03.054235 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:03.054153    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9dwjh"
Sep 15 20:05:03.054344 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:03.054267    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9dwjh" podUID="7e124866-fa5b-4ecb-bca9-075fa8bdb5a5"
Sep 15 20:05:03.746567 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:03.746527    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e124866-fa5b-4ecb-bca9-075fa8bdb5a5-original-pull-secret\") pod \"global-pull-secret-syncer-9dwjh\" (UID: \"7e124866-fa5b-4ecb-bca9-075fa8bdb5a5\") " pod="kube-system/global-pull-secret-syncer-9dwjh"
Sep 15 20:05:03.747103 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:03.746626    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5482d254-b248-4eda-a7d8-37bcb3289d69-metrics-certs\") pod \"network-metrics-daemon-2v4rt\" (UID: \"5482d254-b248-4eda-a7d8-37bcb3289d69\") " pod="openshift-multus/network-metrics-daemon-2v4rt"
Sep 15 20:05:03.747103 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:03.746765    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:05:03.747103 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:03.746829    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5482d254-b248-4eda-a7d8-37bcb3289d69-metrics-certs podName:5482d254-b248-4eda-a7d8-37bcb3289d69 nodeName:}" failed. No retries permitted until 2026-09-15 20:05:11.746808287 +0000 UTC m=+18.137761268 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5482d254-b248-4eda-a7d8-37bcb3289d69-metrics-certs") pod "network-metrics-daemon-2v4rt" (UID: "5482d254-b248-4eda-a7d8-37bcb3289d69") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:05:03.747255 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:03.747150    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 20:05:03.747255 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:03.747221    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e124866-fa5b-4ecb-bca9-075fa8bdb5a5-original-pull-secret podName:7e124866-fa5b-4ecb-bca9-075fa8bdb5a5 nodeName:}" failed. No retries permitted until 2026-09-15 20:05:11.74720248 +0000 UTC m=+18.138155463 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7e124866-fa5b-4ecb-bca9-075fa8bdb5a5-original-pull-secret") pod "global-pull-secret-syncer-9dwjh" (UID: "7e124866-fa5b-4ecb-bca9-075fa8bdb5a5") : object "kube-system"/"original-pull-secret" not registered
Sep 15 20:05:03.847841 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:03.847271    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bnj8\" (UniqueName: \"kubernetes.io/projected/a7a98707-ce3f-4f6c-b723-a19113d90c41-kube-api-access-4bnj8\") pod \"network-check-target-jqbq7\" (UID: \"a7a98707-ce3f-4f6c-b723-a19113d90c41\") " pod="openshift-network-diagnostics/network-check-target-jqbq7"
Sep 15 20:05:03.847841 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:03.847424    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 20:05:03.847841 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:03.847442    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 20:05:03.847841 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:03.847454    2577 projected.go:196] Error preparing data for projected volume kube-api-access-4bnj8 for pod openshift-network-diagnostics/network-check-target-jqbq7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:05:03.847841 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:03.847524    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7a98707-ce3f-4f6c-b723-a19113d90c41-kube-api-access-4bnj8 podName:a7a98707-ce3f-4f6c-b723-a19113d90c41 nodeName:}" failed. No retries permitted until 2026-09-15 20:05:11.847506604 +0000 UTC m=+18.238459580 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4bnj8" (UniqueName: "kubernetes.io/projected/a7a98707-ce3f-4f6c-b723-a19113d90c41-kube-api-access-4bnj8") pod "network-check-target-jqbq7" (UID: "a7a98707-ce3f-4f6c-b723-a19113d90c41") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:05:04.055574 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:04.055496    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqbq7"
Sep 15 20:05:04.055925 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:04.055892    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqbq7" podUID="a7a98707-ce3f-4f6c-b723-a19113d90c41"
Sep 15 20:05:05.054350 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:05.054308    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v4rt"
Sep 15 20:05:05.054350 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:05.054345    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9dwjh"
Sep 15 20:05:05.054902 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:05.054427    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v4rt" podUID="5482d254-b248-4eda-a7d8-37bcb3289d69"
Sep 15 20:05:05.054902 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:05.054558    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9dwjh" podUID="7e124866-fa5b-4ecb-bca9-075fa8bdb5a5"
Sep 15 20:05:06.054181 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:06.053826    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqbq7"
Sep 15 20:05:06.054181 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:06.053991    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqbq7" podUID="a7a98707-ce3f-4f6c-b723-a19113d90c41"
Sep 15 20:05:07.053855 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:07.053815    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v4rt"
Sep 15 20:05:07.054232 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:07.053815    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9dwjh"
Sep 15 20:05:07.054232 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:07.053931    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v4rt" podUID="5482d254-b248-4eda-a7d8-37bcb3289d69"
Sep 15 20:05:07.054232 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:07.054009    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9dwjh" podUID="7e124866-fa5b-4ecb-bca9-075fa8bdb5a5"
Sep 15 20:05:08.054321 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:08.054284    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqbq7"
Sep 15 20:05:08.054816 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:08.054419    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqbq7" podUID="a7a98707-ce3f-4f6c-b723-a19113d90c41"
Sep 15 20:05:09.053775 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:09.053718    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v4rt"
Sep 15 20:05:09.053952 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:09.053718    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9dwjh"
Sep 15 20:05:09.053952 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:09.053894    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v4rt" podUID="5482d254-b248-4eda-a7d8-37bcb3289d69"
Sep 15 20:05:09.054082 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:09.053969    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9dwjh" podUID="7e124866-fa5b-4ecb-bca9-075fa8bdb5a5"
Sep 15 20:05:10.053837 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:10.053801    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqbq7"
Sep 15 20:05:10.054391 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:10.053941    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqbq7" podUID="a7a98707-ce3f-4f6c-b723-a19113d90c41"
Sep 15 20:05:11.053904 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:11.053875    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v4rt"
Sep 15 20:05:11.054285 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:11.053877    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9dwjh"
Sep 15 20:05:11.054285 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:11.054028    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v4rt" podUID="5482d254-b248-4eda-a7d8-37bcb3289d69"
Sep 15 20:05:11.054285 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:11.054082    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9dwjh" podUID="7e124866-fa5b-4ecb-bca9-075fa8bdb5a5"
Sep 15 20:05:11.807029 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:11.806987    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5482d254-b248-4eda-a7d8-37bcb3289d69-metrics-certs\") pod \"network-metrics-daemon-2v4rt\" (UID: \"5482d254-b248-4eda-a7d8-37bcb3289d69\") " pod="openshift-multus/network-metrics-daemon-2v4rt"
Sep 15 20:05:11.807217 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:11.807047    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e124866-fa5b-4ecb-bca9-075fa8bdb5a5-original-pull-secret\") pod \"global-pull-secret-syncer-9dwjh\" (UID: \"7e124866-fa5b-4ecb-bca9-075fa8bdb5a5\") " pod="kube-system/global-pull-secret-syncer-9dwjh"
Sep 15 20:05:11.807217 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:11.807147    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:05:11.807217 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:11.807161    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 20:05:11.807217 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:11.807219    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5482d254-b248-4eda-a7d8-37bcb3289d69-metrics-certs podName:5482d254-b248-4eda-a7d8-37bcb3289d69 nodeName:}" failed. No retries permitted until 2026-09-15 20:05:27.807199195 +0000 UTC m=+34.198152825 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5482d254-b248-4eda-a7d8-37bcb3289d69-metrics-certs") pod "network-metrics-daemon-2v4rt" (UID: "5482d254-b248-4eda-a7d8-37bcb3289d69") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:05:11.807393 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:11.807239    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e124866-fa5b-4ecb-bca9-075fa8bdb5a5-original-pull-secret podName:7e124866-fa5b-4ecb-bca9-075fa8bdb5a5 nodeName:}" failed. No retries permitted until 2026-09-15 20:05:27.807230406 +0000 UTC m=+34.198183400 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7e124866-fa5b-4ecb-bca9-075fa8bdb5a5-original-pull-secret") pod "global-pull-secret-syncer-9dwjh" (UID: "7e124866-fa5b-4ecb-bca9-075fa8bdb5a5") : object "kube-system"/"original-pull-secret" not registered
Sep 15 20:05:11.907857 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:11.907817    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bnj8\" (UniqueName: \"kubernetes.io/projected/a7a98707-ce3f-4f6c-b723-a19113d90c41-kube-api-access-4bnj8\") pod \"network-check-target-jqbq7\" (UID: \"a7a98707-ce3f-4f6c-b723-a19113d90c41\") " pod="openshift-network-diagnostics/network-check-target-jqbq7"
Sep 15 20:05:11.908024 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:11.907999    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 20:05:11.908024 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:11.908019    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 20:05:11.908124 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:11.908029    2577 projected.go:196] Error preparing data for projected volume kube-api-access-4bnj8 for pod openshift-network-diagnostics/network-check-target-jqbq7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:05:11.908124 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:11.908092    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7a98707-ce3f-4f6c-b723-a19113d90c41-kube-api-access-4bnj8 podName:a7a98707-ce3f-4f6c-b723-a19113d90c41 nodeName:}" failed. No retries permitted until 2026-09-15 20:05:27.908073669 +0000 UTC m=+34.299026667 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4bnj8" (UniqueName: "kubernetes.io/projected/a7a98707-ce3f-4f6c-b723-a19113d90c41-kube-api-access-4bnj8") pod "network-check-target-jqbq7" (UID: "a7a98707-ce3f-4f6c-b723-a19113d90c41") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:05:12.053662 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:12.053538    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqbq7"
Sep 15 20:05:12.053662 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:12.053657    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqbq7" podUID="a7a98707-ce3f-4f6c-b723-a19113d90c41"
Sep 15 20:05:13.054532 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:13.054492    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9dwjh"
Sep 15 20:05:13.054532 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:13.054522    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v4rt"
Sep 15 20:05:13.055063 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:13.054629    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9dwjh" podUID="7e124866-fa5b-4ecb-bca9-075fa8bdb5a5"
Sep 15 20:05:13.055063 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:13.054808    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v4rt" podUID="5482d254-b248-4eda-a7d8-37bcb3289d69"
Sep 15 20:05:14.054332 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:14.054301    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqbq7"
Sep 15 20:05:14.054508 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:14.054401    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqbq7" podUID="a7a98707-ce3f-4f6c-b723-a19113d90c41"
Sep 15 20:05:15.054290 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:15.053667    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9dwjh"
Sep 15 20:05:15.054290 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:15.053899    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v4rt"
Sep 15 20:05:15.054290 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:15.054051    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9dwjh" podUID="7e124866-fa5b-4ecb-bca9-075fa8bdb5a5"
Sep 15 20:05:15.054290 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:15.054139    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v4rt" podUID="5482d254-b248-4eda-a7d8-37bcb3289d69"
Sep 15 20:05:15.179602 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:15.179421    2577 generic.go:356] "Generic (PLEG): container finished" podID="fd4ce615-2127-43bd-b843-f1b5a4bfcdc7" containerID="fb16d78aad4b4bc592f6fd66756c1b56f04650f90460be8cf85c64c3f58834f6" exitCode=0
Sep 15 20:05:15.179763 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:15.179505    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kjp4l" event={"ID":"fd4ce615-2127-43bd-b843-f1b5a4bfcdc7","Type":"ContainerDied","Data":"fb16d78aad4b4bc592f6fd66756c1b56f04650f90460be8cf85c64c3f58834f6"}
Sep 15 20:05:15.181433 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:15.181411    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vm6lc" event={"ID":"81e9d392-e3c5-48f6-b98a-4bcaddda3336","Type":"ContainerStarted","Data":"b09c82e9880d246303ce8f60b254ff3654a5f7000a1fd4ba3affaee0dd9197f0"}
Sep 15 20:05:15.182956 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:15.182928    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-k6cn5" event={"ID":"a1d3494a-ca54-44cd-bcc2-bb19321b6f11","Type":"ContainerStarted","Data":"a922ae1c3da7a7331250f61a08494d98eeba2b3d1fc7ac0bc5b95ef35d95c8c5"}
Sep 15 20:05:15.185144 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:15.185120    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm" event={"ID":"87b7bd71-28d6-4214-af3c-44739f696128","Type":"ContainerStarted","Data":"cbb6f1b70902c2101bf70a5976a2554644c117a30566d00947a8c530dd3b5fe2"}
Sep 15 20:05:15.185237 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:15.185185    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm" event={"ID":"87b7bd71-28d6-4214-af3c-44739f696128","Type":"ContainerStarted","Data":"a1e843265a4cd3ee8e71685b22eec6514b80aff9f4b08ac29ef73b343e2c2e16"}
Sep 15 20:05:15.186768 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:15.186742    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-z5wjn" event={"ID":"b6f901c3-0f40-46dc-8406-eaee847b1c10","Type":"ContainerStarted","Data":"e993950c968a77f7b3033efb1d2c3a80a1e9c02d93a4056784cc4b373f1429b0"}
Sep 15 20:05:15.188421 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:15.188393    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqj9r" event={"ID":"c0566e5c-2129-4093-8fd4-d13f98007c00","Type":"ContainerStarted","Data":"88198361d7cb6f7f959d6a8a4c174e7f895495dc5ee906ca880d4e344b142b2b"}
Sep 15 20:05:15.189940 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:15.189918    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-brdnq" event={"ID":"a8a416c3-51a5-4b55-9c73-9aaca4f93242","Type":"ContainerStarted","Data":"2008e55cb9dfc4cfb8f074f5e46ad725d7e7935e67106fd949f107885eb8c795"}
Sep 15 20:05:15.191634 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:15.191602    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qvw2x" event={"ID":"1547d350-30fe-4c0b-9448-1cbd9c4ae5b9","Type":"ContainerStarted","Data":"94cf9e7664c76b4ee5945877b1ec13065f30a1ccf30a08cf2d4e5115feeb9e15"}
Sep 15 20:05:15.202447 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:15.202399    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-72.ec2.internal" podStartSLOduration=20.202384577 podStartE2EDuration="20.202384577s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:04:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:04:59.166757264 +0000 UTC m=+5.557710264" watchObservedRunningTime="2026-09-15 20:05:15.202384577 +0000 UTC m=+21.593337574"
Sep 15 20:05:15.218294 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:15.218250    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-vm6lc" podStartSLOduration=3.4485729210000002 podStartE2EDuration="21.218234648s" totalImagesPullingTime="17.769661727s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:04:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:05:15.217791195 +0000 UTC m=+21.608744196" watchObservedRunningTime="2026-09-15 20:05:15.218234648 +0000 UTC m=+21.609187645"
Sep 15 20:05:15.234973 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:15.234929    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-z5wjn" podStartSLOduration=3.467888791 podStartE2EDuration="21.234914291s" totalImagesPullingTime="17.7670255s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:04:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:05:15.23466491 +0000 UTC m=+21.625617907" watchObservedRunningTime="2026-09-15 20:05:15.234914291 +0000 UTC m=+21.625867286"
Sep 15 20:05:15.255294 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:15.253822    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-brdnq" podStartSLOduration=3.332913438 podStartE2EDuration="21.253800471s" totalImagesPullingTime="17.920887033s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:04:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:05:15.252742422 +0000 UTC m=+21.643695410" watchObservedRunningTime="2026-09-15 20:05:15.253800471 +0000 UTC m=+21.644753487"
Sep 15 20:05:15.287262 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:15.287192    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-k6cn5" podStartSLOduration=3.512748736 podStartE2EDuration="21.287176333s" totalImagesPullingTime="17.774427597s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:04:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:05:15.272746465 +0000 UTC m=+21.663699454" watchObservedRunningTime="2026-09-15 20:05:15.287176333 +0000 UTC m=+21.678129309"
Sep 15 20:05:15.287523 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:15.287502    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qvw2x" podStartSLOduration=3.510301936 podStartE2EDuration="21.287496162s" totalImagesPullingTime="17.777194226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:04:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:05:15.287242726 +0000 UTC m=+21.678195734" watchObservedRunningTime="2026-09-15 20:05:15.287496162 +0000 UTC m=+21.678449186"
Sep 15 20:05:16.053807 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:16.053771    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqbq7"
Sep 15 20:05:16.054015 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:16.053894    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqbq7" podUID="a7a98707-ce3f-4f6c-b723-a19113d90c41"
Sep 15 20:05:16.198492 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:16.198459    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm" event={"ID":"87b7bd71-28d6-4214-af3c-44739f696128","Type":"ContainerStarted","Data":"2229519399d2b89eef05e488db228a59f6f90e498f42665d9875aae13dd47ba0"}
Sep 15 20:05:16.199215 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:16.198551    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm" event={"ID":"87b7bd71-28d6-4214-af3c-44739f696128","Type":"ContainerStarted","Data":"5fe1a4f3c03ddcc38ec4a2db23354e21e27cbcb1ef6a07bcd5c9c2da9be70813"}
Sep 15 20:05:16.199215 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:16.198595    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm" event={"ID":"87b7bd71-28d6-4214-af3c-44739f696128","Type":"ContainerStarted","Data":"72bfedcabadf5028e270e3568c0007931245eba81254c855e92d72e41f29ac40"}
Sep 15 20:05:16.199215 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:16.198638    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm" event={"ID":"87b7bd71-28d6-4214-af3c-44739f696128","Type":"ContainerStarted","Data":"352d3a695639242be8b36595b3141d29b92948d4ba2c4fa5a2ec2786ea325220"}
Sep 15 20:05:16.461798 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:16.461766    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qvw2x"
Sep 15 20:05:16.462383 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:16.462362    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qvw2x"
Sep 15 20:05:16.512701 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:16.512674    2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 15 20:05:17.038691 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:17.038594    2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-15T20:05:16.512695668Z","UUID":"55bb2382-12ac-47d6-a7c7-1a68cea3522d","Handler":null,"Name":"","Endpoint":""}
Sep 15 20:05:17.040064 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:17.040045    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 15 20:05:17.040190 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:17.040076    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 15 20:05:17.053863 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:17.053837    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v4rt"
Sep 15 20:05:17.053996 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:17.053972    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v4rt" podUID="5482d254-b248-4eda-a7d8-37bcb3289d69"
Sep 15 20:05:17.054059 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:17.053988    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9dwjh"
Sep 15 20:05:17.054117 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:17.054087    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9dwjh" podUID="7e124866-fa5b-4ecb-bca9-075fa8bdb5a5"
Sep 15 20:05:17.204913 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:17.204872    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fmkfr" event={"ID":"4b044f60-fa8f-4ae9-9f9b-6722d67b4fb0","Type":"ContainerStarted","Data":"5fd890aaa00273bd8a8c94573a10c9ad0faf5425bf8ead43f21a744b59a17000"}
Sep 15 20:05:17.207387 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:17.207348    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqj9r" event={"ID":"c0566e5c-2129-4093-8fd4-d13f98007c00","Type":"ContainerStarted","Data":"ff84d4b0eb12adc1b6d3921985d042a574b2f962f6f109c5c460abee356185e6"}
Sep 15 20:05:17.207632 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:17.207598    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qvw2x"
Sep 15 20:05:17.208473 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:17.208426    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qvw2x"
Sep 15 20:05:17.220148 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:17.219928    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-fmkfr" podStartSLOduration=5.442379709 podStartE2EDuration="23.219909908s" totalImagesPullingTime="17.777530199s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:04:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:05:17.219244825 +0000 UTC m=+23.610197822" watchObservedRunningTime="2026-09-15 20:05:17.219909908 +0000 UTC m=+23.610862906"
Sep 15 20:05:18.053695 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:18.053655    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqbq7"
Sep 15 20:05:18.053957 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:18.053814    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqbq7" podUID="a7a98707-ce3f-4f6c-b723-a19113d90c41"
Sep 15 20:05:18.214114 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:18.213861    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm" event={"ID":"87b7bd71-28d6-4214-af3c-44739f696128","Type":"ContainerStarted","Data":"6651da39c48f97d79c6235df062898ad0d8e0891ada6ea571702876478a6a0c4"}
Sep 15 20:05:18.216241 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:18.216209    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqj9r" event={"ID":"c0566e5c-2129-4093-8fd4-d13f98007c00","Type":"ContainerStarted","Data":"514b56b041d8b09700012b4c877730c997bbb70f436f47e07a27f153932cb438"}
Sep 15 20:05:19.054191 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:19.054156    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v4rt"
Sep 15 20:05:19.054474 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:19.054201    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9dwjh"
Sep 15 20:05:19.054474 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:19.054299    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v4rt" podUID="5482d254-b248-4eda-a7d8-37bcb3289d69"
Sep 15 20:05:19.054474 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:19.054429    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9dwjh" podUID="7e124866-fa5b-4ecb-bca9-075fa8bdb5a5"
Sep 15 20:05:20.053926 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:20.053902    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqbq7"
Sep 15 20:05:20.054285 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:20.054050    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqbq7" podUID="a7a98707-ce3f-4f6c-b723-a19113d90c41"
Sep 15 20:05:20.223196 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:20.223105    2577 generic.go:356] "Generic (PLEG): container finished" podID="fd4ce615-2127-43bd-b843-f1b5a4bfcdc7" containerID="bf200263e5fb03c2caf23ead6bf3a292849c7e233e675f7b4246be2ff80541d4" exitCode=0
Sep 15 20:05:20.223339 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:20.223188    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kjp4l" event={"ID":"fd4ce615-2127-43bd-b843-f1b5a4bfcdc7","Type":"ContainerDied","Data":"bf200263e5fb03c2caf23ead6bf3a292849c7e233e675f7b4246be2ff80541d4"}
Sep 15 20:05:20.230690 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:20.230665    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm" event={"ID":"87b7bd71-28d6-4214-af3c-44739f696128","Type":"ContainerStarted","Data":"d21749add8950497131c0ddfcbed620013b68f7a9071fbc50c4f0f638612010b"}
Sep 15 20:05:20.230925 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:20.230909    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:05:20.231000 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:20.230958    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:05:20.246611 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:20.246588    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:05:20.249775 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:20.249708    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cqj9r" podStartSLOduration=5.899878977 podStartE2EDuration="26.249693085s" totalImagesPullingTime="20.349814108s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:04:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:05:18.238327211 +0000 UTC m=+24.629280207" watchObservedRunningTime="2026-09-15 20:05:20.249693085 +0000 UTC m=+26.640646079"
Sep 15 20:05:20.279856 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:20.279804    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm" podStartSLOduration=8.472243527 podStartE2EDuration="26.279787815s" totalImagesPullingTime="17.807544288s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:04:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:05:20.278144897 +0000 UTC m=+26.669097893" watchObservedRunningTime="2026-09-15 20:05:20.279787815 +0000 UTC m=+26.670740811"
Sep 15 20:05:21.053889 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:21.053700    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9dwjh"
Sep 15 20:05:21.054077 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:21.053774    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v4rt"
Sep 15 20:05:21.054077 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:21.053986    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9dwjh" podUID="7e124866-fa5b-4ecb-bca9-075fa8bdb5a5"
Sep 15 20:05:21.054077 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:21.054041    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v4rt" podUID="5482d254-b248-4eda-a7d8-37bcb3289d69"
Sep 15 20:05:21.233938 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:21.233909    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:05:21.254470 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:21.254434    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:05:21.920604 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:21.920570    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jqbq7"]
Sep 15 20:05:21.920844 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:21.920690    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqbq7"
Sep 15 20:05:21.920844 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:21.920789    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqbq7" podUID="a7a98707-ce3f-4f6c-b723-a19113d90c41"
Sep 15 20:05:21.921304 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:21.921278    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2v4rt"]
Sep 15 20:05:21.921412 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:21.921385    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v4rt"
Sep 15 20:05:21.921491 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:21.921469    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v4rt" podUID="5482d254-b248-4eda-a7d8-37bcb3289d69"
Sep 15 20:05:21.923285 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:21.923258    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9dwjh"]
Sep 15 20:05:21.923384 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:21.923331    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9dwjh"
Sep 15 20:05:21.923446 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:21.923410    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9dwjh" podUID="7e124866-fa5b-4ecb-bca9-075fa8bdb5a5"
Sep 15 20:05:22.238072 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:22.238034    2577 generic.go:356] "Generic (PLEG): container finished" podID="fd4ce615-2127-43bd-b843-f1b5a4bfcdc7" containerID="5d91c0b01333454d48e50e275a3de6ea4fe1f5a4210e9f63a47657e84f8967e8" exitCode=0
Sep 15 20:05:22.238871 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:22.238108    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kjp4l" event={"ID":"fd4ce615-2127-43bd-b843-f1b5a4bfcdc7","Type":"ContainerDied","Data":"5d91c0b01333454d48e50e275a3de6ea4fe1f5a4210e9f63a47657e84f8967e8"}
Sep 15 20:05:24.055035 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:24.054817    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v4rt"
Sep 15 20:05:24.055495 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:24.054871    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9dwjh"
Sep 15 20:05:24.055495 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:24.055149    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v4rt" podUID="5482d254-b248-4eda-a7d8-37bcb3289d69"
Sep 15 20:05:24.055495 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:24.054903    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqbq7"
Sep 15 20:05:24.055495 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:24.055213    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9dwjh" podUID="7e124866-fa5b-4ecb-bca9-075fa8bdb5a5"
Sep 15 20:05:24.055495 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:24.055304    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqbq7" podUID="a7a98707-ce3f-4f6c-b723-a19113d90c41"
Sep 15 20:05:24.244736 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:24.244686    2577 generic.go:356] "Generic (PLEG): container finished" podID="fd4ce615-2127-43bd-b843-f1b5a4bfcdc7" containerID="f25adbd9032cb522068113cbbaf9e3fc408c5e85fc20e4d311615372f77c7877" exitCode=0
Sep 15 20:05:24.244893 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:24.244766    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kjp4l" event={"ID":"fd4ce615-2127-43bd-b843-f1b5a4bfcdc7","Type":"ContainerDied","Data":"f25adbd9032cb522068113cbbaf9e3fc408c5e85fc20e4d311615372f77c7877"}
Sep 15 20:05:26.053893 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:26.053858    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqbq7"
Sep 15 20:05:26.054350 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:26.053858    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9dwjh"
Sep 15 20:05:26.054350 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:26.053986    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqbq7" podUID="a7a98707-ce3f-4f6c-b723-a19113d90c41"
Sep 15 20:05:26.054350 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:26.053997    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v4rt"
Sep 15 20:05:26.054350 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:26.054131    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9dwjh" podUID="7e124866-fa5b-4ecb-bca9-075fa8bdb5a5"
Sep 15 20:05:26.054350 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:26.054226    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v4rt" podUID="5482d254-b248-4eda-a7d8-37bcb3289d69"
Sep 15 20:05:27.831204 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:27.831173    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5482d254-b248-4eda-a7d8-37bcb3289d69-metrics-certs\") pod \"network-metrics-daemon-2v4rt\" (UID: \"5482d254-b248-4eda-a7d8-37bcb3289d69\") " pod="openshift-multus/network-metrics-daemon-2v4rt"
Sep 15 20:05:27.831669 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:27.831217    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e124866-fa5b-4ecb-bca9-075fa8bdb5a5-original-pull-secret\") pod \"global-pull-secret-syncer-9dwjh\" (UID: \"7e124866-fa5b-4ecb-bca9-075fa8bdb5a5\") " pod="kube-system/global-pull-secret-syncer-9dwjh"
Sep 15 20:05:27.831669 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:27.831338    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 20:05:27.831669 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:27.831348    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:05:27.831669 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:27.831398    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e124866-fa5b-4ecb-bca9-075fa8bdb5a5-original-pull-secret podName:7e124866-fa5b-4ecb-bca9-075fa8bdb5a5 nodeName:}" failed. No retries permitted until 2026-09-15 20:05:59.831382977 +0000 UTC m=+66.222335977 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7e124866-fa5b-4ecb-bca9-075fa8bdb5a5-original-pull-secret") pod "global-pull-secret-syncer-9dwjh" (UID: "7e124866-fa5b-4ecb-bca9-075fa8bdb5a5") : object "kube-system"/"original-pull-secret" not registered
Sep 15 20:05:27.831669 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:27.831413    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5482d254-b248-4eda-a7d8-37bcb3289d69-metrics-certs podName:5482d254-b248-4eda-a7d8-37bcb3289d69 nodeName:}" failed. No retries permitted until 2026-09-15 20:05:59.831406984 +0000 UTC m=+66.222359959 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5482d254-b248-4eda-a7d8-37bcb3289d69-metrics-certs") pod "network-metrics-daemon-2v4rt" (UID: "5482d254-b248-4eda-a7d8-37bcb3289d69") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:05:27.932605 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:27.932570    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bnj8\" (UniqueName: \"kubernetes.io/projected/a7a98707-ce3f-4f6c-b723-a19113d90c41-kube-api-access-4bnj8\") pod \"network-check-target-jqbq7\" (UID: \"a7a98707-ce3f-4f6c-b723-a19113d90c41\") " pod="openshift-network-diagnostics/network-check-target-jqbq7"
Sep 15 20:05:27.932832 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:27.932762    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 20:05:27.932832 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:27.932790    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 20:05:27.932832 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:27.932801    2577 projected.go:196] Error preparing data for projected volume kube-api-access-4bnj8 for pod openshift-network-diagnostics/network-check-target-jqbq7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:05:27.932975 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:27.932856    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7a98707-ce3f-4f6c-b723-a19113d90c41-kube-api-access-4bnj8 podName:a7a98707-ce3f-4f6c-b723-a19113d90c41 nodeName:}" failed. No retries permitted until 2026-09-15 20:05:59.932841811 +0000 UTC m=+66.323794800 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4bnj8" (UniqueName: "kubernetes.io/projected/a7a98707-ce3f-4f6c-b723-a19113d90c41-kube-api-access-4bnj8") pod "network-check-target-jqbq7" (UID: "a7a98707-ce3f-4f6c-b723-a19113d90c41") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:05:28.054634 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:28.054599    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9dwjh"
Sep 15 20:05:28.054634 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:28.054630    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqbq7"
Sep 15 20:05:28.054891 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:28.054600    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v4rt"
Sep 15 20:05:28.054891 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:28.054752    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9dwjh" podUID="7e124866-fa5b-4ecb-bca9-075fa8bdb5a5"
Sep 15 20:05:28.054891 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:28.054877    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2v4rt" podUID="5482d254-b248-4eda-a7d8-37bcb3289d69"
Sep 15 20:05:28.055049 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:28.054980    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqbq7" podUID="a7a98707-ce3f-4f6c-b723-a19113d90c41"
Sep 15 20:05:28.932051 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:28.932013    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-72.ec2.internal" event="NodeReady"
Sep 15 20:05:28.932581 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:28.932182    2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 15 20:05:28.994281 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:28.994239    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-h67tx"]
Sep 15 20:05:28.997261 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:28.997235    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5l278"]
Sep 15 20:05:28.997405 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:28.997378    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-h67tx"
Sep 15 20:05:28.999243 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:28.999218    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5l278"
Sep 15 20:05:29.000115 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:29.000090    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 15 20:05:29.000115 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:29.000107    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 15 20:05:29.000291 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:29.000106    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jwzj9\""
Sep 15 20:05:29.001951 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:29.001929    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 15 20:05:29.002096 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:29.002079    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 15 20:05:29.002161 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:29.002119    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 15 20:05:29.002498 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:29.002478    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-lzsdw\""
Sep 15 20:05:29.006785 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:29.006700    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-h67tx"]
Sep 15 20:05:29.024105 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:29.024076    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5l278"]
Sep 15 20:05:29.143015 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:29.142978    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5360f750-45a7-42d6-8b65-be039f6ee235-config-volume\") pod \"dns-default-h67tx\" (UID: \"5360f750-45a7-42d6-8b65-be039f6ee235\") " pod="openshift-dns/dns-default-h67tx"
Sep 15 20:05:29.143194 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:29.143049    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5360f750-45a7-42d6-8b65-be039f6ee235-tmp-dir\") pod \"dns-default-h67tx\" (UID: \"5360f750-45a7-42d6-8b65-be039f6ee235\") " pod="openshift-dns/dns-default-h67tx"
Sep 15 20:05:29.143194 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:29.143097    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2qpc\" (UniqueName: \"kubernetes.io/projected/5360f750-45a7-42d6-8b65-be039f6ee235-kube-api-access-r2qpc\") pod \"dns-default-h67tx\" (UID: \"5360f750-45a7-42d6-8b65-be039f6ee235\") " pod="openshift-dns/dns-default-h67tx"
Sep 15 20:05:29.143194 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:29.143117    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfs2c\" (UniqueName: \"kubernetes.io/projected/64743302-a5a8-4b6f-b7c7-d574edd19211-kube-api-access-rfs2c\") pod \"ingress-canary-5l278\" (UID: \"64743302-a5a8-4b6f-b7c7-d574edd19211\") " pod="openshift-ingress-canary/ingress-canary-5l278"
Sep 15 20:05:29.143381 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:29.143211    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5360f750-45a7-42d6-8b65-be039f6ee235-metrics-tls\") pod \"dns-default-h67tx\" (UID: \"5360f750-45a7-42d6-8b65-be039f6ee235\") " pod="openshift-dns/dns-default-h67tx"
Sep 15 20:05:29.143381 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:29.143287    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/64743302-a5a8-4b6f-b7c7-d574edd19211-cert\") pod \"ingress-canary-5l278\" (UID: \"64743302-a5a8-4b6f-b7c7-d574edd19211\") " pod="openshift-ingress-canary/ingress-canary-5l278"
Sep 15 20:05:29.243974 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:29.243946    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2qpc\" (UniqueName: \"kubernetes.io/projected/5360f750-45a7-42d6-8b65-be039f6ee235-kube-api-access-r2qpc\") pod \"dns-default-h67tx\" (UID: \"5360f750-45a7-42d6-8b65-be039f6ee235\") " pod="openshift-dns/dns-default-h67tx"
Sep 15 20:05:29.244149 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:29.243985    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfs2c\" (UniqueName: \"kubernetes.io/projected/64743302-a5a8-4b6f-b7c7-d574edd19211-kube-api-access-rfs2c\") pod \"ingress-canary-5l278\" (UID: \"64743302-a5a8-4b6f-b7c7-d574edd19211\") " pod="openshift-ingress-canary/ingress-canary-5l278"
Sep 15 20:05:29.244149 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:29.244010    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5360f750-45a7-42d6-8b65-be039f6ee235-metrics-tls\") pod \"dns-default-h67tx\" (UID: \"5360f750-45a7-42d6-8b65-be039f6ee235\") " pod="openshift-dns/dns-default-h67tx"
Sep 15 20:05:29.244149 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:29.244043    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/64743302-a5a8-4b6f-b7c7-d574edd19211-cert\") pod \"ingress-canary-5l278\" (UID: \"64743302-a5a8-4b6f-b7c7-d574edd19211\") " pod="openshift-ingress-canary/ingress-canary-5l278"
Sep 15 20:05:29.244149 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:29.244069    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5360f750-45a7-42d6-8b65-be039f6ee235-config-volume\") pod \"dns-default-h67tx\" (UID: \"5360f750-45a7-42d6-8b65-be039f6ee235\") " pod="openshift-dns/dns-default-h67tx"
Sep 15 20:05:29.244334 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:29.244159    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 20:05:29.244334 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:29.244250    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5360f750-45a7-42d6-8b65-be039f6ee235-metrics-tls podName:5360f750-45a7-42d6-8b65-be039f6ee235 nodeName:}" failed. No retries permitted until 2026-09-15 20:05:29.744227325 +0000 UTC m=+36.135180321 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5360f750-45a7-42d6-8b65-be039f6ee235-metrics-tls") pod "dns-default-h67tx" (UID: "5360f750-45a7-42d6-8b65-be039f6ee235") : secret "dns-default-metrics-tls" not found
Sep 15 20:05:29.244334 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:29.244287    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5360f750-45a7-42d6-8b65-be039f6ee235-tmp-dir\") pod \"dns-default-h67tx\" (UID: \"5360f750-45a7-42d6-8b65-be039f6ee235\") " pod="openshift-dns/dns-default-h67tx"
Sep 15 20:05:29.244560 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:29.244539    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5360f750-45a7-42d6-8b65-be039f6ee235-config-volume\") pod \"dns-default-h67tx\" (UID: \"5360f750-45a7-42d6-8b65-be039f6ee235\") " pod="openshift-dns/dns-default-h67tx"
Sep 15 20:05:29.244616 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:29.244589    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5360f750-45a7-42d6-8b65-be039f6ee235-tmp-dir\") pod \"dns-default-h67tx\" (UID: \"5360f750-45a7-42d6-8b65-be039f6ee235\") " pod="openshift-dns/dns-default-h67tx"
Sep 15 20:05:29.249373 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:29.249347    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/64743302-a5a8-4b6f-b7c7-d574edd19211-cert\") pod \"ingress-canary-5l278\" (UID: \"64743302-a5a8-4b6f-b7c7-d574edd19211\") " pod="openshift-ingress-canary/ingress-canary-5l278"
Sep 15 20:05:29.253066 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:29.252987    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2qpc\" (UniqueName: \"kubernetes.io/projected/5360f750-45a7-42d6-8b65-be039f6ee235-kube-api-access-r2qpc\") pod \"dns-default-h67tx\" (UID: \"5360f750-45a7-42d6-8b65-be039f6ee235\") " pod="openshift-dns/dns-default-h67tx"
Sep 15 20:05:29.253583 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:29.253543    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfs2c\" (UniqueName: \"kubernetes.io/projected/64743302-a5a8-4b6f-b7c7-d574edd19211-kube-api-access-rfs2c\") pod \"ingress-canary-5l278\" (UID: \"64743302-a5a8-4b6f-b7c7-d574edd19211\") " pod="openshift-ingress-canary/ingress-canary-5l278"
Sep 15 20:05:29.317130 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:29.317104    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5l278"
Sep 15 20:05:29.596842 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:29.596466    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5l278"]
Sep 15 20:05:29.599562 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:05:29.599537    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64743302_a5a8_4b6f_b7c7_d574edd19211.slice/crio-14b038cd592c74a1f8903227cda4b55aa5dddc25599ceb726d70e92da5794bf7 WatchSource:0}: Error finding container 14b038cd592c74a1f8903227cda4b55aa5dddc25599ceb726d70e92da5794bf7: Status 404 returned error can't find the container with id 14b038cd592c74a1f8903227cda4b55aa5dddc25599ceb726d70e92da5794bf7
Sep 15 20:05:29.748484 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:29.748444    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5360f750-45a7-42d6-8b65-be039f6ee235-metrics-tls\") pod \"dns-default-h67tx\" (UID: \"5360f750-45a7-42d6-8b65-be039f6ee235\") " pod="openshift-dns/dns-default-h67tx"
Sep 15 20:05:29.748652 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:29.748614    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 20:05:29.748717 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:29.748700    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5360f750-45a7-42d6-8b65-be039f6ee235-metrics-tls podName:5360f750-45a7-42d6-8b65-be039f6ee235 nodeName:}" failed. No retries permitted until 2026-09-15 20:05:30.748675615 +0000 UTC m=+37.139628611 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5360f750-45a7-42d6-8b65-be039f6ee235-metrics-tls") pod "dns-default-h67tx" (UID: "5360f750-45a7-42d6-8b65-be039f6ee235") : secret "dns-default-metrics-tls" not found
Sep 15 20:05:30.054048 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:30.054009    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9dwjh"
Sep 15 20:05:30.054604 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:30.054009    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v4rt"
Sep 15 20:05:30.054604 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:30.054147    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqbq7"
Sep 15 20:05:30.058116 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:30.058098    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 15 20:05:30.058371 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:30.058349    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 15 20:05:30.058461 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:30.058378    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l9zsw\""
Sep 15 20:05:30.058461 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:30.058395    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fxg67\""
Sep 15 20:05:30.058461 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:30.058354    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 15 20:05:30.058461 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:30.058354    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 15 20:05:30.263685 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:30.263650    2577 generic.go:356] "Generic (PLEG): container finished" podID="fd4ce615-2127-43bd-b843-f1b5a4bfcdc7" containerID="8bc2ebcc406f9019cded67b18a7e2f4d4aa96d6bd37a56e09d5cb2e63362c6b6" exitCode=0
Sep 15 20:05:30.263871 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:30.263743    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kjp4l" event={"ID":"fd4ce615-2127-43bd-b843-f1b5a4bfcdc7","Type":"ContainerDied","Data":"8bc2ebcc406f9019cded67b18a7e2f4d4aa96d6bd37a56e09d5cb2e63362c6b6"}
Sep 15 20:05:30.264983 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:30.264956    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5l278" event={"ID":"64743302-a5a8-4b6f-b7c7-d574edd19211","Type":"ContainerStarted","Data":"14b038cd592c74a1f8903227cda4b55aa5dddc25599ceb726d70e92da5794bf7"}
Sep 15 20:05:30.755864 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:30.755827    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5360f750-45a7-42d6-8b65-be039f6ee235-metrics-tls\") pod \"dns-default-h67tx\" (UID: \"5360f750-45a7-42d6-8b65-be039f6ee235\") " pod="openshift-dns/dns-default-h67tx"
Sep 15 20:05:30.756035 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:30.756016    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 20:05:30.756111 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:30.756097    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5360f750-45a7-42d6-8b65-be039f6ee235-metrics-tls podName:5360f750-45a7-42d6-8b65-be039f6ee235 nodeName:}" failed. No retries permitted until 2026-09-15 20:05:32.756075451 +0000 UTC m=+39.147028429 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5360f750-45a7-42d6-8b65-be039f6ee235-metrics-tls") pod "dns-default-h67tx" (UID: "5360f750-45a7-42d6-8b65-be039f6ee235") : secret "dns-default-metrics-tls" not found
Sep 15 20:05:31.271533 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:31.271498    2577 generic.go:356] "Generic (PLEG): container finished" podID="fd4ce615-2127-43bd-b843-f1b5a4bfcdc7" containerID="d2f4b997511ac82896b357348dd38739c07f024eaa40219f9492bb8fe6e91775" exitCode=0
Sep 15 20:05:31.272235 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:31.271538    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kjp4l" event={"ID":"fd4ce615-2127-43bd-b843-f1b5a4bfcdc7","Type":"ContainerDied","Data":"d2f4b997511ac82896b357348dd38739c07f024eaa40219f9492bb8fe6e91775"}
Sep 15 20:05:32.276938 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:32.276857    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kjp4l" event={"ID":"fd4ce615-2127-43bd-b843-f1b5a4bfcdc7","Type":"ContainerStarted","Data":"ba56b6e71b94823d02ef08ed39b0c75bec7f05fdc9034284744fada6fd735176"}
Sep 15 20:05:32.299903 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:32.299850    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-kjp4l" podStartSLOduration=8.321917307 podStartE2EDuration="38.299835827s" totalImagesPullingTime="29.97791852s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:04:54 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:05:32.299620246 +0000 UTC m=+38.690573242" watchObservedRunningTime="2026-09-15 20:05:32.299835827 +0000 UTC m=+38.690788853"
Sep 15 20:05:32.771868 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:32.771822    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5360f750-45a7-42d6-8b65-be039f6ee235-metrics-tls\") pod \"dns-default-h67tx\" (UID: \"5360f750-45a7-42d6-8b65-be039f6ee235\") " pod="openshift-dns/dns-default-h67tx"
Sep 15 20:05:32.772028 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:32.772002    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 20:05:32.772087 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:32.772077    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5360f750-45a7-42d6-8b65-be039f6ee235-metrics-tls podName:5360f750-45a7-42d6-8b65-be039f6ee235 nodeName:}" failed. No retries permitted until 2026-09-15 20:05:36.772061961 +0000 UTC m=+43.163014957 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5360f750-45a7-42d6-8b65-be039f6ee235-metrics-tls") pod "dns-default-h67tx" (UID: "5360f750-45a7-42d6-8b65-be039f6ee235") : secret "dns-default-metrics-tls" not found
Sep 15 20:05:33.280932 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:33.280892    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5l278" event={"ID":"64743302-a5a8-4b6f-b7c7-d574edd19211","Type":"ContainerStarted","Data":"9535fac4d2928db4f2fb9be2172ef4f90705babb5b9b2034fc4209ce68667563"}
Sep 15 20:05:33.297578 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:33.297531    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5l278" podStartSLOduration=2.44446397 podStartE2EDuration="5.2975138s" totalImagesPullingTime="2.85304983s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:05:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:05:33.297424637 +0000 UTC m=+39.688377636" watchObservedRunningTime="2026-09-15 20:05:33.2975138 +0000 UTC m=+39.688466794"
Sep 15 20:05:36.800152 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:36.800119    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5360f750-45a7-42d6-8b65-be039f6ee235-metrics-tls\") pod \"dns-default-h67tx\" (UID: \"5360f750-45a7-42d6-8b65-be039f6ee235\") " pod="openshift-dns/dns-default-h67tx"
Sep 15 20:05:36.800514 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:36.800253    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 20:05:36.800514 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:36.800303    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5360f750-45a7-42d6-8b65-be039f6ee235-metrics-tls podName:5360f750-45a7-42d6-8b65-be039f6ee235 nodeName:}" failed. No retries permitted until 2026-09-15 20:05:44.800290206 +0000 UTC m=+51.191243195 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5360f750-45a7-42d6-8b65-be039f6ee235-metrics-tls") pod "dns-default-h67tx" (UID: "5360f750-45a7-42d6-8b65-be039f6ee235") : secret "dns-default-metrics-tls" not found
Sep 15 20:05:44.853588 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:44.853544    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5360f750-45a7-42d6-8b65-be039f6ee235-metrics-tls\") pod \"dns-default-h67tx\" (UID: \"5360f750-45a7-42d6-8b65-be039f6ee235\") " pod="openshift-dns/dns-default-h67tx"
Sep 15 20:05:44.854021 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:44.853675    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 20:05:44.854021 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:44.853742    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5360f750-45a7-42d6-8b65-be039f6ee235-metrics-tls podName:5360f750-45a7-42d6-8b65-be039f6ee235 nodeName:}" failed. No retries permitted until 2026-09-15 20:06:00.85371587 +0000 UTC m=+67.244668844 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5360f750-45a7-42d6-8b65-be039f6ee235-metrics-tls") pod "dns-default-h67tx" (UID: "5360f750-45a7-42d6-8b65-be039f6ee235") : secret "dns-default-metrics-tls" not found
Sep 15 20:05:53.255689 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:53.255660    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8n6nm"
Sep 15 20:05:59.861312 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:59.861267    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5482d254-b248-4eda-a7d8-37bcb3289d69-metrics-certs\") pod \"network-metrics-daemon-2v4rt\" (UID: \"5482d254-b248-4eda-a7d8-37bcb3289d69\") " pod="openshift-multus/network-metrics-daemon-2v4rt"
Sep 15 20:05:59.861312 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:59.861314    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e124866-fa5b-4ecb-bca9-075fa8bdb5a5-original-pull-secret\") pod \"global-pull-secret-syncer-9dwjh\" (UID: \"7e124866-fa5b-4ecb-bca9-075fa8bdb5a5\") " pod="kube-system/global-pull-secret-syncer-9dwjh"
Sep 15 20:05:59.864136 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:59.864118    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 15 20:05:59.864196 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:59.864149    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 15 20:05:59.872198 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:59.872171    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 15 20:05:59.872299 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:05:59.872265    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5482d254-b248-4eda-a7d8-37bcb3289d69-metrics-certs podName:5482d254-b248-4eda-a7d8-37bcb3289d69 nodeName:}" failed. No retries permitted until 2026-09-15 20:07:03.872241005 +0000 UTC m=+130.263193998 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5482d254-b248-4eda-a7d8-37bcb3289d69-metrics-certs") pod "network-metrics-daemon-2v4rt" (UID: "5482d254-b248-4eda-a7d8-37bcb3289d69") : secret "metrics-daemon-secret" not found
Sep 15 20:05:59.875106 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:59.875080    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e124866-fa5b-4ecb-bca9-075fa8bdb5a5-original-pull-secret\") pod \"global-pull-secret-syncer-9dwjh\" (UID: \"7e124866-fa5b-4ecb-bca9-075fa8bdb5a5\") " pod="kube-system/global-pull-secret-syncer-9dwjh"
Sep 15 20:05:59.961954 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:59.961910    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bnj8\" (UniqueName: \"kubernetes.io/projected/a7a98707-ce3f-4f6c-b723-a19113d90c41-kube-api-access-4bnj8\") pod \"network-check-target-jqbq7\" (UID: \"a7a98707-ce3f-4f6c-b723-a19113d90c41\") " pod="openshift-network-diagnostics/network-check-target-jqbq7"
Sep 15 20:05:59.964846 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:59.964827    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 15 20:05:59.975535 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:59.975514    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 15 20:05:59.986218 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:05:59.986186    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bnj8\" (UniqueName: \"kubernetes.io/projected/a7a98707-ce3f-4f6c-b723-a19113d90c41-kube-api-access-4bnj8\") pod \"network-check-target-jqbq7\" (UID: \"a7a98707-ce3f-4f6c-b723-a19113d90c41\") " pod="openshift-network-diagnostics/network-check-target-jqbq7"
Sep 15 20:06:00.063941 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:06:00.063908    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9dwjh"
Sep 15 20:06:00.078545 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:06:00.078521    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fxg67\""
Sep 15 20:06:00.085013 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:06:00.084993    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqbq7"
Sep 15 20:06:00.355140 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:06:00.355106    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9dwjh"]
Sep 15 20:06:00.358385 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:06:00.358359    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e124866_fa5b_4ecb_bca9_075fa8bdb5a5.slice/crio-227901db2ccf2533d9a129b6b301e2736fce32461f20897410f27a2bb9850cbf WatchSource:0}: Error finding container 227901db2ccf2533d9a129b6b301e2736fce32461f20897410f27a2bb9850cbf: Status 404 returned error can't find the container with id 227901db2ccf2533d9a129b6b301e2736fce32461f20897410f27a2bb9850cbf
Sep 15 20:06:00.383701 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:06:00.383619    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jqbq7"]
Sep 15 20:06:00.386975 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:06:00.386947    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7a98707_ce3f_4f6c_b723_a19113d90c41.slice/crio-596a6c5cf43f521ec8360654af3a3a1ab86676aebce09382504fa42f12b6218f WatchSource:0}: Error finding container 596a6c5cf43f521ec8360654af3a3a1ab86676aebce09382504fa42f12b6218f: Status 404 returned error can't find the container with id 596a6c5cf43f521ec8360654af3a3a1ab86676aebce09382504fa42f12b6218f
Sep 15 20:06:00.868426 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:06:00.868384    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5360f750-45a7-42d6-8b65-be039f6ee235-metrics-tls\") pod \"dns-default-h67tx\" (UID: \"5360f750-45a7-42d6-8b65-be039f6ee235\") " pod="openshift-dns/dns-default-h67tx"
Sep 15 20:06:00.868989 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:06:00.868560    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 20:06:00.868989 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:06:00.868634    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5360f750-45a7-42d6-8b65-be039f6ee235-metrics-tls podName:5360f750-45a7-42d6-8b65-be039f6ee235 nodeName:}" failed. No retries permitted until 2026-09-15 20:06:32.868614801 +0000 UTC m=+99.259567793 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5360f750-45a7-42d6-8b65-be039f6ee235-metrics-tls") pod "dns-default-h67tx" (UID: "5360f750-45a7-42d6-8b65-be039f6ee235") : secret "dns-default-metrics-tls" not found
Sep 15 20:06:01.349156 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:06:01.349113    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jqbq7" event={"ID":"a7a98707-ce3f-4f6c-b723-a19113d90c41","Type":"ContainerStarted","Data":"596a6c5cf43f521ec8360654af3a3a1ab86676aebce09382504fa42f12b6218f"}
Sep 15 20:06:01.350607 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:06:01.350488    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9dwjh" event={"ID":"7e124866-fa5b-4ecb-bca9-075fa8bdb5a5","Type":"ContainerStarted","Data":"227901db2ccf2533d9a129b6b301e2736fce32461f20897410f27a2bb9850cbf"}
Sep 15 20:06:05.364117 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:06:05.364063    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jqbq7" event={"ID":"a7a98707-ce3f-4f6c-b723-a19113d90c41","Type":"ContainerStarted","Data":"5653df4dfd0d0f60c9b0e10ad5c90f5022e5b46cd02cba8b5b8fbb05743646ed"}
Sep 15 20:06:05.364547 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:06:05.364130    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jqbq7"
Sep 15 20:06:05.365620 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:06:05.365596    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9dwjh" event={"ID":"7e124866-fa5b-4ecb-bca9-075fa8bdb5a5","Type":"ContainerStarted","Data":"f65febbedab72b11b80f1b65882a17bc8eaa65b1c4832d2d06146971256404f5"}
Sep 15 20:06:05.379373 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:06:05.379301    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jqbq7" podStartSLOduration=66.590874571 podStartE2EDuration="1m11.379284342s" totalImagesPullingTime="4.788409771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:04:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:06:05.379184589 +0000 UTC m=+71.770137586" watchObservedRunningTime="2026-09-15 20:06:05.379284342 +0000 UTC m=+71.770237340"
Sep 15 20:06:05.393305 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:06:05.393267    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-9dwjh" podStartSLOduration=66.572236367 podStartE2EDuration="1m11.393254763s" totalImagesPullingTime="4.821018396s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:04:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:06:05.392842645 +0000 UTC m=+71.783795641" watchObservedRunningTime="2026-09-15 20:06:05.393254763 +0000 UTC m=+71.784207759"
Sep 15 20:06:13.111242 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:06:13.111203    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d97c74fd-8c7p7"]
Sep 15 20:06:13.113113 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:06:13.113096    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d97c74fd-8c7p7"
Sep 15 20:06:13.115710 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:06:13.115682    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 15 20:06:13.115855 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:06:13.115682    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 15 20:06:13.117091 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:06:13.117070    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-cq57r\""
Sep 15 20:06:13.117192 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:06:13.117140    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 15 20:06:13.117192 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:06:13.117164    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 15 20:06:13.122173 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:06:13.122144    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d97c74fd-8c7p7"]
Sep 15 20:06:13.155324 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:06:13.155297    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9e57ed9a-14c2-48f4-a961-8494406a4d27-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-86d97c74fd-8c7p7\" (UID: \"9e57ed9a-14c2-48f4-a961-8494406a4d27\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d97c74fd-8c7p7"
Sep 15 20:06:13.155434 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:06:13.155346    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t28wg\" (UniqueName: \"kubernetes.io/projected/9e57ed9a-14c2-48f4-a961-8494406a4d27-kube-api-access-t28wg\") pod \"managed-serviceaccount-addon-agent-86d97c74fd-8c7p7\" (UID: \"9e57ed9a-14c2-48f4-a961-8494406a4d27\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d97c74fd-8c7p7"
Sep 15 20:06:13.256664 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:06:13.256625    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9e57ed9a-14c2-48f4-a961-8494406a4d27-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-86d97c74fd-8c7p7\" (UID: \"9e57ed9a-14c2-48f4-a961-8494406a4d27\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d97c74fd-8c7p7"
Sep 15 20:06:13.256838 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:06:13.256697    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t28wg\" (UniqueName: \"kubernetes.io/projected/9e57ed9a-14c2-48f4-a961-8494406a4d27-kube-api-access-t28wg\") pod \"managed-serviceaccount-addon-agent-86d97c74fd-8c7p7\" (UID: \"9e57ed9a-14c2-48f4-a961-8494406a4d27\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d97c74fd-8c7p7"
Sep 15 20:06:13.259119 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:06:13.259101    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9e57ed9a-14c2-48f4-a961-8494406a4d27-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-86d97c74fd-8c7p7\" (UID: \"9e57ed9a-14c2-48f4-a961-8494406a4d27\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d97c74fd-8c7p7"
Sep 15 20:06:13.267011 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:06:13.266991    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t28wg\" (UniqueName: \"kubernetes.io/projected/9e57ed9a-14c2-48f4-a961-8494406a4d27-kube-api-access-t28wg\") pod \"managed-serviceaccount-addon-agent-86d97c74fd-8c7p7\" (UID: \"9e57ed9a-14c2-48f4-a961-8494406a4d27\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d97c74fd-8c7p7"
Sep 15 20:06:13.433872 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:06:13.433782    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d97c74fd-8c7p7"
Sep 15 20:06:13.700497 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:06:13.700411    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d97c74fd-8c7p7"]
Sep 15 20:06:13.704257 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:06:13.704226    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9e57ed9a_14c2_48f4_a961_8494406a4d27.slice/crio-f4b927ba59f1a8bdcfd6f5e4891f524414867771f9ceb63e378bc9efdd0856b3 WatchSource:0}: Error finding container f4b927ba59f1a8bdcfd6f5e4891f524414867771f9ceb63e378bc9efdd0856b3: Status 404 returned error can't find the container with id f4b927ba59f1a8bdcfd6f5e4891f524414867771f9ceb63e378bc9efdd0856b3
Sep 15 20:06:14.389055 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:06:14.389020    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d97c74fd-8c7p7" event={"ID":"9e57ed9a-14c2-48f4-a961-8494406a4d27","Type":"ContainerStarted","Data":"f4b927ba59f1a8bdcfd6f5e4891f524414867771f9ceb63e378bc9efdd0856b3"}
Sep 15 20:06:16.395765 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:06:16.395714    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d97c74fd-8c7p7" event={"ID":"9e57ed9a-14c2-48f4-a961-8494406a4d27","Type":"ContainerStarted","Data":"154cb089668fb316a0058effeeb3fa22f14bc13d4e63871b929df38aa551acfc"}
Sep 15 20:06:16.414167 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:06:16.414067    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d97c74fd-8c7p7" podStartSLOduration=0.961008403 podStartE2EDuration="3.414050299s" totalImagesPullingTime="2.453041896s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:06:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:06:16.413251108 +0000 UTC m=+82.804204105" watchObservedRunningTime="2026-09-15 20:06:16.414050299 +0000 UTC m=+82.805003346"
Sep 15 20:06:32.882441 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:06:32.882396    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5360f750-45a7-42d6-8b65-be039f6ee235-metrics-tls\") pod \"dns-default-h67tx\" (UID: \"5360f750-45a7-42d6-8b65-be039f6ee235\") " pod="openshift-dns/dns-default-h67tx"
Sep 15 20:06:32.882860 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:06:32.882514    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 20:06:32.882860 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:06:32.882577    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5360f750-45a7-42d6-8b65-be039f6ee235-metrics-tls podName:5360f750-45a7-42d6-8b65-be039f6ee235 nodeName:}" failed. No retries permitted until 2026-09-15 20:07:36.882564648 +0000 UTC m=+163.273517636 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5360f750-45a7-42d6-8b65-be039f6ee235-metrics-tls") pod "dns-default-h67tx" (UID: "5360f750-45a7-42d6-8b65-be039f6ee235") : secret "dns-default-metrics-tls" not found
Sep 15 20:06:36.370777 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:06:36.370743    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jqbq7"
Sep 15 20:07:03.887486 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:07:03.887448    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5482d254-b248-4eda-a7d8-37bcb3289d69-metrics-certs\") pod \"network-metrics-daemon-2v4rt\" (UID: \"5482d254-b248-4eda-a7d8-37bcb3289d69\") " pod="openshift-multus/network-metrics-daemon-2v4rt"
Sep 15 20:07:03.887901 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:07:03.887577    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 15 20:07:03.887901 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:07:03.887651    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5482d254-b248-4eda-a7d8-37bcb3289d69-metrics-certs podName:5482d254-b248-4eda-a7d8-37bcb3289d69 nodeName:}" failed. No retries permitted until 2026-09-15 20:09:05.887633382 +0000 UTC m=+252.278586358 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5482d254-b248-4eda-a7d8-37bcb3289d69-metrics-certs") pod "network-metrics-daemon-2v4rt" (UID: "5482d254-b248-4eda-a7d8-37bcb3289d69") : secret "metrics-daemon-secret" not found
Sep 15 20:07:32.012192 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:07:32.012156    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-h67tx" podUID="5360f750-45a7-42d6-8b65-be039f6ee235"
Sep 15 20:07:32.575624 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:07:32.575590    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-h67tx"
Sep 15 20:07:33.069305 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:07:33.069267    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-2v4rt" podUID="5482d254-b248-4eda-a7d8-37bcb3289d69"
Sep 15 20:07:36.915346 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:07:36.915300    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5360f750-45a7-42d6-8b65-be039f6ee235-metrics-tls\") pod \"dns-default-h67tx\" (UID: \"5360f750-45a7-42d6-8b65-be039f6ee235\") " pod="openshift-dns/dns-default-h67tx"
Sep 15 20:07:36.915783 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:07:36.915443    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 20:07:36.915783 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:07:36.915508    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5360f750-45a7-42d6-8b65-be039f6ee235-metrics-tls podName:5360f750-45a7-42d6-8b65-be039f6ee235 nodeName:}" failed. No retries permitted until 2026-09-15 20:09:38.915492499 +0000 UTC m=+285.306445488 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5360f750-45a7-42d6-8b65-be039f6ee235-metrics-tls") pod "dns-default-h67tx" (UID: "5360f750-45a7-42d6-8b65-be039f6ee235") : secret "dns-default-metrics-tls" not found
Sep 15 20:07:46.054551 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:07:46.054511    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v4rt"
Sep 15 20:08:16.686596 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:16.686509    2577 generic.go:356] "Generic (PLEG): container finished" podID="9e57ed9a-14c2-48f4-a961-8494406a4d27" containerID="154cb089668fb316a0058effeeb3fa22f14bc13d4e63871b929df38aa551acfc" exitCode=255
Sep 15 20:08:16.686596 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:16.686583    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d97c74fd-8c7p7" event={"ID":"9e57ed9a-14c2-48f4-a961-8494406a4d27","Type":"ContainerDied","Data":"154cb089668fb316a0058effeeb3fa22f14bc13d4e63871b929df38aa551acfc"}
Sep 15 20:08:16.692159 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:16.692143    2577 scope.go:117] "RemoveContainer" containerID="154cb089668fb316a0058effeeb3fa22f14bc13d4e63871b929df38aa551acfc"
Sep 15 20:08:17.690860 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:17.690827    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d97c74fd-8c7p7" event={"ID":"9e57ed9a-14c2-48f4-a961-8494406a4d27","Type":"ContainerStarted","Data":"91edd0f142cc8678620b84e814389c75ce0d55eb5378e675e2fb8ba573bf417c"}
Sep 15 20:08:52.112611 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.112575    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-76d959cf57-r9gz6"]
Sep 15 20:08:52.115402 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.115385    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76d959cf57-r9gz6"
Sep 15 20:08:52.120004 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.119981    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 15 20:08:52.121666 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.121345    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 15 20:08:52.121666 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.121628    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-g82mz\""
Sep 15 20:08:52.121909 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.121894    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 15 20:08:52.124845 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.124830    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 15 20:08:52.178496 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.178464    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76d959cf57-r9gz6"]
Sep 15 20:08:52.196735 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.196682    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-zjtlc"]
Sep 15 20:08:52.199835 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.199814    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xsqd9"]
Sep 15 20:08:52.199946 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.199927    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-zjtlc"
Sep 15 20:08:52.202620 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.202601    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-x5v6t"]
Sep 15 20:08:52.202746 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.202716    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xsqd9"
Sep 15 20:08:52.204374 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.204352    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 15 20:08:52.204484 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.204361    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-zhcbm\""
Sep 15 20:08:52.204484 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.204475    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 15 20:08:52.205423 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.205407    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-r227b"]
Sep 15 20:08:52.205511 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.205498    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-x5v6t"
Sep 15 20:08:52.206171 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.206155    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 15 20:08:52.207702 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.207680    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 15 20:08:52.207702 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.207695    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 15 20:08:52.207864 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.207700    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 15 20:08:52.207864 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.207746    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-r7vpc\""
Sep 15 20:08:52.207971 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.207956    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-r227b"
Sep 15 20:08:52.214645 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.214626    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-x5v6t"]
Sep 15 20:08:52.215028 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.215008    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 15 20:08:52.215112 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.215014    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 15 20:08:52.215216 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.215198    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/45f245cb-4670-40a8-9da8-252c030c8c8e-trusted-ca\") pod \"image-registry-76d959cf57-r9gz6\" (UID: \"45f245cb-4670-40a8-9da8-252c030c8c8e\") " pod="openshift-image-registry/image-registry-76d959cf57-r9gz6"
Sep 15 20:08:52.215279 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.215222    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/45f245cb-4670-40a8-9da8-252c030c8c8e-bound-sa-token\") pod \"image-registry-76d959cf57-r9gz6\" (UID: \"45f245cb-4670-40a8-9da8-252c030c8c8e\") " pod="openshift-image-registry/image-registry-76d959cf57-r9gz6"
Sep 15 20:08:52.215279 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.215250    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/45f245cb-4670-40a8-9da8-252c030c8c8e-image-registry-private-configuration\") pod \"image-registry-76d959cf57-r9gz6\" (UID: \"45f245cb-4670-40a8-9da8-252c030c8c8e\") " pod="openshift-image-registry/image-registry-76d959cf57-r9gz6"
Sep 15 20:08:52.215279 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.215268    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/45f245cb-4670-40a8-9da8-252c030c8c8e-installation-pull-secrets\") pod \"image-registry-76d959cf57-r9gz6\" (UID: \"45f245cb-4670-40a8-9da8-252c030c8c8e\") " pod="openshift-image-registry/image-registry-76d959cf57-r9gz6"
Sep 15 20:08:52.215445 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.215325    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/45f245cb-4670-40a8-9da8-252c030c8c8e-registry-tls\") pod \"image-registry-76d959cf57-r9gz6\" (UID: \"45f245cb-4670-40a8-9da8-252c030c8c8e\") " pod="openshift-image-registry/image-registry-76d959cf57-r9gz6"
Sep 15 20:08:52.215445 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.215328    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-lk9zm\""
Sep 15 20:08:52.215445 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.215364    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/45f245cb-4670-40a8-9da8-252c030c8c8e-registry-certificates\") pod \"image-registry-76d959cf57-r9gz6\" (UID: \"45f245cb-4670-40a8-9da8-252c030c8c8e\") " pod="openshift-image-registry/image-registry-76d959cf57-r9gz6"
Sep 15 20:08:52.215445 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.215399    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/45f245cb-4670-40a8-9da8-252c030c8c8e-ca-trust-extracted\") pod \"image-registry-76d959cf57-r9gz6\" (UID: \"45f245cb-4670-40a8-9da8-252c030c8c8e\") " pod="openshift-image-registry/image-registry-76d959cf57-r9gz6"
Sep 15 20:08:52.215445 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.215403    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 15 20:08:52.215445 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.215425    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfjnn\" (UniqueName: \"kubernetes.io/projected/45f245cb-4670-40a8-9da8-252c030c8c8e-kube-api-access-mfjnn\") pod \"image-registry-76d959cf57-r9gz6\" (UID: \"45f245cb-4670-40a8-9da8-252c030c8c8e\") " pod="openshift-image-registry/image-registry-76d959cf57-r9gz6"
Sep 15 20:08:52.215814 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.215795    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-jf5nd\""
Sep 15 20:08:52.215910 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.215893    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 15 20:08:52.219552 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.219531    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-zjtlc"]
Sep 15 20:08:52.220466 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.220446    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xsqd9"]
Sep 15 20:08:52.230364 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.230342    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 15 20:08:52.236574 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.236551    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-r227b"]
Sep 15 20:08:52.316826 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.316785    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/45f245cb-4670-40a8-9da8-252c030c8c8e-trusted-ca\") pod \"image-registry-76d959cf57-r9gz6\" (UID: \"45f245cb-4670-40a8-9da8-252c030c8c8e\") " pod="openshift-image-registry/image-registry-76d959cf57-r9gz6"
Sep 15 20:08:52.316826 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.316820    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/45f245cb-4670-40a8-9da8-252c030c8c8e-bound-sa-token\") pod \"image-registry-76d959cf57-r9gz6\" (UID: \"45f245cb-4670-40a8-9da8-252c030c8c8e\") " pod="openshift-image-registry/image-registry-76d959cf57-r9gz6"
Sep 15 20:08:52.317025 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.316839    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ecab5774-79a4-42eb-9bb5-a1f09b18f3fb-trusted-ca\") pod \"console-operator-696884fb9c-r227b\" (UID: \"ecab5774-79a4-42eb-9bb5-a1f09b18f3fb\") " pod="openshift-console-operator/console-operator-696884fb9c-r227b"
Sep 15 20:08:52.317025 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.316956    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-844lb\" (UniqueName: \"kubernetes.io/projected/b5e2066b-9993-450a-9a18-5bdb8125733e-kube-api-access-844lb\") pod \"network-check-source-6f8b768455-x5v6t\" (UID: \"b5e2066b-9993-450a-9a18-5bdb8125733e\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-x5v6t"
Sep 15 20:08:52.317097 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.317026    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/45f245cb-4670-40a8-9da8-252c030c8c8e-image-registry-private-configuration\") pod \"image-registry-76d959cf57-r9gz6\" (UID: \"45f245cb-4670-40a8-9da8-252c030c8c8e\") " pod="openshift-image-registry/image-registry-76d959cf57-r9gz6"
Sep 15 20:08:52.317097 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.317049    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/45f245cb-4670-40a8-9da8-252c030c8c8e-installation-pull-secrets\") pod \"image-registry-76d959cf57-r9gz6\" (UID: \"45f245cb-4670-40a8-9da8-252c030c8c8e\") " pod="openshift-image-registry/image-registry-76d959cf57-r9gz6"
Sep 15 20:08:52.317097 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.317066    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ecab5774-79a4-42eb-9bb5-a1f09b18f3fb-serving-cert\") pod \"console-operator-696884fb9c-r227b\" (UID: \"ecab5774-79a4-42eb-9bb5-a1f09b18f3fb\") " pod="openshift-console-operator/console-operator-696884fb9c-r227b"
Sep 15 20:08:52.317214 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.317098    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/45f245cb-4670-40a8-9da8-252c030c8c8e-registry-tls\") pod \"image-registry-76d959cf57-r9gz6\" (UID: \"45f245cb-4670-40a8-9da8-252c030c8c8e\") " pod="openshift-image-registry/image-registry-76d959cf57-r9gz6"
Sep 15 20:08:52.317214 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.317115    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/45f245cb-4670-40a8-9da8-252c030c8c8e-registry-certificates\") pod \"image-registry-76d959cf57-r9gz6\" (UID: \"45f245cb-4670-40a8-9da8-252c030c8c8e\") " pod="openshift-image-registry/image-registry-76d959cf57-r9gz6"
Sep 15 20:08:52.317214 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.317130    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ecab5774-79a4-42eb-9bb5-a1f09b18f3fb-config\") pod \"console-operator-696884fb9c-r227b\" (UID: \"ecab5774-79a4-42eb-9bb5-a1f09b18f3fb\") " pod="openshift-console-operator/console-operator-696884fb9c-r227b"
Sep 15 20:08:52.317214 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.317148    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c567a4c3-d608-46e7-b93a-88ad3486e709-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-xsqd9\" (UID: \"c567a4c3-d608-46e7-b93a-88ad3486e709\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xsqd9"
Sep 15 20:08:52.317214 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.317179    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/45f245cb-4670-40a8-9da8-252c030c8c8e-ca-trust-extracted\") pod \"image-registry-76d959cf57-r9gz6\" (UID: \"45f245cb-4670-40a8-9da8-252c030c8c8e\") " pod="openshift-image-registry/image-registry-76d959cf57-r9gz6"
Sep 15 20:08:52.317443 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:08:52.317267    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 15 20:08:52.317443 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:08:52.317286    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76d959cf57-r9gz6: secret "image-registry-tls" not found
Sep 15 20:08:52.317443 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.317306    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfjnn\" (UniqueName: \"kubernetes.io/projected/45f245cb-4670-40a8-9da8-252c030c8c8e-kube-api-access-mfjnn\") pod \"image-registry-76d959cf57-r9gz6\" (UID: \"45f245cb-4670-40a8-9da8-252c030c8c8e\") " pod="openshift-image-registry/image-registry-76d959cf57-r9gz6"
Sep 15 20:08:52.317443 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:08:52.317372    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/45f245cb-4670-40a8-9da8-252c030c8c8e-registry-tls podName:45f245cb-4670-40a8-9da8-252c030c8c8e nodeName:}" failed. No retries permitted until 2026-09-15 20:08:52.817352192 +0000 UTC m=+239.208305183 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/45f245cb-4670-40a8-9da8-252c030c8c8e-registry-tls") pod "image-registry-76d959cf57-r9gz6" (UID: "45f245cb-4670-40a8-9da8-252c030c8c8e") : secret "image-registry-tls" not found
Sep 15 20:08:52.317443 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.317432    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c567a4c3-d608-46e7-b93a-88ad3486e709-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-xsqd9\" (UID: \"c567a4c3-d608-46e7-b93a-88ad3486e709\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xsqd9"
Sep 15 20:08:52.317702 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.317468    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xn2rz\" (UniqueName: \"kubernetes.io/projected/ecab5774-79a4-42eb-9bb5-a1f09b18f3fb-kube-api-access-xn2rz\") pod \"console-operator-696884fb9c-r227b\" (UID: \"ecab5774-79a4-42eb-9bb5-a1f09b18f3fb\") " pod="openshift-console-operator/console-operator-696884fb9c-r227b"
Sep 15 20:08:52.317702 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.317502    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4ch2\" (UniqueName: \"kubernetes.io/projected/920cddf0-219e-4b86-84cd-b4111619a92b-kube-api-access-b4ch2\") pod \"volume-data-source-validator-86695b7cc7-zjtlc\" (UID: \"920cddf0-219e-4b86-84cd-b4111619a92b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-zjtlc"
Sep 15 20:08:52.317702 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.317541    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhwgl\" (UniqueName: \"kubernetes.io/projected/c567a4c3-d608-46e7-b93a-88ad3486e709-kube-api-access-zhwgl\") pod \"kube-storage-version-migrator-operator-6c8f84f897-xsqd9\" (UID: \"c567a4c3-d608-46e7-b93a-88ad3486e709\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xsqd9"
Sep 15 20:08:52.317702 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.317574    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/45f245cb-4670-40a8-9da8-252c030c8c8e-ca-trust-extracted\") pod \"image-registry-76d959cf57-r9gz6\" (UID: \"45f245cb-4670-40a8-9da8-252c030c8c8e\") " pod="openshift-image-registry/image-registry-76d959cf57-r9gz6"
Sep 15 20:08:52.317903 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.317758    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/45f245cb-4670-40a8-9da8-252c030c8c8e-registry-certificates\") pod \"image-registry-76d959cf57-r9gz6\" (UID: \"45f245cb-4670-40a8-9da8-252c030c8c8e\") " pod="openshift-image-registry/image-registry-76d959cf57-r9gz6"
Sep 15 20:08:52.317903 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.317788    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/45f245cb-4670-40a8-9da8-252c030c8c8e-trusted-ca\") pod \"image-registry-76d959cf57-r9gz6\" (UID: \"45f245cb-4670-40a8-9da8-252c030c8c8e\") " pod="openshift-image-registry/image-registry-76d959cf57-r9gz6"
Sep 15 20:08:52.320148 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.320122    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/45f245cb-4670-40a8-9da8-252c030c8c8e-installation-pull-secrets\") pod \"image-registry-76d959cf57-r9gz6\" (UID: \"45f245cb-4670-40a8-9da8-252c030c8c8e\") " pod="openshift-image-registry/image-registry-76d959cf57-r9gz6"
Sep 15 20:08:52.320257 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.320122    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/45f245cb-4670-40a8-9da8-252c030c8c8e-image-registry-private-configuration\") pod \"image-registry-76d959cf57-r9gz6\" (UID: \"45f245cb-4670-40a8-9da8-252c030c8c8e\") " pod="openshift-image-registry/image-registry-76d959cf57-r9gz6"
Sep 15 20:08:52.326151 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.326118    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/45f245cb-4670-40a8-9da8-252c030c8c8e-bound-sa-token\") pod \"image-registry-76d959cf57-r9gz6\" (UID: \"45f245cb-4670-40a8-9da8-252c030c8c8e\") " pod="openshift-image-registry/image-registry-76d959cf57-r9gz6"
Sep 15 20:08:52.326666 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.326650    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfjnn\" (UniqueName: \"kubernetes.io/projected/45f245cb-4670-40a8-9da8-252c030c8c8e-kube-api-access-mfjnn\") pod \"image-registry-76d959cf57-r9gz6\" (UID: \"45f245cb-4670-40a8-9da8-252c030c8c8e\") " pod="openshift-image-registry/image-registry-76d959cf57-r9gz6"
Sep 15 20:08:52.418559 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.418493    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c567a4c3-d608-46e7-b93a-88ad3486e709-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-xsqd9\" (UID: \"c567a4c3-d608-46e7-b93a-88ad3486e709\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xsqd9"
Sep 15 20:08:52.418559 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.418527    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xn2rz\" (UniqueName: \"kubernetes.io/projected/ecab5774-79a4-42eb-9bb5-a1f09b18f3fb-kube-api-access-xn2rz\") pod \"console-operator-696884fb9c-r227b\" (UID: \"ecab5774-79a4-42eb-9bb5-a1f09b18f3fb\") " pod="openshift-console-operator/console-operator-696884fb9c-r227b"
Sep 15 20:08:52.418559 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.418545    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4ch2\" (UniqueName: \"kubernetes.io/projected/920cddf0-219e-4b86-84cd-b4111619a92b-kube-api-access-b4ch2\") pod \"volume-data-source-validator-86695b7cc7-zjtlc\" (UID: \"920cddf0-219e-4b86-84cd-b4111619a92b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-zjtlc"
Sep 15 20:08:52.418559 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.418561    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhwgl\" (UniqueName: \"kubernetes.io/projected/c567a4c3-d608-46e7-b93a-88ad3486e709-kube-api-access-zhwgl\") pod \"kube-storage-version-migrator-operator-6c8f84f897-xsqd9\" (UID: \"c567a4c3-d608-46e7-b93a-88ad3486e709\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xsqd9"
Sep 15 20:08:52.418900 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.418586    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ecab5774-79a4-42eb-9bb5-a1f09b18f3fb-trusted-ca\") pod \"console-operator-696884fb9c-r227b\" (UID: \"ecab5774-79a4-42eb-9bb5-a1f09b18f3fb\") " pod="openshift-console-operator/console-operator-696884fb9c-r227b"
Sep 15 20:08:52.418900 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.418643    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-844lb\" (UniqueName: \"kubernetes.io/projected/b5e2066b-9993-450a-9a18-5bdb8125733e-kube-api-access-844lb\") pod \"network-check-source-6f8b768455-x5v6t\" (UID: \"b5e2066b-9993-450a-9a18-5bdb8125733e\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-x5v6t"
Sep 15 20:08:52.418900 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.418709    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ecab5774-79a4-42eb-9bb5-a1f09b18f3fb-serving-cert\") pod \"console-operator-696884fb9c-r227b\" (UID: \"ecab5774-79a4-42eb-9bb5-a1f09b18f3fb\") " pod="openshift-console-operator/console-operator-696884fb9c-r227b"
Sep 15 20:08:52.418900 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.418797    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ecab5774-79a4-42eb-9bb5-a1f09b18f3fb-config\") pod \"console-operator-696884fb9c-r227b\" (UID: \"ecab5774-79a4-42eb-9bb5-a1f09b18f3fb\") " pod="openshift-console-operator/console-operator-696884fb9c-r227b"
Sep 15 20:08:52.419096 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.418910    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c567a4c3-d608-46e7-b93a-88ad3486e709-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-xsqd9\" (UID: \"c567a4c3-d608-46e7-b93a-88ad3486e709\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xsqd9"
Sep 15 20:08:52.419219 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.419194    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c567a4c3-d608-46e7-b93a-88ad3486e709-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-xsqd9\" (UID: \"c567a4c3-d608-46e7-b93a-88ad3486e709\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xsqd9"
Sep 15 20:08:52.419702 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.419676    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ecab5774-79a4-42eb-9bb5-a1f09b18f3fb-config\") pod \"console-operator-696884fb9c-r227b\" (UID: \"ecab5774-79a4-42eb-9bb5-a1f09b18f3fb\") " pod="openshift-console-operator/console-operator-696884fb9c-r227b"
Sep 15 20:08:52.419999 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.419983    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ecab5774-79a4-42eb-9bb5-a1f09b18f3fb-trusted-ca\") pod \"console-operator-696884fb9c-r227b\" (UID: \"ecab5774-79a4-42eb-9bb5-a1f09b18f3fb\") " pod="openshift-console-operator/console-operator-696884fb9c-r227b"
Sep 15 20:08:52.421075 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.421052    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c567a4c3-d608-46e7-b93a-88ad3486e709-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-xsqd9\" (UID: \"c567a4c3-d608-46e7-b93a-88ad3486e709\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xsqd9"
Sep 15 20:08:52.421161 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.421076    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ecab5774-79a4-42eb-9bb5-a1f09b18f3fb-serving-cert\") pod \"console-operator-696884fb9c-r227b\" (UID: \"ecab5774-79a4-42eb-9bb5-a1f09b18f3fb\") " pod="openshift-console-operator/console-operator-696884fb9c-r227b"
Sep 15 20:08:52.428012 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.427986    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4ch2\" (UniqueName: \"kubernetes.io/projected/920cddf0-219e-4b86-84cd-b4111619a92b-kube-api-access-b4ch2\") pod \"volume-data-source-validator-86695b7cc7-zjtlc\" (UID: \"920cddf0-219e-4b86-84cd-b4111619a92b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-zjtlc"
Sep 15 20:08:52.428012 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.427999    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-844lb\" (UniqueName: \"kubernetes.io/projected/b5e2066b-9993-450a-9a18-5bdb8125733e-kube-api-access-844lb\") pod \"network-check-source-6f8b768455-x5v6t\" (UID: \"b5e2066b-9993-450a-9a18-5bdb8125733e\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-x5v6t"
Sep 15 20:08:52.428770 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.428747    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhwgl\" (UniqueName: \"kubernetes.io/projected/c567a4c3-d608-46e7-b93a-88ad3486e709-kube-api-access-zhwgl\") pod \"kube-storage-version-migrator-operator-6c8f84f897-xsqd9\" (UID: \"c567a4c3-d608-46e7-b93a-88ad3486e709\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xsqd9"
Sep 15 20:08:52.428880 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.428859    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xn2rz\" (UniqueName: \"kubernetes.io/projected/ecab5774-79a4-42eb-9bb5-a1f09b18f3fb-kube-api-access-xn2rz\") pod \"console-operator-696884fb9c-r227b\" (UID: \"ecab5774-79a4-42eb-9bb5-a1f09b18f3fb\") " pod="openshift-console-operator/console-operator-696884fb9c-r227b"
Sep 15 20:08:52.510482 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.510451    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-zjtlc"
Sep 15 20:08:52.516797 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.516779    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xsqd9"
Sep 15 20:08:52.523413 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.523391    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-x5v6t"
Sep 15 20:08:52.531169 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.531144    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-r227b"
Sep 15 20:08:52.791022 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.790993    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-zjtlc"]
Sep 15 20:08:52.793865 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.793836    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xsqd9"]
Sep 15 20:08:52.794311 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:08:52.794214    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod920cddf0_219e_4b86_84cd_b4111619a92b.slice/crio-7f055a69b14d13bb0989b5056d16e1ec8e58b5dcfdda8a7090166b44b92b08f9 WatchSource:0}: Error finding container 7f055a69b14d13bb0989b5056d16e1ec8e58b5dcfdda8a7090166b44b92b08f9: Status 404 returned error can't find the container with id 7f055a69b14d13bb0989b5056d16e1ec8e58b5dcfdda8a7090166b44b92b08f9
Sep 15 20:08:52.796710 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:08:52.796683    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc567a4c3_d608_46e7_b93a_88ad3486e709.slice/crio-66779e0ed92e35727bac4303babc915fe619cf64633d81ab233b7542f9c99cdb WatchSource:0}: Error finding container 66779e0ed92e35727bac4303babc915fe619cf64633d81ab233b7542f9c99cdb: Status 404 returned error can't find the container with id 66779e0ed92e35727bac4303babc915fe619cf64633d81ab233b7542f9c99cdb
Sep 15 20:08:52.796862 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.796843    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-x5v6t"]
Sep 15 20:08:52.801478 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:08:52.801456    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5e2066b_9993_450a_9a18_5bdb8125733e.slice/crio-4de7baf1cc6e2f9583c4b2c69379d79600b54d63299a9848bf4c0b63f1de5970 WatchSource:0}: Error finding container 4de7baf1cc6e2f9583c4b2c69379d79600b54d63299a9848bf4c0b63f1de5970: Status 404 returned error can't find the container with id 4de7baf1cc6e2f9583c4b2c69379d79600b54d63299a9848bf4c0b63f1de5970
Sep 15 20:08:52.821022 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.820998    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/45f245cb-4670-40a8-9da8-252c030c8c8e-registry-tls\") pod \"image-registry-76d959cf57-r9gz6\" (UID: \"45f245cb-4670-40a8-9da8-252c030c8c8e\") " pod="openshift-image-registry/image-registry-76d959cf57-r9gz6"
Sep 15 20:08:52.821164 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:08:52.821146    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 15 20:08:52.821212 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:08:52.821165    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76d959cf57-r9gz6: secret "image-registry-tls" not found
Sep 15 20:08:52.821245 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:08:52.821212    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/45f245cb-4670-40a8-9da8-252c030c8c8e-registry-tls podName:45f245cb-4670-40a8-9da8-252c030c8c8e nodeName:}" failed. No retries permitted until 2026-09-15 20:08:53.821196797 +0000 UTC m=+240.212149776 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/45f245cb-4670-40a8-9da8-252c030c8c8e-registry-tls") pod "image-registry-76d959cf57-r9gz6" (UID: "45f245cb-4670-40a8-9da8-252c030c8c8e") : secret "image-registry-tls" not found
Sep 15 20:08:52.822850 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:52.822830    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-r227b"]
Sep 15 20:08:52.828720 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:08:52.828691    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecab5774_79a4_42eb_9bb5_a1f09b18f3fb.slice/crio-3f74b1b625133af304c9ecb969c2e15974d8944e8f6481835c1bb507a157c6ee WatchSource:0}: Error finding container 3f74b1b625133af304c9ecb969c2e15974d8944e8f6481835c1bb507a157c6ee: Status 404 returned error can't find the container with id 3f74b1b625133af304c9ecb969c2e15974d8944e8f6481835c1bb507a157c6ee
Sep 15 20:08:53.782378 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:53.782338    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-zjtlc" event={"ID":"920cddf0-219e-4b86-84cd-b4111619a92b","Type":"ContainerStarted","Data":"7f055a69b14d13bb0989b5056d16e1ec8e58b5dcfdda8a7090166b44b92b08f9"}
Sep 15 20:08:53.783788 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:53.783758    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-r227b" event={"ID":"ecab5774-79a4-42eb-9bb5-a1f09b18f3fb","Type":"ContainerStarted","Data":"3f74b1b625133af304c9ecb969c2e15974d8944e8f6481835c1bb507a157c6ee"}
Sep 15 20:08:53.785881 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:53.785856    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-x5v6t" event={"ID":"b5e2066b-9993-450a-9a18-5bdb8125733e","Type":"ContainerStarted","Data":"5344e6e61e99ba9d5fa938d31bb608bc5a9f6d897d3a9228bc590ddd68bfbd0d"}
Sep 15 20:08:53.786009 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:53.785896    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-x5v6t" event={"ID":"b5e2066b-9993-450a-9a18-5bdb8125733e","Type":"ContainerStarted","Data":"4de7baf1cc6e2f9583c4b2c69379d79600b54d63299a9848bf4c0b63f1de5970"}
Sep 15 20:08:53.787364 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:53.787341    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xsqd9" event={"ID":"c567a4c3-d608-46e7-b93a-88ad3486e709","Type":"ContainerStarted","Data":"66779e0ed92e35727bac4303babc915fe619cf64633d81ab233b7542f9c99cdb"}
Sep 15 20:08:53.828932 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:53.828883    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/45f245cb-4670-40a8-9da8-252c030c8c8e-registry-tls\") pod \"image-registry-76d959cf57-r9gz6\" (UID: \"45f245cb-4670-40a8-9da8-252c030c8c8e\") " pod="openshift-image-registry/image-registry-76d959cf57-r9gz6"
Sep 15 20:08:53.829090 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:08:53.829034    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 15 20:08:53.829090 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:08:53.829051    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76d959cf57-r9gz6: secret "image-registry-tls" not found
Sep 15 20:08:53.829169 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:08:53.829118    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/45f245cb-4670-40a8-9da8-252c030c8c8e-registry-tls podName:45f245cb-4670-40a8-9da8-252c030c8c8e nodeName:}" failed. No retries permitted until 2026-09-15 20:08:55.829095995 +0000 UTC m=+242.220048972 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/45f245cb-4670-40a8-9da8-252c030c8c8e-registry-tls") pod "image-registry-76d959cf57-r9gz6" (UID: "45f245cb-4670-40a8-9da8-252c030c8c8e") : secret "image-registry-tls" not found
Sep 15 20:08:55.798118 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:55.798075    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-r227b" event={"ID":"ecab5774-79a4-42eb-9bb5-a1f09b18f3fb","Type":"ContainerStarted","Data":"7c727c0c286865e9ed494d51c5ba1ab21c66ddbec2af738de800fa254055afab"}
Sep 15 20:08:55.798585 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:55.798479    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-r227b"
Sep 15 20:08:55.799892 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:55.799828    2577 patch_prober.go:28] interesting pod/console-operator-696884fb9c-r227b container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.133.0.13:8443/readyz\": dial tcp 10.133.0.13:8443: connect: connection refused" start-of-body=
Sep 15 20:08:55.800032 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:55.799885    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-696884fb9c-r227b" podUID="ecab5774-79a4-42eb-9bb5-a1f09b18f3fb" containerName="console-operator" probeResult="failure" output="Get \"https://10.133.0.13:8443/readyz\": dial tcp 10.133.0.13:8443: connect: connection refused"
Sep 15 20:08:55.800406 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:55.800333    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xsqd9" event={"ID":"c567a4c3-d608-46e7-b93a-88ad3486e709","Type":"ContainerStarted","Data":"71cefdc030d1bdcb37ed5679b614af29b1254acbdd1f8971cf5721dfb4abb7fe"}
Sep 15 20:08:55.802477 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:55.802445    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-zjtlc" event={"ID":"920cddf0-219e-4b86-84cd-b4111619a92b","Type":"ContainerStarted","Data":"167d7921c9fb82291f11d93cdb681e101a914fbcee96dad2d6faefb31a2af5e4"}
Sep 15 20:08:55.817228 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:55.817159    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-r227b" podStartSLOduration=0.963717025 podStartE2EDuration="3.817139129s" totalImagesPullingTime="2.853422104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:08:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:08:55.81691718 +0000 UTC m=+242.207870180" watchObservedRunningTime="2026-09-15 20:08:55.817139129 +0000 UTC m=+242.208092123"
Sep 15 20:08:55.817546 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:55.817510    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-x5v6t" podStartSLOduration=3.817498504 podStartE2EDuration="3.817498504s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:08:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:08:53.81050142 +0000 UTC m=+240.201454438" watchObservedRunningTime="2026-09-15 20:08:55.817498504 +0000 UTC m=+242.208451502"
Sep 15 20:08:55.835595 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:55.835547    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-zjtlc" podStartSLOduration=0.948134954 podStartE2EDuration="3.835532704s" totalImagesPullingTime="2.88739775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:08:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:08:55.83487939 +0000 UTC m=+242.225832388" watchObservedRunningTime="2026-09-15 20:08:55.835532704 +0000 UTC m=+242.226485700"
Sep 15 20:08:55.845665 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:55.845633    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/45f245cb-4670-40a8-9da8-252c030c8c8e-registry-tls\") pod \"image-registry-76d959cf57-r9gz6\" (UID: \"45f245cb-4670-40a8-9da8-252c030c8c8e\") " pod="openshift-image-registry/image-registry-76d959cf57-r9gz6"
Sep 15 20:08:55.845778 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:08:55.845741    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 15 20:08:55.845778 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:08:55.845755    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76d959cf57-r9gz6: secret "image-registry-tls" not found
Sep 15 20:08:55.845858 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:08:55.845802    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/45f245cb-4670-40a8-9da8-252c030c8c8e-registry-tls podName:45f245cb-4670-40a8-9da8-252c030c8c8e nodeName:}" failed. No retries permitted until 2026-09-15 20:08:59.845787222 +0000 UTC m=+246.236740198 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/45f245cb-4670-40a8-9da8-252c030c8c8e-registry-tls") pod "image-registry-76d959cf57-r9gz6" (UID: "45f245cb-4670-40a8-9da8-252c030c8c8e") : secret "image-registry-tls" not found
Sep 15 20:08:55.868828 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:55.868786    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xsqd9" podStartSLOduration=0.979103694 podStartE2EDuration="3.868771166s" totalImagesPullingTime="2.889667472s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:08:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:08:55.867869402 +0000 UTC m=+242.258822400" watchObservedRunningTime="2026-09-15 20:08:55.868771166 +0000 UTC m=+242.259724184"
Sep 15 20:08:56.136694 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:56.136660    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-qflpq"]
Sep 15 20:08:56.144518 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:56.144484    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qflpq"
Sep 15 20:08:56.147338 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:56.147271    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 15 20:08:56.147338 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:56.147298    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 15 20:08:56.147338 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:56.147300    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 15 20:08:56.147560 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:56.147300    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 15 20:08:56.147979 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:56.147962    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8swwd\""
Sep 15 20:08:56.155637 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:56.155613    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qflpq"]
Sep 15 20:08:56.248276 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:56.248236    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/866ab681-b97a-4958-a622-85f76910920a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qflpq\" (UID: \"866ab681-b97a-4958-a622-85f76910920a\") " pod="openshift-insights/insights-runtime-extractor-qflpq"
Sep 15 20:08:56.248527 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:56.248295    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/866ab681-b97a-4958-a622-85f76910920a-data-volume\") pod \"insights-runtime-extractor-qflpq\" (UID: \"866ab681-b97a-4958-a622-85f76910920a\") " pod="openshift-insights/insights-runtime-extractor-qflpq"
Sep 15 20:08:56.248527 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:56.248317    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/866ab681-b97a-4958-a622-85f76910920a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qflpq\" (UID: \"866ab681-b97a-4958-a622-85f76910920a\") " pod="openshift-insights/insights-runtime-extractor-qflpq"
Sep 15 20:08:56.248527 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:56.248403    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8276d\" (UniqueName: \"kubernetes.io/projected/866ab681-b97a-4958-a622-85f76910920a-kube-api-access-8276d\") pod \"insights-runtime-extractor-qflpq\" (UID: \"866ab681-b97a-4958-a622-85f76910920a\") " pod="openshift-insights/insights-runtime-extractor-qflpq"
Sep 15 20:08:56.248527 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:56.248448    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/866ab681-b97a-4958-a622-85f76910920a-crio-socket\") pod \"insights-runtime-extractor-qflpq\" (UID: \"866ab681-b97a-4958-a622-85f76910920a\") " pod="openshift-insights/insights-runtime-extractor-qflpq"
Sep 15 20:08:56.349542 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:56.349500    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8276d\" (UniqueName: \"kubernetes.io/projected/866ab681-b97a-4958-a622-85f76910920a-kube-api-access-8276d\") pod \"insights-runtime-extractor-qflpq\" (UID: \"866ab681-b97a-4958-a622-85f76910920a\") " pod="openshift-insights/insights-runtime-extractor-qflpq"
Sep 15 20:08:56.349542 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:56.349544    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/866ab681-b97a-4958-a622-85f76910920a-crio-socket\") pod \"insights-runtime-extractor-qflpq\" (UID: \"866ab681-b97a-4958-a622-85f76910920a\") " pod="openshift-insights/insights-runtime-extractor-qflpq"
Sep 15 20:08:56.349789 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:56.349591    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/866ab681-b97a-4958-a622-85f76910920a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qflpq\" (UID: \"866ab681-b97a-4958-a622-85f76910920a\") " pod="openshift-insights/insights-runtime-extractor-qflpq"
Sep 15 20:08:56.349789 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:56.349625    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/866ab681-b97a-4958-a622-85f76910920a-data-volume\") pod \"insights-runtime-extractor-qflpq\" (UID: \"866ab681-b97a-4958-a622-85f76910920a\") " pod="openshift-insights/insights-runtime-extractor-qflpq"
Sep 15 20:08:56.349789 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:56.349645    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/866ab681-b97a-4958-a622-85f76910920a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qflpq\" (UID: \"866ab681-b97a-4958-a622-85f76910920a\") " pod="openshift-insights/insights-runtime-extractor-qflpq"
Sep 15 20:08:56.349789 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:56.349708    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/866ab681-b97a-4958-a622-85f76910920a-crio-socket\") pod \"insights-runtime-extractor-qflpq\" (UID: \"866ab681-b97a-4958-a622-85f76910920a\") " pod="openshift-insights/insights-runtime-extractor-qflpq"
Sep 15 20:08:56.349789 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:08:56.349752    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 20:08:56.349789 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:08:56.349762    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 20:08:56.349979 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:08:56.349814    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/866ab681-b97a-4958-a622-85f76910920a-insights-runtime-extractor-tls podName:866ab681-b97a-4958-a622-85f76910920a nodeName:}" failed. No retries permitted until 2026-09-15 20:08:56.849796837 +0000 UTC m=+243.240749816 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/866ab681-b97a-4958-a622-85f76910920a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qflpq" (UID: "866ab681-b97a-4958-a622-85f76910920a") : secret "insights-runtime-extractor-tls" not found
Sep 15 20:08:56.349979 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:08:56.349838    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/866ab681-b97a-4958-a622-85f76910920a-kube-rbac-proxy-cm podName:866ab681-b97a-4958-a622-85f76910920a nodeName:}" failed. No retries permitted until 2026-09-15 20:08:56.849828995 +0000 UTC m=+243.240781973 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/866ab681-b97a-4958-a622-85f76910920a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qflpq" (UID: "866ab681-b97a-4958-a622-85f76910920a") : configmap "kube-rbac-proxy" not found
Sep 15 20:08:56.349979 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:56.349969    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/866ab681-b97a-4958-a622-85f76910920a-data-volume\") pod \"insights-runtime-extractor-qflpq\" (UID: \"866ab681-b97a-4958-a622-85f76910920a\") " pod="openshift-insights/insights-runtime-extractor-qflpq"
Sep 15 20:08:56.359794 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:56.359763    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8276d\" (UniqueName: \"kubernetes.io/projected/866ab681-b97a-4958-a622-85f76910920a-kube-api-access-8276d\") pod \"insights-runtime-extractor-qflpq\" (UID: \"866ab681-b97a-4958-a622-85f76910920a\") " pod="openshift-insights/insights-runtime-extractor-qflpq"
Sep 15 20:08:56.807349 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:56.807323    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-r227b_ecab5774-79a4-42eb-9bb5-a1f09b18f3fb/console-operator/0.log"
Sep 15 20:08:56.807804 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:56.807369    2577 generic.go:356] "Generic (PLEG): container finished" podID="ecab5774-79a4-42eb-9bb5-a1f09b18f3fb" containerID="7c727c0c286865e9ed494d51c5ba1ab21c66ddbec2af738de800fa254055afab" exitCode=255
Sep 15 20:08:56.807804 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:56.807470    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-r227b" event={"ID":"ecab5774-79a4-42eb-9bb5-a1f09b18f3fb","Type":"ContainerDied","Data":"7c727c0c286865e9ed494d51c5ba1ab21c66ddbec2af738de800fa254055afab"}
Sep 15 20:08:56.807804 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:56.807747    2577 scope.go:117] "RemoveContainer" containerID="7c727c0c286865e9ed494d51c5ba1ab21c66ddbec2af738de800fa254055afab"
Sep 15 20:08:56.854614 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:56.854588    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/866ab681-b97a-4958-a622-85f76910920a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qflpq\" (UID: \"866ab681-b97a-4958-a622-85f76910920a\") " pod="openshift-insights/insights-runtime-extractor-qflpq"
Sep 15 20:08:56.854754 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:56.854652    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/866ab681-b97a-4958-a622-85f76910920a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qflpq\" (UID: \"866ab681-b97a-4958-a622-85f76910920a\") " pod="openshift-insights/insights-runtime-extractor-qflpq"
Sep 15 20:08:56.854837 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:08:56.854765    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 20:08:56.854890 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:08:56.854862    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/866ab681-b97a-4958-a622-85f76910920a-kube-rbac-proxy-cm podName:866ab681-b97a-4958-a622-85f76910920a nodeName:}" failed. No retries permitted until 2026-09-15 20:08:57.854839592 +0000 UTC m=+244.245792582 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/866ab681-b97a-4958-a622-85f76910920a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qflpq" (UID: "866ab681-b97a-4958-a622-85f76910920a") : configmap "kube-rbac-proxy" not found
Sep 15 20:08:56.855164 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:08:56.855049    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 20:08:56.855164 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:08:56.855119    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/866ab681-b97a-4958-a622-85f76910920a-insights-runtime-extractor-tls podName:866ab681-b97a-4958-a622-85f76910920a nodeName:}" failed. No retries permitted until 2026-09-15 20:08:57.855099117 +0000 UTC m=+244.246052111 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/866ab681-b97a-4958-a622-85f76910920a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qflpq" (UID: "866ab681-b97a-4958-a622-85f76910920a") : secret "insights-runtime-extractor-tls" not found
Sep 15 20:08:56.975491 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:56.975456    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-56d9f"]
Sep 15 20:08:56.978518 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:56.978499    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-56d9f"
Sep 15 20:08:56.984048 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:56.984029    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 15 20:08:56.984127 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:56.984078    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 15 20:08:56.985009 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:56.984989    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-fklwp\""
Sep 15 20:08:57.002683 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:57.002651    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-56d9f"]
Sep 15 20:08:57.056318 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:57.056281    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skvjx\" (UniqueName: \"kubernetes.io/projected/c2db4be2-3083-4ba0-aecc-e3ce368114ce-kube-api-access-skvjx\") pod \"migrator-8b4d697c5-56d9f\" (UID: \"c2db4be2-3083-4ba0-aecc-e3ce368114ce\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-56d9f"
Sep 15 20:08:57.156799 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:57.156655    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skvjx\" (UniqueName: \"kubernetes.io/projected/c2db4be2-3083-4ba0-aecc-e3ce368114ce-kube-api-access-skvjx\") pod \"migrator-8b4d697c5-56d9f\" (UID: \"c2db4be2-3083-4ba0-aecc-e3ce368114ce\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-56d9f"
Sep 15 20:08:57.166472 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:57.166441    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-skvjx\" (UniqueName: \"kubernetes.io/projected/c2db4be2-3083-4ba0-aecc-e3ce368114ce-kube-api-access-skvjx\") pod \"migrator-8b4d697c5-56d9f\" (UID: \"c2db4be2-3083-4ba0-aecc-e3ce368114ce\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-56d9f"
Sep 15 20:08:57.291106 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:57.291070    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-56d9f"
Sep 15 20:08:57.579886 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:57.579843    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-56d9f"]
Sep 15 20:08:57.581801 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:08:57.581764    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2db4be2_3083_4ba0_aecc_e3ce368114ce.slice/crio-30dd737e47ca40498c431f514916fc8c1e604c093217eceb03cdfdf376501e6d WatchSource:0}: Error finding container 30dd737e47ca40498c431f514916fc8c1e604c093217eceb03cdfdf376501e6d: Status 404 returned error can't find the container with id 30dd737e47ca40498c431f514916fc8c1e604c093217eceb03cdfdf376501e6d
Sep 15 20:08:57.812127 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:57.812098    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-r227b_ecab5774-79a4-42eb-9bb5-a1f09b18f3fb/console-operator/1.log"
Sep 15 20:08:57.812519 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:57.812504    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-r227b_ecab5774-79a4-42eb-9bb5-a1f09b18f3fb/console-operator/0.log"
Sep 15 20:08:57.812566 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:57.812541    2577 generic.go:356] "Generic (PLEG): container finished" podID="ecab5774-79a4-42eb-9bb5-a1f09b18f3fb" containerID="ca89939acd787dffdfa91800e536998c056ff37123128f98405a58bdb3a25076" exitCode=255
Sep 15 20:08:57.812649 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:57.812625    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-r227b" event={"ID":"ecab5774-79a4-42eb-9bb5-a1f09b18f3fb","Type":"ContainerDied","Data":"ca89939acd787dffdfa91800e536998c056ff37123128f98405a58bdb3a25076"}
Sep 15 20:08:57.812714 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:57.812700    2577 scope.go:117] "RemoveContainer" containerID="7c727c0c286865e9ed494d51c5ba1ab21c66ddbec2af738de800fa254055afab"
Sep 15 20:08:57.812933 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:57.812916    2577 scope.go:117] "RemoveContainer" containerID="ca89939acd787dffdfa91800e536998c056ff37123128f98405a58bdb3a25076"
Sep 15 20:08:57.813128 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:08:57.813106    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-r227b_openshift-console-operator(ecab5774-79a4-42eb-9bb5-a1f09b18f3fb)\"" pod="openshift-console-operator/console-operator-696884fb9c-r227b" podUID="ecab5774-79a4-42eb-9bb5-a1f09b18f3fb"
Sep 15 20:08:57.814084 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:57.813828    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-56d9f" event={"ID":"c2db4be2-3083-4ba0-aecc-e3ce368114ce","Type":"ContainerStarted","Data":"30dd737e47ca40498c431f514916fc8c1e604c093217eceb03cdfdf376501e6d"}
Sep 15 20:08:57.861446 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:57.861352    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/866ab681-b97a-4958-a622-85f76910920a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qflpq\" (UID: \"866ab681-b97a-4958-a622-85f76910920a\") " pod="openshift-insights/insights-runtime-extractor-qflpq"
Sep 15 20:08:57.861605 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:08:57.861444    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 20:08:57.861605 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:08:57.861501    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/866ab681-b97a-4958-a622-85f76910920a-kube-rbac-proxy-cm podName:866ab681-b97a-4958-a622-85f76910920a nodeName:}" failed. No retries permitted until 2026-09-15 20:08:59.861484789 +0000 UTC m=+246.252437767 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/866ab681-b97a-4958-a622-85f76910920a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qflpq" (UID: "866ab681-b97a-4958-a622-85f76910920a") : configmap "kube-rbac-proxy" not found
Sep 15 20:08:57.861605 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:57.861558    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/866ab681-b97a-4958-a622-85f76910920a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qflpq\" (UID: \"866ab681-b97a-4958-a622-85f76910920a\") " pod="openshift-insights/insights-runtime-extractor-qflpq"
Sep 15 20:08:57.861798 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:08:57.861642    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 20:08:57.861798 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:08:57.861693    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/866ab681-b97a-4958-a622-85f76910920a-insights-runtime-extractor-tls podName:866ab681-b97a-4958-a622-85f76910920a nodeName:}" failed. No retries permitted until 2026-09-15 20:08:59.861682455 +0000 UTC m=+246.252635431 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/866ab681-b97a-4958-a622-85f76910920a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qflpq" (UID: "866ab681-b97a-4958-a622-85f76910920a") : secret "insights-runtime-extractor-tls" not found
Sep 15 20:08:58.256001 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:58.255967    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-z5wjn_b6f901c3-0f40-46dc-8406-eaee847b1c10/dns-node-resolver/0.log"
Sep 15 20:08:58.819119 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:58.819092    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-r227b_ecab5774-79a4-42eb-9bb5-a1f09b18f3fb/console-operator/1.log"
Sep 15 20:08:58.819473 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:58.819454    2577 scope.go:117] "RemoveContainer" containerID="ca89939acd787dffdfa91800e536998c056ff37123128f98405a58bdb3a25076"
Sep 15 20:08:58.819632 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:08:58.819613    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-r227b_openshift-console-operator(ecab5774-79a4-42eb-9bb5-a1f09b18f3fb)\"" pod="openshift-console-operator/console-operator-696884fb9c-r227b" podUID="ecab5774-79a4-42eb-9bb5-a1f09b18f3fb"
Sep 15 20:08:59.256164 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:59.256141    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vm6lc_81e9d392-e3c5-48f6-b98a-4bcaddda3336/node-ca/0.log"
Sep 15 20:08:59.824884 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:59.824845    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-56d9f" event={"ID":"c2db4be2-3083-4ba0-aecc-e3ce368114ce","Type":"ContainerStarted","Data":"c5798533f5e972b1decd9106de81e0a1c9ff59a16d669051343a19895848d688"}
Sep 15 20:08:59.825354 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:59.824927    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-56d9f" event={"ID":"c2db4be2-3083-4ba0-aecc-e3ce368114ce","Type":"ContainerStarted","Data":"10315fcc611582e8bce1981369b9105a2e1c313d6fe332b37eebf6fefcfe2b76"}
Sep 15 20:08:59.844168 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:59.844116    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-56d9f" podStartSLOduration=2.580384753 podStartE2EDuration="3.844100712s" totalImagesPullingTime="1.263715959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:08:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:08:59.842437088 +0000 UTC m=+246.233390097" watchObservedRunningTime="2026-09-15 20:08:59.844100712 +0000 UTC m=+246.235053708"
Sep 15 20:08:59.861637 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:59.861610    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5l278_64743302-a5a8-4b6f-b7c7-d574edd19211/serve-healthcheck-canary/0.log"
Sep 15 20:08:59.877906 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:59.877880    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/866ab681-b97a-4958-a622-85f76910920a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qflpq\" (UID: \"866ab681-b97a-4958-a622-85f76910920a\") " pod="openshift-insights/insights-runtime-extractor-qflpq"
Sep 15 20:08:59.877989 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:59.877920    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/45f245cb-4670-40a8-9da8-252c030c8c8e-registry-tls\") pod \"image-registry-76d959cf57-r9gz6\" (UID: \"45f245cb-4670-40a8-9da8-252c030c8c8e\") " pod="openshift-image-registry/image-registry-76d959cf57-r9gz6"
Sep 15 20:08:59.877989 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:08:59.877943    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/866ab681-b97a-4958-a622-85f76910920a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qflpq\" (UID: \"866ab681-b97a-4958-a622-85f76910920a\") " pod="openshift-insights/insights-runtime-extractor-qflpq"
Sep 15 20:08:59.878069 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:08:59.878013    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 20:08:59.878069 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:08:59.878031    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 15 20:08:59.878069 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:08:59.878045    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76d959cf57-r9gz6: secret "image-registry-tls" not found
Sep 15 20:08:59.878069 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:08:59.878062    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 20:08:59.878253 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:08:59.878089    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/866ab681-b97a-4958-a622-85f76910920a-kube-rbac-proxy-cm podName:866ab681-b97a-4958-a622-85f76910920a nodeName:}" failed. No retries permitted until 2026-09-15 20:09:03.878070856 +0000 UTC m=+250.269023846 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/866ab681-b97a-4958-a622-85f76910920a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qflpq" (UID: "866ab681-b97a-4958-a622-85f76910920a") : configmap "kube-rbac-proxy" not found
Sep 15 20:08:59.878253 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:08:59.878111    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/866ab681-b97a-4958-a622-85f76910920a-insights-runtime-extractor-tls podName:866ab681-b97a-4958-a622-85f76910920a nodeName:}" failed. No retries permitted until 2026-09-15 20:09:03.878096241 +0000 UTC m=+250.269049217 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/866ab681-b97a-4958-a622-85f76910920a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qflpq" (UID: "866ab681-b97a-4958-a622-85f76910920a") : secret "insights-runtime-extractor-tls" not found
Sep 15 20:08:59.878253 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:08:59.878129    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/45f245cb-4670-40a8-9da8-252c030c8c8e-registry-tls podName:45f245cb-4670-40a8-9da8-252c030c8c8e nodeName:}" failed. No retries permitted until 2026-09-15 20:09:07.878119861 +0000 UTC m=+254.269072838 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/45f245cb-4670-40a8-9da8-252c030c8c8e-registry-tls") pod "image-registry-76d959cf57-r9gz6" (UID: "45f245cb-4670-40a8-9da8-252c030c8c8e") : secret "image-registry-tls" not found
Sep 15 20:09:02.531403 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:02.531370    2577 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-r227b"
Sep 15 20:09:02.531848 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:02.531831    2577 scope.go:117] "RemoveContainer" containerID="ca89939acd787dffdfa91800e536998c056ff37123128f98405a58bdb3a25076"
Sep 15 20:09:02.532005 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:09:02.531986    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-r227b_openshift-console-operator(ecab5774-79a4-42eb-9bb5-a1f09b18f3fb)\"" pod="openshift-console-operator/console-operator-696884fb9c-r227b" podUID="ecab5774-79a4-42eb-9bb5-a1f09b18f3fb"
Sep 15 20:09:03.908684 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:03.908644    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/866ab681-b97a-4958-a622-85f76910920a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qflpq\" (UID: \"866ab681-b97a-4958-a622-85f76910920a\") " pod="openshift-insights/insights-runtime-extractor-qflpq"
Sep 15 20:09:03.909085 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:03.908703    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/866ab681-b97a-4958-a622-85f76910920a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qflpq\" (UID: \"866ab681-b97a-4958-a622-85f76910920a\") " pod="openshift-insights/insights-runtime-extractor-qflpq"
Sep 15 20:09:03.909085 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:09:03.908775    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 20:09:03.909085 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:09:03.908801    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 20:09:03.909085 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:09:03.908842    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/866ab681-b97a-4958-a622-85f76910920a-insights-runtime-extractor-tls podName:866ab681-b97a-4958-a622-85f76910920a nodeName:}" failed. No retries permitted until 2026-09-15 20:09:11.908829144 +0000 UTC m=+258.299782132 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/866ab681-b97a-4958-a622-85f76910920a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qflpq" (UID: "866ab681-b97a-4958-a622-85f76910920a") : secret "insights-runtime-extractor-tls" not found
Sep 15 20:09:03.909085 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:09:03.908855    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/866ab681-b97a-4958-a622-85f76910920a-kube-rbac-proxy-cm podName:866ab681-b97a-4958-a622-85f76910920a nodeName:}" failed. No retries permitted until 2026-09-15 20:09:11.908848691 +0000 UTC m=+258.299801666 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/866ab681-b97a-4958-a622-85f76910920a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qflpq" (UID: "866ab681-b97a-4958-a622-85f76910920a") : configmap "kube-rbac-proxy" not found
Sep 15 20:09:05.798550 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:05.798494    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-r227b"
Sep 15 20:09:05.799000 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:05.798898    2577 scope.go:117] "RemoveContainer" containerID="ca89939acd787dffdfa91800e536998c056ff37123128f98405a58bdb3a25076"
Sep 15 20:09:05.799057 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:09:05.799038    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-r227b_openshift-console-operator(ecab5774-79a4-42eb-9bb5-a1f09b18f3fb)\"" pod="openshift-console-operator/console-operator-696884fb9c-r227b" podUID="ecab5774-79a4-42eb-9bb5-a1f09b18f3fb"
Sep 15 20:09:05.924237 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:05.924205    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5482d254-b248-4eda-a7d8-37bcb3289d69-metrics-certs\") pod \"network-metrics-daemon-2v4rt\" (UID: \"5482d254-b248-4eda-a7d8-37bcb3289d69\") " pod="openshift-multus/network-metrics-daemon-2v4rt"
Sep 15 20:09:05.924443 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:09:05.924407    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 15 20:09:05.925291 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:09:05.925265    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5482d254-b248-4eda-a7d8-37bcb3289d69-metrics-certs podName:5482d254-b248-4eda-a7d8-37bcb3289d69 nodeName:}" failed. No retries permitted until 2026-09-15 20:11:07.92523576 +0000 UTC m=+374.316188751 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5482d254-b248-4eda-a7d8-37bcb3289d69-metrics-certs") pod "network-metrics-daemon-2v4rt" (UID: "5482d254-b248-4eda-a7d8-37bcb3289d69") : secret "metrics-daemon-secret" not found
Sep 15 20:09:07.939515 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:07.939472    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/45f245cb-4670-40a8-9da8-252c030c8c8e-registry-tls\") pod \"image-registry-76d959cf57-r9gz6\" (UID: \"45f245cb-4670-40a8-9da8-252c030c8c8e\") " pod="openshift-image-registry/image-registry-76d959cf57-r9gz6"
Sep 15 20:09:07.941830 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:07.941806    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/45f245cb-4670-40a8-9da8-252c030c8c8e-registry-tls\") pod \"image-registry-76d959cf57-r9gz6\" (UID: \"45f245cb-4670-40a8-9da8-252c030c8c8e\") " pod="openshift-image-registry/image-registry-76d959cf57-r9gz6"
Sep 15 20:09:08.026951 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:08.026916    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-g82mz\""
Sep 15 20:09:08.034966 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:08.034946    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76d959cf57-r9gz6"
Sep 15 20:09:08.307621 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:08.307588    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76d959cf57-r9gz6"]
Sep 15 20:09:08.311162 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:09:08.311134    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45f245cb_4670_40a8_9da8_252c030c8c8e.slice/crio-456a3df06b3080ae5a03f1e42350edcdcc534d0521c72b9fe333d5e553b4eaa2 WatchSource:0}: Error finding container 456a3df06b3080ae5a03f1e42350edcdcc534d0521c72b9fe333d5e553b4eaa2: Status 404 returned error can't find the container with id 456a3df06b3080ae5a03f1e42350edcdcc534d0521c72b9fe333d5e553b4eaa2
Sep 15 20:09:08.854288 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:08.854247    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76d959cf57-r9gz6" event={"ID":"45f245cb-4670-40a8-9da8-252c030c8c8e","Type":"ContainerStarted","Data":"738f5a1f89b35226f4a390cdd3efa4300bf0147f1a14298a8cc74f714e3c9367"}
Sep 15 20:09:08.854490 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:08.854300    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76d959cf57-r9gz6" event={"ID":"45f245cb-4670-40a8-9da8-252c030c8c8e","Type":"ContainerStarted","Data":"456a3df06b3080ae5a03f1e42350edcdcc534d0521c72b9fe333d5e553b4eaa2"}
Sep 15 20:09:08.854490 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:08.854382    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-76d959cf57-r9gz6"
Sep 15 20:09:08.875412 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:08.875363    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-76d959cf57-r9gz6" podStartSLOduration=16.875347768 podStartE2EDuration="16.875347768s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:08:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:09:08.87438461 +0000 UTC m=+255.265337628" watchObservedRunningTime="2026-09-15 20:09:08.875347768 +0000 UTC m=+255.266300765"
Sep 15 20:09:11.974687 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:11.974646    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/866ab681-b97a-4958-a622-85f76910920a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qflpq\" (UID: \"866ab681-b97a-4958-a622-85f76910920a\") " pod="openshift-insights/insights-runtime-extractor-qflpq"
Sep 15 20:09:11.974687 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:11.974695    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/866ab681-b97a-4958-a622-85f76910920a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qflpq\" (UID: \"866ab681-b97a-4958-a622-85f76910920a\") " pod="openshift-insights/insights-runtime-extractor-qflpq"
Sep 15 20:09:11.975283 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:09:11.974819    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 20:09:11.975283 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:09:11.974880    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/866ab681-b97a-4958-a622-85f76910920a-insights-runtime-extractor-tls podName:866ab681-b97a-4958-a622-85f76910920a nodeName:}" failed. No retries permitted until 2026-09-15 20:09:27.974866015 +0000 UTC m=+274.365818990 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/866ab681-b97a-4958-a622-85f76910920a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qflpq" (UID: "866ab681-b97a-4958-a622-85f76910920a") : secret "insights-runtime-extractor-tls" not found
Sep 15 20:09:11.975283 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:11.975201    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/866ab681-b97a-4958-a622-85f76910920a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qflpq\" (UID: \"866ab681-b97a-4958-a622-85f76910920a\") " pod="openshift-insights/insights-runtime-extractor-qflpq"
Sep 15 20:09:19.054881 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:19.054794    2577 scope.go:117] "RemoveContainer" containerID="ca89939acd787dffdfa91800e536998c056ff37123128f98405a58bdb3a25076"
Sep 15 20:09:19.887572 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:19.887540    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-r227b_ecab5774-79a4-42eb-9bb5-a1f09b18f3fb/console-operator/1.log"
Sep 15 20:09:19.887752 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:19.887610    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-r227b" event={"ID":"ecab5774-79a4-42eb-9bb5-a1f09b18f3fb","Type":"ContainerStarted","Data":"41814cbd6f153d3ac65af7997ba42ed4824826816abc785d959ee9ff83cefaba"}
Sep 15 20:09:19.887967 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:19.887948    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-r227b"
Sep 15 20:09:19.963206 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:19.963169    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-r227b"
Sep 15 20:09:21.212741 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:21.212703    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-76d959cf57-r9gz6"]
Sep 15 20:09:21.273584 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:21.273550    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6d4544bc7f-64cxr"]
Sep 15 20:09:21.276517 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:21.276503    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d4544bc7f-64cxr"
Sep 15 20:09:21.299674 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:21.299640    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d4544bc7f-64cxr"]
Sep 15 20:09:21.446332 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:21.446298    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5d9260ed-5dcc-488e-9d13-a49675580c16-bound-sa-token\") pod \"image-registry-6d4544bc7f-64cxr\" (UID: \"5d9260ed-5dcc-488e-9d13-a49675580c16\") " pod="openshift-image-registry/image-registry-6d4544bc7f-64cxr"
Sep 15 20:09:21.446508 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:21.446340    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5d9260ed-5dcc-488e-9d13-a49675580c16-image-registry-private-configuration\") pod \"image-registry-6d4544bc7f-64cxr\" (UID: \"5d9260ed-5dcc-488e-9d13-a49675580c16\") " pod="openshift-image-registry/image-registry-6d4544bc7f-64cxr"
Sep 15 20:09:21.446508 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:21.446374    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5d9260ed-5dcc-488e-9d13-a49675580c16-registry-certificates\") pod \"image-registry-6d4544bc7f-64cxr\" (UID: \"5d9260ed-5dcc-488e-9d13-a49675580c16\") " pod="openshift-image-registry/image-registry-6d4544bc7f-64cxr"
Sep 15 20:09:21.446508 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:21.446432    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5d9260ed-5dcc-488e-9d13-a49675580c16-installation-pull-secrets\") pod \"image-registry-6d4544bc7f-64cxr\" (UID: \"5d9260ed-5dcc-488e-9d13-a49675580c16\") " pod="openshift-image-registry/image-registry-6d4544bc7f-64cxr"
Sep 15 20:09:21.446508 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:21.446453    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5d9260ed-5dcc-488e-9d13-a49675580c16-registry-tls\") pod \"image-registry-6d4544bc7f-64cxr\" (UID: \"5d9260ed-5dcc-488e-9d13-a49675580c16\") " pod="openshift-image-registry/image-registry-6d4544bc7f-64cxr"
Sep 15 20:09:21.446508 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:21.446467    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5d9260ed-5dcc-488e-9d13-a49675580c16-ca-trust-extracted\") pod \"image-registry-6d4544bc7f-64cxr\" (UID: \"5d9260ed-5dcc-488e-9d13-a49675580c16\") " pod="openshift-image-registry/image-registry-6d4544bc7f-64cxr"
Sep 15 20:09:21.446508 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:21.446490    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tct72\" (UniqueName: \"kubernetes.io/projected/5d9260ed-5dcc-488e-9d13-a49675580c16-kube-api-access-tct72\") pod \"image-registry-6d4544bc7f-64cxr\" (UID: \"5d9260ed-5dcc-488e-9d13-a49675580c16\") " pod="openshift-image-registry/image-registry-6d4544bc7f-64cxr"
Sep 15 20:09:21.446788 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:21.446530    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5d9260ed-5dcc-488e-9d13-a49675580c16-trusted-ca\") pod \"image-registry-6d4544bc7f-64cxr\" (UID: \"5d9260ed-5dcc-488e-9d13-a49675580c16\") " pod="openshift-image-registry/image-registry-6d4544bc7f-64cxr"
Sep 15 20:09:21.547538 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:21.547447    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5d9260ed-5dcc-488e-9d13-a49675580c16-trusted-ca\") pod \"image-registry-6d4544bc7f-64cxr\" (UID: \"5d9260ed-5dcc-488e-9d13-a49675580c16\") " pod="openshift-image-registry/image-registry-6d4544bc7f-64cxr"
Sep 15 20:09:21.547538 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:21.547492    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5d9260ed-5dcc-488e-9d13-a49675580c16-bound-sa-token\") pod \"image-registry-6d4544bc7f-64cxr\" (UID: \"5d9260ed-5dcc-488e-9d13-a49675580c16\") " pod="openshift-image-registry/image-registry-6d4544bc7f-64cxr"
Sep 15 20:09:21.547538 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:21.547513    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5d9260ed-5dcc-488e-9d13-a49675580c16-image-registry-private-configuration\") pod \"image-registry-6d4544bc7f-64cxr\" (UID: \"5d9260ed-5dcc-488e-9d13-a49675580c16\") " pod="openshift-image-registry/image-registry-6d4544bc7f-64cxr"
Sep 15 20:09:21.547848 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:21.547545    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5d9260ed-5dcc-488e-9d13-a49675580c16-registry-certificates\") pod \"image-registry-6d4544bc7f-64cxr\" (UID: \"5d9260ed-5dcc-488e-9d13-a49675580c16\") " pod="openshift-image-registry/image-registry-6d4544bc7f-64cxr"
Sep 15 20:09:21.547848 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:21.547605    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5d9260ed-5dcc-488e-9d13-a49675580c16-installation-pull-secrets\") pod \"image-registry-6d4544bc7f-64cxr\" (UID: \"5d9260ed-5dcc-488e-9d13-a49675580c16\") " pod="openshift-image-registry/image-registry-6d4544bc7f-64cxr"
Sep 15 20:09:21.547848 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:21.547637    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5d9260ed-5dcc-488e-9d13-a49675580c16-registry-tls\") pod \"image-registry-6d4544bc7f-64cxr\" (UID: \"5d9260ed-5dcc-488e-9d13-a49675580c16\") " pod="openshift-image-registry/image-registry-6d4544bc7f-64cxr"
Sep 15 20:09:21.547848 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:21.547659    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5d9260ed-5dcc-488e-9d13-a49675580c16-ca-trust-extracted\") pod \"image-registry-6d4544bc7f-64cxr\" (UID: \"5d9260ed-5dcc-488e-9d13-a49675580c16\") " pod="openshift-image-registry/image-registry-6d4544bc7f-64cxr"
Sep 15 20:09:21.547848 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:21.547684    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tct72\" (UniqueName: \"kubernetes.io/projected/5d9260ed-5dcc-488e-9d13-a49675580c16-kube-api-access-tct72\") pod \"image-registry-6d4544bc7f-64cxr\" (UID: \"5d9260ed-5dcc-488e-9d13-a49675580c16\") " pod="openshift-image-registry/image-registry-6d4544bc7f-64cxr"
Sep 15 20:09:21.548176 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:21.548148    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5d9260ed-5dcc-488e-9d13-a49675580c16-ca-trust-extracted\") pod \"image-registry-6d4544bc7f-64cxr\" (UID: \"5d9260ed-5dcc-488e-9d13-a49675580c16\") " pod="openshift-image-registry/image-registry-6d4544bc7f-64cxr"
Sep 15 20:09:21.548645 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:21.548623    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5d9260ed-5dcc-488e-9d13-a49675580c16-registry-certificates\") pod \"image-registry-6d4544bc7f-64cxr\" (UID: \"5d9260ed-5dcc-488e-9d13-a49675580c16\") " pod="openshift-image-registry/image-registry-6d4544bc7f-64cxr"
Sep 15 20:09:21.548858 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:21.548833    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5d9260ed-5dcc-488e-9d13-a49675580c16-trusted-ca\") pod \"image-registry-6d4544bc7f-64cxr\" (UID: \"5d9260ed-5dcc-488e-9d13-a49675580c16\") " pod="openshift-image-registry/image-registry-6d4544bc7f-64cxr"
Sep 15 20:09:21.550149 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:21.550125    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5d9260ed-5dcc-488e-9d13-a49675580c16-installation-pull-secrets\") pod \"image-registry-6d4544bc7f-64cxr\" (UID: \"5d9260ed-5dcc-488e-9d13-a49675580c16\") " pod="openshift-image-registry/image-registry-6d4544bc7f-64cxr"
Sep 15 20:09:21.550237 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:21.550148    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5d9260ed-5dcc-488e-9d13-a49675580c16-image-registry-private-configuration\") pod \"image-registry-6d4544bc7f-64cxr\" (UID: \"5d9260ed-5dcc-488e-9d13-a49675580c16\") " pod="openshift-image-registry/image-registry-6d4544bc7f-64cxr"
Sep 15 20:09:21.550286 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:21.550231    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5d9260ed-5dcc-488e-9d13-a49675580c16-registry-tls\") pod \"image-registry-6d4544bc7f-64cxr\" (UID: \"5d9260ed-5dcc-488e-9d13-a49675580c16\") " pod="openshift-image-registry/image-registry-6d4544bc7f-64cxr"
Sep 15 20:09:21.557225 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:21.557207    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5d9260ed-5dcc-488e-9d13-a49675580c16-bound-sa-token\") pod \"image-registry-6d4544bc7f-64cxr\" (UID: \"5d9260ed-5dcc-488e-9d13-a49675580c16\") " pod="openshift-image-registry/image-registry-6d4544bc7f-64cxr"
Sep 15 20:09:21.557485 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:21.557470    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tct72\" (UniqueName: \"kubernetes.io/projected/5d9260ed-5dcc-488e-9d13-a49675580c16-kube-api-access-tct72\") pod \"image-registry-6d4544bc7f-64cxr\" (UID: \"5d9260ed-5dcc-488e-9d13-a49675580c16\") " pod="openshift-image-registry/image-registry-6d4544bc7f-64cxr"
Sep 15 20:09:21.584901 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:21.584870    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d4544bc7f-64cxr"
Sep 15 20:09:21.858821 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:21.858744    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d4544bc7f-64cxr"]
Sep 15 20:09:21.861115 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:09:21.861095    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d9260ed_5dcc_488e_9d13_a49675580c16.slice/crio-4446eaafc5d8c010e7bdfd45c45a800491a0e6ae7e6fc76ca3ba927368e99587 WatchSource:0}: Error finding container 4446eaafc5d8c010e7bdfd45c45a800491a0e6ae7e6fc76ca3ba927368e99587: Status 404 returned error can't find the container with id 4446eaafc5d8c010e7bdfd45c45a800491a0e6ae7e6fc76ca3ba927368e99587
Sep 15 20:09:21.894048 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:21.894016    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d4544bc7f-64cxr" event={"ID":"5d9260ed-5dcc-488e-9d13-a49675580c16","Type":"ContainerStarted","Data":"4446eaafc5d8c010e7bdfd45c45a800491a0e6ae7e6fc76ca3ba927368e99587"}
Sep 15 20:09:22.899495 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:22.899459    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d4544bc7f-64cxr" event={"ID":"5d9260ed-5dcc-488e-9d13-a49675580c16","Type":"ContainerStarted","Data":"1008f346a4c8c76aeb870dc65e9eab2fbcb3fb49c4ec691b82133394196538aa"}
Sep 15 20:09:22.899940 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:22.899591    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6d4544bc7f-64cxr"
Sep 15 20:09:22.920557 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:22.920512    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6d4544bc7f-64cxr" podStartSLOduration=1.920499486 podStartE2EDuration="1.920499486s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:09:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:09:22.919618656 +0000 UTC m=+269.310571652" watchObservedRunningTime="2026-09-15 20:09:22.920499486 +0000 UTC m=+269.311452482"
Sep 15 20:09:27.997642 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:27.997601    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/866ab681-b97a-4958-a622-85f76910920a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qflpq\" (UID: \"866ab681-b97a-4958-a622-85f76910920a\") " pod="openshift-insights/insights-runtime-extractor-qflpq"
Sep 15 20:09:27.999907 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:27.999883    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/866ab681-b97a-4958-a622-85f76910920a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qflpq\" (UID: \"866ab681-b97a-4958-a622-85f76910920a\") " pod="openshift-insights/insights-runtime-extractor-qflpq"
Sep 15 20:09:28.253997 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:28.253915    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qflpq"
Sep 15 20:09:28.528187 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:28.528100    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qflpq"]
Sep 15 20:09:28.531976 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:09:28.531945    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod866ab681_b97a_4958_a622_85f76910920a.slice/crio-4af11418c8170e44a0cb80c4add15cee26b7f19394aea7bde66daa5314c522e2 WatchSource:0}: Error finding container 4af11418c8170e44a0cb80c4add15cee26b7f19394aea7bde66daa5314c522e2: Status 404 returned error can't find the container with id 4af11418c8170e44a0cb80c4add15cee26b7f19394aea7bde66daa5314c522e2
Sep 15 20:09:28.921953 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:28.921868    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qflpq" event={"ID":"866ab681-b97a-4958-a622-85f76910920a","Type":"ContainerStarted","Data":"84a6fa61b31f96b2640dd724ed544cb0d860463906f75a0f7b2b05ac48295a5b"}
Sep 15 20:09:28.921953 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:28.921927    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qflpq" event={"ID":"866ab681-b97a-4958-a622-85f76910920a","Type":"ContainerStarted","Data":"4af11418c8170e44a0cb80c4add15cee26b7f19394aea7bde66daa5314c522e2"}
Sep 15 20:09:30.930020 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:30.929934    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qflpq" event={"ID":"866ab681-b97a-4958-a622-85f76910920a","Type":"ContainerStarted","Data":"4a33298c006a7da29b1a93f152a960c02345651e3027daec1aad2fce9dca5ab4"}
Sep 15 20:09:31.219167 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:31.219133    2577 patch_prober.go:28] interesting pod/image-registry-76d959cf57-r9gz6 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 15 20:09:31.219373 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:31.219193    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-76d959cf57-r9gz6" podUID="45f245cb-4670-40a8-9da8-252c030c8c8e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 20:09:31.935895 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:31.935855    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qflpq" event={"ID":"866ab681-b97a-4958-a622-85f76910920a","Type":"ContainerStarted","Data":"19efe9e7c076bd58dd4bf0ee591583d4979af8e11951c9ac8a858f243b22970a"}
Sep 15 20:09:31.958700 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:31.958654    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-qflpq" podStartSLOduration=32.847505601 podStartE2EDuration="35.958639824s" totalImagesPullingTime="3.111134223s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:08:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:09:31.958500395 +0000 UTC m=+278.349453393" watchObservedRunningTime="2026-09-15 20:09:31.958639824 +0000 UTC m=+278.349592821"
Sep 15 20:09:34.658639 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:34.658604    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-696cd69c49-76fh6"]
Sep 15 20:09:34.660503 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:34.660486    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-696cd69c49-76fh6"
Sep 15 20:09:34.664171 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:34.664150    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 15 20:09:34.665149 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:34.665128    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 15 20:09:34.665251 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:34.665132    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 15 20:09:34.665251 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:34.665163    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 15 20:09:34.665251 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:34.665169    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-2kzkh\""
Sep 15 20:09:34.665493 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:34.665477    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 15 20:09:34.665580 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:34.665481    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 15 20:09:34.665580 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:34.665531    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 15 20:09:34.670176 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:34.670157    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 15 20:09:34.673802 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:34.673775    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-696cd69c49-76fh6"]
Sep 15 20:09:34.750485 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:34.750445    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-trusted-ca-bundle\") pod \"console-696cd69c49-76fh6\" (UID: \"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b\") " pod="openshift-console/console-696cd69c49-76fh6"
Sep 15 20:09:34.750688 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:34.750501    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-console-config\") pod \"console-696cd69c49-76fh6\" (UID: \"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b\") " pod="openshift-console/console-696cd69c49-76fh6"
Sep 15 20:09:34.750688 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:34.750578    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjfpp\" (UniqueName: \"kubernetes.io/projected/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-kube-api-access-rjfpp\") pod \"console-696cd69c49-76fh6\" (UID: \"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b\") " pod="openshift-console/console-696cd69c49-76fh6"
Sep 15 20:09:34.750688 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:34.750668    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-console-oauth-config\") pod \"console-696cd69c49-76fh6\" (UID: \"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b\") " pod="openshift-console/console-696cd69c49-76fh6"
Sep 15 20:09:34.750892 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:34.750693    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-console-serving-cert\") pod \"console-696cd69c49-76fh6\" (UID: \"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b\") " pod="openshift-console/console-696cd69c49-76fh6"
Sep 15 20:09:34.750892 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:34.750786    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-oauth-serving-cert\") pod \"console-696cd69c49-76fh6\" (UID: \"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b\") " pod="openshift-console/console-696cd69c49-76fh6"
Sep 15 20:09:34.750892 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:34.750824    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-service-ca\") pod \"console-696cd69c49-76fh6\" (UID: \"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b\") " pod="openshift-console/console-696cd69c49-76fh6"
Sep 15 20:09:34.851968 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:34.851931    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-console-oauth-config\") pod \"console-696cd69c49-76fh6\" (UID: \"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b\") " pod="openshift-console/console-696cd69c49-76fh6"
Sep 15 20:09:34.851968 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:34.851967    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-console-serving-cert\") pod \"console-696cd69c49-76fh6\" (UID: \"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b\") " pod="openshift-console/console-696cd69c49-76fh6"
Sep 15 20:09:34.852208 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:34.851986    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-oauth-serving-cert\") pod \"console-696cd69c49-76fh6\" (UID: \"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b\") " pod="openshift-console/console-696cd69c49-76fh6"
Sep 15 20:09:34.852208 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:34.852004    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-service-ca\") pod \"console-696cd69c49-76fh6\" (UID: \"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b\") " pod="openshift-console/console-696cd69c49-76fh6"
Sep 15 20:09:34.852208 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:34.852031    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-trusted-ca-bundle\") pod \"console-696cd69c49-76fh6\" (UID: \"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b\") " pod="openshift-console/console-696cd69c49-76fh6"
Sep 15 20:09:34.852208 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:34.852071    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-console-config\") pod \"console-696cd69c49-76fh6\" (UID: \"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b\") " pod="openshift-console/console-696cd69c49-76fh6"
Sep 15 20:09:34.852208 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:34.852108    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjfpp\" (UniqueName: \"kubernetes.io/projected/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-kube-api-access-rjfpp\") pod \"console-696cd69c49-76fh6\" (UID: \"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b\") " pod="openshift-console/console-696cd69c49-76fh6"
Sep 15 20:09:34.852839 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:34.852816    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-oauth-serving-cert\") pod \"console-696cd69c49-76fh6\" (UID: \"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b\") " pod="openshift-console/console-696cd69c49-76fh6"
Sep 15 20:09:34.853000 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:34.852975    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-service-ca\") pod \"console-696cd69c49-76fh6\" (UID: \"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b\") " pod="openshift-console/console-696cd69c49-76fh6"
Sep 15 20:09:34.853057 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:34.852983    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-console-config\") pod \"console-696cd69c49-76fh6\" (UID: \"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b\") " pod="openshift-console/console-696cd69c49-76fh6"
Sep 15 20:09:34.853206 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:34.853188    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-trusted-ca-bundle\") pod \"console-696cd69c49-76fh6\" (UID: \"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b\") " pod="openshift-console/console-696cd69c49-76fh6"
Sep 15 20:09:34.854634 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:34.854607    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-console-oauth-config\") pod \"console-696cd69c49-76fh6\" (UID: \"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b\") " pod="openshift-console/console-696cd69c49-76fh6"
Sep 15 20:09:34.854741 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:34.854683    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-console-serving-cert\") pod \"console-696cd69c49-76fh6\" (UID: \"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b\") " pod="openshift-console/console-696cd69c49-76fh6"
Sep 15 20:09:34.862100 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:34.862076    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjfpp\" (UniqueName: \"kubernetes.io/projected/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-kube-api-access-rjfpp\") pod \"console-696cd69c49-76fh6\" (UID: \"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b\") " pod="openshift-console/console-696cd69c49-76fh6"
Sep 15 20:09:34.970057 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:34.970023    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-696cd69c49-76fh6"
Sep 15 20:09:35.242639 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:35.242556    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-696cd69c49-76fh6"]
Sep 15 20:09:35.245262 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:09:35.245236    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeba45072_9a03_4268_a4d7_3c2ae1d7cf8b.slice/crio-9a069dff79b632117d91967a0c6a394cf33dd61686384c10ec7ce4f8b39b5490 WatchSource:0}: Error finding container 9a069dff79b632117d91967a0c6a394cf33dd61686384c10ec7ce4f8b39b5490: Status 404 returned error can't find the container with id 9a069dff79b632117d91967a0c6a394cf33dd61686384c10ec7ce4f8b39b5490
Sep 15 20:09:35.576289 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:09:35.576180    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-h67tx" podUID="5360f750-45a7-42d6-8b65-be039f6ee235"
Sep 15 20:09:35.951352 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:35.951278    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-h67tx"
Sep 15 20:09:35.951718 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:35.951279    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-696cd69c49-76fh6" event={"ID":"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b","Type":"ContainerStarted","Data":"9a069dff79b632117d91967a0c6a394cf33dd61686384c10ec7ce4f8b39b5490"}
Sep 15 20:09:36.268950 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:36.268916    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-cnlkd"]
Sep 15 20:09:36.271449 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:36.271430    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cnlkd"
Sep 15 20:09:36.274987 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:36.274962    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 15 20:09:36.275854 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:36.275831    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 15 20:09:36.276118 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:36.276104    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9jdsl\""
Sep 15 20:09:36.276260 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:36.276239    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 15 20:09:36.276546 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:36.276239    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 15 20:09:36.276632 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:36.276615    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 15 20:09:36.276632 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:36.276620    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 15 20:09:36.364711 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:36.364680    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/605e5608-f132-43d8-a691-c56f2f342c9c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cnlkd\" (UID: \"605e5608-f132-43d8-a691-c56f2f342c9c\") " pod="openshift-monitoring/node-exporter-cnlkd"
Sep 15 20:09:36.364909 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:36.364761    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/605e5608-f132-43d8-a691-c56f2f342c9c-node-exporter-tls\") pod \"node-exporter-cnlkd\" (UID: \"605e5608-f132-43d8-a691-c56f2f342c9c\") " pod="openshift-monitoring/node-exporter-cnlkd"
Sep 15 20:09:36.364909 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:36.364808    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/605e5608-f132-43d8-a691-c56f2f342c9c-sys\") pod \"node-exporter-cnlkd\" (UID: \"605e5608-f132-43d8-a691-c56f2f342c9c\") " pod="openshift-monitoring/node-exporter-cnlkd"
Sep 15 20:09:36.364909 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:36.364830    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/605e5608-f132-43d8-a691-c56f2f342c9c-metrics-client-ca\") pod \"node-exporter-cnlkd\" (UID: \"605e5608-f132-43d8-a691-c56f2f342c9c\") " pod="openshift-monitoring/node-exporter-cnlkd"
Sep 15 20:09:36.364909 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:36.364879    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/605e5608-f132-43d8-a691-c56f2f342c9c-node-exporter-textfile\") pod \"node-exporter-cnlkd\" (UID: \"605e5608-f132-43d8-a691-c56f2f342c9c\") " pod="openshift-monitoring/node-exporter-cnlkd"
Sep 15 20:09:36.364909 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:36.364912    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/605e5608-f132-43d8-a691-c56f2f342c9c-root\") pod \"node-exporter-cnlkd\" (UID: \"605e5608-f132-43d8-a691-c56f2f342c9c\") " pod="openshift-monitoring/node-exporter-cnlkd"
Sep 15 20:09:36.365109 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:36.364934    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/605e5608-f132-43d8-a691-c56f2f342c9c-node-exporter-accelerators-collector-config\") pod \"node-exporter-cnlkd\" (UID: \"605e5608-f132-43d8-a691-c56f2f342c9c\") " pod="openshift-monitoring/node-exporter-cnlkd"
Sep 15 20:09:36.365109 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:36.364955    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvjpp\" (UniqueName: \"kubernetes.io/projected/605e5608-f132-43d8-a691-c56f2f342c9c-kube-api-access-zvjpp\") pod \"node-exporter-cnlkd\" (UID: \"605e5608-f132-43d8-a691-c56f2f342c9c\") " pod="openshift-monitoring/node-exporter-cnlkd"
Sep 15 20:09:36.365109 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:36.364984    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/605e5608-f132-43d8-a691-c56f2f342c9c-node-exporter-wtmp\") pod \"node-exporter-cnlkd\" (UID: \"605e5608-f132-43d8-a691-c56f2f342c9c\") " pod="openshift-monitoring/node-exporter-cnlkd"
Sep 15 20:09:36.466256 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:36.466217    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/605e5608-f132-43d8-a691-c56f2f342c9c-node-exporter-wtmp\") pod \"node-exporter-cnlkd\" (UID: \"605e5608-f132-43d8-a691-c56f2f342c9c\") " pod="openshift-monitoring/node-exporter-cnlkd"
Sep 15 20:09:36.466433 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:36.466271    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/605e5608-f132-43d8-a691-c56f2f342c9c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cnlkd\" (UID: \"605e5608-f132-43d8-a691-c56f2f342c9c\") " pod="openshift-monitoring/node-exporter-cnlkd"
Sep 15 20:09:36.466433 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:36.466296    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/605e5608-f132-43d8-a691-c56f2f342c9c-node-exporter-tls\") pod \"node-exporter-cnlkd\" (UID: \"605e5608-f132-43d8-a691-c56f2f342c9c\") " pod="openshift-monitoring/node-exporter-cnlkd"
Sep 15 20:09:36.466433 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:36.466335    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/605e5608-f132-43d8-a691-c56f2f342c9c-sys\") pod \"node-exporter-cnlkd\" (UID: \"605e5608-f132-43d8-a691-c56f2f342c9c\") " pod="openshift-monitoring/node-exporter-cnlkd"
Sep 15 20:09:36.466433 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:36.466351    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/605e5608-f132-43d8-a691-c56f2f342c9c-metrics-client-ca\") pod \"node-exporter-cnlkd\" (UID: \"605e5608-f132-43d8-a691-c56f2f342c9c\") " pod="openshift-monitoring/node-exporter-cnlkd"
Sep 15 20:09:36.466433 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:36.466370    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/605e5608-f132-43d8-a691-c56f2f342c9c-node-exporter-textfile\") pod \"node-exporter-cnlkd\" (UID: \"605e5608-f132-43d8-a691-c56f2f342c9c\") " pod="openshift-monitoring/node-exporter-cnlkd"
Sep 15 20:09:36.466433 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:36.466387    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/605e5608-f132-43d8-a691-c56f2f342c9c-root\") pod \"node-exporter-cnlkd\" (UID: \"605e5608-f132-43d8-a691-c56f2f342c9c\") " pod="openshift-monitoring/node-exporter-cnlkd"
Sep 15 20:09:36.466433 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:36.466403    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/605e5608-f132-43d8-a691-c56f2f342c9c-node-exporter-accelerators-collector-config\") pod \"node-exporter-cnlkd\" (UID: \"605e5608-f132-43d8-a691-c56f2f342c9c\") " pod="openshift-monitoring/node-exporter-cnlkd"
Sep 15 20:09:36.466433 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:36.466419    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/605e5608-f132-43d8-a691-c56f2f342c9c-node-exporter-wtmp\") pod \"node-exporter-cnlkd\" (UID: \"605e5608-f132-43d8-a691-c56f2f342c9c\") " pod="openshift-monitoring/node-exporter-cnlkd"
Sep 15 20:09:36.466853 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:36.466425    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvjpp\" (UniqueName: \"kubernetes.io/projected/605e5608-f132-43d8-a691-c56f2f342c9c-kube-api-access-zvjpp\") pod \"node-exporter-cnlkd\" (UID: \"605e5608-f132-43d8-a691-c56f2f342c9c\") " pod="openshift-monitoring/node-exporter-cnlkd"
Sep 15 20:09:36.466853 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:36.466466    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/605e5608-f132-43d8-a691-c56f2f342c9c-root\") pod \"node-exporter-cnlkd\" (UID: \"605e5608-f132-43d8-a691-c56f2f342c9c\") " pod="openshift-monitoring/node-exporter-cnlkd"
Sep 15 20:09:36.466853 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:09:36.466481    2577 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 15 20:09:36.466853 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:36.466425    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/605e5608-f132-43d8-a691-c56f2f342c9c-sys\") pod \"node-exporter-cnlkd\" (UID: \"605e5608-f132-43d8-a691-c56f2f342c9c\") " pod="openshift-monitoring/node-exporter-cnlkd"
Sep 15 20:09:36.466853 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:09:36.466555    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/605e5608-f132-43d8-a691-c56f2f342c9c-node-exporter-tls podName:605e5608-f132-43d8-a691-c56f2f342c9c nodeName:}" failed. No retries permitted until 2026-09-15 20:09:36.966532245 +0000 UTC m=+283.357485223 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/605e5608-f132-43d8-a691-c56f2f342c9c-node-exporter-tls") pod "node-exporter-cnlkd" (UID: "605e5608-f132-43d8-a691-c56f2f342c9c") : secret "node-exporter-tls" not found
Sep 15 20:09:36.466853 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:36.466797    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/605e5608-f132-43d8-a691-c56f2f342c9c-node-exporter-textfile\") pod \"node-exporter-cnlkd\" (UID: \"605e5608-f132-43d8-a691-c56f2f342c9c\") " pod="openshift-monitoring/node-exporter-cnlkd"
Sep 15 20:09:36.467154 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:36.467011    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/605e5608-f132-43d8-a691-c56f2f342c9c-node-exporter-accelerators-collector-config\") pod \"node-exporter-cnlkd\" (UID: \"605e5608-f132-43d8-a691-c56f2f342c9c\") " pod="openshift-monitoring/node-exporter-cnlkd"
Sep 15 20:09:36.467154 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:36.467141    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/605e5608-f132-43d8-a691-c56f2f342c9c-metrics-client-ca\") pod \"node-exporter-cnlkd\" (UID: \"605e5608-f132-43d8-a691-c56f2f342c9c\") " pod="openshift-monitoring/node-exporter-cnlkd"
Sep 15 20:09:36.468623 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:36.468606    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/605e5608-f132-43d8-a691-c56f2f342c9c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cnlkd\" (UID: \"605e5608-f132-43d8-a691-c56f2f342c9c\") " pod="openshift-monitoring/node-exporter-cnlkd"
Sep 15 20:09:36.480083 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:36.480061    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvjpp\" (UniqueName: \"kubernetes.io/projected/605e5608-f132-43d8-a691-c56f2f342c9c-kube-api-access-zvjpp\") pod \"node-exporter-cnlkd\" (UID: \"605e5608-f132-43d8-a691-c56f2f342c9c\") " pod="openshift-monitoring/node-exporter-cnlkd"
Sep 15 20:09:36.970333 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:36.970300    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/605e5608-f132-43d8-a691-c56f2f342c9c-node-exporter-tls\") pod \"node-exporter-cnlkd\" (UID: \"605e5608-f132-43d8-a691-c56f2f342c9c\") " pod="openshift-monitoring/node-exporter-cnlkd"
Sep 15 20:09:36.972880 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:36.972856    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/605e5608-f132-43d8-a691-c56f2f342c9c-node-exporter-tls\") pod \"node-exporter-cnlkd\" (UID: \"605e5608-f132-43d8-a691-c56f2f342c9c\") " pod="openshift-monitoring/node-exporter-cnlkd"
Sep 15 20:09:37.181062 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.181029    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cnlkd"
Sep 15 20:09:37.191489 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:09:37.191459    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod605e5608_f132_43d8_a691_c56f2f342c9c.slice/crio-64c5ddbf8e7f47970a441f789c70eb983ac8b05556d0e8da7206ccb667168b20 WatchSource:0}: Error finding container 64c5ddbf8e7f47970a441f789c70eb983ac8b05556d0e8da7206ccb667168b20: Status 404 returned error can't find the container with id 64c5ddbf8e7f47970a441f789c70eb983ac8b05556d0e8da7206ccb667168b20
Sep 15 20:09:37.369136 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.369047    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 15 20:09:37.377583 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.377557    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.380321 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.380098    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 15 20:09:37.380321 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.380192    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 15 20:09:37.380321 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.380199    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 15 20:09:37.380321 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.380213    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-xsvw7\""
Sep 15 20:09:37.380321 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.380228    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 15 20:09:37.380321 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.380261    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 15 20:09:37.380906 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.380651    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 15 20:09:37.380906 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.380785    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 15 20:09:37.380906 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.380788    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 15 20:09:37.381044 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.381012    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 15 20:09:37.388441 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.388417    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 15 20:09:37.476828 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.476794    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.476828 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.476836    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.477048 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.476877    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.477048 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.476894    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e94af346-ecec-410c-8144-f476cb7d77d2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.477048 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.476934    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-web-config\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.477048 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.476953    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-config-volume\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.477048 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.476973    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e94af346-ecec-410c-8144-f476cb7d77d2-config-out\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.477048 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.476998    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e94af346-ecec-410c-8144-f476cb7d77d2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.477048 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.477043    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.477286 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.477110    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e94af346-ecec-410c-8144-f476cb7d77d2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.477286 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.477129    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xk7v\" (UniqueName: \"kubernetes.io/projected/e94af346-ecec-410c-8144-f476cb7d77d2-kube-api-access-5xk7v\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.477286 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.477163    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e94af346-ecec-410c-8144-f476cb7d77d2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.477286 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.477188    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.577828 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.577799    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e94af346-ecec-410c-8144-f476cb7d77d2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.578008 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.577839    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.578008 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.577867    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.578008 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.577889    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.578008 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.577922    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.578008 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.577946    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e94af346-ecec-410c-8144-f476cb7d77d2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.578008 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.577977    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-web-config\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.578008 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.578002    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-config-volume\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.578328 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.578023    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e94af346-ecec-410c-8144-f476cb7d77d2-config-out\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.578328 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.578046    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e94af346-ecec-410c-8144-f476cb7d77d2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.578328 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.578076    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.578328 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.578139    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e94af346-ecec-410c-8144-f476cb7d77d2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.578328 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.578159    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xk7v\" (UniqueName: \"kubernetes.io/projected/e94af346-ecec-410c-8144-f476cb7d77d2-kube-api-access-5xk7v\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.579189 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.578811    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e94af346-ecec-410c-8144-f476cb7d77d2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.579951 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.579664    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e94af346-ecec-410c-8144-f476cb7d77d2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.580067 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.580047    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e94af346-ecec-410c-8144-f476cb7d77d2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.582279 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.582145    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.582279 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.582159    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e94af346-ecec-410c-8144-f476cb7d77d2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.582279 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.582244    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.582604 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.582572    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.583098 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.583054    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-config-volume\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.583222 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.583200    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.584075 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.584043    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.584444 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.584423    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-web-config\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.584643 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.584619    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e94af346-ecec-410c-8144-f476cb7d77d2-config-out\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.589407 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.589389    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xk7v\" (UniqueName: \"kubernetes.io/projected/e94af346-ecec-410c-8144-f476cb7d77d2-kube-api-access-5xk7v\") pod \"alertmanager-main-0\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.689484 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.689403    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:09:37.960648 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.960611    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cnlkd" event={"ID":"605e5608-f132-43d8-a691-c56f2f342c9c","Type":"ContainerStarted","Data":"64c5ddbf8e7f47970a441f789c70eb983ac8b05556d0e8da7206ccb667168b20"}
Sep 15 20:09:37.980354 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:37.980324    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 15 20:09:38.729907 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:09:38.729874    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode94af346_ecec_410c_8144_f476cb7d77d2.slice/crio-b9a008507e0324b9d0182b7fb247edc7d460661a724ed5f2b5eefcb169a36dcf WatchSource:0}: Error finding container b9a008507e0324b9d0182b7fb247edc7d460661a724ed5f2b5eefcb169a36dcf: Status 404 returned error can't find the container with id b9a008507e0324b9d0182b7fb247edc7d460661a724ed5f2b5eefcb169a36dcf
Sep 15 20:09:38.965418 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:38.965386    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-696cd69c49-76fh6" event={"ID":"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b","Type":"ContainerStarted","Data":"f29c6ec6cd86dec83e186c5c73f88e5346dc62f172c5753adb1ced6fe4cd066b"}
Sep 15 20:09:38.966470 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:38.966448    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e94af346-ecec-410c-8144-f476cb7d77d2","Type":"ContainerStarted","Data":"b9a008507e0324b9d0182b7fb247edc7d460661a724ed5f2b5eefcb169a36dcf"}
Sep 15 20:09:38.987695 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:38.987580    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-696cd69c49-76fh6" podStartSLOduration=1.4784101139999999 podStartE2EDuration="4.987562613s" totalImagesPullingTime="3.509152499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:09:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:09:38.985851588 +0000 UTC m=+285.376804578" watchObservedRunningTime="2026-09-15 20:09:38.987562613 +0000 UTC m=+285.378515609"
Sep 15 20:09:38.991144 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:38.991108    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5360f750-45a7-42d6-8b65-be039f6ee235-metrics-tls\") pod \"dns-default-h67tx\" (UID: \"5360f750-45a7-42d6-8b65-be039f6ee235\") " pod="openshift-dns/dns-default-h67tx"
Sep 15 20:09:38.993582 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:38.993553    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5360f750-45a7-42d6-8b65-be039f6ee235-metrics-tls\") pod \"dns-default-h67tx\" (UID: \"5360f750-45a7-42d6-8b65-be039f6ee235\") " pod="openshift-dns/dns-default-h67tx"
Sep 15 20:09:39.254452 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.254373    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jwzj9\""
Sep 15 20:09:39.262759 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.262703    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-h67tx"
Sep 15 20:09:39.282194 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.282159    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7686f988d4-slqm5"]
Sep 15 20:09:39.285528 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.285177    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7686f988d4-slqm5"
Sep 15 20:09:39.287804 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.287768    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 15 20:09:39.288360 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.288329    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-fvsz2\""
Sep 15 20:09:39.288469 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.288395    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 15 20:09:39.288566 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.288533    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 15 20:09:39.288566 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.288551    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 15 20:09:39.289346 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.288927    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 15 20:09:39.289346 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.289237    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-7ccko6s7a7uen\""
Sep 15 20:09:39.302855 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.302829    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7686f988d4-slqm5"]
Sep 15 20:09:39.394523 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.394490    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ec8c4c90-e728-48af-9caf-0e11b18e85db-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7686f988d4-slqm5\" (UID: \"ec8c4c90-e728-48af-9caf-0e11b18e85db\") " pod="openshift-monitoring/thanos-querier-7686f988d4-slqm5"
Sep 15 20:09:39.394708 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.394557    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ec8c4c90-e728-48af-9caf-0e11b18e85db-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7686f988d4-slqm5\" (UID: \"ec8c4c90-e728-48af-9caf-0e11b18e85db\") " pod="openshift-monitoring/thanos-querier-7686f988d4-slqm5"
Sep 15 20:09:39.394708 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.394611    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ec8c4c90-e728-48af-9caf-0e11b18e85db-secret-grpc-tls\") pod \"thanos-querier-7686f988d4-slqm5\" (UID: \"ec8c4c90-e728-48af-9caf-0e11b18e85db\") " pod="openshift-monitoring/thanos-querier-7686f988d4-slqm5"
Sep 15 20:09:39.394708 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.394668    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ec8c4c90-e728-48af-9caf-0e11b18e85db-secret-thanos-querier-tls\") pod \"thanos-querier-7686f988d4-slqm5\" (UID: \"ec8c4c90-e728-48af-9caf-0e11b18e85db\") " pod="openshift-monitoring/thanos-querier-7686f988d4-slqm5"
Sep 15 20:09:39.394708 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.394702    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77qbj\" (UniqueName: \"kubernetes.io/projected/ec8c4c90-e728-48af-9caf-0e11b18e85db-kube-api-access-77qbj\") pod \"thanos-querier-7686f988d4-slqm5\" (UID: \"ec8c4c90-e728-48af-9caf-0e11b18e85db\") " pod="openshift-monitoring/thanos-querier-7686f988d4-slqm5"
Sep 15 20:09:39.394960 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.394754    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ec8c4c90-e728-48af-9caf-0e11b18e85db-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7686f988d4-slqm5\" (UID: \"ec8c4c90-e728-48af-9caf-0e11b18e85db\") " pod="openshift-monitoring/thanos-querier-7686f988d4-slqm5"
Sep 15 20:09:39.394960 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.394797    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ec8c4c90-e728-48af-9caf-0e11b18e85db-metrics-client-ca\") pod \"thanos-querier-7686f988d4-slqm5\" (UID: \"ec8c4c90-e728-48af-9caf-0e11b18e85db\") " pod="openshift-monitoring/thanos-querier-7686f988d4-slqm5"
Sep 15 20:09:39.394960 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.394865    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ec8c4c90-e728-48af-9caf-0e11b18e85db-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7686f988d4-slqm5\" (UID: \"ec8c4c90-e728-48af-9caf-0e11b18e85db\") " pod="openshift-monitoring/thanos-querier-7686f988d4-slqm5"
Sep 15 20:09:39.496879 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.496555    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ec8c4c90-e728-48af-9caf-0e11b18e85db-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7686f988d4-slqm5\" (UID: \"ec8c4c90-e728-48af-9caf-0e11b18e85db\") " pod="openshift-monitoring/thanos-querier-7686f988d4-slqm5"
Sep 15 20:09:39.496879 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.496618    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ec8c4c90-e728-48af-9caf-0e11b18e85db-metrics-client-ca\") pod \"thanos-querier-7686f988d4-slqm5\" (UID: \"ec8c4c90-e728-48af-9caf-0e11b18e85db\") " pod="openshift-monitoring/thanos-querier-7686f988d4-slqm5"
Sep 15 20:09:39.496879 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.496672    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ec8c4c90-e728-48af-9caf-0e11b18e85db-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7686f988d4-slqm5\" (UID: \"ec8c4c90-e728-48af-9caf-0e11b18e85db\") " pod="openshift-monitoring/thanos-querier-7686f988d4-slqm5"
Sep 15 20:09:39.496879 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.496698    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ec8c4c90-e728-48af-9caf-0e11b18e85db-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7686f988d4-slqm5\" (UID: \"ec8c4c90-e728-48af-9caf-0e11b18e85db\") " pod="openshift-monitoring/thanos-querier-7686f988d4-slqm5"
Sep 15 20:09:39.496879 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.496772    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ec8c4c90-e728-48af-9caf-0e11b18e85db-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7686f988d4-slqm5\" (UID: \"ec8c4c90-e728-48af-9caf-0e11b18e85db\") " pod="openshift-monitoring/thanos-querier-7686f988d4-slqm5"
Sep 15 20:09:39.496879 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.496829    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ec8c4c90-e728-48af-9caf-0e11b18e85db-secret-grpc-tls\") pod \"thanos-querier-7686f988d4-slqm5\" (UID: \"ec8c4c90-e728-48af-9caf-0e11b18e85db\") " pod="openshift-monitoring/thanos-querier-7686f988d4-slqm5"
Sep 15 20:09:39.496879 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.496882    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ec8c4c90-e728-48af-9caf-0e11b18e85db-secret-thanos-querier-tls\") pod \"thanos-querier-7686f988d4-slqm5\" (UID: \"ec8c4c90-e728-48af-9caf-0e11b18e85db\") " pod="openshift-monitoring/thanos-querier-7686f988d4-slqm5"
Sep 15 20:09:39.497328 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.496929    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77qbj\" (UniqueName: \"kubernetes.io/projected/ec8c4c90-e728-48af-9caf-0e11b18e85db-kube-api-access-77qbj\") pod \"thanos-querier-7686f988d4-slqm5\" (UID: \"ec8c4c90-e728-48af-9caf-0e11b18e85db\") " pod="openshift-monitoring/thanos-querier-7686f988d4-slqm5"
Sep 15 20:09:39.498869 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.498018    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ec8c4c90-e728-48af-9caf-0e11b18e85db-metrics-client-ca\") pod \"thanos-querier-7686f988d4-slqm5\" (UID: \"ec8c4c90-e728-48af-9caf-0e11b18e85db\") " pod="openshift-monitoring/thanos-querier-7686f988d4-slqm5"
Sep 15 20:09:39.506445 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.506342    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77qbj\" (UniqueName: \"kubernetes.io/projected/ec8c4c90-e728-48af-9caf-0e11b18e85db-kube-api-access-77qbj\") pod \"thanos-querier-7686f988d4-slqm5\" (UID: \"ec8c4c90-e728-48af-9caf-0e11b18e85db\") " pod="openshift-monitoring/thanos-querier-7686f988d4-slqm5"
Sep 15 20:09:39.506445 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.506428    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ec8c4c90-e728-48af-9caf-0e11b18e85db-secret-thanos-querier-tls\") pod \"thanos-querier-7686f988d4-slqm5\" (UID: \"ec8c4c90-e728-48af-9caf-0e11b18e85db\") " pod="openshift-monitoring/thanos-querier-7686f988d4-slqm5"
Sep 15 20:09:39.507427 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.506894    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ec8c4c90-e728-48af-9caf-0e11b18e85db-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7686f988d4-slqm5\" (UID: \"ec8c4c90-e728-48af-9caf-0e11b18e85db\") " pod="openshift-monitoring/thanos-querier-7686f988d4-slqm5"
Sep 15 20:09:39.507427 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.507217    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ec8c4c90-e728-48af-9caf-0e11b18e85db-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7686f988d4-slqm5\" (UID: \"ec8c4c90-e728-48af-9caf-0e11b18e85db\") " pod="openshift-monitoring/thanos-querier-7686f988d4-slqm5"
Sep 15 20:09:39.507427 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.507357    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ec8c4c90-e728-48af-9caf-0e11b18e85db-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7686f988d4-slqm5\" (UID: \"ec8c4c90-e728-48af-9caf-0e11b18e85db\") " pod="openshift-monitoring/thanos-querier-7686f988d4-slqm5"
Sep 15 20:09:39.507943 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.507903    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ec8c4c90-e728-48af-9caf-0e11b18e85db-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7686f988d4-slqm5\" (UID: \"ec8c4c90-e728-48af-9caf-0e11b18e85db\") " pod="openshift-monitoring/thanos-querier-7686f988d4-slqm5"
Sep 15 20:09:39.508774 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.508752    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ec8c4c90-e728-48af-9caf-0e11b18e85db-secret-grpc-tls\") pod \"thanos-querier-7686f988d4-slqm5\" (UID: \"ec8c4c90-e728-48af-9caf-0e11b18e85db\") " pod="openshift-monitoring/thanos-querier-7686f988d4-slqm5"
Sep 15 20:09:39.598284 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.598187    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7686f988d4-slqm5"
Sep 15 20:09:39.728676 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.728645    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-h67tx"]
Sep 15 20:09:39.765127 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:09:39.765053    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5360f750_45a7_42d6_8b65_be039f6ee235.slice/crio-388a1d59f9f3459dc5d0419e45f00613f00aec67f043085953c82797cec41ce2 WatchSource:0}: Error finding container 388a1d59f9f3459dc5d0419e45f00613f00aec67f043085953c82797cec41ce2: Status 404 returned error can't find the container with id 388a1d59f9f3459dc5d0419e45f00613f00aec67f043085953c82797cec41ce2
Sep 15 20:09:39.971096 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.971056    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h67tx" event={"ID":"5360f750-45a7-42d6-8b65-be039f6ee235","Type":"ContainerStarted","Data":"388a1d59f9f3459dc5d0419e45f00613f00aec67f043085953c82797cec41ce2"}
Sep 15 20:09:39.972624 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.972592    2577 generic.go:356] "Generic (PLEG): container finished" podID="605e5608-f132-43d8-a691-c56f2f342c9c" containerID="afb5b645cfed23c8d97ada0b036121a3da47401143534c0706b1d8eff1ffd77f" exitCode=0
Sep 15 20:09:39.972760 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.972675    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cnlkd" event={"ID":"605e5608-f132-43d8-a691-c56f2f342c9c","Type":"ContainerDied","Data":"afb5b645cfed23c8d97ada0b036121a3da47401143534c0706b1d8eff1ffd77f"}
Sep 15 20:09:39.974344 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.974318    2577 generic.go:356] "Generic (PLEG): container finished" podID="e94af346-ecec-410c-8144-f476cb7d77d2" containerID="659f55e21f1157e8db85745e7f4218fa2b65ff00f1b55fd5c53da781e17d706d" exitCode=0
Sep 15 20:09:39.974451 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:39.974400    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e94af346-ecec-410c-8144-f476cb7d77d2","Type":"ContainerDied","Data":"659f55e21f1157e8db85745e7f4218fa2b65ff00f1b55fd5c53da781e17d706d"}
Sep 15 20:09:40.043694 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:40.043633    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7686f988d4-slqm5"]
Sep 15 20:09:40.046095 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:09:40.046069    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec8c4c90_e728_48af_9caf_0e11b18e85db.slice/crio-d022345a8e3941701012f11fae0e8e43b14bf2fba263ac0155f5a36e853685f9 WatchSource:0}: Error finding container d022345a8e3941701012f11fae0e8e43b14bf2fba263ac0155f5a36e853685f9: Status 404 returned error can't find the container with id d022345a8e3941701012f11fae0e8e43b14bf2fba263ac0155f5a36e853685f9
Sep 15 20:09:40.803376 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:40.803326    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-599ffb4745-n48qx"]
Sep 15 20:09:40.807032 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:40.807003    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-599ffb4745-n48qx"
Sep 15 20:09:40.810580 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:40.809945    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 15 20:09:40.811029 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:40.811007    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 15 20:09:40.811255 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:40.811239    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-pmfbd\""
Sep 15 20:09:40.811486 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:40.811469    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 15 20:09:40.811694 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:40.811677    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 15 20:09:40.812419 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:40.812399    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-fauqiipn97lc0\""
Sep 15 20:09:40.818015 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:40.817879    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-599ffb4745-n48qx"]
Sep 15 20:09:40.914447 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:40.914412    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/fb2a17c4-0ab8-47ec-ad13-694ca3e2c537-audit-log\") pod \"metrics-server-599ffb4745-n48qx\" (UID: \"fb2a17c4-0ab8-47ec-ad13-694ca3e2c537\") " pod="openshift-monitoring/metrics-server-599ffb4745-n48qx"
Sep 15 20:09:40.914756 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:40.914474    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb2a17c4-0ab8-47ec-ad13-694ca3e2c537-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-599ffb4745-n48qx\" (UID: \"fb2a17c4-0ab8-47ec-ad13-694ca3e2c537\") " pod="openshift-monitoring/metrics-server-599ffb4745-n48qx"
Sep 15 20:09:40.914756 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:40.914509    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/fb2a17c4-0ab8-47ec-ad13-694ca3e2c537-secret-metrics-server-client-certs\") pod \"metrics-server-599ffb4745-n48qx\" (UID: \"fb2a17c4-0ab8-47ec-ad13-694ca3e2c537\") " pod="openshift-monitoring/metrics-server-599ffb4745-n48qx"
Sep 15 20:09:40.914756 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:40.914588    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fb2a17c4-0ab8-47ec-ad13-694ca3e2c537-client-ca-bundle\") pod \"metrics-server-599ffb4745-n48qx\" (UID: \"fb2a17c4-0ab8-47ec-ad13-694ca3e2c537\") " pod="openshift-monitoring/metrics-server-599ffb4745-n48qx"
Sep 15 20:09:40.914756 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:40.914625    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/fb2a17c4-0ab8-47ec-ad13-694ca3e2c537-metrics-server-audit-profiles\") pod \"metrics-server-599ffb4745-n48qx\" (UID: \"fb2a17c4-0ab8-47ec-ad13-694ca3e2c537\") " pod="openshift-monitoring/metrics-server-599ffb4745-n48qx"
Sep 15 20:09:40.914756 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:40.914653    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/fb2a17c4-0ab8-47ec-ad13-694ca3e2c537-secret-metrics-server-tls\") pod \"metrics-server-599ffb4745-n48qx\" (UID: \"fb2a17c4-0ab8-47ec-ad13-694ca3e2c537\") " pod="openshift-monitoring/metrics-server-599ffb4745-n48qx"
Sep 15 20:09:40.915048 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:40.914750    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22jzg\" (UniqueName: \"kubernetes.io/projected/fb2a17c4-0ab8-47ec-ad13-694ca3e2c537-kube-api-access-22jzg\") pod \"metrics-server-599ffb4745-n48qx\" (UID: \"fb2a17c4-0ab8-47ec-ad13-694ca3e2c537\") " pod="openshift-monitoring/metrics-server-599ffb4745-n48qx"
Sep 15 20:09:40.981389 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:40.981267    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7686f988d4-slqm5" event={"ID":"ec8c4c90-e728-48af-9caf-0e11b18e85db","Type":"ContainerStarted","Data":"d022345a8e3941701012f11fae0e8e43b14bf2fba263ac0155f5a36e853685f9"}
Sep 15 20:09:40.985874 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:40.985841    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cnlkd" event={"ID":"605e5608-f132-43d8-a691-c56f2f342c9c","Type":"ContainerStarted","Data":"252412ff2e504f710fecfe86b43a8c5118ea0f4eb327f86296d60b64f2cfcf74"}
Sep 15 20:09:40.986015 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:40.985954    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cnlkd" event={"ID":"605e5608-f132-43d8-a691-c56f2f342c9c","Type":"ContainerStarted","Data":"f5e413e217eb86041a6cc16d82c8a360d2f01cfae26f6d0a1535e8efc3f24f09"}
Sep 15 20:09:41.015331 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:41.015294    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/fb2a17c4-0ab8-47ec-ad13-694ca3e2c537-audit-log\") pod \"metrics-server-599ffb4745-n48qx\" (UID: \"fb2a17c4-0ab8-47ec-ad13-694ca3e2c537\") " pod="openshift-monitoring/metrics-server-599ffb4745-n48qx"
Sep 15 20:09:41.015510 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:41.015358    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb2a17c4-0ab8-47ec-ad13-694ca3e2c537-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-599ffb4745-n48qx\" (UID: \"fb2a17c4-0ab8-47ec-ad13-694ca3e2c537\") " pod="openshift-monitoring/metrics-server-599ffb4745-n48qx"
Sep 15 20:09:41.015510 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:41.015407    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/fb2a17c4-0ab8-47ec-ad13-694ca3e2c537-secret-metrics-server-client-certs\") pod \"metrics-server-599ffb4745-n48qx\" (UID: \"fb2a17c4-0ab8-47ec-ad13-694ca3e2c537\") " pod="openshift-monitoring/metrics-server-599ffb4745-n48qx"
Sep 15 20:09:41.015510 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:41.015494    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fb2a17c4-0ab8-47ec-ad13-694ca3e2c537-client-ca-bundle\") pod \"metrics-server-599ffb4745-n48qx\" (UID: \"fb2a17c4-0ab8-47ec-ad13-694ca3e2c537\") " pod="openshift-monitoring/metrics-server-599ffb4745-n48qx"
Sep 15 20:09:41.015680 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:41.015532    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/fb2a17c4-0ab8-47ec-ad13-694ca3e2c537-metrics-server-audit-profiles\") pod \"metrics-server-599ffb4745-n48qx\" (UID: \"fb2a17c4-0ab8-47ec-ad13-694ca3e2c537\") " pod="openshift-monitoring/metrics-server-599ffb4745-n48qx"
Sep 15 20:09:41.015680 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:41.015562    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/fb2a17c4-0ab8-47ec-ad13-694ca3e2c537-secret-metrics-server-tls\") pod \"metrics-server-599ffb4745-n48qx\" (UID: \"fb2a17c4-0ab8-47ec-ad13-694ca3e2c537\") " pod="openshift-monitoring/metrics-server-599ffb4745-n48qx"
Sep 15 20:09:41.015680 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:41.015635    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22jzg\" (UniqueName: \"kubernetes.io/projected/fb2a17c4-0ab8-47ec-ad13-694ca3e2c537-kube-api-access-22jzg\") pod \"metrics-server-599ffb4745-n48qx\" (UID: \"fb2a17c4-0ab8-47ec-ad13-694ca3e2c537\") " pod="openshift-monitoring/metrics-server-599ffb4745-n48qx"
Sep 15 20:09:41.017492 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:41.017464    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/fb2a17c4-0ab8-47ec-ad13-694ca3e2c537-audit-log\") pod \"metrics-server-599ffb4745-n48qx\" (UID: \"fb2a17c4-0ab8-47ec-ad13-694ca3e2c537\") " pod="openshift-monitoring/metrics-server-599ffb4745-n48qx"
Sep 15 20:09:41.018514 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:41.018485    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb2a17c4-0ab8-47ec-ad13-694ca3e2c537-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-599ffb4745-n48qx\" (UID: \"fb2a17c4-0ab8-47ec-ad13-694ca3e2c537\") " pod="openshift-monitoring/metrics-server-599ffb4745-n48qx"
Sep 15 20:09:41.018948 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:41.018886    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/fb2a17c4-0ab8-47ec-ad13-694ca3e2c537-metrics-server-audit-profiles\") pod \"metrics-server-599ffb4745-n48qx\" (UID: \"fb2a17c4-0ab8-47ec-ad13-694ca3e2c537\") " pod="openshift-monitoring/metrics-server-599ffb4745-n48qx"
Sep 15 20:09:41.021283 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:41.021237    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/fb2a17c4-0ab8-47ec-ad13-694ca3e2c537-secret-metrics-server-client-certs\") pod \"metrics-server-599ffb4745-n48qx\" (UID: \"fb2a17c4-0ab8-47ec-ad13-694ca3e2c537\") " pod="openshift-monitoring/metrics-server-599ffb4745-n48qx"
Sep 15 20:09:41.021930 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:41.021907    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fb2a17c4-0ab8-47ec-ad13-694ca3e2c537-client-ca-bundle\") pod \"metrics-server-599ffb4745-n48qx\" (UID: \"fb2a17c4-0ab8-47ec-ad13-694ca3e2c537\") " pod="openshift-monitoring/metrics-server-599ffb4745-n48qx"
Sep 15 20:09:41.022510 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:41.022487    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/fb2a17c4-0ab8-47ec-ad13-694ca3e2c537-secret-metrics-server-tls\") pod \"metrics-server-599ffb4745-n48qx\" (UID: \"fb2a17c4-0ab8-47ec-ad13-694ca3e2c537\") " pod="openshift-monitoring/metrics-server-599ffb4745-n48qx"
Sep 15 20:09:41.029709 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:41.029606    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-cnlkd" podStartSLOduration=2.67459806 podStartE2EDuration="5.029569256s" totalImagesPullingTime="2.283855144s" totalInitContainerRuntime="71.116052ms" isStatefulPod=true podCreationTimestamp="2026-09-15 20:09:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:09:41.02913792 +0000 UTC m=+287.420090920" watchObservedRunningTime="2026-09-15 20:09:41.029569256 +0000 UTC m=+287.420522254"
Sep 15 20:09:41.060837 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:41.060748    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22jzg\" (UniqueName: \"kubernetes.io/projected/fb2a17c4-0ab8-47ec-ad13-694ca3e2c537-kube-api-access-22jzg\") pod \"metrics-server-599ffb4745-n48qx\" (UID: \"fb2a17c4-0ab8-47ec-ad13-694ca3e2c537\") " pod="openshift-monitoring/metrics-server-599ffb4745-n48qx"
Sep 15 20:09:41.123115 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:41.123081    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-599ffb4745-n48qx"
Sep 15 20:09:41.223989 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:41.223942    2577 patch_prober.go:28] interesting pod/image-registry-76d959cf57-r9gz6 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 15 20:09:41.224211 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:41.224006    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-76d959cf57-r9gz6" podUID="45f245cb-4670-40a8-9da8-252c030c8c8e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 20:09:41.591313 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:41.591269    2577 patch_prober.go:28] interesting pod/image-registry-6d4544bc7f-64cxr container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 15 20:09:41.591498 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:41.591335    2577 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6d4544bc7f-64cxr" podUID="5d9260ed-5dcc-488e-9d13-a49675580c16" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 20:09:42.675763 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:42.675718    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-599ffb4745-n48qx"]
Sep 15 20:09:42.679752 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:09:42.679703    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb2a17c4_0ab8_47ec_ad13_694ca3e2c537.slice/crio-1a5524b5577bd0d421446d1da973a1a342163aa5d1fd48eeafe7ec70aeed131d WatchSource:0}: Error finding container 1a5524b5577bd0d421446d1da973a1a342163aa5d1fd48eeafe7ec70aeed131d: Status 404 returned error can't find the container with id 1a5524b5577bd0d421446d1da973a1a342163aa5d1fd48eeafe7ec70aeed131d
Sep 15 20:09:42.996769 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:42.996717    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h67tx" event={"ID":"5360f750-45a7-42d6-8b65-be039f6ee235","Type":"ContainerStarted","Data":"41b7ca7b662a06fcfd43bf5fc2c5231f3e04aae8a4877cc183640ea82459004a"}
Sep 15 20:09:42.996962 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:42.996798    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h67tx" event={"ID":"5360f750-45a7-42d6-8b65-be039f6ee235","Type":"ContainerStarted","Data":"e4359760218f86a58618bcf1758303be1e58d0ad9bd55ff2b2162834a9701ddb"}
Sep 15 20:09:42.996962 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:42.996867    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-h67tx"
Sep 15 20:09:42.998066 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:42.998044    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-599ffb4745-n48qx" event={"ID":"fb2a17c4-0ab8-47ec-ad13-694ca3e2c537","Type":"ContainerStarted","Data":"1a5524b5577bd0d421446d1da973a1a342163aa5d1fd48eeafe7ec70aeed131d"}
Sep 15 20:09:43.001330 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:43.001310    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e94af346-ecec-410c-8144-f476cb7d77d2","Type":"ContainerStarted","Data":"570abc5cda96b60e043cf444fd970ada3ddbcbebf56b166783b6a71e4d6170fe"}
Sep 15 20:09:43.001433 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:43.001362    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e94af346-ecec-410c-8144-f476cb7d77d2","Type":"ContainerStarted","Data":"ab2c1377a2b9628488919c567d9f01c999ddfdbe8ab86e4f33553b66495b9b67"}
Sep 15 20:09:43.001433 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:43.001395    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e94af346-ecec-410c-8144-f476cb7d77d2","Type":"ContainerStarted","Data":"2cd63a99cefdc575f85064414c6d661e42ff773a98993f0d2789f18119802e0a"}
Sep 15 20:09:43.001548 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:43.001433    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e94af346-ecec-410c-8144-f476cb7d77d2","Type":"ContainerStarted","Data":"6a115174f8cd6e398a67850f51dc9273e57f9236987e5a55c597b0aaabb237a1"}
Sep 15 20:09:43.001548 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:43.001468    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e94af346-ecec-410c-8144-f476cb7d77d2","Type":"ContainerStarted","Data":"e851b66a2251782befc506d13126a4809caa747fa8de4eaf338f56c95ef1d54c"}
Sep 15 20:09:43.003565 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:43.003545    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7686f988d4-slqm5" event={"ID":"ec8c4c90-e728-48af-9caf-0e11b18e85db","Type":"ContainerStarted","Data":"9985c6fada62844c939783f0cb97c73e2c74b5610a67d410b88dd410fe304703"}
Sep 15 20:09:43.003651 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:43.003592    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7686f988d4-slqm5" event={"ID":"ec8c4c90-e728-48af-9caf-0e11b18e85db","Type":"ContainerStarted","Data":"96d3382e1d5b91be0abd2523592015a28f4e097711ff4c7290b9816583b5b1bd"}
Sep 15 20:09:43.003651 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:43.003622    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7686f988d4-slqm5" event={"ID":"ec8c4c90-e728-48af-9caf-0e11b18e85db","Type":"ContainerStarted","Data":"a5eb51150aad6801d4f267bdee897b2e8d684e564243b7553494277285af72d9"}
Sep 15 20:09:43.019538 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:43.019490    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-h67tx" podStartSLOduration=252.393565668 podStartE2EDuration="4m15.019473333s" totalImagesPullingTime="2.625907665s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:05:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:09:43.017941344 +0000 UTC m=+289.408894354" watchObservedRunningTime="2026-09-15 20:09:43.019473333 +0000 UTC m=+289.410426332"
Sep 15 20:09:43.909485 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:43.909441    2577 patch_prober.go:28] interesting pod/image-registry-6d4544bc7f-64cxr container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 15 20:09:43.909899 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:43.909510    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6d4544bc7f-64cxr" podUID="5d9260ed-5dcc-488e-9d13-a49675580c16" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 20:09:44.971137 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:44.971087    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-696cd69c49-76fh6"
Sep 15 20:09:44.971534 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:44.971166    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-696cd69c49-76fh6"
Sep 15 20:09:44.978848 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:44.978825    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-696cd69c49-76fh6"
Sep 15 20:09:45.016935 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:45.016896    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7686f988d4-slqm5" event={"ID":"ec8c4c90-e728-48af-9caf-0e11b18e85db","Type":"ContainerStarted","Data":"85be0c612a79ac8a3d287a567244399fa50adc265b2983e4e346dab73bd30234"}
Sep 15 20:09:45.017117 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:45.016991    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7686f988d4-slqm5"
Sep 15 20:09:45.017117 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:45.017036    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7686f988d4-slqm5" event={"ID":"ec8c4c90-e728-48af-9caf-0e11b18e85db","Type":"ContainerStarted","Data":"5421c73650235c5fd24a6cfdb15121de9bc3d688f2433fb39fd7cf1ebe7b3de3"}
Sep 15 20:09:45.017117 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:45.017077    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7686f988d4-slqm5" event={"ID":"ec8c4c90-e728-48af-9caf-0e11b18e85db","Type":"ContainerStarted","Data":"a23cd1d56ecb92e18a120d24832519480a79cd96df5703b8397a91d41f80abbc"}
Sep 15 20:09:45.018600 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:45.018567    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-599ffb4745-n48qx" event={"ID":"fb2a17c4-0ab8-47ec-ad13-694ca3e2c537","Type":"ContainerStarted","Data":"3ee48b7e1d39d95a6f0a80cbb1c26320f16abd88187f0c29e50619e0975ccf7d"}
Sep 15 20:09:45.022099 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:45.022071    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e94af346-ecec-410c-8144-f476cb7d77d2","Type":"ContainerStarted","Data":"2191b466827c49d42f6bc5ba4cb2a0cd7dd7ecf5f761a991a63322e42e212eea"}
Sep 15 20:09:45.026020 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:45.026000    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-696cd69c49-76fh6"
Sep 15 20:09:45.038779 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:45.038713    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7686f988d4-slqm5" podStartSLOduration=2.218437541 podStartE2EDuration="6.038700488s" totalImagesPullingTime="3.820262947s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:09:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:09:45.038125873 +0000 UTC m=+291.429078882" watchObservedRunningTime="2026-09-15 20:09:45.038700488 +0000 UTC m=+291.429653485"
Sep 15 20:09:45.055223 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:45.055184    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-599ffb4745-n48qx" podStartSLOduration=3.628954547 podStartE2EDuration="5.055170314s" totalImagesPullingTime="1.426215767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:09:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:09:45.054756 +0000 UTC m=+291.445708998" watchObservedRunningTime="2026-09-15 20:09:45.055170314 +0000 UTC m=+291.446123348"
Sep 15 20:09:45.102710 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:45.102660    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.252708294 podStartE2EDuration="8.102646093s" totalImagesPullingTime="4.830216933s" totalInitContainerRuntime="19.720866ms" isStatefulPod=false podCreationTimestamp="2026-09-15 20:09:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:09:45.10084061 +0000 UTC m=+291.491793607" watchObservedRunningTime="2026-09-15 20:09:45.102646093 +0000 UTC m=+291.493599090"
Sep 15 20:09:46.231833 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:46.231794    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-76d959cf57-r9gz6" podUID="45f245cb-4670-40a8-9da8-252c030c8c8e" containerName="registry" containerID="cri-o://738f5a1f89b35226f4a390cdd3efa4300bf0147f1a14298a8cc74f714e3c9367" gracePeriod=30
Sep 15 20:09:46.474826 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:46.474803    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76d959cf57-r9gz6"
Sep 15 20:09:46.568551 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:46.568472    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/45f245cb-4670-40a8-9da8-252c030c8c8e-registry-tls\") pod \"45f245cb-4670-40a8-9da8-252c030c8c8e\" (UID: \"45f245cb-4670-40a8-9da8-252c030c8c8e\") "
Sep 15 20:09:46.568551 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:46.568518    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/45f245cb-4670-40a8-9da8-252c030c8c8e-bound-sa-token\") pod \"45f245cb-4670-40a8-9da8-252c030c8c8e\" (UID: \"45f245cb-4670-40a8-9da8-252c030c8c8e\") "
Sep 15 20:09:46.568551 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:46.568545    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/45f245cb-4670-40a8-9da8-252c030c8c8e-trusted-ca\") pod \"45f245cb-4670-40a8-9da8-252c030c8c8e\" (UID: \"45f245cb-4670-40a8-9da8-252c030c8c8e\") "
Sep 15 20:09:46.568827 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:46.568667    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/45f245cb-4670-40a8-9da8-252c030c8c8e-ca-trust-extracted\") pod \"45f245cb-4670-40a8-9da8-252c030c8c8e\" (UID: \"45f245cb-4670-40a8-9da8-252c030c8c8e\") "
Sep 15 20:09:46.568827 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:46.568712    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/45f245cb-4670-40a8-9da8-252c030c8c8e-installation-pull-secrets\") pod \"45f245cb-4670-40a8-9da8-252c030c8c8e\" (UID: \"45f245cb-4670-40a8-9da8-252c030c8c8e\") "
Sep 15 20:09:46.568827 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:46.568768    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mfjnn\" (UniqueName: \"kubernetes.io/projected/45f245cb-4670-40a8-9da8-252c030c8c8e-kube-api-access-mfjnn\") pod \"45f245cb-4670-40a8-9da8-252c030c8c8e\" (UID: \"45f245cb-4670-40a8-9da8-252c030c8c8e\") "
Sep 15 20:09:46.568827 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:46.568813    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/45f245cb-4670-40a8-9da8-252c030c8c8e-registry-certificates\") pod \"45f245cb-4670-40a8-9da8-252c030c8c8e\" (UID: \"45f245cb-4670-40a8-9da8-252c030c8c8e\") "
Sep 15 20:09:46.569043 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:46.568846    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/45f245cb-4670-40a8-9da8-252c030c8c8e-image-registry-private-configuration\") pod \"45f245cb-4670-40a8-9da8-252c030c8c8e\" (UID: \"45f245cb-4670-40a8-9da8-252c030c8c8e\") "
Sep 15 20:09:46.569043 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:46.569017    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45f245cb-4670-40a8-9da8-252c030c8c8e-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "45f245cb-4670-40a8-9da8-252c030c8c8e" (UID: "45f245cb-4670-40a8-9da8-252c030c8c8e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:09:46.569197 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:46.569177    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/45f245cb-4670-40a8-9da8-252c030c8c8e-trusted-ca\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:09:46.569461 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:46.569421    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45f245cb-4670-40a8-9da8-252c030c8c8e-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "45f245cb-4670-40a8-9da8-252c030c8c8e" (UID: "45f245cb-4670-40a8-9da8-252c030c8c8e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:09:46.571097 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:46.571047    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45f245cb-4670-40a8-9da8-252c030c8c8e-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "45f245cb-4670-40a8-9da8-252c030c8c8e" (UID: "45f245cb-4670-40a8-9da8-252c030c8c8e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:09:46.571248 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:46.571224    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45f245cb-4670-40a8-9da8-252c030c8c8e-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "45f245cb-4670-40a8-9da8-252c030c8c8e" (UID: "45f245cb-4670-40a8-9da8-252c030c8c8e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:09:46.571388 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:46.571369    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45f245cb-4670-40a8-9da8-252c030c8c8e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "45f245cb-4670-40a8-9da8-252c030c8c8e" (UID: "45f245cb-4670-40a8-9da8-252c030c8c8e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:09:46.571469 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:46.571449    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45f245cb-4670-40a8-9da8-252c030c8c8e-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "45f245cb-4670-40a8-9da8-252c030c8c8e" (UID: "45f245cb-4670-40a8-9da8-252c030c8c8e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:09:46.571544 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:46.571471    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45f245cb-4670-40a8-9da8-252c030c8c8e-kube-api-access-mfjnn" (OuterVolumeSpecName: "kube-api-access-mfjnn") pod "45f245cb-4670-40a8-9da8-252c030c8c8e" (UID: "45f245cb-4670-40a8-9da8-252c030c8c8e"). InnerVolumeSpecName "kube-api-access-mfjnn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:09:46.576918 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:46.576893    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45f245cb-4670-40a8-9da8-252c030c8c8e-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "45f245cb-4670-40a8-9da8-252c030c8c8e" (UID: "45f245cb-4670-40a8-9da8-252c030c8c8e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:09:46.669961 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:46.669922    2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/45f245cb-4670-40a8-9da8-252c030c8c8e-ca-trust-extracted\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:09:46.669961 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:46.669953    2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/45f245cb-4670-40a8-9da8-252c030c8c8e-installation-pull-secrets\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:09:46.669961 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:46.669964    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfjnn\" (UniqueName: \"kubernetes.io/projected/45f245cb-4670-40a8-9da8-252c030c8c8e-kube-api-access-mfjnn\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:09:46.669961 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:46.669973    2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/45f245cb-4670-40a8-9da8-252c030c8c8e-registry-certificates\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:09:46.670212 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:46.669983    2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/45f245cb-4670-40a8-9da8-252c030c8c8e-image-registry-private-configuration\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:09:46.670212 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:46.669992    2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/45f245cb-4670-40a8-9da8-252c030c8c8e-registry-tls\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:09:46.670212 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:46.670001    2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/45f245cb-4670-40a8-9da8-252c030c8c8e-bound-sa-token\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:09:47.034286 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:47.034245    2577 generic.go:356] "Generic (PLEG): container finished" podID="45f245cb-4670-40a8-9da8-252c030c8c8e" containerID="738f5a1f89b35226f4a390cdd3efa4300bf0147f1a14298a8cc74f714e3c9367" exitCode=0
Sep 15 20:09:47.034286 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:47.034288    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76d959cf57-r9gz6"
Sep 15 20:09:47.034474 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:47.034307    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76d959cf57-r9gz6" event={"ID":"45f245cb-4670-40a8-9da8-252c030c8c8e","Type":"ContainerDied","Data":"738f5a1f89b35226f4a390cdd3efa4300bf0147f1a14298a8cc74f714e3c9367"}
Sep 15 20:09:47.034474 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:47.034352    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76d959cf57-r9gz6" event={"ID":"45f245cb-4670-40a8-9da8-252c030c8c8e","Type":"ContainerDied","Data":"456a3df06b3080ae5a03f1e42350edcdcc534d0521c72b9fe333d5e553b4eaa2"}
Sep 15 20:09:47.034474 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:47.034375    2577 scope.go:117] "RemoveContainer" containerID="738f5a1f89b35226f4a390cdd3efa4300bf0147f1a14298a8cc74f714e3c9367"
Sep 15 20:09:47.043397 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:47.043380    2577 scope.go:117] "RemoveContainer" containerID="738f5a1f89b35226f4a390cdd3efa4300bf0147f1a14298a8cc74f714e3c9367"
Sep 15 20:09:47.043679 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:09:47.043653    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"738f5a1f89b35226f4a390cdd3efa4300bf0147f1a14298a8cc74f714e3c9367\": container with ID starting with 738f5a1f89b35226f4a390cdd3efa4300bf0147f1a14298a8cc74f714e3c9367 not found: ID does not exist" containerID="738f5a1f89b35226f4a390cdd3efa4300bf0147f1a14298a8cc74f714e3c9367"
Sep 15 20:09:47.043719 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:47.043687    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"738f5a1f89b35226f4a390cdd3efa4300bf0147f1a14298a8cc74f714e3c9367"} err="failed to get container status \"738f5a1f89b35226f4a390cdd3efa4300bf0147f1a14298a8cc74f714e3c9367\": rpc error: code = NotFound desc = could not find container \"738f5a1f89b35226f4a390cdd3efa4300bf0147f1a14298a8cc74f714e3c9367\": container with ID starting with 738f5a1f89b35226f4a390cdd3efa4300bf0147f1a14298a8cc74f714e3c9367 not found: ID does not exist"
Sep 15 20:09:47.064843 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:47.064818    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-76d959cf57-r9gz6"]
Sep 15 20:09:47.074253 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:47.074229    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-76d959cf57-r9gz6"]
Sep 15 20:09:48.058225 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:48.058174    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45f245cb-4670-40a8-9da8-252c030c8c8e" path="/var/lib/kubelet/pods/45f245cb-4670-40a8-9da8-252c030c8c8e/volumes"
Sep 15 20:09:49.055424 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:09:49.055384    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-2v4rt" podUID="5482d254-b248-4eda-a7d8-37bcb3289d69"
Sep 15 20:09:51.034100 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:51.034069    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7686f988d4-slqm5"
Sep 15 20:09:51.588873 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:51.588838    2577 patch_prober.go:28] interesting pod/image-registry-6d4544bc7f-64cxr container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 15 20:09:51.589032 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:51.588901    2577 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6d4544bc7f-64cxr" podUID="5d9260ed-5dcc-488e-9d13-a49675580c16" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 20:09:52.102573 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:52.102533    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-696cd69c49-76fh6"]
Sep 15 20:09:53.012898 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:53.012861    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-h67tx"
Sep 15 20:09:53.909202 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:53.909170    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6d4544bc7f-64cxr"
Sep 15 20:09:53.994210 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:53.994179    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-r227b_ecab5774-79a4-42eb-9bb5-a1f09b18f3fb/console-operator/1.log"
Sep 15 20:09:53.994692 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:53.994667    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-r227b_ecab5774-79a4-42eb-9bb5-a1f09b18f3fb/console-operator/1.log"
Sep 15 20:09:54.007364 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:09:54.007336    2577 kubelet.go:1648] "Image garbage collection succeeded"
Sep 15 20:10:01.123676 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:01.123626    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-599ffb4745-n48qx"
Sep 15 20:10:01.126299 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:01.123698    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-599ffb4745-n48qx"
Sep 15 20:10:04.056097 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:04.056055    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v4rt"
Sep 15 20:10:07.115490 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:07.115454    2577 generic.go:356] "Generic (PLEG): container finished" podID="c567a4c3-d608-46e7-b93a-88ad3486e709" containerID="71cefdc030d1bdcb37ed5679b614af29b1254acbdd1f8971cf5721dfb4abb7fe" exitCode=0
Sep 15 20:10:07.115892 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:07.115512    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xsqd9" event={"ID":"c567a4c3-d608-46e7-b93a-88ad3486e709","Type":"ContainerDied","Data":"71cefdc030d1bdcb37ed5679b614af29b1254acbdd1f8971cf5721dfb4abb7fe"}
Sep 15 20:10:07.115892 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:07.115823    2577 scope.go:117] "RemoveContainer" containerID="71cefdc030d1bdcb37ed5679b614af29b1254acbdd1f8971cf5721dfb4abb7fe"
Sep 15 20:10:08.122037 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:08.122001    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xsqd9" event={"ID":"c567a4c3-d608-46e7-b93a-88ad3486e709","Type":"ContainerStarted","Data":"beeeff8809cf0f4fbb89ba77f33e8b24d21a152bb7b851d608d047b4728f03ec"}
Sep 15 20:10:17.122514 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:17.122470    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-696cd69c49-76fh6" podUID="eba45072-9a03-4268-a4d7-3c2ae1d7cf8b" containerName="console" containerID="cri-o://f29c6ec6cd86dec83e186c5c73f88e5346dc62f172c5753adb1ced6fe4cd066b" gracePeriod=15
Sep 15 20:10:17.369223 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:17.369186    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-696cd69c49-76fh6"
Sep 15 20:10:17.435871 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:17.435791    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-oauth-serving-cert\") pod \"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b\" (UID: \"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b\") "
Sep 15 20:10:17.435871 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:17.435831    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-console-oauth-config\") pod \"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b\" (UID: \"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b\") "
Sep 15 20:10:17.435871 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:17.435865    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjfpp\" (UniqueName: \"kubernetes.io/projected/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-kube-api-access-rjfpp\") pod \"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b\" (UID: \"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b\") "
Sep 15 20:10:17.436122 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:17.435884    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-service-ca\") pod \"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b\" (UID: \"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b\") "
Sep 15 20:10:17.436122 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:17.435934    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-trusted-ca-bundle\") pod \"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b\" (UID: \"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b\") "
Sep 15 20:10:17.436122 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:17.435974    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-console-config\") pod \"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b\" (UID: \"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b\") "
Sep 15 20:10:17.436122 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:17.436031    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-console-serving-cert\") pod \"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b\" (UID: \"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b\") "
Sep 15 20:10:17.436312 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:17.436235    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "eba45072-9a03-4268-a4d7-3c2ae1d7cf8b" (UID: "eba45072-9a03-4268-a4d7-3c2ae1d7cf8b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:10:17.436581 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:17.436463    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-console-config" (OuterVolumeSpecName: "console-config") pod "eba45072-9a03-4268-a4d7-3c2ae1d7cf8b" (UID: "eba45072-9a03-4268-a4d7-3c2ae1d7cf8b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:10:17.436710 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:17.436641    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-service-ca" (OuterVolumeSpecName: "service-ca") pod "eba45072-9a03-4268-a4d7-3c2ae1d7cf8b" (UID: "eba45072-9a03-4268-a4d7-3c2ae1d7cf8b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:10:17.436710 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:17.436674    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "eba45072-9a03-4268-a4d7-3c2ae1d7cf8b" (UID: "eba45072-9a03-4268-a4d7-3c2ae1d7cf8b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:10:17.438352 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:17.438321    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "eba45072-9a03-4268-a4d7-3c2ae1d7cf8b" (UID: "eba45072-9a03-4268-a4d7-3c2ae1d7cf8b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:10:17.438447 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:17.438354    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-kube-api-access-rjfpp" (OuterVolumeSpecName: "kube-api-access-rjfpp") pod "eba45072-9a03-4268-a4d7-3c2ae1d7cf8b" (UID: "eba45072-9a03-4268-a4d7-3c2ae1d7cf8b"). InnerVolumeSpecName "kube-api-access-rjfpp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:10:17.438629 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:17.438613    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "eba45072-9a03-4268-a4d7-3c2ae1d7cf8b" (UID: "eba45072-9a03-4268-a4d7-3c2ae1d7cf8b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:10:17.537469 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:17.537429    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-console-serving-cert\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:10:17.537469 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:17.537470    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-oauth-serving-cert\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:10:17.537654 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:17.537485    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-console-oauth-config\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:10:17.537654 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:17.537498    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjfpp\" (UniqueName: \"kubernetes.io/projected/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-kube-api-access-rjfpp\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:10:17.537654 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:17.537513    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-service-ca\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:10:17.537654 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:17.537527    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-trusted-ca-bundle\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:10:17.537654 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:17.537540    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b-console-config\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:10:18.162187 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:18.162153    2577 generic.go:356] "Generic (PLEG): container finished" podID="eba45072-9a03-4268-a4d7-3c2ae1d7cf8b" containerID="f29c6ec6cd86dec83e186c5c73f88e5346dc62f172c5753adb1ced6fe4cd066b" exitCode=0
Sep 15 20:10:18.162579 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:18.162199    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-696cd69c49-76fh6"
Sep 15 20:10:18.162579 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:18.162237    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-696cd69c49-76fh6" event={"ID":"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b","Type":"ContainerDied","Data":"f29c6ec6cd86dec83e186c5c73f88e5346dc62f172c5753adb1ced6fe4cd066b"}
Sep 15 20:10:18.162579 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:18.162289    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-696cd69c49-76fh6" event={"ID":"eba45072-9a03-4268-a4d7-3c2ae1d7cf8b","Type":"ContainerDied","Data":"9a069dff79b632117d91967a0c6a394cf33dd61686384c10ec7ce4f8b39b5490"}
Sep 15 20:10:18.162579 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:18.162311    2577 scope.go:117] "RemoveContainer" containerID="f29c6ec6cd86dec83e186c5c73f88e5346dc62f172c5753adb1ced6fe4cd066b"
Sep 15 20:10:18.171305 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:18.171287    2577 scope.go:117] "RemoveContainer" containerID="f29c6ec6cd86dec83e186c5c73f88e5346dc62f172c5753adb1ced6fe4cd066b"
Sep 15 20:10:18.171652 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:10:18.171622    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f29c6ec6cd86dec83e186c5c73f88e5346dc62f172c5753adb1ced6fe4cd066b\": container with ID starting with f29c6ec6cd86dec83e186c5c73f88e5346dc62f172c5753adb1ced6fe4cd066b not found: ID does not exist" containerID="f29c6ec6cd86dec83e186c5c73f88e5346dc62f172c5753adb1ced6fe4cd066b"
Sep 15 20:10:18.171779 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:18.171659    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f29c6ec6cd86dec83e186c5c73f88e5346dc62f172c5753adb1ced6fe4cd066b"} err="failed to get container status \"f29c6ec6cd86dec83e186c5c73f88e5346dc62f172c5753adb1ced6fe4cd066b\": rpc error: code = NotFound desc = could not find container \"f29c6ec6cd86dec83e186c5c73f88e5346dc62f172c5753adb1ced6fe4cd066b\": container with ID starting with f29c6ec6cd86dec83e186c5c73f88e5346dc62f172c5753adb1ced6fe4cd066b not found: ID does not exist"
Sep 15 20:10:18.180484 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:18.180460    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-696cd69c49-76fh6"]
Sep 15 20:10:18.185079 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:18.185055    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-696cd69c49-76fh6"]
Sep 15 20:10:20.063626 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:20.063594    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eba45072-9a03-4268-a4d7-3c2ae1d7cf8b" path="/var/lib/kubelet/pods/eba45072-9a03-4268-a4d7-3c2ae1d7cf8b/volumes"
Sep 15 20:10:21.129295 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:21.129263    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-599ffb4745-n48qx"
Sep 15 20:10:21.133030 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:21.133007    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-599ffb4745-n48qx"
Sep 15 20:10:56.647300 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:56.647267    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 15 20:10:56.647795 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:56.647568    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e94af346-ecec-410c-8144-f476cb7d77d2" containerName="alertmanager" containerID="cri-o://e851b66a2251782befc506d13126a4809caa747fa8de4eaf338f56c95ef1d54c" gracePeriod=120
Sep 15 20:10:56.647795 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:56.647624    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e94af346-ecec-410c-8144-f476cb7d77d2" containerName="kube-rbac-proxy-metric" containerID="cri-o://570abc5cda96b60e043cf444fd970ada3ddbcbebf56b166783b6a71e4d6170fe" gracePeriod=120
Sep 15 20:10:56.647795 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:56.647675    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e94af346-ecec-410c-8144-f476cb7d77d2" containerName="prom-label-proxy" containerID="cri-o://2191b466827c49d42f6bc5ba4cb2a0cd7dd7ecf5f761a991a63322e42e212eea" gracePeriod=120
Sep 15 20:10:56.647795 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:56.647652    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e94af346-ecec-410c-8144-f476cb7d77d2" containerName="kube-rbac-proxy" containerID="cri-o://ab2c1377a2b9628488919c567d9f01c999ddfdbe8ab86e4f33553b66495b9b67" gracePeriod=120
Sep 15 20:10:56.647795 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:56.647631    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e94af346-ecec-410c-8144-f476cb7d77d2" containerName="kube-rbac-proxy-web" containerID="cri-o://2cd63a99cefdc575f85064414c6d661e42ff773a98993f0d2789f18119802e0a" gracePeriod=120
Sep 15 20:10:56.647795 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:56.647695    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e94af346-ecec-410c-8144-f476cb7d77d2" containerName="config-reloader" containerID="cri-o://6a115174f8cd6e398a67850f51dc9273e57f9236987e5a55c597b0aaabb237a1" gracePeriod=120
Sep 15 20:10:57.312761 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.312709    2577 generic.go:356] "Generic (PLEG): container finished" podID="e94af346-ecec-410c-8144-f476cb7d77d2" containerID="2191b466827c49d42f6bc5ba4cb2a0cd7dd7ecf5f761a991a63322e42e212eea" exitCode=0
Sep 15 20:10:57.312761 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.312751    2577 generic.go:356] "Generic (PLEG): container finished" podID="e94af346-ecec-410c-8144-f476cb7d77d2" containerID="570abc5cda96b60e043cf444fd970ada3ddbcbebf56b166783b6a71e4d6170fe" exitCode=0
Sep 15 20:10:57.312761 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.312759    2577 generic.go:356] "Generic (PLEG): container finished" podID="e94af346-ecec-410c-8144-f476cb7d77d2" containerID="ab2c1377a2b9628488919c567d9f01c999ddfdbe8ab86e4f33553b66495b9b67" exitCode=0
Sep 15 20:10:57.312761 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.312755    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e94af346-ecec-410c-8144-f476cb7d77d2","Type":"ContainerDied","Data":"2191b466827c49d42f6bc5ba4cb2a0cd7dd7ecf5f761a991a63322e42e212eea"}
Sep 15 20:10:57.313021 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.312822    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e94af346-ecec-410c-8144-f476cb7d77d2","Type":"ContainerDied","Data":"570abc5cda96b60e043cf444fd970ada3ddbcbebf56b166783b6a71e4d6170fe"}
Sep 15 20:10:57.313021 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.312764    2577 generic.go:356] "Generic (PLEG): container finished" podID="e94af346-ecec-410c-8144-f476cb7d77d2" containerID="6a115174f8cd6e398a67850f51dc9273e57f9236987e5a55c597b0aaabb237a1" exitCode=0
Sep 15 20:10:57.313021 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.312847    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e94af346-ecec-410c-8144-f476cb7d77d2","Type":"ContainerDied","Data":"ab2c1377a2b9628488919c567d9f01c999ddfdbe8ab86e4f33553b66495b9b67"}
Sep 15 20:10:57.313021 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.312876    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e94af346-ecec-410c-8144-f476cb7d77d2","Type":"ContainerDied","Data":"6a115174f8cd6e398a67850f51dc9273e57f9236987e5a55c597b0aaabb237a1"}
Sep 15 20:10:57.313021 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.312899    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e94af346-ecec-410c-8144-f476cb7d77d2","Type":"ContainerDied","Data":"e851b66a2251782befc506d13126a4809caa747fa8de4eaf338f56c95ef1d54c"}
Sep 15 20:10:57.313021 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.312847    2577 generic.go:356] "Generic (PLEG): container finished" podID="e94af346-ecec-410c-8144-f476cb7d77d2" containerID="e851b66a2251782befc506d13126a4809caa747fa8de4eaf338f56c95ef1d54c" exitCode=0
Sep 15 20:10:57.892240 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.892212    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:57.977369 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.977287    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e94af346-ecec-410c-8144-f476cb7d77d2-metrics-client-ca\") pod \"e94af346-ecec-410c-8144-f476cb7d77d2\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") "
Sep 15 20:10:57.977369 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.977334    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-secret-alertmanager-kube-rbac-proxy\") pod \"e94af346-ecec-410c-8144-f476cb7d77d2\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") "
Sep 15 20:10:57.977369 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.977361    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e94af346-ecec-410c-8144-f476cb7d77d2-alertmanager-main-db\") pod \"e94af346-ecec-410c-8144-f476cb7d77d2\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") "
Sep 15 20:10:57.977589 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.977518    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e94af346-ecec-410c-8144-f476cb7d77d2-alertmanager-trusted-ca-bundle\") pod \"e94af346-ecec-410c-8144-f476cb7d77d2\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") "
Sep 15 20:10:57.977589 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.977555    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-web-config\") pod \"e94af346-ecec-410c-8144-f476cb7d77d2\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") "
Sep 15 20:10:57.977589 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.977581    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e94af346-ecec-410c-8144-f476cb7d77d2-config-out\") pod \"e94af346-ecec-410c-8144-f476cb7d77d2\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") "
Sep 15 20:10:57.977753 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.977608    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-secret-alertmanager-main-tls\") pod \"e94af346-ecec-410c-8144-f476cb7d77d2\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") "
Sep 15 20:10:57.977753 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.977643    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5xk7v\" (UniqueName: \"kubernetes.io/projected/e94af346-ecec-410c-8144-f476cb7d77d2-kube-api-access-5xk7v\") pod \"e94af346-ecec-410c-8144-f476cb7d77d2\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") "
Sep 15 20:10:57.977753 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.977671    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e94af346-ecec-410c-8144-f476cb7d77d2-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "e94af346-ecec-410c-8144-f476cb7d77d2" (UID: "e94af346-ecec-410c-8144-f476cb7d77d2"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:10:57.977753 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.977679    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-secret-alertmanager-kube-rbac-proxy-web\") pod \"e94af346-ecec-410c-8144-f476cb7d77d2\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") "
Sep 15 20:10:57.977753 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.977706    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"e94af346-ecec-410c-8144-f476cb7d77d2\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") "
Sep 15 20:10:57.977753 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.977712    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e94af346-ecec-410c-8144-f476cb7d77d2-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "e94af346-ecec-410c-8144-f476cb7d77d2" (UID: "e94af346-ecec-410c-8144-f476cb7d77d2"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:10:57.978795 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.977941    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e94af346-ecec-410c-8144-f476cb7d77d2-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "e94af346-ecec-410c-8144-f476cb7d77d2" (UID: "e94af346-ecec-410c-8144-f476cb7d77d2"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:10:57.978795 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.977761    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-cluster-tls-config\") pod \"e94af346-ecec-410c-8144-f476cb7d77d2\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") "
Sep 15 20:10:57.978795 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.978469    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e94af346-ecec-410c-8144-f476cb7d77d2-tls-assets\") pod \"e94af346-ecec-410c-8144-f476cb7d77d2\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") "
Sep 15 20:10:57.978795 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.978502    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-config-volume\") pod \"e94af346-ecec-410c-8144-f476cb7d77d2\" (UID: \"e94af346-ecec-410c-8144-f476cb7d77d2\") "
Sep 15 20:10:57.979098 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.978802    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e94af346-ecec-410c-8144-f476cb7d77d2-metrics-client-ca\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:10:57.979098 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.978821    2577 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e94af346-ecec-410c-8144-f476cb7d77d2-alertmanager-main-db\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:10:57.979098 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.978838    2577 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e94af346-ecec-410c-8144-f476cb7d77d2-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:10:57.980483 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.980445    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "e94af346-ecec-410c-8144-f476cb7d77d2" (UID: "e94af346-ecec-410c-8144-f476cb7d77d2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:10:57.981862 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.981823    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "e94af346-ecec-410c-8144-f476cb7d77d2" (UID: "e94af346-ecec-410c-8144-f476cb7d77d2"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:10:57.982217 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.982182    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e94af346-ecec-410c-8144-f476cb7d77d2-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "e94af346-ecec-410c-8144-f476cb7d77d2" (UID: "e94af346-ecec-410c-8144-f476cb7d77d2"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:10:57.982502 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.982463    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "e94af346-ecec-410c-8144-f476cb7d77d2" (UID: "e94af346-ecec-410c-8144-f476cb7d77d2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:10:57.982769 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.982710    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "e94af346-ecec-410c-8144-f476cb7d77d2" (UID: "e94af346-ecec-410c-8144-f476cb7d77d2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:10:57.982891 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.982782    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e94af346-ecec-410c-8144-f476cb7d77d2-config-out" (OuterVolumeSpecName: "config-out") pod "e94af346-ecec-410c-8144-f476cb7d77d2" (UID: "e94af346-ecec-410c-8144-f476cb7d77d2"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:10:57.983332 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.983310    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e94af346-ecec-410c-8144-f476cb7d77d2-kube-api-access-5xk7v" (OuterVolumeSpecName: "kube-api-access-5xk7v") pod "e94af346-ecec-410c-8144-f476cb7d77d2" (UID: "e94af346-ecec-410c-8144-f476cb7d77d2"). InnerVolumeSpecName "kube-api-access-5xk7v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:10:57.983415 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.983390    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-config-volume" (OuterVolumeSpecName: "config-volume") pod "e94af346-ecec-410c-8144-f476cb7d77d2" (UID: "e94af346-ecec-410c-8144-f476cb7d77d2"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:10:57.985254 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.985223    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "e94af346-ecec-410c-8144-f476cb7d77d2" (UID: "e94af346-ecec-410c-8144-f476cb7d77d2"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:10:57.992981 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:57.992951    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-web-config" (OuterVolumeSpecName: "web-config") pod "e94af346-ecec-410c-8144-f476cb7d77d2" (UID: "e94af346-ecec-410c-8144-f476cb7d77d2"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:10:58.079849 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.079817    2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:10:58.079849 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.079844    2577 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-web-config\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:10:58.079849 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.079854    2577 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e94af346-ecec-410c-8144-f476cb7d77d2-config-out\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:10:58.080045 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.079864    2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-secret-alertmanager-main-tls\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:10:58.080045 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.079873    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5xk7v\" (UniqueName: \"kubernetes.io/projected/e94af346-ecec-410c-8144-f476cb7d77d2-kube-api-access-5xk7v\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:10:58.080045 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.079882    2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:10:58.080045 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.079891    2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:10:58.080045 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.079902    2577 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-cluster-tls-config\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:10:58.080045 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.079910    2577 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e94af346-ecec-410c-8144-f476cb7d77d2-tls-assets\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:10:58.080045 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.079919    2577 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e94af346-ecec-410c-8144-f476cb7d77d2-config-volume\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:10:58.320164 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.320077    2577 generic.go:356] "Generic (PLEG): container finished" podID="e94af346-ecec-410c-8144-f476cb7d77d2" containerID="2cd63a99cefdc575f85064414c6d661e42ff773a98993f0d2789f18119802e0a" exitCode=0
Sep 15 20:10:58.320164 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.320150    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.320340 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.320159    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e94af346-ecec-410c-8144-f476cb7d77d2","Type":"ContainerDied","Data":"2cd63a99cefdc575f85064414c6d661e42ff773a98993f0d2789f18119802e0a"}
Sep 15 20:10:58.320340 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.320230    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e94af346-ecec-410c-8144-f476cb7d77d2","Type":"ContainerDied","Data":"b9a008507e0324b9d0182b7fb247edc7d460661a724ed5f2b5eefcb169a36dcf"}
Sep 15 20:10:58.320340 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.320264    2577 scope.go:117] "RemoveContainer" containerID="2191b466827c49d42f6bc5ba4cb2a0cd7dd7ecf5f761a991a63322e42e212eea"
Sep 15 20:10:58.328068 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.328050    2577 scope.go:117] "RemoveContainer" containerID="570abc5cda96b60e043cf444fd970ada3ddbcbebf56b166783b6a71e4d6170fe"
Sep 15 20:10:58.335364 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.335346    2577 scope.go:117] "RemoveContainer" containerID="ab2c1377a2b9628488919c567d9f01c999ddfdbe8ab86e4f33553b66495b9b67"
Sep 15 20:10:58.340278 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.340255    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 15 20:10:58.342562 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.342544    2577 scope.go:117] "RemoveContainer" containerID="2cd63a99cefdc575f85064414c6d661e42ff773a98993f0d2789f18119802e0a"
Sep 15 20:10:58.345518 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.345496    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 15 20:10:58.349744 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.349712    2577 scope.go:117] "RemoveContainer" containerID="6a115174f8cd6e398a67850f51dc9273e57f9236987e5a55c597b0aaabb237a1"
Sep 15 20:10:58.356709 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.356690    2577 scope.go:117] "RemoveContainer" containerID="e851b66a2251782befc506d13126a4809caa747fa8de4eaf338f56c95ef1d54c"
Sep 15 20:10:58.363477 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.363458    2577 scope.go:117] "RemoveContainer" containerID="659f55e21f1157e8db85745e7f4218fa2b65ff00f1b55fd5c53da781e17d706d"
Sep 15 20:10:58.370563 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.370541    2577 scope.go:117] "RemoveContainer" containerID="2191b466827c49d42f6bc5ba4cb2a0cd7dd7ecf5f761a991a63322e42e212eea"
Sep 15 20:10:58.370899 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:10:58.370870    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2191b466827c49d42f6bc5ba4cb2a0cd7dd7ecf5f761a991a63322e42e212eea\": container with ID starting with 2191b466827c49d42f6bc5ba4cb2a0cd7dd7ecf5f761a991a63322e42e212eea not found: ID does not exist" containerID="2191b466827c49d42f6bc5ba4cb2a0cd7dd7ecf5f761a991a63322e42e212eea"
Sep 15 20:10:58.370979 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.370901    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2191b466827c49d42f6bc5ba4cb2a0cd7dd7ecf5f761a991a63322e42e212eea"} err="failed to get container status \"2191b466827c49d42f6bc5ba4cb2a0cd7dd7ecf5f761a991a63322e42e212eea\": rpc error: code = NotFound desc = could not find container \"2191b466827c49d42f6bc5ba4cb2a0cd7dd7ecf5f761a991a63322e42e212eea\": container with ID starting with 2191b466827c49d42f6bc5ba4cb2a0cd7dd7ecf5f761a991a63322e42e212eea not found: ID does not exist"
Sep 15 20:10:58.370979 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.370923    2577 scope.go:117] "RemoveContainer" containerID="570abc5cda96b60e043cf444fd970ada3ddbcbebf56b166783b6a71e4d6170fe"
Sep 15 20:10:58.371189 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.371172    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 15 20:10:58.371267 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:10:58.371178    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"570abc5cda96b60e043cf444fd970ada3ddbcbebf56b166783b6a71e4d6170fe\": container with ID starting with 570abc5cda96b60e043cf444fd970ada3ddbcbebf56b166783b6a71e4d6170fe not found: ID does not exist" containerID="570abc5cda96b60e043cf444fd970ada3ddbcbebf56b166783b6a71e4d6170fe"
Sep 15 20:10:58.371304 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.371275    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"570abc5cda96b60e043cf444fd970ada3ddbcbebf56b166783b6a71e4d6170fe"} err="failed to get container status \"570abc5cda96b60e043cf444fd970ada3ddbcbebf56b166783b6a71e4d6170fe\": rpc error: code = NotFound desc = could not find container \"570abc5cda96b60e043cf444fd970ada3ddbcbebf56b166783b6a71e4d6170fe\": container with ID starting with 570abc5cda96b60e043cf444fd970ada3ddbcbebf56b166783b6a71e4d6170fe not found: ID does not exist"
Sep 15 20:10:58.371304 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.371291    2577 scope.go:117] "RemoveContainer" containerID="ab2c1377a2b9628488919c567d9f01c999ddfdbe8ab86e4f33553b66495b9b67"
Sep 15 20:10:58.371465 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.371449    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eba45072-9a03-4268-a4d7-3c2ae1d7cf8b" containerName="console"
Sep 15 20:10:58.371533 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.371467    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="eba45072-9a03-4268-a4d7-3c2ae1d7cf8b" containerName="console"
Sep 15 20:10:58.371533 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.371482    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e94af346-ecec-410c-8144-f476cb7d77d2" containerName="config-reloader"
Sep 15 20:10:58.371533 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.371489    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e94af346-ecec-410c-8144-f476cb7d77d2" containerName="config-reloader"
Sep 15 20:10:58.371533 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.371497    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e94af346-ecec-410c-8144-f476cb7d77d2" containerName="kube-rbac-proxy-web"
Sep 15 20:10:58.371533 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.371503    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e94af346-ecec-410c-8144-f476cb7d77d2" containerName="kube-rbac-proxy-web"
Sep 15 20:10:58.371533 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.371510    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e94af346-ecec-410c-8144-f476cb7d77d2" containerName="kube-rbac-proxy-metric"
Sep 15 20:10:58.371533 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.371515    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e94af346-ecec-410c-8144-f476cb7d77d2" containerName="kube-rbac-proxy-metric"
Sep 15 20:10:58.371533 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.371526    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e94af346-ecec-410c-8144-f476cb7d77d2" containerName="prom-label-proxy"
Sep 15 20:10:58.371533 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.371532    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e94af346-ecec-410c-8144-f476cb7d77d2" containerName="prom-label-proxy"
Sep 15 20:10:58.371842 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.371541    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45f245cb-4670-40a8-9da8-252c030c8c8e" containerName="registry"
Sep 15 20:10:58.371842 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.371546    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="45f245cb-4670-40a8-9da8-252c030c8c8e" containerName="registry"
Sep 15 20:10:58.371842 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.371551    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e94af346-ecec-410c-8144-f476cb7d77d2" containerName="init-config-reloader"
Sep 15 20:10:58.371842 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.371556    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e94af346-ecec-410c-8144-f476cb7d77d2" containerName="init-config-reloader"
Sep 15 20:10:58.371842 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.371565    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e94af346-ecec-410c-8144-f476cb7d77d2" containerName="alertmanager"
Sep 15 20:10:58.371842 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.371574    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e94af346-ecec-410c-8144-f476cb7d77d2" containerName="alertmanager"
Sep 15 20:10:58.371842 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.371586    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e94af346-ecec-410c-8144-f476cb7d77d2" containerName="kube-rbac-proxy"
Sep 15 20:10:58.371842 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:10:58.371583    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab2c1377a2b9628488919c567d9f01c999ddfdbe8ab86e4f33553b66495b9b67\": container with ID starting with ab2c1377a2b9628488919c567d9f01c999ddfdbe8ab86e4f33553b66495b9b67 not found: ID does not exist" containerID="ab2c1377a2b9628488919c567d9f01c999ddfdbe8ab86e4f33553b66495b9b67"
Sep 15 20:10:58.371842 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.371613    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab2c1377a2b9628488919c567d9f01c999ddfdbe8ab86e4f33553b66495b9b67"} err="failed to get container status \"ab2c1377a2b9628488919c567d9f01c999ddfdbe8ab86e4f33553b66495b9b67\": rpc error: code = NotFound desc = could not find container \"ab2c1377a2b9628488919c567d9f01c999ddfdbe8ab86e4f33553b66495b9b67\": container with ID starting with ab2c1377a2b9628488919c567d9f01c999ddfdbe8ab86e4f33553b66495b9b67 not found: ID does not exist"
Sep 15 20:10:58.371842 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.371629    2577 scope.go:117] "RemoveContainer" containerID="2cd63a99cefdc575f85064414c6d661e42ff773a98993f0d2789f18119802e0a"
Sep 15 20:10:58.371842 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.371591    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e94af346-ecec-410c-8144-f476cb7d77d2" containerName="kube-rbac-proxy"
Sep 15 20:10:58.371842 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.371765    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e94af346-ecec-410c-8144-f476cb7d77d2" containerName="alertmanager"
Sep 15 20:10:58.371842 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.371778    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e94af346-ecec-410c-8144-f476cb7d77d2" containerName="config-reloader"
Sep 15 20:10:58.371842 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.371789    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="45f245cb-4670-40a8-9da8-252c030c8c8e" containerName="registry"
Sep 15 20:10:58.371842 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.371798    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e94af346-ecec-410c-8144-f476cb7d77d2" containerName="kube-rbac-proxy"
Sep 15 20:10:58.371842 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.371807    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e94af346-ecec-410c-8144-f476cb7d77d2" containerName="kube-rbac-proxy-metric"
Sep 15 20:10:58.371842 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.371817    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="eba45072-9a03-4268-a4d7-3c2ae1d7cf8b" containerName="console"
Sep 15 20:10:58.371842 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.371826    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e94af346-ecec-410c-8144-f476cb7d77d2" containerName="kube-rbac-proxy-web"
Sep 15 20:10:58.371842 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.371836    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e94af346-ecec-410c-8144-f476cb7d77d2" containerName="prom-label-proxy"
Sep 15 20:10:58.372455 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:10:58.371885    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cd63a99cefdc575f85064414c6d661e42ff773a98993f0d2789f18119802e0a\": container with ID starting with 2cd63a99cefdc575f85064414c6d661e42ff773a98993f0d2789f18119802e0a not found: ID does not exist" containerID="2cd63a99cefdc575f85064414c6d661e42ff773a98993f0d2789f18119802e0a"
Sep 15 20:10:58.372455 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.371905    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cd63a99cefdc575f85064414c6d661e42ff773a98993f0d2789f18119802e0a"} err="failed to get container status \"2cd63a99cefdc575f85064414c6d661e42ff773a98993f0d2789f18119802e0a\": rpc error: code = NotFound desc = could not find container \"2cd63a99cefdc575f85064414c6d661e42ff773a98993f0d2789f18119802e0a\": container with ID starting with 2cd63a99cefdc575f85064414c6d661e42ff773a98993f0d2789f18119802e0a not found: ID does not exist"
Sep 15 20:10:58.372455 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.371922    2577 scope.go:117] "RemoveContainer" containerID="6a115174f8cd6e398a67850f51dc9273e57f9236987e5a55c597b0aaabb237a1"
Sep 15 20:10:58.372455 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:10:58.372140    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a115174f8cd6e398a67850f51dc9273e57f9236987e5a55c597b0aaabb237a1\": container with ID starting with 6a115174f8cd6e398a67850f51dc9273e57f9236987e5a55c597b0aaabb237a1 not found: ID does not exist" containerID="6a115174f8cd6e398a67850f51dc9273e57f9236987e5a55c597b0aaabb237a1"
Sep 15 20:10:58.372455 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.372158    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a115174f8cd6e398a67850f51dc9273e57f9236987e5a55c597b0aaabb237a1"} err="failed to get container status \"6a115174f8cd6e398a67850f51dc9273e57f9236987e5a55c597b0aaabb237a1\": rpc error: code = NotFound desc = could not find container \"6a115174f8cd6e398a67850f51dc9273e57f9236987e5a55c597b0aaabb237a1\": container with ID starting with 6a115174f8cd6e398a67850f51dc9273e57f9236987e5a55c597b0aaabb237a1 not found: ID does not exist"
Sep 15 20:10:58.372455 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.372169    2577 scope.go:117] "RemoveContainer" containerID="e851b66a2251782befc506d13126a4809caa747fa8de4eaf338f56c95ef1d54c"
Sep 15 20:10:58.372455 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:10:58.372410    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e851b66a2251782befc506d13126a4809caa747fa8de4eaf338f56c95ef1d54c\": container with ID starting with e851b66a2251782befc506d13126a4809caa747fa8de4eaf338f56c95ef1d54c not found: ID does not exist" containerID="e851b66a2251782befc506d13126a4809caa747fa8de4eaf338f56c95ef1d54c"
Sep 15 20:10:58.372455 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.372425    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e851b66a2251782befc506d13126a4809caa747fa8de4eaf338f56c95ef1d54c"} err="failed to get container status \"e851b66a2251782befc506d13126a4809caa747fa8de4eaf338f56c95ef1d54c\": rpc error: code = NotFound desc = could not find container \"e851b66a2251782befc506d13126a4809caa747fa8de4eaf338f56c95ef1d54c\": container with ID starting with e851b66a2251782befc506d13126a4809caa747fa8de4eaf338f56c95ef1d54c not found: ID does not exist"
Sep 15 20:10:58.372455 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.372438    2577 scope.go:117] "RemoveContainer" containerID="659f55e21f1157e8db85745e7f4218fa2b65ff00f1b55fd5c53da781e17d706d"
Sep 15 20:10:58.372828 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:10:58.372643    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"659f55e21f1157e8db85745e7f4218fa2b65ff00f1b55fd5c53da781e17d706d\": container with ID starting with 659f55e21f1157e8db85745e7f4218fa2b65ff00f1b55fd5c53da781e17d706d not found: ID does not exist" containerID="659f55e21f1157e8db85745e7f4218fa2b65ff00f1b55fd5c53da781e17d706d"
Sep 15 20:10:58.372828 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.372662    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"659f55e21f1157e8db85745e7f4218fa2b65ff00f1b55fd5c53da781e17d706d"} err="failed to get container status \"659f55e21f1157e8db85745e7f4218fa2b65ff00f1b55fd5c53da781e17d706d\": rpc error: code = NotFound desc = could not find container \"659f55e21f1157e8db85745e7f4218fa2b65ff00f1b55fd5c53da781e17d706d\": container with ID starting with 659f55e21f1157e8db85745e7f4218fa2b65ff00f1b55fd5c53da781e17d706d not found: ID does not exist"
Sep 15 20:10:58.375270 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.375255    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.378009 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.377990    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 15 20:10:58.378108 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.378059    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-xsvw7\""
Sep 15 20:10:58.378179 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.378159    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 15 20:10:58.378272 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.378257    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 15 20:10:58.378370 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.378354    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 15 20:10:58.378871 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.378856    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 15 20:10:58.378942 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.378927    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 15 20:10:58.379009 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.378932    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 15 20:10:58.379086 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.379070    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 15 20:10:58.389441 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.389415    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 15 20:10:58.391179 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.391157    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 15 20:10:58.485798 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.485749    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/93569234-ca8e-4e34-acc8-1d413f92b96e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.485798 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.485796    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/93569234-ca8e-4e34-acc8-1d413f92b96e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.486027 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.485821    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/93569234-ca8e-4e34-acc8-1d413f92b96e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.486027 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.485893    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/93569234-ca8e-4e34-acc8-1d413f92b96e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.486027 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.485936    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/93569234-ca8e-4e34-acc8-1d413f92b96e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.486027 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.485984    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2bnm\" (UniqueName: \"kubernetes.io/projected/93569234-ca8e-4e34-acc8-1d413f92b96e-kube-api-access-j2bnm\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.486027 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.486020    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/93569234-ca8e-4e34-acc8-1d413f92b96e-web-config\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.486187 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.486040    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/93569234-ca8e-4e34-acc8-1d413f92b96e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.486187 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.486067    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93569234-ca8e-4e34-acc8-1d413f92b96e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.486187 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.486133    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/93569234-ca8e-4e34-acc8-1d413f92b96e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.486187 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.486162    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/93569234-ca8e-4e34-acc8-1d413f92b96e-config-volume\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.486187 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.486176    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/93569234-ca8e-4e34-acc8-1d413f92b96e-config-out\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.486361 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.486189    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/93569234-ca8e-4e34-acc8-1d413f92b96e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.587674 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.587569    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/93569234-ca8e-4e34-acc8-1d413f92b96e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.587875 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.587674    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/93569234-ca8e-4e34-acc8-1d413f92b96e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.587875 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.587716    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/93569234-ca8e-4e34-acc8-1d413f92b96e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.587875 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.587779    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2bnm\" (UniqueName: \"kubernetes.io/projected/93569234-ca8e-4e34-acc8-1d413f92b96e-kube-api-access-j2bnm\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.587875 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.587810    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/93569234-ca8e-4e34-acc8-1d413f92b96e-web-config\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.587875 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.587827    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/93569234-ca8e-4e34-acc8-1d413f92b96e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.587875 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.587846    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93569234-ca8e-4e34-acc8-1d413f92b96e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.588185 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.587906    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/93569234-ca8e-4e34-acc8-1d413f92b96e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.588185 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.587951    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/93569234-ca8e-4e34-acc8-1d413f92b96e-config-volume\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.588185 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.587973    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/93569234-ca8e-4e34-acc8-1d413f92b96e-config-out\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.588185 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.587996    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/93569234-ca8e-4e34-acc8-1d413f92b96e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.588185 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.588033    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/93569234-ca8e-4e34-acc8-1d413f92b96e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.588185 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.588059    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/93569234-ca8e-4e34-acc8-1d413f92b96e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.588692 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.588664    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/93569234-ca8e-4e34-acc8-1d413f92b96e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.589309 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.589283    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93569234-ca8e-4e34-acc8-1d413f92b96e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.589440 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.589289    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/93569234-ca8e-4e34-acc8-1d413f92b96e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.590841 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.590802    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/93569234-ca8e-4e34-acc8-1d413f92b96e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.590949 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.590834    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/93569234-ca8e-4e34-acc8-1d413f92b96e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.591016 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.590960    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/93569234-ca8e-4e34-acc8-1d413f92b96e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.591283 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.591252    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/93569234-ca8e-4e34-acc8-1d413f92b96e-web-config\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.591283 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.591271    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/93569234-ca8e-4e34-acc8-1d413f92b96e-config-volume\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.591613 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.591592    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/93569234-ca8e-4e34-acc8-1d413f92b96e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.591816 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.591796    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/93569234-ca8e-4e34-acc8-1d413f92b96e-config-out\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.591960 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.591941    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/93569234-ca8e-4e34-acc8-1d413f92b96e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.592633 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.592611    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/93569234-ca8e-4e34-acc8-1d413f92b96e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.596634 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.596614    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2bnm\" (UniqueName: \"kubernetes.io/projected/93569234-ca8e-4e34-acc8-1d413f92b96e-kube-api-access-j2bnm\") pod \"alertmanager-main-0\" (UID: \"93569234-ca8e-4e34-acc8-1d413f92b96e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.686827 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.686790    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 15 20:10:58.975555 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:58.975522    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 15 20:10:58.979153 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:10:58.979121    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93569234_ca8e_4e34_acc8_1d413f92b96e.slice/crio-b8f2b7417646062acb69d4b2b948672b6c959190b2d068358e9e878493cd43ef WatchSource:0}: Error finding container b8f2b7417646062acb69d4b2b948672b6c959190b2d068358e9e878493cd43ef: Status 404 returned error can't find the container with id b8f2b7417646062acb69d4b2b948672b6c959190b2d068358e9e878493cd43ef
Sep 15 20:10:59.325742 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:59.325641    2577 generic.go:356] "Generic (PLEG): container finished" podID="93569234-ca8e-4e34-acc8-1d413f92b96e" containerID="2636c3d019c479e1e6d21681306096ad5ed602262686b0971d378a4f62005908" exitCode=0
Sep 15 20:10:59.325742 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:59.325685    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"93569234-ca8e-4e34-acc8-1d413f92b96e","Type":"ContainerDied","Data":"2636c3d019c479e1e6d21681306096ad5ed602262686b0971d378a4f62005908"}
Sep 15 20:10:59.325998 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:10:59.325777    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"93569234-ca8e-4e34-acc8-1d413f92b96e","Type":"ContainerStarted","Data":"b8f2b7417646062acb69d4b2b948672b6c959190b2d068358e9e878493cd43ef"}
Sep 15 20:11:00.058890 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:11:00.058856    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e94af346-ecec-410c-8144-f476cb7d77d2" path="/var/lib/kubelet/pods/e94af346-ecec-410c-8144-f476cb7d77d2/volumes"
Sep 15 20:11:00.335170 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:11:00.335087    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"93569234-ca8e-4e34-acc8-1d413f92b96e","Type":"ContainerStarted","Data":"18263fe7d8739c03962aa6ac4eefde3fc5b0dc9f74e6b5b8290468a92d8656ab"}
Sep 15 20:11:00.335170 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:11:00.335150    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"93569234-ca8e-4e34-acc8-1d413f92b96e","Type":"ContainerStarted","Data":"a4513ad68bc8c21322b2f4e17b0df1d5a3a75d5380807cd91411273d13202174"}
Sep 15 20:11:00.335346 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:11:00.335178    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"93569234-ca8e-4e34-acc8-1d413f92b96e","Type":"ContainerStarted","Data":"9656137fcaf51553dacb0259eccd5ea22b93c23f93f980cd4eccd67c2c6da4a1"}
Sep 15 20:11:00.335346 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:11:00.335205    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"93569234-ca8e-4e34-acc8-1d413f92b96e","Type":"ContainerStarted","Data":"2f72407b05ff726c2a9a9de247a429c06e13ad608a17363089a909184e066465"}
Sep 15 20:11:00.335346 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:11:00.335236    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"93569234-ca8e-4e34-acc8-1d413f92b96e","Type":"ContainerStarted","Data":"e93f2d60afe179754271d35737ec17f7056f17a0df0fe4c712f6486f77d595b5"}
Sep 15 20:11:00.335346 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:11:00.335260    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"93569234-ca8e-4e34-acc8-1d413f92b96e","Type":"ContainerStarted","Data":"32f36647978e0846f0efbf13eb1456d133261fab4ed349cef8523f9561257e9b"}
Sep 15 20:11:00.362565 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:11:00.362507    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.346111737 podStartE2EDuration="2.36248987s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.378133ms" isStatefulPod=false podCreationTimestamp="2026-09-15 20:10:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:11:00.361118117 +0000 UTC m=+366.752071113" watchObservedRunningTime="2026-09-15 20:11:00.36248987 +0000 UTC m=+366.753442924"
Sep 15 20:11:07.967714 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:11:07.967661    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5482d254-b248-4eda-a7d8-37bcb3289d69-metrics-certs\") pod \"network-metrics-daemon-2v4rt\" (UID: \"5482d254-b248-4eda-a7d8-37bcb3289d69\") " pod="openshift-multus/network-metrics-daemon-2v4rt"
Sep 15 20:11:07.970093 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:11:07.970069    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5482d254-b248-4eda-a7d8-37bcb3289d69-metrics-certs\") pod \"network-metrics-daemon-2v4rt\" (UID: \"5482d254-b248-4eda-a7d8-37bcb3289d69\") " pod="openshift-multus/network-metrics-daemon-2v4rt"
Sep 15 20:11:08.260666 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:11:08.260578    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l9zsw\""
Sep 15 20:11:08.266852 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:11:08.266832    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2v4rt"
Sep 15 20:11:08.537047 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:11:08.537016    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2v4rt"]
Sep 15 20:11:08.539373 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:11:08.539343    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5482d254_b248_4eda_a7d8_37bcb3289d69.slice/crio-deec3f36a28512d01a04903f46d38b426f1ef7a4522c96a9bcd6141d6f347051 WatchSource:0}: Error finding container deec3f36a28512d01a04903f46d38b426f1ef7a4522c96a9bcd6141d6f347051: Status 404 returned error can't find the container with id deec3f36a28512d01a04903f46d38b426f1ef7a4522c96a9bcd6141d6f347051
Sep 15 20:11:08.541268 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:11:08.541248    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 20:11:09.373341 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:11:09.373289    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2v4rt" event={"ID":"5482d254-b248-4eda-a7d8-37bcb3289d69","Type":"ContainerStarted","Data":"deec3f36a28512d01a04903f46d38b426f1ef7a4522c96a9bcd6141d6f347051"}
Sep 15 20:11:10.378538 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:11:10.378498    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2v4rt" event={"ID":"5482d254-b248-4eda-a7d8-37bcb3289d69","Type":"ContainerStarted","Data":"a2f8da68eb026085febe19ddbf01d560039be1a67211cf9ded306c6afe434db2"}
Sep 15 20:11:10.379039 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:11:10.378558    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2v4rt" event={"ID":"5482d254-b248-4eda-a7d8-37bcb3289d69","Type":"ContainerStarted","Data":"f5ef022d5ab1f97435582c5abd7c3ab9ca51a10a4533b9dafe44442b5f0e546e"}
Sep 15 20:11:10.397138 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:11:10.397078    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-2v4rt" podStartSLOduration=375.246274086 podStartE2EDuration="6m16.397061218s" totalImagesPullingTime="1.150787132s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:04:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:11:10.396047931 +0000 UTC m=+376.787000927" watchObservedRunningTime="2026-09-15 20:11:10.397061218 +0000 UTC m=+376.788014215"
Sep 15 20:11:55.184348 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:11:55.184311    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nc575"]
Sep 15 20:11:55.187372 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:11:55.187354    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nc575"
Sep 15 20:11:55.190218 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:11:55.190198    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 15 20:11:55.190346 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:11:55.190262    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 15 20:11:55.191097 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:11:55.191082    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-k79st\""
Sep 15 20:11:55.196260 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:11:55.196234    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nc575"]
Sep 15 20:11:55.356800 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:11:55.356762    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae70af2b-265e-42aa-995c-56c795343ce7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nc575\" (UID: \"ae70af2b-265e-42aa-995c-56c795343ce7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nc575"
Sep 15 20:11:55.356800 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:11:55.356810    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lslws\" (UniqueName: \"kubernetes.io/projected/ae70af2b-265e-42aa-995c-56c795343ce7-kube-api-access-lslws\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nc575\" (UID: \"ae70af2b-265e-42aa-995c-56c795343ce7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nc575"
Sep 15 20:11:55.357021 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:11:55.356880    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae70af2b-265e-42aa-995c-56c795343ce7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nc575\" (UID: \"ae70af2b-265e-42aa-995c-56c795343ce7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nc575"
Sep 15 20:11:55.457774 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:11:55.457743    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae70af2b-265e-42aa-995c-56c795343ce7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nc575\" (UID: \"ae70af2b-265e-42aa-995c-56c795343ce7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nc575"
Sep 15 20:11:55.457936 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:11:55.457792    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lslws\" (UniqueName: \"kubernetes.io/projected/ae70af2b-265e-42aa-995c-56c795343ce7-kube-api-access-lslws\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nc575\" (UID: \"ae70af2b-265e-42aa-995c-56c795343ce7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nc575"
Sep 15 20:11:55.457936 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:11:55.457858    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae70af2b-265e-42aa-995c-56c795343ce7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nc575\" (UID: \"ae70af2b-265e-42aa-995c-56c795343ce7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nc575"
Sep 15 20:11:55.458150 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:11:55.458131    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae70af2b-265e-42aa-995c-56c795343ce7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nc575\" (UID: \"ae70af2b-265e-42aa-995c-56c795343ce7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nc575"
Sep 15 20:11:55.458191 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:11:55.458160    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae70af2b-265e-42aa-995c-56c795343ce7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nc575\" (UID: \"ae70af2b-265e-42aa-995c-56c795343ce7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nc575"
Sep 15 20:11:55.467146 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:11:55.467114    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lslws\" (UniqueName: \"kubernetes.io/projected/ae70af2b-265e-42aa-995c-56c795343ce7-kube-api-access-lslws\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nc575\" (UID: \"ae70af2b-265e-42aa-995c-56c795343ce7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nc575"
Sep 15 20:11:55.498049 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:11:55.498019    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nc575"
Sep 15 20:11:55.770996 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:11:55.770961    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nc575"]
Sep 15 20:11:55.773638 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:11:55.773611    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae70af2b_265e_42aa_995c_56c795343ce7.slice/crio-3c357bee824952d228f792c5a2b676ca5ff181de90696c72dc15e8129130d30f WatchSource:0}: Error finding container 3c357bee824952d228f792c5a2b676ca5ff181de90696c72dc15e8129130d30f: Status 404 returned error can't find the container with id 3c357bee824952d228f792c5a2b676ca5ff181de90696c72dc15e8129130d30f
Sep 15 20:11:56.565575 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:11:56.565530    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nc575" event={"ID":"ae70af2b-265e-42aa-995c-56c795343ce7","Type":"ContainerStarted","Data":"3c357bee824952d228f792c5a2b676ca5ff181de90696c72dc15e8129130d30f"}
Sep 15 20:12:03.597032 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:03.596989    2577 generic.go:356] "Generic (PLEG): container finished" podID="ae70af2b-265e-42aa-995c-56c795343ce7" containerID="99d8f00f502a46df22ab948403ef1365fc904a93a5f242916442e208b7e45078" exitCode=0
Sep 15 20:12:03.597486 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:03.597081    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nc575" event={"ID":"ae70af2b-265e-42aa-995c-56c795343ce7","Type":"ContainerDied","Data":"99d8f00f502a46df22ab948403ef1365fc904a93a5f242916442e208b7e45078"}
Sep 15 20:12:06.613010 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:06.612974    2577 generic.go:356] "Generic (PLEG): container finished" podID="ae70af2b-265e-42aa-995c-56c795343ce7" containerID="dafc9416bb208f0ee69a8620ba3f7b099b177f470039a5131656d25c395824b8" exitCode=0
Sep 15 20:12:06.613394 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:06.613025    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nc575" event={"ID":"ae70af2b-265e-42aa-995c-56c795343ce7","Type":"ContainerDied","Data":"dafc9416bb208f0ee69a8620ba3f7b099b177f470039a5131656d25c395824b8"}
Sep 15 20:12:13.648606 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:13.648565    2577 generic.go:356] "Generic (PLEG): container finished" podID="ae70af2b-265e-42aa-995c-56c795343ce7" containerID="efc549c6a394ae542474b1d43b1b77d17e6b2bc2dd79fd6101556467b07e59c4" exitCode=0
Sep 15 20:12:13.648606 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:13.648611    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nc575" event={"ID":"ae70af2b-265e-42aa-995c-56c795343ce7","Type":"ContainerDied","Data":"efc549c6a394ae542474b1d43b1b77d17e6b2bc2dd79fd6101556467b07e59c4"}
Sep 15 20:12:14.790297 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:14.790270    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nc575"
Sep 15 20:12:14.830232 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:14.830197    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lslws\" (UniqueName: \"kubernetes.io/projected/ae70af2b-265e-42aa-995c-56c795343ce7-kube-api-access-lslws\") pod \"ae70af2b-265e-42aa-995c-56c795343ce7\" (UID: \"ae70af2b-265e-42aa-995c-56c795343ce7\") "
Sep 15 20:12:14.830389 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:14.830272    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae70af2b-265e-42aa-995c-56c795343ce7-bundle\") pod \"ae70af2b-265e-42aa-995c-56c795343ce7\" (UID: \"ae70af2b-265e-42aa-995c-56c795343ce7\") "
Sep 15 20:12:14.830389 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:14.830295    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae70af2b-265e-42aa-995c-56c795343ce7-util\") pod \"ae70af2b-265e-42aa-995c-56c795343ce7\" (UID: \"ae70af2b-265e-42aa-995c-56c795343ce7\") "
Sep 15 20:12:14.830980 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:14.830952    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae70af2b-265e-42aa-995c-56c795343ce7-bundle" (OuterVolumeSpecName: "bundle") pod "ae70af2b-265e-42aa-995c-56c795343ce7" (UID: "ae70af2b-265e-42aa-995c-56c795343ce7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:12:14.832460 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:14.832422    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae70af2b-265e-42aa-995c-56c795343ce7-kube-api-access-lslws" (OuterVolumeSpecName: "kube-api-access-lslws") pod "ae70af2b-265e-42aa-995c-56c795343ce7" (UID: "ae70af2b-265e-42aa-995c-56c795343ce7"). InnerVolumeSpecName "kube-api-access-lslws". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:12:14.834129 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:14.834099    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae70af2b-265e-42aa-995c-56c795343ce7-util" (OuterVolumeSpecName: "util") pod "ae70af2b-265e-42aa-995c-56c795343ce7" (UID: "ae70af2b-265e-42aa-995c-56c795343ce7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:12:14.931297 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:14.931216    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae70af2b-265e-42aa-995c-56c795343ce7-bundle\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:12:14.931297 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:14.931244    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae70af2b-265e-42aa-995c-56c795343ce7-util\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:12:14.931297 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:14.931253    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lslws\" (UniqueName: \"kubernetes.io/projected/ae70af2b-265e-42aa-995c-56c795343ce7-kube-api-access-lslws\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:12:15.659101 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:15.659058    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nc575" event={"ID":"ae70af2b-265e-42aa-995c-56c795343ce7","Type":"ContainerDied","Data":"3c357bee824952d228f792c5a2b676ca5ff181de90696c72dc15e8129130d30f"}
Sep 15 20:12:15.659101 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:15.659102    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nc575"
Sep 15 20:12:15.659310 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:15.659116    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3c357bee824952d228f792c5a2b676ca5ff181de90696c72dc15e8129130d30f"
Sep 15 20:12:23.217199 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:23.217118    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t47kp"]
Sep 15 20:12:23.217619 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:23.217410    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae70af2b-265e-42aa-995c-56c795343ce7" containerName="pull"
Sep 15 20:12:23.217619 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:23.217421    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae70af2b-265e-42aa-995c-56c795343ce7" containerName="pull"
Sep 15 20:12:23.217619 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:23.217432    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae70af2b-265e-42aa-995c-56c795343ce7" containerName="extract"
Sep 15 20:12:23.217619 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:23.217438    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae70af2b-265e-42aa-995c-56c795343ce7" containerName="extract"
Sep 15 20:12:23.217619 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:23.217445    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae70af2b-265e-42aa-995c-56c795343ce7" containerName="util"
Sep 15 20:12:23.217619 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:23.217450    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae70af2b-265e-42aa-995c-56c795343ce7" containerName="util"
Sep 15 20:12:23.217619 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:23.217519    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae70af2b-265e-42aa-995c-56c795343ce7" containerName="extract"
Sep 15 20:12:23.219560 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:23.219543    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t47kp"
Sep 15 20:12:23.222472 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:23.222445    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 15 20:12:23.222598 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:23.222449    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 15 20:12:23.222655 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:23.222604    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-rbvpl\""
Sep 15 20:12:23.233213 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:23.233181    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t47kp"]
Sep 15 20:12:23.300813 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:23.300774    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9d85x\" (UniqueName: \"kubernetes.io/projected/e0aa45e1-b958-419a-9160-1e8bd2789fd1-kube-api-access-9d85x\") pod \"cert-manager-operator-controller-manager-55c444bf86-t47kp\" (UID: \"e0aa45e1-b958-419a-9160-1e8bd2789fd1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t47kp"
Sep 15 20:12:23.301001 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:23.300820    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e0aa45e1-b958-419a-9160-1e8bd2789fd1-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-t47kp\" (UID: \"e0aa45e1-b958-419a-9160-1e8bd2789fd1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t47kp"
Sep 15 20:12:23.402125 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:23.402077    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9d85x\" (UniqueName: \"kubernetes.io/projected/e0aa45e1-b958-419a-9160-1e8bd2789fd1-kube-api-access-9d85x\") pod \"cert-manager-operator-controller-manager-55c444bf86-t47kp\" (UID: \"e0aa45e1-b958-419a-9160-1e8bd2789fd1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t47kp"
Sep 15 20:12:23.402125 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:23.402130    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e0aa45e1-b958-419a-9160-1e8bd2789fd1-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-t47kp\" (UID: \"e0aa45e1-b958-419a-9160-1e8bd2789fd1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t47kp"
Sep 15 20:12:23.402530 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:23.402512    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e0aa45e1-b958-419a-9160-1e8bd2789fd1-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-t47kp\" (UID: \"e0aa45e1-b958-419a-9160-1e8bd2789fd1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t47kp"
Sep 15 20:12:23.411362 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:23.411337    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9d85x\" (UniqueName: \"kubernetes.io/projected/e0aa45e1-b958-419a-9160-1e8bd2789fd1-kube-api-access-9d85x\") pod \"cert-manager-operator-controller-manager-55c444bf86-t47kp\" (UID: \"e0aa45e1-b958-419a-9160-1e8bd2789fd1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t47kp"
Sep 15 20:12:23.528527 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:23.528438    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t47kp"
Sep 15 20:12:23.845007 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:12:23.844964    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0aa45e1_b958_419a_9160_1e8bd2789fd1.slice/crio-f563eed346a440a7282b28b1a10c2de0f7a4560e3f45865d1c88c58db5bfc1af WatchSource:0}: Error finding container f563eed346a440a7282b28b1a10c2de0f7a4560e3f45865d1c88c58db5bfc1af: Status 404 returned error can't find the container with id f563eed346a440a7282b28b1a10c2de0f7a4560e3f45865d1c88c58db5bfc1af
Sep 15 20:12:23.847251 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:23.847227    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t47kp"]
Sep 15 20:12:24.695739 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:24.695689    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t47kp" event={"ID":"e0aa45e1-b958-419a-9160-1e8bd2789fd1","Type":"ContainerStarted","Data":"f563eed346a440a7282b28b1a10c2de0f7a4560e3f45865d1c88c58db5bfc1af"}
Sep 15 20:12:26.705914 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:26.705870    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t47kp" event={"ID":"e0aa45e1-b958-419a-9160-1e8bd2789fd1","Type":"ContainerStarted","Data":"ab5225ede6b076d30ef065be17619fb86d22b76f3701b92e5ede656a9ca3bb15"}
Sep 15 20:12:26.726794 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:26.726720    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t47kp" podStartSLOduration=1.6301620589999999 podStartE2EDuration="3.726704908s" totalImagesPullingTime="2.096542849s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:12:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:12:26.726240544 +0000 UTC m=+453.117193542" watchObservedRunningTime="2026-09-15 20:12:26.726704908 +0000 UTC m=+453.117657904"
Sep 15 20:12:27.619199 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:27.619162    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2pv4l"]
Sep 15 20:12:27.621905 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:27.621881    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2pv4l"
Sep 15 20:12:27.625193 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:27.625169    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 15 20:12:27.625307 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:27.625222    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-k79st\""
Sep 15 20:12:27.628543 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:27.628524    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 15 20:12:27.636637 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:27.636613    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2pv4l"]
Sep 15 20:12:27.739948 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:27.739905    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b2a93012-95ed-413b-a54d-b3369438a9c6-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2pv4l\" (UID: \"b2a93012-95ed-413b-a54d-b3369438a9c6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2pv4l"
Sep 15 20:12:27.739948 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:27.739956    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnf84\" (UniqueName: \"kubernetes.io/projected/b2a93012-95ed-413b-a54d-b3369438a9c6-kube-api-access-vnf84\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2pv4l\" (UID: \"b2a93012-95ed-413b-a54d-b3369438a9c6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2pv4l"
Sep 15 20:12:27.740369 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:27.739976    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b2a93012-95ed-413b-a54d-b3369438a9c6-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2pv4l\" (UID: \"b2a93012-95ed-413b-a54d-b3369438a9c6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2pv4l"
Sep 15 20:12:27.841023 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:27.840981    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnf84\" (UniqueName: \"kubernetes.io/projected/b2a93012-95ed-413b-a54d-b3369438a9c6-kube-api-access-vnf84\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2pv4l\" (UID: \"b2a93012-95ed-413b-a54d-b3369438a9c6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2pv4l"
Sep 15 20:12:27.841023 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:27.841024    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b2a93012-95ed-413b-a54d-b3369438a9c6-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2pv4l\" (UID: \"b2a93012-95ed-413b-a54d-b3369438a9c6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2pv4l"
Sep 15 20:12:27.841236 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:27.841080    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b2a93012-95ed-413b-a54d-b3369438a9c6-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2pv4l\" (UID: \"b2a93012-95ed-413b-a54d-b3369438a9c6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2pv4l"
Sep 15 20:12:27.841431 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:27.841411    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b2a93012-95ed-413b-a54d-b3369438a9c6-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2pv4l\" (UID: \"b2a93012-95ed-413b-a54d-b3369438a9c6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2pv4l"
Sep 15 20:12:27.841513 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:27.841496    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b2a93012-95ed-413b-a54d-b3369438a9c6-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2pv4l\" (UID: \"b2a93012-95ed-413b-a54d-b3369438a9c6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2pv4l"
Sep 15 20:12:27.850239 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:27.850214    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnf84\" (UniqueName: \"kubernetes.io/projected/b2a93012-95ed-413b-a54d-b3369438a9c6-kube-api-access-vnf84\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2pv4l\" (UID: \"b2a93012-95ed-413b-a54d-b3369438a9c6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2pv4l"
Sep 15 20:12:27.931079 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:27.930986    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2pv4l"
Sep 15 20:12:28.099890 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:28.099853    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-cdcd2"]
Sep 15 20:12:28.102417 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:28.102400    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-cdcd2"
Sep 15 20:12:28.104984 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:28.104958    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 15 20:12:28.105117 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:28.105022    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 15 20:12:28.105117 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:28.105067    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-9vx68\""
Sep 15 20:12:28.113844 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:28.113818    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-cdcd2"]
Sep 15 20:12:28.206465 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:28.206434    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2pv4l"]
Sep 15 20:12:28.209314 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:12:28.209283    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2a93012_95ed_413b_a54d_b3369438a9c6.slice/crio-63e73ee1ea8b64f7005cdc517f1c34c5ab08d2b294b8c2507e4a97a14f252e8a WatchSource:0}: Error finding container 63e73ee1ea8b64f7005cdc517f1c34c5ab08d2b294b8c2507e4a97a14f252e8a: Status 404 returned error can't find the container with id 63e73ee1ea8b64f7005cdc517f1c34c5ab08d2b294b8c2507e4a97a14f252e8a
Sep 15 20:12:28.244405 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:28.244367    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcwmf\" (UniqueName: \"kubernetes.io/projected/a4a56179-6066-4f99-8ad8-249d5843c6f5-kube-api-access-hcwmf\") pod \"cert-manager-webhook-6486ffd997-cdcd2\" (UID: \"a4a56179-6066-4f99-8ad8-249d5843c6f5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cdcd2"
Sep 15 20:12:28.244405 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:28.244420    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a4a56179-6066-4f99-8ad8-249d5843c6f5-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-cdcd2\" (UID: \"a4a56179-6066-4f99-8ad8-249d5843c6f5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cdcd2"
Sep 15 20:12:28.345520 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:28.345475    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcwmf\" (UniqueName: \"kubernetes.io/projected/a4a56179-6066-4f99-8ad8-249d5843c6f5-kube-api-access-hcwmf\") pod \"cert-manager-webhook-6486ffd997-cdcd2\" (UID: \"a4a56179-6066-4f99-8ad8-249d5843c6f5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cdcd2"
Sep 15 20:12:28.345716 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:28.345528    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a4a56179-6066-4f99-8ad8-249d5843c6f5-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-cdcd2\" (UID: \"a4a56179-6066-4f99-8ad8-249d5843c6f5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cdcd2"
Sep 15 20:12:28.354685 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:28.354660    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a4a56179-6066-4f99-8ad8-249d5843c6f5-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-cdcd2\" (UID: \"a4a56179-6066-4f99-8ad8-249d5843c6f5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cdcd2"
Sep 15 20:12:28.354877 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:28.354860    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcwmf\" (UniqueName: \"kubernetes.io/projected/a4a56179-6066-4f99-8ad8-249d5843c6f5-kube-api-access-hcwmf\") pod \"cert-manager-webhook-6486ffd997-cdcd2\" (UID: \"a4a56179-6066-4f99-8ad8-249d5843c6f5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cdcd2"
Sep 15 20:12:28.412047 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:28.412009    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-cdcd2"
Sep 15 20:12:28.685539 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:28.685495    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-cdcd2"]
Sep 15 20:12:28.688238 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:12:28.688207    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda4a56179_6066_4f99_8ad8_249d5843c6f5.slice/crio-b1cd2cff48d742709866ac906abba7a0cb77f7deb746dc51173ccba8a7624561 WatchSource:0}: Error finding container b1cd2cff48d742709866ac906abba7a0cb77f7deb746dc51173ccba8a7624561: Status 404 returned error can't find the container with id b1cd2cff48d742709866ac906abba7a0cb77f7deb746dc51173ccba8a7624561
Sep 15 20:12:28.716938 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:28.716841    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-cdcd2" event={"ID":"a4a56179-6066-4f99-8ad8-249d5843c6f5","Type":"ContainerStarted","Data":"b1cd2cff48d742709866ac906abba7a0cb77f7deb746dc51173ccba8a7624561"}
Sep 15 20:12:28.718373 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:28.718342    2577 generic.go:356] "Generic (PLEG): container finished" podID="b2a93012-95ed-413b-a54d-b3369438a9c6" containerID="d9642dde8266a5b29f1202fee8bb9e6ec25ca93e6947f694421a5e4ec7f20605" exitCode=0
Sep 15 20:12:28.718507 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:28.718433    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2pv4l" event={"ID":"b2a93012-95ed-413b-a54d-b3369438a9c6","Type":"ContainerDied","Data":"d9642dde8266a5b29f1202fee8bb9e6ec25ca93e6947f694421a5e4ec7f20605"}
Sep 15 20:12:28.718568 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:28.718523    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2pv4l" event={"ID":"b2a93012-95ed-413b-a54d-b3369438a9c6","Type":"ContainerStarted","Data":"63e73ee1ea8b64f7005cdc517f1c34c5ab08d2b294b8c2507e4a97a14f252e8a"}
Sep 15 20:12:32.741240 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:32.741193    2577 generic.go:356] "Generic (PLEG): container finished" podID="b2a93012-95ed-413b-a54d-b3369438a9c6" containerID="a52e7606cc1835ddb114cc099cde54332dceacdabcdda86934582220c6bd5296" exitCode=0
Sep 15 20:12:32.741667 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:32.741276    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2pv4l" event={"ID":"b2a93012-95ed-413b-a54d-b3369438a9c6","Type":"ContainerDied","Data":"a52e7606cc1835ddb114cc099cde54332dceacdabcdda86934582220c6bd5296"}
Sep 15 20:12:32.742889 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:32.742868    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-cdcd2" event={"ID":"a4a56179-6066-4f99-8ad8-249d5843c6f5","Type":"ContainerStarted","Data":"be0a622459a468e5a8952813935f749e27b3c3b7c8a2005b44dce7d8d0bee231"}
Sep 15 20:12:32.743011 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:32.742995    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-cdcd2"
Sep 15 20:12:32.776027 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:32.775963    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-cdcd2" podStartSLOduration=1.31634072 podStartE2EDuration="4.775943878s" totalImagesPullingTime="3.459603158s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:12:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:12:32.774857253 +0000 UTC m=+459.165810254" watchObservedRunningTime="2026-09-15 20:12:32.775943878 +0000 UTC m=+459.166896875"
Sep 15 20:12:33.749346 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:33.749311    2577 generic.go:356] "Generic (PLEG): container finished" podID="b2a93012-95ed-413b-a54d-b3369438a9c6" containerID="fa67ee729ae35fe03e91d5c7547db8d5b62ae72404067087067cce067291e91d" exitCode=0
Sep 15 20:12:33.749705 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:33.749398    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2pv4l" event={"ID":"b2a93012-95ed-413b-a54d-b3369438a9c6","Type":"ContainerDied","Data":"fa67ee729ae35fe03e91d5c7547db8d5b62ae72404067087067cce067291e91d"}
Sep 15 20:12:34.883685 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:34.883659    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2pv4l"
Sep 15 20:12:35.007207 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:35.007121    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b2a93012-95ed-413b-a54d-b3369438a9c6-util\") pod \"b2a93012-95ed-413b-a54d-b3369438a9c6\" (UID: \"b2a93012-95ed-413b-a54d-b3369438a9c6\") "
Sep 15 20:12:35.007207 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:35.007186    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b2a93012-95ed-413b-a54d-b3369438a9c6-bundle\") pod \"b2a93012-95ed-413b-a54d-b3369438a9c6\" (UID: \"b2a93012-95ed-413b-a54d-b3369438a9c6\") "
Sep 15 20:12:35.007393 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:35.007246    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vnf84\" (UniqueName: \"kubernetes.io/projected/b2a93012-95ed-413b-a54d-b3369438a9c6-kube-api-access-vnf84\") pod \"b2a93012-95ed-413b-a54d-b3369438a9c6\" (UID: \"b2a93012-95ed-413b-a54d-b3369438a9c6\") "
Sep 15 20:12:35.007598 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:35.007564    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2a93012-95ed-413b-a54d-b3369438a9c6-bundle" (OuterVolumeSpecName: "bundle") pod "b2a93012-95ed-413b-a54d-b3369438a9c6" (UID: "b2a93012-95ed-413b-a54d-b3369438a9c6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:12:35.009266 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:35.009237    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2a93012-95ed-413b-a54d-b3369438a9c6-kube-api-access-vnf84" (OuterVolumeSpecName: "kube-api-access-vnf84") pod "b2a93012-95ed-413b-a54d-b3369438a9c6" (UID: "b2a93012-95ed-413b-a54d-b3369438a9c6"). InnerVolumeSpecName "kube-api-access-vnf84". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:12:35.011398 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:35.011377    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2a93012-95ed-413b-a54d-b3369438a9c6-util" (OuterVolumeSpecName: "util") pod "b2a93012-95ed-413b-a54d-b3369438a9c6" (UID: "b2a93012-95ed-413b-a54d-b3369438a9c6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:12:35.107841 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:35.107802    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b2a93012-95ed-413b-a54d-b3369438a9c6-bundle\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:12:35.107841 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:35.107833    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vnf84\" (UniqueName: \"kubernetes.io/projected/b2a93012-95ed-413b-a54d-b3369438a9c6-kube-api-access-vnf84\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:12:35.107841 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:35.107843    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b2a93012-95ed-413b-a54d-b3369438a9c6-util\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:12:35.239402 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:35.239365    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-mwzdt"]
Sep 15 20:12:35.239687 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:35.239674    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2a93012-95ed-413b-a54d-b3369438a9c6" containerName="extract"
Sep 15 20:12:35.239753 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:35.239689    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2a93012-95ed-413b-a54d-b3369438a9c6" containerName="extract"
Sep 15 20:12:35.239753 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:35.239700    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2a93012-95ed-413b-a54d-b3369438a9c6" containerName="util"
Sep 15 20:12:35.239753 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:35.239706    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2a93012-95ed-413b-a54d-b3369438a9c6" containerName="util"
Sep 15 20:12:35.239753 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:35.239742    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2a93012-95ed-413b-a54d-b3369438a9c6" containerName="pull"
Sep 15 20:12:35.239753 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:35.239748    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2a93012-95ed-413b-a54d-b3369438a9c6" containerName="pull"
Sep 15 20:12:35.239916 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:35.239797    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2a93012-95ed-413b-a54d-b3369438a9c6" containerName="extract"
Sep 15 20:12:35.241940 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:35.241917    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-mwzdt"
Sep 15 20:12:35.244301 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:35.244267    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-nd45l\""
Sep 15 20:12:35.251804 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:35.251780    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-mwzdt"]
Sep 15 20:12:35.309490 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:35.309390    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g94th\" (UniqueName: \"kubernetes.io/projected/7346f5f4-3574-4f00-8d07-d4fbcc1b8ba2-kube-api-access-g94th\") pod \"cert-manager-cainjector-559cdd8c87-mwzdt\" (UID: \"7346f5f4-3574-4f00-8d07-d4fbcc1b8ba2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-mwzdt"
Sep 15 20:12:35.309490 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:35.309441    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7346f5f4-3574-4f00-8d07-d4fbcc1b8ba2-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-mwzdt\" (UID: \"7346f5f4-3574-4f00-8d07-d4fbcc1b8ba2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-mwzdt"
Sep 15 20:12:35.409965 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:35.409927    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7346f5f4-3574-4f00-8d07-d4fbcc1b8ba2-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-mwzdt\" (UID: \"7346f5f4-3574-4f00-8d07-d4fbcc1b8ba2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-mwzdt"
Sep 15 20:12:35.410131 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:35.410063    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g94th\" (UniqueName: \"kubernetes.io/projected/7346f5f4-3574-4f00-8d07-d4fbcc1b8ba2-kube-api-access-g94th\") pod \"cert-manager-cainjector-559cdd8c87-mwzdt\" (UID: \"7346f5f4-3574-4f00-8d07-d4fbcc1b8ba2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-mwzdt"
Sep 15 20:12:35.418284 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:35.418258    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g94th\" (UniqueName: \"kubernetes.io/projected/7346f5f4-3574-4f00-8d07-d4fbcc1b8ba2-kube-api-access-g94th\") pod \"cert-manager-cainjector-559cdd8c87-mwzdt\" (UID: \"7346f5f4-3574-4f00-8d07-d4fbcc1b8ba2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-mwzdt"
Sep 15 20:12:35.418800 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:35.418780    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7346f5f4-3574-4f00-8d07-d4fbcc1b8ba2-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-mwzdt\" (UID: \"7346f5f4-3574-4f00-8d07-d4fbcc1b8ba2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-mwzdt"
Sep 15 20:12:35.551078 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:35.551040    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-mwzdt"
Sep 15 20:12:35.760546 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:35.760519    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2pv4l"
Sep 15 20:12:35.760546 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:35.760542    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2pv4l" event={"ID":"b2a93012-95ed-413b-a54d-b3369438a9c6","Type":"ContainerDied","Data":"63e73ee1ea8b64f7005cdc517f1c34c5ab08d2b294b8c2507e4a97a14f252e8a"}
Sep 15 20:12:35.760764 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:35.760583    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="63e73ee1ea8b64f7005cdc517f1c34c5ab08d2b294b8c2507e4a97a14f252e8a"
Sep 15 20:12:35.821785 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:35.821757    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-mwzdt"]
Sep 15 20:12:35.823832 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:12:35.823800    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7346f5f4_3574_4f00_8d07_d4fbcc1b8ba2.slice/crio-931b13aa2dacd67e15e2ac65c6ebf1a0f9bebc15cd2db1f7380176288df77157 WatchSource:0}: Error finding container 931b13aa2dacd67e15e2ac65c6ebf1a0f9bebc15cd2db1f7380176288df77157: Status 404 returned error can't find the container with id 931b13aa2dacd67e15e2ac65c6ebf1a0f9bebc15cd2db1f7380176288df77157
Sep 15 20:12:36.766908 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:36.766863    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-mwzdt" event={"ID":"7346f5f4-3574-4f00-8d07-d4fbcc1b8ba2","Type":"ContainerStarted","Data":"001c0861ab24a7b95b8707a5fe8dbd39bae54d22b070e36d3e83e8b26f9da8cd"}
Sep 15 20:12:36.766908 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:36.766912    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-mwzdt" event={"ID":"7346f5f4-3574-4f00-8d07-d4fbcc1b8ba2","Type":"ContainerStarted","Data":"931b13aa2dacd67e15e2ac65c6ebf1a0f9bebc15cd2db1f7380176288df77157"}
Sep 15 20:12:36.784614 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:36.784557    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-mwzdt" podStartSLOduration=1.784540104 podStartE2EDuration="1.784540104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:12:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:12:36.783101345 +0000 UTC m=+463.174054343" watchObservedRunningTime="2026-09-15 20:12:36.784540104 +0000 UTC m=+463.175493100"
Sep 15 20:12:38.752038 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:38.752006    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-cdcd2"
Sep 15 20:12:45.280385 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:45.280346    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bt4f9"]
Sep 15 20:12:45.285422 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:45.285400    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bt4f9"
Sep 15 20:12:45.287989 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:45.287966    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 15 20:12:45.288093 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:45.287968    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-j4cr2\""
Sep 15 20:12:45.288793 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:45.288773    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 15 20:12:45.294010 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:45.293988    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bt4f9"]
Sep 15 20:12:45.395967 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:45.395925    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msjtl\" (UniqueName: \"kubernetes.io/projected/dfba85cc-ef2b-4d44-b886-8821a367c70e-kube-api-access-msjtl\") pod \"openshift-lws-operator-fd8ccff4c-bt4f9\" (UID: \"dfba85cc-ef2b-4d44-b886-8821a367c70e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bt4f9"
Sep 15 20:12:45.396145 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:45.396041    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dfba85cc-ef2b-4d44-b886-8821a367c70e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-bt4f9\" (UID: \"dfba85cc-ef2b-4d44-b886-8821a367c70e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bt4f9"
Sep 15 20:12:45.497175 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:45.497126    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dfba85cc-ef2b-4d44-b886-8821a367c70e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-bt4f9\" (UID: \"dfba85cc-ef2b-4d44-b886-8821a367c70e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bt4f9"
Sep 15 20:12:45.497356 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:45.497205    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msjtl\" (UniqueName: \"kubernetes.io/projected/dfba85cc-ef2b-4d44-b886-8821a367c70e-kube-api-access-msjtl\") pod \"openshift-lws-operator-fd8ccff4c-bt4f9\" (UID: \"dfba85cc-ef2b-4d44-b886-8821a367c70e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bt4f9"
Sep 15 20:12:45.497642 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:45.497621    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dfba85cc-ef2b-4d44-b886-8821a367c70e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-bt4f9\" (UID: \"dfba85cc-ef2b-4d44-b886-8821a367c70e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bt4f9"
Sep 15 20:12:45.505622 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:45.505597    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msjtl\" (UniqueName: \"kubernetes.io/projected/dfba85cc-ef2b-4d44-b886-8821a367c70e-kube-api-access-msjtl\") pod \"openshift-lws-operator-fd8ccff4c-bt4f9\" (UID: \"dfba85cc-ef2b-4d44-b886-8821a367c70e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bt4f9"
Sep 15 20:12:45.595856 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:45.595768    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bt4f9"
Sep 15 20:12:45.867094 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:45.867005    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bt4f9"]
Sep 15 20:12:45.870482 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:12:45.870452    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddfba85cc_ef2b_4d44_b886_8821a367c70e.slice/crio-00996a48941f360bbf51777f11729fe4df8981c1c708ad92077f5f35f26fd1f7 WatchSource:0}: Error finding container 00996a48941f360bbf51777f11729fe4df8981c1c708ad92077f5f35f26fd1f7: Status 404 returned error can't find the container with id 00996a48941f360bbf51777f11729fe4df8981c1c708ad92077f5f35f26fd1f7
Sep 15 20:12:46.813852 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:46.813812    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bt4f9" event={"ID":"dfba85cc-ef2b-4d44-b886-8821a367c70e","Type":"ContainerStarted","Data":"00996a48941f360bbf51777f11729fe4df8981c1c708ad92077f5f35f26fd1f7"}
Sep 15 20:12:47.826316 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:47.826227    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bt4f9" event={"ID":"dfba85cc-ef2b-4d44-b886-8821a367c70e","Type":"ContainerStarted","Data":"0454ee3b0b0e5c3da3da221c80272000d4ec502cb87a77072a555ca65a219e86"}
Sep 15 20:12:47.843705 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:47.843644    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bt4f9" podStartSLOduration=1.2335848839999999 podStartE2EDuration="2.843629082s" totalImagesPullingTime="1.610044198s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:12:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:12:47.842120997 +0000 UTC m=+474.233073996" watchObservedRunningTime="2026-09-15 20:12:47.843629082 +0000 UTC m=+474.234582079"
Sep 15 20:12:50.726183 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:50.726147    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwspnt"]
Sep 15 20:12:50.728388 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:50.728371    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwspnt"
Sep 15 20:12:50.731155 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:50.731133    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 15 20:12:50.731280 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:50.731219    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 15 20:12:50.732083 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:50.732064    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-k79st\""
Sep 15 20:12:50.738418 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:50.738396    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwspnt"]
Sep 15 20:12:50.745396 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:50.745373    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb3e6227-682e-455e-9f1f-2cda3a6d4e3f-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwspnt\" (UID: \"bb3e6227-682e-455e-9f1f-2cda3a6d4e3f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwspnt"
Sep 15 20:12:50.745516 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:50.745403    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs4kf\" (UniqueName: \"kubernetes.io/projected/bb3e6227-682e-455e-9f1f-2cda3a6d4e3f-kube-api-access-zs4kf\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwspnt\" (UID: \"bb3e6227-682e-455e-9f1f-2cda3a6d4e3f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwspnt"
Sep 15 20:12:50.745516 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:50.745441    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb3e6227-682e-455e-9f1f-2cda3a6d4e3f-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwspnt\" (UID: \"bb3e6227-682e-455e-9f1f-2cda3a6d4e3f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwspnt"
Sep 15 20:12:50.846035 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:50.845987    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zs4kf\" (UniqueName: \"kubernetes.io/projected/bb3e6227-682e-455e-9f1f-2cda3a6d4e3f-kube-api-access-zs4kf\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwspnt\" (UID: \"bb3e6227-682e-455e-9f1f-2cda3a6d4e3f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwspnt"
Sep 15 20:12:50.846247 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:50.846053    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb3e6227-682e-455e-9f1f-2cda3a6d4e3f-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwspnt\" (UID: \"bb3e6227-682e-455e-9f1f-2cda3a6d4e3f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwspnt"
Sep 15 20:12:50.846247 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:50.846103    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb3e6227-682e-455e-9f1f-2cda3a6d4e3f-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwspnt\" (UID: \"bb3e6227-682e-455e-9f1f-2cda3a6d4e3f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwspnt"
Sep 15 20:12:50.846430 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:50.846414    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb3e6227-682e-455e-9f1f-2cda3a6d4e3f-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwspnt\" (UID: \"bb3e6227-682e-455e-9f1f-2cda3a6d4e3f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwspnt"
Sep 15 20:12:50.846477 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:50.846448    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb3e6227-682e-455e-9f1f-2cda3a6d4e3f-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwspnt\" (UID: \"bb3e6227-682e-455e-9f1f-2cda3a6d4e3f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwspnt"
Sep 15 20:12:50.854355 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:50.854325    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs4kf\" (UniqueName: \"kubernetes.io/projected/bb3e6227-682e-455e-9f1f-2cda3a6d4e3f-kube-api-access-zs4kf\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwspnt\" (UID: \"bb3e6227-682e-455e-9f1f-2cda3a6d4e3f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwspnt"
Sep 15 20:12:51.038021 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:51.037932    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwspnt"
Sep 15 20:12:51.308237 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:51.308210    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwspnt"]
Sep 15 20:12:51.310500 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:12:51.310471    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb3e6227_682e_455e_9f1f_2cda3a6d4e3f.slice/crio-b8af55741db227f361f4769eb461363dbdb192e39e510bd96567e251d7abd5fd WatchSource:0}: Error finding container b8af55741db227f361f4769eb461363dbdb192e39e510bd96567e251d7abd5fd: Status 404 returned error can't find the container with id b8af55741db227f361f4769eb461363dbdb192e39e510bd96567e251d7abd5fd
Sep 15 20:12:51.844634 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:51.844600    2577 generic.go:356] "Generic (PLEG): container finished" podID="bb3e6227-682e-455e-9f1f-2cda3a6d4e3f" containerID="9186bd1899da5affca7b5038400bea5bbf2c7cd36ad00daf3894d417564901ce" exitCode=0
Sep 15 20:12:51.845040 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:51.844676    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwspnt" event={"ID":"bb3e6227-682e-455e-9f1f-2cda3a6d4e3f","Type":"ContainerDied","Data":"9186bd1899da5affca7b5038400bea5bbf2c7cd36ad00daf3894d417564901ce"}
Sep 15 20:12:51.845040 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:51.844746    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwspnt" event={"ID":"bb3e6227-682e-455e-9f1f-2cda3a6d4e3f","Type":"ContainerStarted","Data":"b8af55741db227f361f4769eb461363dbdb192e39e510bd96567e251d7abd5fd"}
Sep 15 20:12:52.851631 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:52.851528    2577 generic.go:356] "Generic (PLEG): container finished" podID="bb3e6227-682e-455e-9f1f-2cda3a6d4e3f" containerID="97a7f08be237f315986fc62ad67c6111bcbe5bafffd33cd69f50f9f69b043426" exitCode=0
Sep 15 20:12:52.851631 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:52.851615    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwspnt" event={"ID":"bb3e6227-682e-455e-9f1f-2cda3a6d4e3f","Type":"ContainerDied","Data":"97a7f08be237f315986fc62ad67c6111bcbe5bafffd33cd69f50f9f69b043426"}
Sep 15 20:12:53.858867 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:53.858832    2577 generic.go:356] "Generic (PLEG): container finished" podID="bb3e6227-682e-455e-9f1f-2cda3a6d4e3f" containerID="4b07557da98d6647bbef4741af633584ca9fde454066f629dba8b1b561ebcb85" exitCode=0
Sep 15 20:12:53.859258 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:53.858919    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwspnt" event={"ID":"bb3e6227-682e-455e-9f1f-2cda3a6d4e3f","Type":"ContainerDied","Data":"4b07557da98d6647bbef4741af633584ca9fde454066f629dba8b1b561ebcb85"}
Sep 15 20:12:54.988717 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:54.988694    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwspnt"
Sep 15 20:12:55.077041 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:55.077000    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb3e6227-682e-455e-9f1f-2cda3a6d4e3f-util\") pod \"bb3e6227-682e-455e-9f1f-2cda3a6d4e3f\" (UID: \"bb3e6227-682e-455e-9f1f-2cda3a6d4e3f\") "
Sep 15 20:12:55.077041 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:55.077043    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zs4kf\" (UniqueName: \"kubernetes.io/projected/bb3e6227-682e-455e-9f1f-2cda3a6d4e3f-kube-api-access-zs4kf\") pod \"bb3e6227-682e-455e-9f1f-2cda3a6d4e3f\" (UID: \"bb3e6227-682e-455e-9f1f-2cda3a6d4e3f\") "
Sep 15 20:12:55.077293 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:55.077077    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb3e6227-682e-455e-9f1f-2cda3a6d4e3f-bundle\") pod \"bb3e6227-682e-455e-9f1f-2cda3a6d4e3f\" (UID: \"bb3e6227-682e-455e-9f1f-2cda3a6d4e3f\") "
Sep 15 20:12:55.077914 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:55.077888    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb3e6227-682e-455e-9f1f-2cda3a6d4e3f-bundle" (OuterVolumeSpecName: "bundle") pod "bb3e6227-682e-455e-9f1f-2cda3a6d4e3f" (UID: "bb3e6227-682e-455e-9f1f-2cda3a6d4e3f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:12:55.079341 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:55.079322    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb3e6227-682e-455e-9f1f-2cda3a6d4e3f-kube-api-access-zs4kf" (OuterVolumeSpecName: "kube-api-access-zs4kf") pod "bb3e6227-682e-455e-9f1f-2cda3a6d4e3f" (UID: "bb3e6227-682e-455e-9f1f-2cda3a6d4e3f"). InnerVolumeSpecName "kube-api-access-zs4kf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:12:55.082444 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:55.082403    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb3e6227-682e-455e-9f1f-2cda3a6d4e3f-util" (OuterVolumeSpecName: "util") pod "bb3e6227-682e-455e-9f1f-2cda3a6d4e3f" (UID: "bb3e6227-682e-455e-9f1f-2cda3a6d4e3f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:12:55.177951 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:55.177863    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb3e6227-682e-455e-9f1f-2cda3a6d4e3f-util\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:12:55.177951 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:55.177891    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zs4kf\" (UniqueName: \"kubernetes.io/projected/bb3e6227-682e-455e-9f1f-2cda3a6d4e3f-kube-api-access-zs4kf\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:12:55.177951 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:55.177901    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb3e6227-682e-455e-9f1f-2cda3a6d4e3f-bundle\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:12:55.871299 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:55.871264    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwspnt" event={"ID":"bb3e6227-682e-455e-9f1f-2cda3a6d4e3f","Type":"ContainerDied","Data":"b8af55741db227f361f4769eb461363dbdb192e39e510bd96567e251d7abd5fd"}
Sep 15 20:12:55.871446 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:55.871307    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwspnt"
Sep 15 20:12:55.871446 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:12:55.871311    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b8af55741db227f361f4769eb461363dbdb192e39e510bd96567e251d7abd5fd"
Sep 15 20:13:01.962078 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:01.962042    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08pc6r"]
Sep 15 20:13:01.962467 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:01.962377    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb3e6227-682e-455e-9f1f-2cda3a6d4e3f" containerName="util"
Sep 15 20:13:01.962467 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:01.962390    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb3e6227-682e-455e-9f1f-2cda3a6d4e3f" containerName="util"
Sep 15 20:13:01.962467 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:01.962403    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb3e6227-682e-455e-9f1f-2cda3a6d4e3f" containerName="extract"
Sep 15 20:13:01.962467 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:01.962409    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb3e6227-682e-455e-9f1f-2cda3a6d4e3f" containerName="extract"
Sep 15 20:13:01.962467 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:01.962417    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb3e6227-682e-455e-9f1f-2cda3a6d4e3f" containerName="pull"
Sep 15 20:13:01.962467 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:01.962424    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb3e6227-682e-455e-9f1f-2cda3a6d4e3f" containerName="pull"
Sep 15 20:13:01.962653 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:01.962488    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb3e6227-682e-455e-9f1f-2cda3a6d4e3f" containerName="extract"
Sep 15 20:13:01.965737 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:01.965711    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08pc6r"
Sep 15 20:13:01.968534 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:01.968509    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 15 20:13:01.968711 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:01.968697    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-k79st\""
Sep 15 20:13:01.969624 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:01.969607    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 15 20:13:01.976328 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:01.976301    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08pc6r"]
Sep 15 20:13:02.034922 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:02.034885    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0c02e78-eadb-40a7-a931-a24b2c575b94-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08pc6r\" (UID: \"c0c02e78-eadb-40a7-a931-a24b2c575b94\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08pc6r"
Sep 15 20:13:02.034922 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:02.034925    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0c02e78-eadb-40a7-a931-a24b2c575b94-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08pc6r\" (UID: \"c0c02e78-eadb-40a7-a931-a24b2c575b94\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08pc6r"
Sep 15 20:13:02.035150 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:02.034953    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsnls\" (UniqueName: \"kubernetes.io/projected/c0c02e78-eadb-40a7-a931-a24b2c575b94-kube-api-access-tsnls\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08pc6r\" (UID: \"c0c02e78-eadb-40a7-a931-a24b2c575b94\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08pc6r"
Sep 15 20:13:02.136128 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:02.136081    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0c02e78-eadb-40a7-a931-a24b2c575b94-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08pc6r\" (UID: \"c0c02e78-eadb-40a7-a931-a24b2c575b94\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08pc6r"
Sep 15 20:13:02.136128 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:02.136125    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0c02e78-eadb-40a7-a931-a24b2c575b94-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08pc6r\" (UID: \"c0c02e78-eadb-40a7-a931-a24b2c575b94\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08pc6r"
Sep 15 20:13:02.136389 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:02.136157    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsnls\" (UniqueName: \"kubernetes.io/projected/c0c02e78-eadb-40a7-a931-a24b2c575b94-kube-api-access-tsnls\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08pc6r\" (UID: \"c0c02e78-eadb-40a7-a931-a24b2c575b94\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08pc6r"
Sep 15 20:13:02.136497 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:02.136476    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0c02e78-eadb-40a7-a931-a24b2c575b94-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08pc6r\" (UID: \"c0c02e78-eadb-40a7-a931-a24b2c575b94\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08pc6r"
Sep 15 20:13:02.136536 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:02.136508    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0c02e78-eadb-40a7-a931-a24b2c575b94-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08pc6r\" (UID: \"c0c02e78-eadb-40a7-a931-a24b2c575b94\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08pc6r"
Sep 15 20:13:02.146748 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:02.146705    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsnls\" (UniqueName: \"kubernetes.io/projected/c0c02e78-eadb-40a7-a931-a24b2c575b94-kube-api-access-tsnls\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08pc6r\" (UID: \"c0c02e78-eadb-40a7-a931-a24b2c575b94\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08pc6r"
Sep 15 20:13:02.275494 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:02.275408    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08pc6r"
Sep 15 20:13:02.563083 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:02.563056    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08pc6r"]
Sep 15 20:13:02.565052 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:13:02.565019    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0c02e78_eadb_40a7_a931_a24b2c575b94.slice/crio-baa601db3f554de6f67b2a843c61151d04163e15084ef5c339b3e77276a9bde2 WatchSource:0}: Error finding container baa601db3f554de6f67b2a843c61151d04163e15084ef5c339b3e77276a9bde2: Status 404 returned error can't find the container with id baa601db3f554de6f67b2a843c61151d04163e15084ef5c339b3e77276a9bde2
Sep 15 20:13:02.905939 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:02.905849    2577 generic.go:356] "Generic (PLEG): container finished" podID="c0c02e78-eadb-40a7-a931-a24b2c575b94" containerID="13250e1e69860e1a3b118d655cb0fe1ec13662b67572d5bb59717bc4c42e1f18" exitCode=0
Sep 15 20:13:02.906076 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:02.905932    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08pc6r" event={"ID":"c0c02e78-eadb-40a7-a931-a24b2c575b94","Type":"ContainerDied","Data":"13250e1e69860e1a3b118d655cb0fe1ec13662b67572d5bb59717bc4c42e1f18"}
Sep 15 20:13:02.906076 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:02.906014    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08pc6r" event={"ID":"c0c02e78-eadb-40a7-a931-a24b2c575b94","Type":"ContainerStarted","Data":"baa601db3f554de6f67b2a843c61151d04163e15084ef5c339b3e77276a9bde2"}
Sep 15 20:13:03.912620 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:03.912585    2577 generic.go:356] "Generic (PLEG): container finished" podID="c0c02e78-eadb-40a7-a931-a24b2c575b94" containerID="18e4ee339e1066bdcbcd40d445916075969f851b2fde8d4318cc8d5bf8dcdeaa" exitCode=0
Sep 15 20:13:03.912995 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:03.912656    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08pc6r" event={"ID":"c0c02e78-eadb-40a7-a931-a24b2c575b94","Type":"ContainerDied","Data":"18e4ee339e1066bdcbcd40d445916075969f851b2fde8d4318cc8d5bf8dcdeaa"}
Sep 15 20:13:04.438860 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:04.438776    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-75fcf566-jwslv"]
Sep 15 20:13:04.441410 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:04.441391    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-jwslv"
Sep 15 20:13:04.445024 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:04.445001    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 15 20:13:04.445148 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:04.445116    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 15 20:13:04.445205 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:04.445134    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 15 20:13:04.445277 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:04.445262    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 15 20:13:04.445785 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:04.445770    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-d6xtv\""
Sep 15 20:13:04.454637 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:04.454613    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2db0feff-46a2-4a97-aac2-ccb349c31ea0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-75fcf566-jwslv\" (UID: \"2db0feff-46a2-4a97-aac2-ccb349c31ea0\") " pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-jwslv"
Sep 15 20:13:04.454748 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:04.454661    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2db0feff-46a2-4a97-aac2-ccb349c31ea0-webhook-cert\") pod \"opendatahub-operator-controller-manager-75fcf566-jwslv\" (UID: \"2db0feff-46a2-4a97-aac2-ccb349c31ea0\") " pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-jwslv"
Sep 15 20:13:04.454748 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:04.454714    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vn4t\" (UniqueName: \"kubernetes.io/projected/2db0feff-46a2-4a97-aac2-ccb349c31ea0-kube-api-access-5vn4t\") pod \"opendatahub-operator-controller-manager-75fcf566-jwslv\" (UID: \"2db0feff-46a2-4a97-aac2-ccb349c31ea0\") " pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-jwslv"
Sep 15 20:13:04.454826 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:04.454772    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2db0feff-46a2-4a97-aac2-ccb349c31ea0-manifests\") pod \"opendatahub-operator-controller-manager-75fcf566-jwslv\" (UID: \"2db0feff-46a2-4a97-aac2-ccb349c31ea0\") " pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-jwslv"
Sep 15 20:13:04.454865 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:04.454834    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2db0feff-46a2-4a97-aac2-ccb349c31ea0-tmp\") pod \"opendatahub-operator-controller-manager-75fcf566-jwslv\" (UID: \"2db0feff-46a2-4a97-aac2-ccb349c31ea0\") " pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-jwslv"
Sep 15 20:13:04.460330 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:04.460304    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-75fcf566-jwslv"]
Sep 15 20:13:04.556250 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:04.556216    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2db0feff-46a2-4a97-aac2-ccb349c31ea0-tmp\") pod \"opendatahub-operator-controller-manager-75fcf566-jwslv\" (UID: \"2db0feff-46a2-4a97-aac2-ccb349c31ea0\") " pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-jwslv"
Sep 15 20:13:04.556461 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:04.556274    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2db0feff-46a2-4a97-aac2-ccb349c31ea0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-75fcf566-jwslv\" (UID: \"2db0feff-46a2-4a97-aac2-ccb349c31ea0\") " pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-jwslv"
Sep 15 20:13:04.556461 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:04.556327    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2db0feff-46a2-4a97-aac2-ccb349c31ea0-webhook-cert\") pod \"opendatahub-operator-controller-manager-75fcf566-jwslv\" (UID: \"2db0feff-46a2-4a97-aac2-ccb349c31ea0\") " pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-jwslv"
Sep 15 20:13:04.556461 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:04.556357    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vn4t\" (UniqueName: \"kubernetes.io/projected/2db0feff-46a2-4a97-aac2-ccb349c31ea0-kube-api-access-5vn4t\") pod \"opendatahub-operator-controller-manager-75fcf566-jwslv\" (UID: \"2db0feff-46a2-4a97-aac2-ccb349c31ea0\") " pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-jwslv"
Sep 15 20:13:04.556461 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:04.556387    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2db0feff-46a2-4a97-aac2-ccb349c31ea0-manifests\") pod \"opendatahub-operator-controller-manager-75fcf566-jwslv\" (UID: \"2db0feff-46a2-4a97-aac2-ccb349c31ea0\") " pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-jwslv"
Sep 15 20:13:04.556804 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:04.556778    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2db0feff-46a2-4a97-aac2-ccb349c31ea0-manifests\") pod \"opendatahub-operator-controller-manager-75fcf566-jwslv\" (UID: \"2db0feff-46a2-4a97-aac2-ccb349c31ea0\") " pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-jwslv"
Sep 15 20:13:04.558684 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:04.558664    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2db0feff-46a2-4a97-aac2-ccb349c31ea0-tmp\") pod \"opendatahub-operator-controller-manager-75fcf566-jwslv\" (UID: \"2db0feff-46a2-4a97-aac2-ccb349c31ea0\") " pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-jwslv"
Sep 15 20:13:04.558849 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:04.558828    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2db0feff-46a2-4a97-aac2-ccb349c31ea0-webhook-cert\") pod \"opendatahub-operator-controller-manager-75fcf566-jwslv\" (UID: \"2db0feff-46a2-4a97-aac2-ccb349c31ea0\") " pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-jwslv"
Sep 15 20:13:04.559087 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:04.559069    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2db0feff-46a2-4a97-aac2-ccb349c31ea0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-75fcf566-jwslv\" (UID: \"2db0feff-46a2-4a97-aac2-ccb349c31ea0\") " pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-jwslv"
Sep 15 20:13:04.572440 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:04.572415    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vn4t\" (UniqueName: \"kubernetes.io/projected/2db0feff-46a2-4a97-aac2-ccb349c31ea0-kube-api-access-5vn4t\") pod \"opendatahub-operator-controller-manager-75fcf566-jwslv\" (UID: \"2db0feff-46a2-4a97-aac2-ccb349c31ea0\") " pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-jwslv"
Sep 15 20:13:04.751821 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:04.751779    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-jwslv"
Sep 15 20:13:04.919817 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:04.919777    2577 generic.go:356] "Generic (PLEG): container finished" podID="c0c02e78-eadb-40a7-a931-a24b2c575b94" containerID="43a4a5db18b7d23b8189d7e3a91a1e02bcca20d1f6070ef80bc4d4f77723e124" exitCode=0
Sep 15 20:13:04.920186 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:04.919825    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08pc6r" event={"ID":"c0c02e78-eadb-40a7-a931-a24b2c575b94","Type":"ContainerDied","Data":"43a4a5db18b7d23b8189d7e3a91a1e02bcca20d1f6070ef80bc4d4f77723e124"}
Sep 15 20:13:05.039316 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:05.039229    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-75fcf566-jwslv"]
Sep 15 20:13:05.042215 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:13:05.042184    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2db0feff_46a2_4a97_aac2_ccb349c31ea0.slice/crio-1b4a2ed1e0072f94fb609b2decc9a9fce910a0f79392029591154cc7443969cb WatchSource:0}: Error finding container 1b4a2ed1e0072f94fb609b2decc9a9fce910a0f79392029591154cc7443969cb: Status 404 returned error can't find the container with id 1b4a2ed1e0072f94fb609b2decc9a9fce910a0f79392029591154cc7443969cb
Sep 15 20:13:05.926699 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:05.926659    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-jwslv" event={"ID":"2db0feff-46a2-4a97-aac2-ccb349c31ea0","Type":"ContainerStarted","Data":"1b4a2ed1e0072f94fb609b2decc9a9fce910a0f79392029591154cc7443969cb"}
Sep 15 20:13:06.081220 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:06.081185    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08pc6r"
Sep 15 20:13:06.173249 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:06.173215    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tsnls\" (UniqueName: \"kubernetes.io/projected/c0c02e78-eadb-40a7-a931-a24b2c575b94-kube-api-access-tsnls\") pod \"c0c02e78-eadb-40a7-a931-a24b2c575b94\" (UID: \"c0c02e78-eadb-40a7-a931-a24b2c575b94\") "
Sep 15 20:13:06.173413 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:06.173320    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0c02e78-eadb-40a7-a931-a24b2c575b94-bundle\") pod \"c0c02e78-eadb-40a7-a931-a24b2c575b94\" (UID: \"c0c02e78-eadb-40a7-a931-a24b2c575b94\") "
Sep 15 20:13:06.173413 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:06.173346    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0c02e78-eadb-40a7-a931-a24b2c575b94-util\") pod \"c0c02e78-eadb-40a7-a931-a24b2c575b94\" (UID: \"c0c02e78-eadb-40a7-a931-a24b2c575b94\") "
Sep 15 20:13:06.174193 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:06.174157    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0c02e78-eadb-40a7-a931-a24b2c575b94-bundle" (OuterVolumeSpecName: "bundle") pod "c0c02e78-eadb-40a7-a931-a24b2c575b94" (UID: "c0c02e78-eadb-40a7-a931-a24b2c575b94"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:13:06.176052 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:06.176019    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0c02e78-eadb-40a7-a931-a24b2c575b94-kube-api-access-tsnls" (OuterVolumeSpecName: "kube-api-access-tsnls") pod "c0c02e78-eadb-40a7-a931-a24b2c575b94" (UID: "c0c02e78-eadb-40a7-a931-a24b2c575b94"). InnerVolumeSpecName "kube-api-access-tsnls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:13:06.180739 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:06.180660    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0c02e78-eadb-40a7-a931-a24b2c575b94-util" (OuterVolumeSpecName: "util") pod "c0c02e78-eadb-40a7-a931-a24b2c575b94" (UID: "c0c02e78-eadb-40a7-a931-a24b2c575b94"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:13:06.274250 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:06.274208    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tsnls\" (UniqueName: \"kubernetes.io/projected/c0c02e78-eadb-40a7-a931-a24b2c575b94-kube-api-access-tsnls\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:13:06.274250 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:06.274245    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0c02e78-eadb-40a7-a931-a24b2c575b94-bundle\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:13:06.274489 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:06.274261    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0c02e78-eadb-40a7-a931-a24b2c575b94-util\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:13:06.935156 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:06.935116    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08pc6r" event={"ID":"c0c02e78-eadb-40a7-a931-a24b2c575b94","Type":"ContainerDied","Data":"baa601db3f554de6f67b2a843c61151d04163e15084ef5c339b3e77276a9bde2"}
Sep 15 20:13:06.935642 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:06.935174    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08pc6r"
Sep 15 20:13:06.935642 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:06.935200    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="baa601db3f554de6f67b2a843c61151d04163e15084ef5c339b3e77276a9bde2"
Sep 15 20:13:08.948147 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:08.948106    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-jwslv" event={"ID":"2db0feff-46a2-4a97-aac2-ccb349c31ea0","Type":"ContainerStarted","Data":"5b15283da9f0c419a8ff702048dcb4f6161adb2d97ef91e5b150f9cc7e5a4b37"}
Sep 15 20:13:09.954474 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:09.954438    2577 generic.go:356] "Generic (PLEG): container finished" podID="2db0feff-46a2-4a97-aac2-ccb349c31ea0" containerID="5b15283da9f0c419a8ff702048dcb4f6161adb2d97ef91e5b150f9cc7e5a4b37" exitCode=0
Sep 15 20:13:09.954866 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:09.954519    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-jwslv" event={"ID":"2db0feff-46a2-4a97-aac2-ccb349c31ea0","Type":"ContainerDied","Data":"5b15283da9f0c419a8ff702048dcb4f6161adb2d97ef91e5b150f9cc7e5a4b37"}
Sep 15 20:13:10.961577 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:10.961544    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-jwslv" event={"ID":"2db0feff-46a2-4a97-aac2-ccb349c31ea0","Type":"ContainerStarted","Data":"293ce3e3f51744282507e991e05047cf5c750b56a16d02c79d31cc4064ca8b95"}
Sep 15 20:13:10.961983 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:10.961666    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-jwslv"
Sep 15 20:13:10.985343 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:10.985289    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-jwslv" podStartSLOduration=2.334884408 podStartE2EDuration="6.985271772s" totalImagesPullingTime="3.608007483s" totalInitContainerRuntime="1.042379881s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:13:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:13:10.984879042 +0000 UTC m=+497.375832041" watchObservedRunningTime="2026-09-15 20:13:10.985271772 +0000 UTC m=+497.376224768"
Sep 15 20:13:21.969582 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:21.969547    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-75fcf566-jwslv"
Sep 15 20:13:35.433890 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:35.433849    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-565c49d7c-smwm7"]
Sep 15 20:13:35.434287 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:35.434180    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0c02e78-eadb-40a7-a931-a24b2c575b94" containerName="extract"
Sep 15 20:13:35.434287 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:35.434193    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0c02e78-eadb-40a7-a931-a24b2c575b94" containerName="extract"
Sep 15 20:13:35.434287 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:35.434212    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0c02e78-eadb-40a7-a931-a24b2c575b94" containerName="util"
Sep 15 20:13:35.434287 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:35.434217    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0c02e78-eadb-40a7-a931-a24b2c575b94" containerName="util"
Sep 15 20:13:35.434287 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:35.434225    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0c02e78-eadb-40a7-a931-a24b2c575b94" containerName="pull"
Sep 15 20:13:35.434287 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:35.434231    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0c02e78-eadb-40a7-a931-a24b2c575b94" containerName="pull"
Sep 15 20:13:35.434287 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:35.434283    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0c02e78-eadb-40a7-a931-a24b2c575b94" containerName="extract"
Sep 15 20:13:35.439835 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:35.439812    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-565c49d7c-smwm7"
Sep 15 20:13:35.442405 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:35.442381    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 15 20:13:35.442564 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:35.442545    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 15 20:13:35.442652 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:35.442630    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 15 20:13:35.442704 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:35.442643    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-bztbx\""
Sep 15 20:13:35.442956 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:35.442942    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 15 20:13:35.449129 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:35.449108    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-565c49d7c-smwm7"]
Sep 15 20:13:35.532912 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:35.532865    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdvkk\" (UniqueName: \"kubernetes.io/projected/d92f8a14-740a-4a09-87e9-3e39da9d2788-kube-api-access-kdvkk\") pod \"kube-auth-proxy-565c49d7c-smwm7\" (UID: \"d92f8a14-740a-4a09-87e9-3e39da9d2788\") " pod="openshift-ingress/kube-auth-proxy-565c49d7c-smwm7"
Sep 15 20:13:35.533086 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:35.532923    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d92f8a14-740a-4a09-87e9-3e39da9d2788-tls-certs\") pod \"kube-auth-proxy-565c49d7c-smwm7\" (UID: \"d92f8a14-740a-4a09-87e9-3e39da9d2788\") " pod="openshift-ingress/kube-auth-proxy-565c49d7c-smwm7"
Sep 15 20:13:35.533086 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:35.532950    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d92f8a14-740a-4a09-87e9-3e39da9d2788-tmp\") pod \"kube-auth-proxy-565c49d7c-smwm7\" (UID: \"d92f8a14-740a-4a09-87e9-3e39da9d2788\") " pod="openshift-ingress/kube-auth-proxy-565c49d7c-smwm7"
Sep 15 20:13:35.625023 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:35.624989    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-b4sg4"]
Sep 15 20:13:35.627312 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:35.627289    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-b4sg4"
Sep 15 20:13:35.629913 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:35.629891    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-6xnxn\""
Sep 15 20:13:35.630493 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:35.630478    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 15 20:13:35.633452 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:35.633423    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdvkk\" (UniqueName: \"kubernetes.io/projected/d92f8a14-740a-4a09-87e9-3e39da9d2788-kube-api-access-kdvkk\") pod \"kube-auth-proxy-565c49d7c-smwm7\" (UID: \"d92f8a14-740a-4a09-87e9-3e39da9d2788\") " pod="openshift-ingress/kube-auth-proxy-565c49d7c-smwm7"
Sep 15 20:13:35.633560 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:35.633476    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d92f8a14-740a-4a09-87e9-3e39da9d2788-tls-certs\") pod \"kube-auth-proxy-565c49d7c-smwm7\" (UID: \"d92f8a14-740a-4a09-87e9-3e39da9d2788\") " pod="openshift-ingress/kube-auth-proxy-565c49d7c-smwm7"
Sep 15 20:13:35.633560 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:35.633498    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d92f8a14-740a-4a09-87e9-3e39da9d2788-tmp\") pod \"kube-auth-proxy-565c49d7c-smwm7\" (UID: \"d92f8a14-740a-4a09-87e9-3e39da9d2788\") " pod="openshift-ingress/kube-auth-proxy-565c49d7c-smwm7"
Sep 15 20:13:35.636131 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:35.636101    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d92f8a14-740a-4a09-87e9-3e39da9d2788-tls-certs\") pod \"kube-auth-proxy-565c49d7c-smwm7\" (UID: \"d92f8a14-740a-4a09-87e9-3e39da9d2788\") " pod="openshift-ingress/kube-auth-proxy-565c49d7c-smwm7"
Sep 15 20:13:35.636250 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:35.636230    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d92f8a14-740a-4a09-87e9-3e39da9d2788-tmp\") pod \"kube-auth-proxy-565c49d7c-smwm7\" (UID: \"d92f8a14-740a-4a09-87e9-3e39da9d2788\") " pod="openshift-ingress/kube-auth-proxy-565c49d7c-smwm7"
Sep 15 20:13:35.640299 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:35.640274    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-b4sg4"]
Sep 15 20:13:35.642931 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:35.642906    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdvkk\" (UniqueName: \"kubernetes.io/projected/d92f8a14-740a-4a09-87e9-3e39da9d2788-kube-api-access-kdvkk\") pod \"kube-auth-proxy-565c49d7c-smwm7\" (UID: \"d92f8a14-740a-4a09-87e9-3e39da9d2788\") " pod="openshift-ingress/kube-auth-proxy-565c49d7c-smwm7"
Sep 15 20:13:35.734609 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:35.734558    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vjsh\" (UniqueName: \"kubernetes.io/projected/8e8dc63c-00bd-4917-8463-1795690501b1-kube-api-access-7vjsh\") pod \"odh-model-controller-7998c67dc4-b4sg4\" (UID: \"8e8dc63c-00bd-4917-8463-1795690501b1\") " pod="opendatahub/odh-model-controller-7998c67dc4-b4sg4"
Sep 15 20:13:35.734819 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:35.734618    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e8dc63c-00bd-4917-8463-1795690501b1-cert\") pod \"odh-model-controller-7998c67dc4-b4sg4\" (UID: \"8e8dc63c-00bd-4917-8463-1795690501b1\") " pod="opendatahub/odh-model-controller-7998c67dc4-b4sg4"
Sep 15 20:13:35.751188 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:35.751164    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-565c49d7c-smwm7"
Sep 15 20:13:35.835772 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:35.835712    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vjsh\" (UniqueName: \"kubernetes.io/projected/8e8dc63c-00bd-4917-8463-1795690501b1-kube-api-access-7vjsh\") pod \"odh-model-controller-7998c67dc4-b4sg4\" (UID: \"8e8dc63c-00bd-4917-8463-1795690501b1\") " pod="opendatahub/odh-model-controller-7998c67dc4-b4sg4"
Sep 15 20:13:35.835934 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:35.835784    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e8dc63c-00bd-4917-8463-1795690501b1-cert\") pod \"odh-model-controller-7998c67dc4-b4sg4\" (UID: \"8e8dc63c-00bd-4917-8463-1795690501b1\") " pod="opendatahub/odh-model-controller-7998c67dc4-b4sg4"
Sep 15 20:13:35.838339 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:35.838309    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e8dc63c-00bd-4917-8463-1795690501b1-cert\") pod \"odh-model-controller-7998c67dc4-b4sg4\" (UID: \"8e8dc63c-00bd-4917-8463-1795690501b1\") " pod="opendatahub/odh-model-controller-7998c67dc4-b4sg4"
Sep 15 20:13:35.846195 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:35.846171    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vjsh\" (UniqueName: \"kubernetes.io/projected/8e8dc63c-00bd-4917-8463-1795690501b1-kube-api-access-7vjsh\") pod \"odh-model-controller-7998c67dc4-b4sg4\" (UID: \"8e8dc63c-00bd-4917-8463-1795690501b1\") " pod="opendatahub/odh-model-controller-7998c67dc4-b4sg4"
Sep 15 20:13:35.938349 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:35.938311    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-b4sg4"
Sep 15 20:13:36.025950 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:36.025717    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-565c49d7c-smwm7"]
Sep 15 20:13:36.028262 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:13:36.028216    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd92f8a14_740a_4a09_87e9_3e39da9d2788.slice/crio-1b5bc2f7a42a47d6b1b776ad182a686dbc25aaabbd78f0394a18463113c52444 WatchSource:0}: Error finding container 1b5bc2f7a42a47d6b1b776ad182a686dbc25aaabbd78f0394a18463113c52444: Status 404 returned error can't find the container with id 1b5bc2f7a42a47d6b1b776ad182a686dbc25aaabbd78f0394a18463113c52444
Sep 15 20:13:36.083289 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:36.083257    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-565c49d7c-smwm7" event={"ID":"d92f8a14-740a-4a09-87e9-3e39da9d2788","Type":"ContainerStarted","Data":"1b5bc2f7a42a47d6b1b776ad182a686dbc25aaabbd78f0394a18463113c52444"}
Sep 15 20:13:36.139067 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:36.139024    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-p4gsl"]
Sep 15 20:13:36.142099 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:36.142075    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-p4gsl"
Sep 15 20:13:36.145456 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:36.145434    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 15 20:13:36.145456 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:36.145452    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-r29sw\""
Sep 15 20:13:36.158055 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:36.158025    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-p4gsl"]
Sep 15 20:13:36.214857 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:36.214831    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-b4sg4"]
Sep 15 20:13:36.217258 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:13:36.217233    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e8dc63c_00bd_4917_8463_1795690501b1.slice/crio-122f348b1ad211e491fcd616bd7d74860434c8d7af2610ec00e68eeb663b1a8d WatchSource:0}: Error finding container 122f348b1ad211e491fcd616bd7d74860434c8d7af2610ec00e68eeb663b1a8d: Status 404 returned error can't find the container with id 122f348b1ad211e491fcd616bd7d74860434c8d7af2610ec00e68eeb663b1a8d
Sep 15 20:13:36.239494 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:36.239415    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwgz2\" (UniqueName: \"kubernetes.io/projected/f7d0a0cc-441e-4de1-93f7-99bf335416d2-kube-api-access-lwgz2\") pod \"model-serving-api-54d89bd5fc-p4gsl\" (UID: \"f7d0a0cc-441e-4de1-93f7-99bf335416d2\") " pod="opendatahub/model-serving-api-54d89bd5fc-p4gsl"
Sep 15 20:13:36.239494 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:36.239450    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7d0a0cc-441e-4de1-93f7-99bf335416d2-tls-certs\") pod \"model-serving-api-54d89bd5fc-p4gsl\" (UID: \"f7d0a0cc-441e-4de1-93f7-99bf335416d2\") " pod="opendatahub/model-serving-api-54d89bd5fc-p4gsl"
Sep 15 20:13:36.340657 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:36.340616    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwgz2\" (UniqueName: \"kubernetes.io/projected/f7d0a0cc-441e-4de1-93f7-99bf335416d2-kube-api-access-lwgz2\") pod \"model-serving-api-54d89bd5fc-p4gsl\" (UID: \"f7d0a0cc-441e-4de1-93f7-99bf335416d2\") " pod="opendatahub/model-serving-api-54d89bd5fc-p4gsl"
Sep 15 20:13:36.340657 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:36.340658    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7d0a0cc-441e-4de1-93f7-99bf335416d2-tls-certs\") pod \"model-serving-api-54d89bd5fc-p4gsl\" (UID: \"f7d0a0cc-441e-4de1-93f7-99bf335416d2\") " pod="opendatahub/model-serving-api-54d89bd5fc-p4gsl"
Sep 15 20:13:36.340948 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:13:36.340820    2577 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 15 20:13:36.340948 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:13:36.340889    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7d0a0cc-441e-4de1-93f7-99bf335416d2-tls-certs podName:f7d0a0cc-441e-4de1-93f7-99bf335416d2 nodeName:}" failed. No retries permitted until 2026-09-15 20:13:36.840868227 +0000 UTC m=+523.231821215 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/f7d0a0cc-441e-4de1-93f7-99bf335416d2-tls-certs") pod "model-serving-api-54d89bd5fc-p4gsl" (UID: "f7d0a0cc-441e-4de1-93f7-99bf335416d2") : secret "model-serving-api-tls" not found
Sep 15 20:13:36.350207 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:36.350180    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwgz2\" (UniqueName: \"kubernetes.io/projected/f7d0a0cc-441e-4de1-93f7-99bf335416d2-kube-api-access-lwgz2\") pod \"model-serving-api-54d89bd5fc-p4gsl\" (UID: \"f7d0a0cc-441e-4de1-93f7-99bf335416d2\") " pod="opendatahub/model-serving-api-54d89bd5fc-p4gsl"
Sep 15 20:13:36.846737 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:36.846691    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7d0a0cc-441e-4de1-93f7-99bf335416d2-tls-certs\") pod \"model-serving-api-54d89bd5fc-p4gsl\" (UID: \"f7d0a0cc-441e-4de1-93f7-99bf335416d2\") " pod="opendatahub/model-serving-api-54d89bd5fc-p4gsl"
Sep 15 20:13:36.849881 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:36.849831    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7d0a0cc-441e-4de1-93f7-99bf335416d2-tls-certs\") pod \"model-serving-api-54d89bd5fc-p4gsl\" (UID: \"f7d0a0cc-441e-4de1-93f7-99bf335416d2\") " pod="opendatahub/model-serving-api-54d89bd5fc-p4gsl"
Sep 15 20:13:36.873294 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:36.873249    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq"]
Sep 15 20:13:36.875779 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:36.875756    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq"
Sep 15 20:13:36.879538 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:36.879504    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 15 20:13:36.879538 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:36.879505    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 15 20:13:36.880037 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:36.879899    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-d5rk6\""
Sep 15 20:13:36.880037 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:36.879898    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 15 20:13:36.880037 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:36.879995    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 15 20:13:36.903035 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:36.902979    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq"]
Sep 15 20:13:37.049196 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:37.049156    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-fxgpq\" (UID: \"7e4b53d8-f12f-4683-aa41-09f6a4b5df55\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq"
Sep 15 20:13:37.049449 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:37.049207    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-fxgpq\" (UID: \"7e4b53d8-f12f-4683-aa41-09f6a4b5df55\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq"
Sep 15 20:13:37.049449 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:37.049263    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-fxgpq\" (UID: \"7e4b53d8-f12f-4683-aa41-09f6a4b5df55\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq"
Sep 15 20:13:37.049449 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:37.049348    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-fxgpq\" (UID: \"7e4b53d8-f12f-4683-aa41-09f6a4b5df55\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq"
Sep 15 20:13:37.049449 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:37.049408    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-fxgpq\" (UID: \"7e4b53d8-f12f-4683-aa41-09f6a4b5df55\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq"
Sep 15 20:13:37.049675 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:37.049485    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w9hm\" (UniqueName: \"kubernetes.io/projected/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-kube-api-access-6w9hm\") pod \"istiod-openshift-gateway-7b7995d5cd-fxgpq\" (UID: \"7e4b53d8-f12f-4683-aa41-09f6a4b5df55\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq"
Sep 15 20:13:37.049675 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:37.049516    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-fxgpq\" (UID: \"7e4b53d8-f12f-4683-aa41-09f6a4b5df55\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq"
Sep 15 20:13:37.053323 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:37.053263    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-p4gsl"
Sep 15 20:13:37.094111 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:37.094043    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-b4sg4" event={"ID":"8e8dc63c-00bd-4917-8463-1795690501b1","Type":"ContainerStarted","Data":"122f348b1ad211e491fcd616bd7d74860434c8d7af2610ec00e68eeb663b1a8d"}
Sep 15 20:13:37.155262 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:37.151994    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-fxgpq\" (UID: \"7e4b53d8-f12f-4683-aa41-09f6a4b5df55\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq"
Sep 15 20:13:37.155262 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:37.152081    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-fxgpq\" (UID: \"7e4b53d8-f12f-4683-aa41-09f6a4b5df55\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq"
Sep 15 20:13:37.155262 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:37.152121    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-fxgpq\" (UID: \"7e4b53d8-f12f-4683-aa41-09f6a4b5df55\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq"
Sep 15 20:13:37.155262 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:37.152194    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w9hm\" (UniqueName: \"kubernetes.io/projected/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-kube-api-access-6w9hm\") pod \"istiod-openshift-gateway-7b7995d5cd-fxgpq\" (UID: \"7e4b53d8-f12f-4683-aa41-09f6a4b5df55\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq"
Sep 15 20:13:37.155262 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:37.152230    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-fxgpq\" (UID: \"7e4b53d8-f12f-4683-aa41-09f6a4b5df55\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq"
Sep 15 20:13:37.155262 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:37.152290    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-fxgpq\" (UID: \"7e4b53d8-f12f-4683-aa41-09f6a4b5df55\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq"
Sep 15 20:13:37.155262 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:37.152326    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-fxgpq\" (UID: \"7e4b53d8-f12f-4683-aa41-09f6a4b5df55\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq"
Sep 15 20:13:37.155262 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:37.153146    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-fxgpq\" (UID: \"7e4b53d8-f12f-4683-aa41-09f6a4b5df55\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq"
Sep 15 20:13:37.158745 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:37.158677    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-fxgpq\" (UID: \"7e4b53d8-f12f-4683-aa41-09f6a4b5df55\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq"
Sep 15 20:13:37.159004 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:37.158979    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-fxgpq\" (UID: \"7e4b53d8-f12f-4683-aa41-09f6a4b5df55\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq"
Sep 15 20:13:37.159278 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:37.159257    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-fxgpq\" (UID: \"7e4b53d8-f12f-4683-aa41-09f6a4b5df55\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq"
Sep 15 20:13:37.162622 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:37.162532    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-fxgpq\" (UID: \"7e4b53d8-f12f-4683-aa41-09f6a4b5df55\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq"
Sep 15 20:13:37.169766 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:37.169739    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-fxgpq\" (UID: \"7e4b53d8-f12f-4683-aa41-09f6a4b5df55\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq"
Sep 15 20:13:37.171587 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:37.171478    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w9hm\" (UniqueName: \"kubernetes.io/projected/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-kube-api-access-6w9hm\") pod \"istiod-openshift-gateway-7b7995d5cd-fxgpq\" (UID: \"7e4b53d8-f12f-4683-aa41-09f6a4b5df55\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq"
Sep 15 20:13:37.188459 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:37.188189    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq"
Sep 15 20:13:37.366178 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:37.365996    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-p4gsl"]
Sep 15 20:13:37.370189 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:13:37.370156    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7d0a0cc_441e_4de1_93f7_99bf335416d2.slice/crio-893c2a0cbf641724c61bcdeb103775221a4c95a449c0870a25c43049a5960d40 WatchSource:0}: Error finding container 893c2a0cbf641724c61bcdeb103775221a4c95a449c0870a25c43049a5960d40: Status 404 returned error can't find the container with id 893c2a0cbf641724c61bcdeb103775221a4c95a449c0870a25c43049a5960d40
Sep 15 20:13:37.512076 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:37.512038    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq"]
Sep 15 20:13:37.517801 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:13:37.517763    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e4b53d8_f12f_4683_aa41_09f6a4b5df55.slice/crio-8492a8b4632a4cad24379f65cc7c0cfd13c040e007624c059ba02e56063387e3 WatchSource:0}: Error finding container 8492a8b4632a4cad24379f65cc7c0cfd13c040e007624c059ba02e56063387e3: Status 404 returned error can't find the container with id 8492a8b4632a4cad24379f65cc7c0cfd13c040e007624c059ba02e56063387e3
Sep 15 20:13:38.103240 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:38.103165    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq" event={"ID":"7e4b53d8-f12f-4683-aa41-09f6a4b5df55","Type":"ContainerStarted","Data":"8492a8b4632a4cad24379f65cc7c0cfd13c040e007624c059ba02e56063387e3"}
Sep 15 20:13:38.105205 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:38.105173    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-p4gsl" event={"ID":"f7d0a0cc-441e-4de1-93f7-99bf335416d2","Type":"ContainerStarted","Data":"893c2a0cbf641724c61bcdeb103775221a4c95a449c0870a25c43049a5960d40"}
Sep 15 20:13:41.124695 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:41.124590    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-p4gsl" event={"ID":"f7d0a0cc-441e-4de1-93f7-99bf335416d2","Type":"ContainerStarted","Data":"50100aca78c6722c449ca65d772d2405e66e4af762e81dd663badfbfb62be33d"}
Sep 15 20:13:41.125183 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:41.124762    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-p4gsl"
Sep 15 20:13:41.126455 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:41.126425    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-b4sg4" event={"ID":"8e8dc63c-00bd-4917-8463-1795690501b1","Type":"ContainerStarted","Data":"c7bc66fb2bc5d4997601aabcb51bda4a54bb03203601600a684d77700bd9bc58"}
Sep 15 20:13:41.126593 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:41.126480    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-b4sg4"
Sep 15 20:13:41.128145 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:41.128123    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-565c49d7c-smwm7" event={"ID":"d92f8a14-740a-4a09-87e9-3e39da9d2788","Type":"ContainerStarted","Data":"94dac2a563a544b5e8ddd0845079b9f90ed47fbc319d081f17de55e410fea405"}
Sep 15 20:13:41.160270 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:41.160212    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-p4gsl" podStartSLOduration=1.68561187 podStartE2EDuration="5.160190182s" totalImagesPullingTime="3.474578312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:13:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:13:41.141850593 +0000 UTC m=+527.532803615" watchObservedRunningTime="2026-09-15 20:13:41.160190182 +0000 UTC m=+527.551143179"
Sep 15 20:13:41.160946 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:41.160910    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-565c49d7c-smwm7" podStartSLOduration=1.34650972 podStartE2EDuration="6.160900887s" totalImagesPullingTime="4.814391167s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:13:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:13:41.159013226 +0000 UTC m=+527.549966224" watchObservedRunningTime="2026-09-15 20:13:41.160900887 +0000 UTC m=+527.551853913"
Sep 15 20:13:41.177576 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:41.177502    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-b4sg4" podStartSLOduration=1.540477369 podStartE2EDuration="6.177484212s" totalImagesPullingTime="4.637006843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:13:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:13:41.175090261 +0000 UTC m=+527.566043258" watchObservedRunningTime="2026-09-15 20:13:41.177484212 +0000 UTC m=+527.568437209"
Sep 15 20:13:43.056404 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:43.056353    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 20:13:43.056705 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:43.056448    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 20:13:43.141349 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:43.141304    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq" event={"ID":"7e4b53d8-f12f-4683-aa41-09f6a4b5df55","Type":"ContainerStarted","Data":"1ec7ae552b7f59608f99956c8cc3d0aed283c79c9bac55399a757d5422f9c3dd"}
Sep 15 20:13:43.141549 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:43.141433    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq"
Sep 15 20:13:43.166931 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:43.166874    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq" podStartSLOduration=1.63146595 podStartE2EDuration="7.166856829s" totalImagesPullingTime="5.535390879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:13:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:13:43.166497675 +0000 UTC m=+529.557450672" watchObservedRunningTime="2026-09-15 20:13:43.166856829 +0000 UTC m=+529.557809826"
Sep 15 20:13:44.105319 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:44.105282    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f"]
Sep 15 20:13:44.107642 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:44.107624    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f"
Sep 15 20:13:44.115759 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:44.115718    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-mzppk\""
Sep 15 20:13:44.116180 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:44.116157    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrz4w\" (UniqueName: \"kubernetes.io/projected/6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf-kube-api-access-mrz4w\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799csl74f\" (UID: \"6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f"
Sep 15 20:13:44.116276 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:44.116199    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799csl74f\" (UID: \"6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f"
Sep 15 20:13:44.116325 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:44.116272    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799csl74f\" (UID: \"6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f"
Sep 15 20:13:44.116325 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:44.116312    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799csl74f\" (UID: \"6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f"
Sep 15 20:13:44.116388 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:44.116344    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799csl74f\" (UID: \"6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f"
Sep 15 20:13:44.116388 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:44.116371    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799csl74f\" (UID: \"6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f"
Sep 15 20:13:44.116452 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:44.116391    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799csl74f\" (UID: \"6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f"
Sep 15 20:13:44.116452 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:44.116414    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799csl74f\" (UID: \"6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f"
Sep 15 20:13:44.116515 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:44.116459    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799csl74f\" (UID: \"6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f"
Sep 15 20:13:44.149416 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:44.149384    2577 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-fxgpq container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 20:13:44.149698 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:44.149666    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq" podUID="7e4b53d8-f12f-4683-aa41-09f6a4b5df55" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 20:13:44.150971 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:44.150948    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f"]
Sep 15 20:13:44.217255 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:44.217211    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799csl74f\" (UID: \"6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f"
Sep 15 20:13:44.217447 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:44.217283    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799csl74f\" (UID: \"6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f"
Sep 15 20:13:44.217447 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:44.217333    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799csl74f\" (UID: \"6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f"
Sep 15 20:13:44.217447 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:44.217358    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799csl74f\" (UID: \"6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f"
Sep 15 20:13:44.217447 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:44.217386    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799csl74f\" (UID: \"6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f"
Sep 15 20:13:44.217447 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:44.217431    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799csl74f\" (UID: \"6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f"
Sep 15 20:13:44.217695 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:44.217642    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799csl74f\" (UID: \"6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f"
Sep 15 20:13:44.217777 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:44.217704    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799csl74f\" (UID: \"6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f"
Sep 15 20:13:44.217840 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:44.217791    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrz4w\" (UniqueName: \"kubernetes.io/projected/6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf-kube-api-access-mrz4w\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799csl74f\" (UID: \"6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f"
Sep 15 20:13:44.218002 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:44.217976    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799csl74f\" (UID: \"6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f"
Sep 15 20:13:44.218075 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:44.218017    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799csl74f\" (UID: \"6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f"
Sep 15 20:13:44.218165 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:44.218138    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799csl74f\" (UID: \"6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f"
Sep 15 20:13:44.218243 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:44.218161    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799csl74f\" (UID: \"6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f"
Sep 15 20:13:44.218243 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:44.218197    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799csl74f\" (UID: \"6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f"
Sep 15 20:13:44.219950 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:44.219933    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799csl74f\" (UID: \"6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f"
Sep 15 20:13:44.220430 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:44.220407    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799csl74f\" (UID: \"6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f"
Sep 15 20:13:44.229372 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:44.229345    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799csl74f\" (UID: \"6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f"
Sep 15 20:13:44.233765 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:44.233710    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrz4w\" (UniqueName: \"kubernetes.io/projected/6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf-kube-api-access-mrz4w\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799csl74f\" (UID: \"6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f"
Sep 15 20:13:44.417817 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:44.417719    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f"
Sep 15 20:13:44.775532 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:13:44.775504    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e72cd0b_7ee7_418d_96fd_fb2c31b99ecf.slice/crio-92d3797c08b4c57d1d882762a34feca87a2e1ed48f3c0232f5c0fa08e2d393a8 WatchSource:0}: Error finding container 92d3797c08b4c57d1d882762a34feca87a2e1ed48f3c0232f5c0fa08e2d393a8: Status 404 returned error can't find the container with id 92d3797c08b4c57d1d882762a34feca87a2e1ed48f3c0232f5c0fa08e2d393a8
Sep 15 20:13:44.777478 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:44.777456    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f"]
Sep 15 20:13:45.153652 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:45.153558    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f" event={"ID":"6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf","Type":"ContainerStarted","Data":"92d3797c08b4c57d1d882762a34feca87a2e1ed48f3c0232f5c0fa08e2d393a8"}
Sep 15 20:13:45.908020 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:45.907981    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-bmnwk"]
Sep 15 20:13:45.911184 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:45.911161    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bmnwk"
Sep 15 20:13:45.914222 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:45.914197    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 15 20:13:45.914616 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:45.914597    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-l2rsk\""
Sep 15 20:13:45.927829 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:45.927797    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-8gvgr"]
Sep 15 20:13:45.930089 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:45.930068    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-bmnwk"]
Sep 15 20:13:45.930221 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:45.930176    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8gvgr"
Sep 15 20:13:45.933822 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:45.933796    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 15 20:13:45.934041 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:45.934012    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-qjxx2\""
Sep 15 20:13:45.934189 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:45.934162    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8b2eb956-9488-4880-a693-464ef27c2e72-cert\") pod \"kserve-controller-manager-d6f4b65b8-bmnwk\" (UID: \"8b2eb956-9488-4880-a693-464ef27c2e72\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bmnwk"
Sep 15 20:13:45.934300 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:45.934283    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvz2j\" (UniqueName: \"kubernetes.io/projected/8b2eb956-9488-4880-a693-464ef27c2e72-kube-api-access-wvz2j\") pod \"kserve-controller-manager-d6f4b65b8-bmnwk\" (UID: \"8b2eb956-9488-4880-a693-464ef27c2e72\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bmnwk"
Sep 15 20:13:45.950089 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:45.950057    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-8gvgr"]
Sep 15 20:13:46.035668 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:46.035622    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svdw2\" (UniqueName: \"kubernetes.io/projected/c53c751d-5dda-4a25-9112-8c3d9f89c665-kube-api-access-svdw2\") pod \"llmisvc-controller-manager-777c6bd7d5-8gvgr\" (UID: \"c53c751d-5dda-4a25-9112-8c3d9f89c665\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8gvgr"
Sep 15 20:13:46.035881 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:46.035702    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvz2j\" (UniqueName: \"kubernetes.io/projected/8b2eb956-9488-4880-a693-464ef27c2e72-kube-api-access-wvz2j\") pod \"kserve-controller-manager-d6f4b65b8-bmnwk\" (UID: \"8b2eb956-9488-4880-a693-464ef27c2e72\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bmnwk"
Sep 15 20:13:46.035881 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:46.035790    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c53c751d-5dda-4a25-9112-8c3d9f89c665-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-8gvgr\" (UID: \"c53c751d-5dda-4a25-9112-8c3d9f89c665\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8gvgr"
Sep 15 20:13:46.035881 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:46.035868    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8b2eb956-9488-4880-a693-464ef27c2e72-cert\") pod \"kserve-controller-manager-d6f4b65b8-bmnwk\" (UID: \"8b2eb956-9488-4880-a693-464ef27c2e72\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bmnwk"
Sep 15 20:13:46.036059 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:13:46.035969    2577 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Sep 15 20:13:46.036059 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:13:46.036027    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b2eb956-9488-4880-a693-464ef27c2e72-cert podName:8b2eb956-9488-4880-a693-464ef27c2e72 nodeName:}" failed. No retries permitted until 2026-09-15 20:13:46.536008414 +0000 UTC m=+532.926961388 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/8b2eb956-9488-4880-a693-464ef27c2e72-cert") pod "kserve-controller-manager-d6f4b65b8-bmnwk" (UID: "8b2eb956-9488-4880-a693-464ef27c2e72") : secret "kserve-webhook-server-cert" not found
Sep 15 20:13:46.049858 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:46.049823    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvz2j\" (UniqueName: \"kubernetes.io/projected/8b2eb956-9488-4880-a693-464ef27c2e72-kube-api-access-wvz2j\") pod \"kserve-controller-manager-d6f4b65b8-bmnwk\" (UID: \"8b2eb956-9488-4880-a693-464ef27c2e72\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bmnwk"
Sep 15 20:13:46.136926 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:46.136885    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svdw2\" (UniqueName: \"kubernetes.io/projected/c53c751d-5dda-4a25-9112-8c3d9f89c665-kube-api-access-svdw2\") pod \"llmisvc-controller-manager-777c6bd7d5-8gvgr\" (UID: \"c53c751d-5dda-4a25-9112-8c3d9f89c665\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8gvgr"
Sep 15 20:13:46.137094 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:46.137013    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c53c751d-5dda-4a25-9112-8c3d9f89c665-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-8gvgr\" (UID: \"c53c751d-5dda-4a25-9112-8c3d9f89c665\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8gvgr"
Sep 15 20:13:46.140422 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:46.140371    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c53c751d-5dda-4a25-9112-8c3d9f89c665-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-8gvgr\" (UID: \"c53c751d-5dda-4a25-9112-8c3d9f89c665\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8gvgr"
Sep 15 20:13:46.147235 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:46.147184    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svdw2\" (UniqueName: \"kubernetes.io/projected/c53c751d-5dda-4a25-9112-8c3d9f89c665-kube-api-access-svdw2\") pod \"llmisvc-controller-manager-777c6bd7d5-8gvgr\" (UID: \"c53c751d-5dda-4a25-9112-8c3d9f89c665\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8gvgr"
Sep 15 20:13:46.241766 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:46.241708    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8gvgr"
Sep 15 20:13:46.529938 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:46.529904    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-8gvgr"]
Sep 15 20:13:46.532272 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:13:46.532240    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc53c751d_5dda_4a25_9112_8c3d9f89c665.slice/crio-0e874af168050c6e40d711c58106af8bd58fe4055590510bcfce472ae4222232 WatchSource:0}: Error finding container 0e874af168050c6e40d711c58106af8bd58fe4055590510bcfce472ae4222232: Status 404 returned error can't find the container with id 0e874af168050c6e40d711c58106af8bd58fe4055590510bcfce472ae4222232
Sep 15 20:13:46.540287 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:46.540261    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8b2eb956-9488-4880-a693-464ef27c2e72-cert\") pod \"kserve-controller-manager-d6f4b65b8-bmnwk\" (UID: \"8b2eb956-9488-4880-a693-464ef27c2e72\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bmnwk"
Sep 15 20:13:46.543375 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:46.543353    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8b2eb956-9488-4880-a693-464ef27c2e72-cert\") pod \"kserve-controller-manager-d6f4b65b8-bmnwk\" (UID: \"8b2eb956-9488-4880-a693-464ef27c2e72\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bmnwk"
Sep 15 20:13:46.822195 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:46.822091    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bmnwk"
Sep 15 20:13:47.148155 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:47.148128    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq"
Sep 15 20:13:47.148600 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:47.148576    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq"]
Sep 15 20:13:47.151359 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:47.151332    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-bmnwk"]
Sep 15 20:13:47.167495 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:47.167457    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8gvgr" event={"ID":"c53c751d-5dda-4a25-9112-8c3d9f89c665","Type":"ContainerStarted","Data":"0e874af168050c6e40d711c58106af8bd58fe4055590510bcfce472ae4222232"}
Sep 15 20:13:47.167655 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:47.167559    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq" podUID="7e4b53d8-f12f-4683-aa41-09f6a4b5df55" containerName="discovery" containerID="cri-o://1ec7ae552b7f59608f99956c8cc3d0aed283c79c9bac55399a757d5422f9c3dd" gracePeriod=30
Sep 15 20:13:47.913506 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:13:47.913472    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b2eb956_9488_4880_a693_464ef27c2e72.slice/crio-fa669329affe897d63cf453f84065c433565a7090cd89bdc430adfb5b3f00a06 WatchSource:0}: Error finding container fa669329affe897d63cf453f84065c433565a7090cd89bdc430adfb5b3f00a06: Status 404 returned error can't find the container with id fa669329affe897d63cf453f84065c433565a7090cd89bdc430adfb5b3f00a06
Sep 15 20:13:47.953963 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:47.953912    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 20:13:47.954067 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:47.953996    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 20:13:47.954067 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:47.954035    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 20:13:48.074227 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:48.074198    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq"
Sep 15 20:13:48.157095 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:48.157070    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-local-certs\") pod \"7e4b53d8-f12f-4683-aa41-09f6a4b5df55\" (UID: \"7e4b53d8-f12f-4683-aa41-09f6a4b5df55\") "
Sep 15 20:13:48.157283 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:48.157111    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-istio-csr-dns-cert\") pod \"7e4b53d8-f12f-4683-aa41-09f6a4b5df55\" (UID: \"7e4b53d8-f12f-4683-aa41-09f6a4b5df55\") "
Sep 15 20:13:48.157283 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:48.157147    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6w9hm\" (UniqueName: \"kubernetes.io/projected/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-kube-api-access-6w9hm\") pod \"7e4b53d8-f12f-4683-aa41-09f6a4b5df55\" (UID: \"7e4b53d8-f12f-4683-aa41-09f6a4b5df55\") "
Sep 15 20:13:48.157413 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:48.157313    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-istio-token\") pod \"7e4b53d8-f12f-4683-aa41-09f6a4b5df55\" (UID: \"7e4b53d8-f12f-4683-aa41-09f6a4b5df55\") "
Sep 15 20:13:48.157413 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:48.157349    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-istio-csr-ca-configmap\") pod \"7e4b53d8-f12f-4683-aa41-09f6a4b5df55\" (UID: \"7e4b53d8-f12f-4683-aa41-09f6a4b5df55\") "
Sep 15 20:13:48.157529 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:48.157418    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-istio-kubeconfig\") pod \"7e4b53d8-f12f-4683-aa41-09f6a4b5df55\" (UID: \"7e4b53d8-f12f-4683-aa41-09f6a4b5df55\") "
Sep 15 20:13:48.157529 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:48.157448    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-cacerts\") pod \"7e4b53d8-f12f-4683-aa41-09f6a4b5df55\" (UID: \"7e4b53d8-f12f-4683-aa41-09f6a4b5df55\") "
Sep 15 20:13:48.157837 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:48.157802    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "7e4b53d8-f12f-4683-aa41-09f6a4b5df55" (UID: "7e4b53d8-f12f-4683-aa41-09f6a4b5df55"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:13:48.159982 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:48.159949    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "7e4b53d8-f12f-4683-aa41-09f6a4b5df55" (UID: "7e4b53d8-f12f-4683-aa41-09f6a4b5df55"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:13:48.160134 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:48.160111    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-local-certs" (OuterVolumeSpecName: "local-certs") pod "7e4b53d8-f12f-4683-aa41-09f6a4b5df55" (UID: "7e4b53d8-f12f-4683-aa41-09f6a4b5df55"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:13:48.160200 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:48.160180    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-kube-api-access-6w9hm" (OuterVolumeSpecName: "kube-api-access-6w9hm") pod "7e4b53d8-f12f-4683-aa41-09f6a4b5df55" (UID: "7e4b53d8-f12f-4683-aa41-09f6a4b5df55"). InnerVolumeSpecName "kube-api-access-6w9hm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:13:48.160337 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:48.160298    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-istio-token" (OuterVolumeSpecName: "istio-token") pod "7e4b53d8-f12f-4683-aa41-09f6a4b5df55" (UID: "7e4b53d8-f12f-4683-aa41-09f6a4b5df55"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:13:48.160451 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:48.160344    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-cacerts" (OuterVolumeSpecName: "cacerts") pod "7e4b53d8-f12f-4683-aa41-09f6a4b5df55" (UID: "7e4b53d8-f12f-4683-aa41-09f6a4b5df55"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:13:48.160533 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:48.160511    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "7e4b53d8-f12f-4683-aa41-09f6a4b5df55" (UID: "7e4b53d8-f12f-4683-aa41-09f6a4b5df55"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:13:48.174517 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:48.174434    2577 generic.go:356] "Generic (PLEG): container finished" podID="7e4b53d8-f12f-4683-aa41-09f6a4b5df55" containerID="1ec7ae552b7f59608f99956c8cc3d0aed283c79c9bac55399a757d5422f9c3dd" exitCode=0
Sep 15 20:13:48.174649 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:48.174595    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq"
Sep 15 20:13:48.175271 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:48.175228    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq" event={"ID":"7e4b53d8-f12f-4683-aa41-09f6a4b5df55","Type":"ContainerDied","Data":"1ec7ae552b7f59608f99956c8cc3d0aed283c79c9bac55399a757d5422f9c3dd"}
Sep 15 20:13:48.175409 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:48.175313    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq" event={"ID":"7e4b53d8-f12f-4683-aa41-09f6a4b5df55","Type":"ContainerDied","Data":"8492a8b4632a4cad24379f65cc7c0cfd13c040e007624c059ba02e56063387e3"}
Sep 15 20:13:48.175409 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:48.175353    2577 scope.go:117] "RemoveContainer" containerID="1ec7ae552b7f59608f99956c8cc3d0aed283c79c9bac55399a757d5422f9c3dd"
Sep 15 20:13:48.177367 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:48.177303    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bmnwk" event={"ID":"8b2eb956-9488-4880-a693-464ef27c2e72","Type":"ContainerStarted","Data":"fa669329affe897d63cf453f84065c433565a7090cd89bdc430adfb5b3f00a06"}
Sep 15 20:13:48.179048 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:48.179023    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f" event={"ID":"6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf","Type":"ContainerStarted","Data":"6fc4b871f27af4f5eb71d9f7f175a8cb4f5ec8dc0fff7c6ed96196787b6cd0da"}
Sep 15 20:13:48.200973 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:48.200951    2577 scope.go:117] "RemoveContainer" containerID="1ec7ae552b7f59608f99956c8cc3d0aed283c79c9bac55399a757d5422f9c3dd"
Sep 15 20:13:48.201357 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:13:48.201333    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ec7ae552b7f59608f99956c8cc3d0aed283c79c9bac55399a757d5422f9c3dd\": container with ID starting with 1ec7ae552b7f59608f99956c8cc3d0aed283c79c9bac55399a757d5422f9c3dd not found: ID does not exist" containerID="1ec7ae552b7f59608f99956c8cc3d0aed283c79c9bac55399a757d5422f9c3dd"
Sep 15 20:13:48.201467 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:48.201366    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ec7ae552b7f59608f99956c8cc3d0aed283c79c9bac55399a757d5422f9c3dd"} err="failed to get container status \"1ec7ae552b7f59608f99956c8cc3d0aed283c79c9bac55399a757d5422f9c3dd\": rpc error: code = NotFound desc = could not find container \"1ec7ae552b7f59608f99956c8cc3d0aed283c79c9bac55399a757d5422f9c3dd\": container with ID starting with 1ec7ae552b7f59608f99956c8cc3d0aed283c79c9bac55399a757d5422f9c3dd not found: ID does not exist"
Sep 15 20:13:48.204406 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:48.204333    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f" podStartSLOduration=1.028242417 podStartE2EDuration="4.204313983s" totalImagesPullingTime="3.176071566s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:13:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:13:48.202640041 +0000 UTC m=+534.593593107" watchObservedRunningTime="2026-09-15 20:13:48.204313983 +0000 UTC m=+534.595266982"
Sep 15 20:13:48.223526 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:48.223494    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq"]
Sep 15 20:13:48.228657 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:48.228623    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fxgpq"]
Sep 15 20:13:48.259142 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:48.258818    2577 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-istio-csr-dns-cert\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:13:48.259142 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:48.258854    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6w9hm\" (UniqueName: \"kubernetes.io/projected/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-kube-api-access-6w9hm\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:13:48.259142 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:48.258870    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-istio-token\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:13:48.259142 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:48.258884    2577 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-istio-csr-ca-configmap\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:13:48.259142 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:48.258896    2577 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-istio-kubeconfig\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:13:48.259142 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:48.258908    2577 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-cacerts\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:13:48.259142 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:48.258920    2577 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e4b53d8-f12f-4683-aa41-09f6a4b5df55-local-certs\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:13:48.418191 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:48.418152    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f"
Sep 15 20:13:49.423664 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:49.423626    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f"
Sep 15 20:13:50.062503 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:50.062464    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e4b53d8-f12f-4683-aa41-09f6a4b5df55" path="/var/lib/kubelet/pods/7e4b53d8-f12f-4683-aa41-09f6a4b5df55/volumes"
Sep 15 20:13:50.193206 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:50.193171    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f"
Sep 15 20:13:50.195189 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:50.195156    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799csl74f"
Sep 15 20:13:51.200911 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:51.200810    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8gvgr" event={"ID":"c53c751d-5dda-4a25-9112-8c3d9f89c665","Type":"ContainerStarted","Data":"a35425415d245f89f165c435453139a3eb59be04e345b64b015582520f795fce"}
Sep 15 20:13:51.201337 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:51.200908    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8gvgr"
Sep 15 20:13:51.202489 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:51.202466    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bmnwk" event={"ID":"8b2eb956-9488-4880-a693-464ef27c2e72","Type":"ContainerStarted","Data":"1a3045d4d279f66941e8f01369e40c44f2a6ba62bf15a39b571850ec0c05ccfa"}
Sep 15 20:13:51.202642 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:51.202623    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bmnwk"
Sep 15 20:13:51.235121 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:51.235070    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8gvgr" podStartSLOduration=1.981567311 podStartE2EDuration="6.235050192s" totalImagesPullingTime="4.253482881s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:13:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:13:51.233840029 +0000 UTC m=+537.624793026" watchObservedRunningTime="2026-09-15 20:13:51.235050192 +0000 UTC m=+537.626003190"
Sep 15 20:13:51.262964 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:51.262910    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bmnwk" podStartSLOduration=3.388673112 podStartE2EDuration="6.262893472s" totalImagesPullingTime="2.87422036s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:13:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:13:51.260321492 +0000 UTC m=+537.651274489" watchObservedRunningTime="2026-09-15 20:13:51.262893472 +0000 UTC m=+537.653846489"
Sep 15 20:13:52.136610 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:52.136577    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-b4sg4"
Sep 15 20:13:52.138660 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:13:52.138636    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-p4gsl"
Sep 15 20:14:22.210713 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:22.210678    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8gvgr"
Sep 15 20:14:22.214336 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:22.214311    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bmnwk"
Sep 15 20:14:37.483777 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:37.483709    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f89cb66dd-rhb7p"]
Sep 15 20:14:37.484557 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:37.484523    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e4b53d8-f12f-4683-aa41-09f6a4b5df55" containerName="discovery"
Sep 15 20:14:37.484557 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:37.484559    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e4b53d8-f12f-4683-aa41-09f6a4b5df55" containerName="discovery"
Sep 15 20:14:37.484736 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:37.484659    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e4b53d8-f12f-4683-aa41-09f6a4b5df55" containerName="discovery"
Sep 15 20:14:37.491575 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:37.491554    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f89cb66dd-rhb7p"
Sep 15 20:14:37.494443 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:37.494423    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 15 20:14:37.494561 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:37.494471    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 15 20:14:37.495843 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:37.495825    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-2kzkh\""
Sep 15 20:14:37.496055 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:37.495917    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 15 20:14:37.496055 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:37.495874    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 15 20:14:37.496055 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:37.495903    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 15 20:14:37.496055 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:37.495874    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 15 20:14:37.496500 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:37.496217    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 15 20:14:37.501551 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:37.501528    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 15 20:14:37.502330 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:37.502308    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f89cb66dd-rhb7p"]
Sep 15 20:14:37.512534 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:37.512507    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4193569c-6ce6-4671-952b-e1601ee2f70b-trusted-ca-bundle\") pod \"console-7f89cb66dd-rhb7p\" (UID: \"4193569c-6ce6-4671-952b-e1601ee2f70b\") " pod="openshift-console/console-7f89cb66dd-rhb7p"
Sep 15 20:14:37.512696 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:37.512571    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4193569c-6ce6-4671-952b-e1601ee2f70b-service-ca\") pod \"console-7f89cb66dd-rhb7p\" (UID: \"4193569c-6ce6-4671-952b-e1601ee2f70b\") " pod="openshift-console/console-7f89cb66dd-rhb7p"
Sep 15 20:14:37.512696 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:37.512607    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4193569c-6ce6-4671-952b-e1601ee2f70b-console-config\") pod \"console-7f89cb66dd-rhb7p\" (UID: \"4193569c-6ce6-4671-952b-e1601ee2f70b\") " pod="openshift-console/console-7f89cb66dd-rhb7p"
Sep 15 20:14:37.512696 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:37.512642    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4193569c-6ce6-4671-952b-e1601ee2f70b-console-serving-cert\") pod \"console-7f89cb66dd-rhb7p\" (UID: \"4193569c-6ce6-4671-952b-e1601ee2f70b\") " pod="openshift-console/console-7f89cb66dd-rhb7p"
Sep 15 20:14:37.512696 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:37.512668    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4193569c-6ce6-4671-952b-e1601ee2f70b-console-oauth-config\") pod \"console-7f89cb66dd-rhb7p\" (UID: \"4193569c-6ce6-4671-952b-e1601ee2f70b\") " pod="openshift-console/console-7f89cb66dd-rhb7p"
Sep 15 20:14:37.512959 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:37.512744    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4193569c-6ce6-4671-952b-e1601ee2f70b-oauth-serving-cert\") pod \"console-7f89cb66dd-rhb7p\" (UID: \"4193569c-6ce6-4671-952b-e1601ee2f70b\") " pod="openshift-console/console-7f89cb66dd-rhb7p"
Sep 15 20:14:37.512959 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:37.512768    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5n8j5\" (UniqueName: \"kubernetes.io/projected/4193569c-6ce6-4671-952b-e1601ee2f70b-kube-api-access-5n8j5\") pod \"console-7f89cb66dd-rhb7p\" (UID: \"4193569c-6ce6-4671-952b-e1601ee2f70b\") " pod="openshift-console/console-7f89cb66dd-rhb7p"
Sep 15 20:14:37.613601 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:37.613563    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4193569c-6ce6-4671-952b-e1601ee2f70b-service-ca\") pod \"console-7f89cb66dd-rhb7p\" (UID: \"4193569c-6ce6-4671-952b-e1601ee2f70b\") " pod="openshift-console/console-7f89cb66dd-rhb7p"
Sep 15 20:14:37.613601 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:37.613608    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4193569c-6ce6-4671-952b-e1601ee2f70b-console-config\") pod \"console-7f89cb66dd-rhb7p\" (UID: \"4193569c-6ce6-4671-952b-e1601ee2f70b\") " pod="openshift-console/console-7f89cb66dd-rhb7p"
Sep 15 20:14:37.613913 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:37.613638    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4193569c-6ce6-4671-952b-e1601ee2f70b-console-serving-cert\") pod \"console-7f89cb66dd-rhb7p\" (UID: \"4193569c-6ce6-4671-952b-e1601ee2f70b\") " pod="openshift-console/console-7f89cb66dd-rhb7p"
Sep 15 20:14:37.613913 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:37.613662    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4193569c-6ce6-4671-952b-e1601ee2f70b-console-oauth-config\") pod \"console-7f89cb66dd-rhb7p\" (UID: \"4193569c-6ce6-4671-952b-e1601ee2f70b\") " pod="openshift-console/console-7f89cb66dd-rhb7p"
Sep 15 20:14:37.613913 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:37.613715    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4193569c-6ce6-4671-952b-e1601ee2f70b-oauth-serving-cert\") pod \"console-7f89cb66dd-rhb7p\" (UID: \"4193569c-6ce6-4671-952b-e1601ee2f70b\") " pod="openshift-console/console-7f89cb66dd-rhb7p"
Sep 15 20:14:37.613913 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:37.613777    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5n8j5\" (UniqueName: \"kubernetes.io/projected/4193569c-6ce6-4671-952b-e1601ee2f70b-kube-api-access-5n8j5\") pod \"console-7f89cb66dd-rhb7p\" (UID: \"4193569c-6ce6-4671-952b-e1601ee2f70b\") " pod="openshift-console/console-7f89cb66dd-rhb7p"
Sep 15 20:14:37.613913 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:37.613821    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4193569c-6ce6-4671-952b-e1601ee2f70b-trusted-ca-bundle\") pod \"console-7f89cb66dd-rhb7p\" (UID: \"4193569c-6ce6-4671-952b-e1601ee2f70b\") " pod="openshift-console/console-7f89cb66dd-rhb7p"
Sep 15 20:14:37.614449 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:37.614424    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4193569c-6ce6-4671-952b-e1601ee2f70b-service-ca\") pod \"console-7f89cb66dd-rhb7p\" (UID: \"4193569c-6ce6-4671-952b-e1601ee2f70b\") " pod="openshift-console/console-7f89cb66dd-rhb7p"
Sep 15 20:14:37.614558 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:37.614529    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4193569c-6ce6-4671-952b-e1601ee2f70b-console-config\") pod \"console-7f89cb66dd-rhb7p\" (UID: \"4193569c-6ce6-4671-952b-e1601ee2f70b\") " pod="openshift-console/console-7f89cb66dd-rhb7p"
Sep 15 20:14:37.614600 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:37.614577    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4193569c-6ce6-4671-952b-e1601ee2f70b-oauth-serving-cert\") pod \"console-7f89cb66dd-rhb7p\" (UID: \"4193569c-6ce6-4671-952b-e1601ee2f70b\") " pod="openshift-console/console-7f89cb66dd-rhb7p"
Sep 15 20:14:37.614810 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:37.614787    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4193569c-6ce6-4671-952b-e1601ee2f70b-trusted-ca-bundle\") pod \"console-7f89cb66dd-rhb7p\" (UID: \"4193569c-6ce6-4671-952b-e1601ee2f70b\") " pod="openshift-console/console-7f89cb66dd-rhb7p"
Sep 15 20:14:37.616175 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:37.616144    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4193569c-6ce6-4671-952b-e1601ee2f70b-console-oauth-config\") pod \"console-7f89cb66dd-rhb7p\" (UID: \"4193569c-6ce6-4671-952b-e1601ee2f70b\") " pod="openshift-console/console-7f89cb66dd-rhb7p"
Sep 15 20:14:37.616306 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:37.616289    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4193569c-6ce6-4671-952b-e1601ee2f70b-console-serving-cert\") pod \"console-7f89cb66dd-rhb7p\" (UID: \"4193569c-6ce6-4671-952b-e1601ee2f70b\") " pod="openshift-console/console-7f89cb66dd-rhb7p"
Sep 15 20:14:37.623436 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:37.623414    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5n8j5\" (UniqueName: \"kubernetes.io/projected/4193569c-6ce6-4671-952b-e1601ee2f70b-kube-api-access-5n8j5\") pod \"console-7f89cb66dd-rhb7p\" (UID: \"4193569c-6ce6-4671-952b-e1601ee2f70b\") " pod="openshift-console/console-7f89cb66dd-rhb7p"
Sep 15 20:14:37.802762 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:37.802635    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f89cb66dd-rhb7p"
Sep 15 20:14:38.077926 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:38.077852    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f89cb66dd-rhb7p"]
Sep 15 20:14:38.080122 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:14:38.080094    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4193569c_6ce6_4671_952b_e1601ee2f70b.slice/crio-06f14cb939883a9a64560e35820169073961ad46f0b6f5366c8232b41aaac248 WatchSource:0}: Error finding container 06f14cb939883a9a64560e35820169073961ad46f0b6f5366c8232b41aaac248: Status 404 returned error can't find the container with id 06f14cb939883a9a64560e35820169073961ad46f0b6f5366c8232b41aaac248
Sep 15 20:14:38.455412 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:38.455385    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f89cb66dd-rhb7p" event={"ID":"4193569c-6ce6-4671-952b-e1601ee2f70b","Type":"ContainerStarted","Data":"e63caca78360f83b6aa7258a3c1f7fc5437489983933de5c95e16521572b015e"}
Sep 15 20:14:38.455564 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:38.455433    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f89cb66dd-rhb7p" event={"ID":"4193569c-6ce6-4671-952b-e1601ee2f70b","Type":"ContainerStarted","Data":"06f14cb939883a9a64560e35820169073961ad46f0b6f5366c8232b41aaac248"}
Sep 15 20:14:38.491016 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:38.490963    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7f89cb66dd-rhb7p" podStartSLOduration=1.490944704 podStartE2EDuration="1.490944704s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:14:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:14:38.489631202 +0000 UTC m=+584.880584199" watchObservedRunningTime="2026-09-15 20:14:38.490944704 +0000 UTC m=+584.881897701"
Sep 15 20:14:40.729269 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:40.729230    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsv2h"]
Sep 15 20:14:40.732019 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:40.732001    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsv2h"
Sep 15 20:14:40.734614 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:40.734593    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 15 20:14:40.735612 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:40.735586    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 15 20:14:40.735681 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:40.735617    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-k79st\""
Sep 15 20:14:40.742084 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:40.742056    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsv2h"]
Sep 15 20:14:40.832540 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:40.832504    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cp7nrn"]
Sep 15 20:14:40.835045 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:40.835020    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cp7nrn"
Sep 15 20:14:40.844978 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:40.844944    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e99ab7d7-2145-4c7b-8138-6473d37226cb-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsv2h\" (UID: \"e99ab7d7-2145-4c7b-8138-6473d37226cb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsv2h"
Sep 15 20:14:40.845111 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:40.845000    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pz2fl\" (UniqueName: \"kubernetes.io/projected/e99ab7d7-2145-4c7b-8138-6473d37226cb-kube-api-access-pz2fl\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsv2h\" (UID: \"e99ab7d7-2145-4c7b-8138-6473d37226cb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsv2h"
Sep 15 20:14:40.845188 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:40.845100    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e99ab7d7-2145-4c7b-8138-6473d37226cb-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsv2h\" (UID: \"e99ab7d7-2145-4c7b-8138-6473d37226cb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsv2h"
Sep 15 20:14:40.845313 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:40.845296    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cp7nrn"]
Sep 15 20:14:40.931041 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:40.931002    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abvlzh"]
Sep 15 20:14:40.933558 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:40.933536    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abvlzh"
Sep 15 20:14:40.944516 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:40.944489    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abvlzh"]
Sep 15 20:14:40.946432 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:40.946407    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e99ab7d7-2145-4c7b-8138-6473d37226cb-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsv2h\" (UID: \"e99ab7d7-2145-4c7b-8138-6473d37226cb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsv2h"
Sep 15 20:14:40.946515 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:40.946450    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ccd7ea5-c061-4a89-9885-117aa68abbe6-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cp7nrn\" (UID: \"6ccd7ea5-c061-4a89-9885-117aa68abbe6\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cp7nrn"
Sep 15 20:14:40.946581 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:40.946521    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ccd7ea5-c061-4a89-9885-117aa68abbe6-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cp7nrn\" (UID: \"6ccd7ea5-c061-4a89-9885-117aa68abbe6\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cp7nrn"
Sep 15 20:14:40.946666 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:40.946641    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnshw\" (UniqueName: \"kubernetes.io/projected/6ccd7ea5-c061-4a89-9885-117aa68abbe6-kube-api-access-vnshw\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cp7nrn\" (UID: \"6ccd7ea5-c061-4a89-9885-117aa68abbe6\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cp7nrn"
Sep 15 20:14:40.946757 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:40.946739    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e99ab7d7-2145-4c7b-8138-6473d37226cb-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsv2h\" (UID: \"e99ab7d7-2145-4c7b-8138-6473d37226cb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsv2h"
Sep 15 20:14:40.946825 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:40.946777    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e99ab7d7-2145-4c7b-8138-6473d37226cb-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsv2h\" (UID: \"e99ab7d7-2145-4c7b-8138-6473d37226cb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsv2h"
Sep 15 20:14:40.946825 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:40.946789    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz2fl\" (UniqueName: \"kubernetes.io/projected/e99ab7d7-2145-4c7b-8138-6473d37226cb-kube-api-access-pz2fl\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsv2h\" (UID: \"e99ab7d7-2145-4c7b-8138-6473d37226cb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsv2h"
Sep 15 20:14:40.947039 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:40.947015    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e99ab7d7-2145-4c7b-8138-6473d37226cb-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsv2h\" (UID: \"e99ab7d7-2145-4c7b-8138-6473d37226cb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsv2h"
Sep 15 20:14:40.955296 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:40.955272    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pz2fl\" (UniqueName: \"kubernetes.io/projected/e99ab7d7-2145-4c7b-8138-6473d37226cb-kube-api-access-pz2fl\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsv2h\" (UID: \"e99ab7d7-2145-4c7b-8138-6473d37226cb\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsv2h"
Sep 15 20:14:41.033310 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:41.033228    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dqxgd"]
Sep 15 20:14:41.035962 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:41.035943    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dqxgd"
Sep 15 20:14:41.042301 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:41.042274    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsv2h"
Sep 15 20:14:41.045754 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:41.045707    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dqxgd"]
Sep 15 20:14:41.048190 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:41.048169    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nj9j\" (UniqueName: \"kubernetes.io/projected/d2d2bfef-d821-483a-995f-f3560ece2755-kube-api-access-9nj9j\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abvlzh\" (UID: \"d2d2bfef-d821-483a-995f-f3560ece2755\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abvlzh"
Sep 15 20:14:41.048304 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:41.048214    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ccd7ea5-c061-4a89-9885-117aa68abbe6-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cp7nrn\" (UID: \"6ccd7ea5-c061-4a89-9885-117aa68abbe6\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cp7nrn"
Sep 15 20:14:41.048304 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:41.048252    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2d2bfef-d821-483a-995f-f3560ece2755-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abvlzh\" (UID: \"d2d2bfef-d821-483a-995f-f3560ece2755\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abvlzh"
Sep 15 20:14:41.048304 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:41.048292    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ccd7ea5-c061-4a89-9885-117aa68abbe6-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cp7nrn\" (UID: \"6ccd7ea5-c061-4a89-9885-117aa68abbe6\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cp7nrn"
Sep 15 20:14:41.048474 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:41.048318    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2d2bfef-d821-483a-995f-f3560ece2755-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abvlzh\" (UID: \"d2d2bfef-d821-483a-995f-f3560ece2755\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abvlzh"
Sep 15 20:14:41.048474 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:41.048347    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnshw\" (UniqueName: \"kubernetes.io/projected/6ccd7ea5-c061-4a89-9885-117aa68abbe6-kube-api-access-vnshw\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cp7nrn\" (UID: \"6ccd7ea5-c061-4a89-9885-117aa68abbe6\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cp7nrn"
Sep 15 20:14:41.048679 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:41.048656    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ccd7ea5-c061-4a89-9885-117aa68abbe6-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cp7nrn\" (UID: \"6ccd7ea5-c061-4a89-9885-117aa68abbe6\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cp7nrn"
Sep 15 20:14:41.048756 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:41.048693    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ccd7ea5-c061-4a89-9885-117aa68abbe6-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cp7nrn\" (UID: \"6ccd7ea5-c061-4a89-9885-117aa68abbe6\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cp7nrn"
Sep 15 20:14:41.060147 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:41.060109    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnshw\" (UniqueName: \"kubernetes.io/projected/6ccd7ea5-c061-4a89-9885-117aa68abbe6-kube-api-access-vnshw\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cp7nrn\" (UID: \"6ccd7ea5-c061-4a89-9885-117aa68abbe6\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cp7nrn"
Sep 15 20:14:41.149030 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:41.148989    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cp7nrn"
Sep 15 20:14:41.149441 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:41.149400    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nj9j\" (UniqueName: \"kubernetes.io/projected/d2d2bfef-d821-483a-995f-f3560ece2755-kube-api-access-9nj9j\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abvlzh\" (UID: \"d2d2bfef-d821-483a-995f-f3560ece2755\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abvlzh"
Sep 15 20:14:41.149547 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:41.149473    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lkcx\" (UniqueName: \"kubernetes.io/projected/791a176b-d640-43d1-8c96-a7d9b2023624-kube-api-access-8lkcx\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dqxgd\" (UID: \"791a176b-d640-43d1-8c96-a7d9b2023624\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dqxgd"
Sep 15 20:14:41.149709 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:41.149682    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2d2bfef-d821-483a-995f-f3560ece2755-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abvlzh\" (UID: \"d2d2bfef-d821-483a-995f-f3560ece2755\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abvlzh"
Sep 15 20:14:41.149813 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:41.149763    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/791a176b-d640-43d1-8c96-a7d9b2023624-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dqxgd\" (UID: \"791a176b-d640-43d1-8c96-a7d9b2023624\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dqxgd"
Sep 15 20:14:41.149871 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:41.149833    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2d2bfef-d821-483a-995f-f3560ece2755-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abvlzh\" (UID: \"d2d2bfef-d821-483a-995f-f3560ece2755\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abvlzh"
Sep 15 20:14:41.149871 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:41.149861    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/791a176b-d640-43d1-8c96-a7d9b2023624-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dqxgd\" (UID: \"791a176b-d640-43d1-8c96-a7d9b2023624\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dqxgd"
Sep 15 20:14:41.150117 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:41.150093    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2d2bfef-d821-483a-995f-f3560ece2755-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abvlzh\" (UID: \"d2d2bfef-d821-483a-995f-f3560ece2755\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abvlzh"
Sep 15 20:14:41.150258 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:41.150236    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2d2bfef-d821-483a-995f-f3560ece2755-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abvlzh\" (UID: \"d2d2bfef-d821-483a-995f-f3560ece2755\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abvlzh"
Sep 15 20:14:41.158701 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:41.158683    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nj9j\" (UniqueName: \"kubernetes.io/projected/d2d2bfef-d821-483a-995f-f3560ece2755-kube-api-access-9nj9j\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abvlzh\" (UID: \"d2d2bfef-d821-483a-995f-f3560ece2755\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abvlzh"
Sep 15 20:14:41.242658 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:41.242625    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abvlzh"
Sep 15 20:14:41.251210 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:41.251079    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/791a176b-d640-43d1-8c96-a7d9b2023624-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dqxgd\" (UID: \"791a176b-d640-43d1-8c96-a7d9b2023624\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dqxgd"
Sep 15 20:14:41.251210 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:41.251125    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/791a176b-d640-43d1-8c96-a7d9b2023624-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dqxgd\" (UID: \"791a176b-d640-43d1-8c96-a7d9b2023624\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dqxgd"
Sep 15 20:14:41.251210 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:41.251175    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lkcx\" (UniqueName: \"kubernetes.io/projected/791a176b-d640-43d1-8c96-a7d9b2023624-kube-api-access-8lkcx\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dqxgd\" (UID: \"791a176b-d640-43d1-8c96-a7d9b2023624\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dqxgd"
Sep 15 20:14:41.251643 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:41.251618    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/791a176b-d640-43d1-8c96-a7d9b2023624-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dqxgd\" (UID: \"791a176b-d640-43d1-8c96-a7d9b2023624\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dqxgd"
Sep 15 20:14:41.251750 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:41.251682    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/791a176b-d640-43d1-8c96-a7d9b2023624-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dqxgd\" (UID: \"791a176b-d640-43d1-8c96-a7d9b2023624\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dqxgd"
Sep 15 20:14:41.260660 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:41.260635    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lkcx\" (UniqueName: \"kubernetes.io/projected/791a176b-d640-43d1-8c96-a7d9b2023624-kube-api-access-8lkcx\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dqxgd\" (UID: \"791a176b-d640-43d1-8c96-a7d9b2023624\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dqxgd"
Sep 15 20:14:41.324618 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:41.324590    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsv2h"]
Sep 15 20:14:41.327349 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:14:41.327313    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode99ab7d7_2145_4c7b_8138_6473d37226cb.slice/crio-690e72e5b93087f15068f75fcecfaf5dc10b4e0673ccd9fe842c49f34ecef06a WatchSource:0}: Error finding container 690e72e5b93087f15068f75fcecfaf5dc10b4e0673ccd9fe842c49f34ecef06a: Status 404 returned error can't find the container with id 690e72e5b93087f15068f75fcecfaf5dc10b4e0673ccd9fe842c49f34ecef06a
Sep 15 20:14:41.346596 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:41.346568    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dqxgd"
Sep 15 20:14:41.428133 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:41.428110    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cp7nrn"]
Sep 15 20:14:41.472301 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:41.472266    2577 generic.go:356] "Generic (PLEG): container finished" podID="e99ab7d7-2145-4c7b-8138-6473d37226cb" containerID="2bf374ee3123f93948dc6079966138187477720507d9cc68603b6a60d13fcf91" exitCode=0
Sep 15 20:14:41.472488 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:41.472344    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsv2h" event={"ID":"e99ab7d7-2145-4c7b-8138-6473d37226cb","Type":"ContainerDied","Data":"2bf374ee3123f93948dc6079966138187477720507d9cc68603b6a60d13fcf91"}
Sep 15 20:14:41.472488 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:41.472392    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsv2h" event={"ID":"e99ab7d7-2145-4c7b-8138-6473d37226cb","Type":"ContainerStarted","Data":"690e72e5b93087f15068f75fcecfaf5dc10b4e0673ccd9fe842c49f34ecef06a"}
Sep 15 20:14:41.486604 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:14:41.486574    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ccd7ea5_c061_4a89_9885_117aa68abbe6.slice/crio-401848b6b5a7ded3274a48f3556c2bfd8aa155c2474257336dcd6022efaa14cd WatchSource:0}: Error finding container 401848b6b5a7ded3274a48f3556c2bfd8aa155c2474257336dcd6022efaa14cd: Status 404 returned error can't find the container with id 401848b6b5a7ded3274a48f3556c2bfd8aa155c2474257336dcd6022efaa14cd
Sep 15 20:14:41.516773 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:41.516748    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abvlzh"]
Sep 15 20:14:41.519871 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:14:41.519840    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2d2bfef_d821_483a_995f_f3560ece2755.slice/crio-67b59d28d5e09360c49f9a72c88cc682d8c0826586fa1bbc52d27426c17cd53b WatchSource:0}: Error finding container 67b59d28d5e09360c49f9a72c88cc682d8c0826586fa1bbc52d27426c17cd53b: Status 404 returned error can't find the container with id 67b59d28d5e09360c49f9a72c88cc682d8c0826586fa1bbc52d27426c17cd53b
Sep 15 20:14:41.623313 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:41.623285    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dqxgd"]
Sep 15 20:14:41.701785 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:14:41.701745    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod791a176b_d640_43d1_8c96_a7d9b2023624.slice/crio-1985cff09d7a218c32d10087bda7c538b13b658c57d63f58cbbb5367988c5f65 WatchSource:0}: Error finding container 1985cff09d7a218c32d10087bda7c538b13b658c57d63f58cbbb5367988c5f65: Status 404 returned error can't find the container with id 1985cff09d7a218c32d10087bda7c538b13b658c57d63f58cbbb5367988c5f65
Sep 15 20:14:42.479085 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:42.479052    2577 generic.go:356] "Generic (PLEG): container finished" podID="d2d2bfef-d821-483a-995f-f3560ece2755" containerID="12e2a701bc83a55be9aead5cb819a7d1b29ea81fb0baa701445ddcbc558a3ad9" exitCode=0
Sep 15 20:14:42.479507 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:42.479131    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abvlzh" event={"ID":"d2d2bfef-d821-483a-995f-f3560ece2755","Type":"ContainerDied","Data":"12e2a701bc83a55be9aead5cb819a7d1b29ea81fb0baa701445ddcbc558a3ad9"}
Sep 15 20:14:42.479507 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:42.479190    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abvlzh" event={"ID":"d2d2bfef-d821-483a-995f-f3560ece2755","Type":"ContainerStarted","Data":"67b59d28d5e09360c49f9a72c88cc682d8c0826586fa1bbc52d27426c17cd53b"}
Sep 15 20:14:42.480810 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:42.480764    2577 generic.go:356] "Generic (PLEG): container finished" podID="791a176b-d640-43d1-8c96-a7d9b2023624" containerID="af38d4890cf1b6c703163f04500e690328cf75a556bcb179cb7bb76a0cdcd696" exitCode=0
Sep 15 20:14:42.480889 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:42.480855    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dqxgd" event={"ID":"791a176b-d640-43d1-8c96-a7d9b2023624","Type":"ContainerDied","Data":"af38d4890cf1b6c703163f04500e690328cf75a556bcb179cb7bb76a0cdcd696"}
Sep 15 20:14:42.480960 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:42.480901    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dqxgd" event={"ID":"791a176b-d640-43d1-8c96-a7d9b2023624","Type":"ContainerStarted","Data":"1985cff09d7a218c32d10087bda7c538b13b658c57d63f58cbbb5367988c5f65"}
Sep 15 20:14:42.482687 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:42.482660    2577 generic.go:356] "Generic (PLEG): container finished" podID="6ccd7ea5-c061-4a89-9885-117aa68abbe6" containerID="e981501557411946f4c21e8aec07b3146dd7bc59a6b9d09063f8671601732a1d" exitCode=0
Sep 15 20:14:42.482933 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:42.482756    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cp7nrn" event={"ID":"6ccd7ea5-c061-4a89-9885-117aa68abbe6","Type":"ContainerDied","Data":"e981501557411946f4c21e8aec07b3146dd7bc59a6b9d09063f8671601732a1d"}
Sep 15 20:14:42.482933 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:42.482793    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cp7nrn" event={"ID":"6ccd7ea5-c061-4a89-9885-117aa68abbe6","Type":"ContainerStarted","Data":"401848b6b5a7ded3274a48f3556c2bfd8aa155c2474257336dcd6022efaa14cd"}
Sep 15 20:14:43.496310 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:43.496281    2577 generic.go:356] "Generic (PLEG): container finished" podID="e99ab7d7-2145-4c7b-8138-6473d37226cb" containerID="c7ff20b59f93dfe763283b98688a1680f3e3528bb84a70ee9686372be596f233" exitCode=0
Sep 15 20:14:43.496771 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:43.496356    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsv2h" event={"ID":"e99ab7d7-2145-4c7b-8138-6473d37226cb","Type":"ContainerDied","Data":"c7ff20b59f93dfe763283b98688a1680f3e3528bb84a70ee9686372be596f233"}
Sep 15 20:14:43.498515 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:43.498491    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dqxgd" event={"ID":"791a176b-d640-43d1-8c96-a7d9b2023624","Type":"ContainerStarted","Data":"fc88eb34f2bdec3d28dbf6ed1a184c670ba10bbdbc4e95ea80279eded75dcf1b"}
Sep 15 20:14:44.508644 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:44.508608    2577 generic.go:356] "Generic (PLEG): container finished" podID="e99ab7d7-2145-4c7b-8138-6473d37226cb" containerID="4707e12466522008558db622446ec1d42d1b13ea473b7ad0ef33ddd7d3cb10be" exitCode=0
Sep 15 20:14:44.509170 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:44.508699    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsv2h" event={"ID":"e99ab7d7-2145-4c7b-8138-6473d37226cb","Type":"ContainerDied","Data":"4707e12466522008558db622446ec1d42d1b13ea473b7ad0ef33ddd7d3cb10be"}
Sep 15 20:14:44.510600 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:44.510573    2577 generic.go:356] "Generic (PLEG): container finished" podID="d2d2bfef-d821-483a-995f-f3560ece2755" containerID="2d11caee77a182aedb2fa7f110cf3bef4246c69f465220b70345a8f3f450b990" exitCode=0
Sep 15 20:14:44.510747 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:44.510651    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abvlzh" event={"ID":"d2d2bfef-d821-483a-995f-f3560ece2755","Type":"ContainerDied","Data":"2d11caee77a182aedb2fa7f110cf3bef4246c69f465220b70345a8f3f450b990"}
Sep 15 20:14:44.512759 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:44.512733    2577 generic.go:356] "Generic (PLEG): container finished" podID="791a176b-d640-43d1-8c96-a7d9b2023624" containerID="fc88eb34f2bdec3d28dbf6ed1a184c670ba10bbdbc4e95ea80279eded75dcf1b" exitCode=0
Sep 15 20:14:44.512864 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:44.512820    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dqxgd" event={"ID":"791a176b-d640-43d1-8c96-a7d9b2023624","Type":"ContainerDied","Data":"fc88eb34f2bdec3d28dbf6ed1a184c670ba10bbdbc4e95ea80279eded75dcf1b"}
Sep 15 20:14:44.514934 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:44.514913    2577 generic.go:356] "Generic (PLEG): container finished" podID="6ccd7ea5-c061-4a89-9885-117aa68abbe6" containerID="3f30cb4562dc25a4f8b378510b914e321bb23a47763dd27dbeae6b90e5477825" exitCode=0
Sep 15 20:14:44.515033 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:44.514976    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cp7nrn" event={"ID":"6ccd7ea5-c061-4a89-9885-117aa68abbe6","Type":"ContainerDied","Data":"3f30cb4562dc25a4f8b378510b914e321bb23a47763dd27dbeae6b90e5477825"}
Sep 15 20:14:44.793050 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:14:44.793015    2577 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ccd7ea5_c061_4a89_9885_117aa68abbe6.slice/crio-625e3d0fb7633f608e1262adb72b02cfeb02ef84e339e154398b99f9d163cfec.scope\": RecentStats: unable to find data in memory cache]"
Sep 15 20:14:45.523213 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:45.523177    2577 generic.go:356] "Generic (PLEG): container finished" podID="d2d2bfef-d821-483a-995f-f3560ece2755" containerID="53b3af993f0fe7f2720c3b7693fe47a77fb8dc99ea032a948349e2f7e116cf98" exitCode=0
Sep 15 20:14:45.523638 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:45.523253    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abvlzh" event={"ID":"d2d2bfef-d821-483a-995f-f3560ece2755","Type":"ContainerDied","Data":"53b3af993f0fe7f2720c3b7693fe47a77fb8dc99ea032a948349e2f7e116cf98"}
Sep 15 20:14:45.525331 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:45.525302    2577 generic.go:356] "Generic (PLEG): container finished" podID="791a176b-d640-43d1-8c96-a7d9b2023624" containerID="c1cdfff52739dee8b9d06ad6f14d2ab6e07d51599d2c9bfb53af8e7ecffbe37f" exitCode=0
Sep 15 20:14:45.525459 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:45.525380    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dqxgd" event={"ID":"791a176b-d640-43d1-8c96-a7d9b2023624","Type":"ContainerDied","Data":"c1cdfff52739dee8b9d06ad6f14d2ab6e07d51599d2c9bfb53af8e7ecffbe37f"}
Sep 15 20:14:45.527358 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:45.527335    2577 generic.go:356] "Generic (PLEG): container finished" podID="6ccd7ea5-c061-4a89-9885-117aa68abbe6" containerID="625e3d0fb7633f608e1262adb72b02cfeb02ef84e339e154398b99f9d163cfec" exitCode=0
Sep 15 20:14:45.527481 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:45.527404    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cp7nrn" event={"ID":"6ccd7ea5-c061-4a89-9885-117aa68abbe6","Type":"ContainerDied","Data":"625e3d0fb7633f608e1262adb72b02cfeb02ef84e339e154398b99f9d163cfec"}
Sep 15 20:14:45.661335 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:45.661306    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsv2h"
Sep 15 20:14:45.697917 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:45.697878    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e99ab7d7-2145-4c7b-8138-6473d37226cb-util\") pod \"e99ab7d7-2145-4c7b-8138-6473d37226cb\" (UID: \"e99ab7d7-2145-4c7b-8138-6473d37226cb\") "
Sep 15 20:14:45.698102 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:45.697959    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pz2fl\" (UniqueName: \"kubernetes.io/projected/e99ab7d7-2145-4c7b-8138-6473d37226cb-kube-api-access-pz2fl\") pod \"e99ab7d7-2145-4c7b-8138-6473d37226cb\" (UID: \"e99ab7d7-2145-4c7b-8138-6473d37226cb\") "
Sep 15 20:14:45.698102 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:45.697998    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e99ab7d7-2145-4c7b-8138-6473d37226cb-bundle\") pod \"e99ab7d7-2145-4c7b-8138-6473d37226cb\" (UID: \"e99ab7d7-2145-4c7b-8138-6473d37226cb\") "
Sep 15 20:14:45.698508 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:45.698477    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e99ab7d7-2145-4c7b-8138-6473d37226cb-bundle" (OuterVolumeSpecName: "bundle") pod "e99ab7d7-2145-4c7b-8138-6473d37226cb" (UID: "e99ab7d7-2145-4c7b-8138-6473d37226cb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:14:45.700217 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:45.700186    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e99ab7d7-2145-4c7b-8138-6473d37226cb-kube-api-access-pz2fl" (OuterVolumeSpecName: "kube-api-access-pz2fl") pod "e99ab7d7-2145-4c7b-8138-6473d37226cb" (UID: "e99ab7d7-2145-4c7b-8138-6473d37226cb"). InnerVolumeSpecName "kube-api-access-pz2fl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:14:45.704954 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:45.704905    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e99ab7d7-2145-4c7b-8138-6473d37226cb-util" (OuterVolumeSpecName: "util") pod "e99ab7d7-2145-4c7b-8138-6473d37226cb" (UID: "e99ab7d7-2145-4c7b-8138-6473d37226cb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:14:45.799369 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:45.799274    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e99ab7d7-2145-4c7b-8138-6473d37226cb-util\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:14:45.799369 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:45.799312    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pz2fl\" (UniqueName: \"kubernetes.io/projected/e99ab7d7-2145-4c7b-8138-6473d37226cb-kube-api-access-pz2fl\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:14:45.799369 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:45.799322    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e99ab7d7-2145-4c7b-8138-6473d37226cb-bundle\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:14:46.534700 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:46.534668    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsv2h"
Sep 15 20:14:46.535152 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:46.534698    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsv2h" event={"ID":"e99ab7d7-2145-4c7b-8138-6473d37226cb","Type":"ContainerDied","Data":"690e72e5b93087f15068f75fcecfaf5dc10b4e0673ccd9fe842c49f34ecef06a"}
Sep 15 20:14:46.535152 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:46.534790    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="690e72e5b93087f15068f75fcecfaf5dc10b4e0673ccd9fe842c49f34ecef06a"
Sep 15 20:14:46.710174 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:46.710136    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cp7nrn"
Sep 15 20:14:46.712835 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:46.712810    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dqxgd"
Sep 15 20:14:46.729695 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:46.729672    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abvlzh"
Sep 15 20:14:46.807351 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:46.807265    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2d2bfef-d821-483a-995f-f3560ece2755-bundle\") pod \"d2d2bfef-d821-483a-995f-f3560ece2755\" (UID: \"d2d2bfef-d821-483a-995f-f3560ece2755\") "
Sep 15 20:14:46.807519 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:46.807352    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ccd7ea5-c061-4a89-9885-117aa68abbe6-bundle\") pod \"6ccd7ea5-c061-4a89-9885-117aa68abbe6\" (UID: \"6ccd7ea5-c061-4a89-9885-117aa68abbe6\") "
Sep 15 20:14:46.807519 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:46.807392    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8lkcx\" (UniqueName: \"kubernetes.io/projected/791a176b-d640-43d1-8c96-a7d9b2023624-kube-api-access-8lkcx\") pod \"791a176b-d640-43d1-8c96-a7d9b2023624\" (UID: \"791a176b-d640-43d1-8c96-a7d9b2023624\") "
Sep 15 20:14:46.807519 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:46.807438    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2d2bfef-d821-483a-995f-f3560ece2755-util\") pod \"d2d2bfef-d821-483a-995f-f3560ece2755\" (UID: \"d2d2bfef-d821-483a-995f-f3560ece2755\") "
Sep 15 20:14:46.807519 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:46.807483    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9nj9j\" (UniqueName: \"kubernetes.io/projected/d2d2bfef-d821-483a-995f-f3560ece2755-kube-api-access-9nj9j\") pod \"d2d2bfef-d821-483a-995f-f3560ece2755\" (UID: \"d2d2bfef-d821-483a-995f-f3560ece2755\") "
Sep 15 20:14:46.807774 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:46.807529    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/791a176b-d640-43d1-8c96-a7d9b2023624-bundle\") pod \"791a176b-d640-43d1-8c96-a7d9b2023624\" (UID: \"791a176b-d640-43d1-8c96-a7d9b2023624\") "
Sep 15 20:14:46.807774 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:46.807557    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/791a176b-d640-43d1-8c96-a7d9b2023624-util\") pod \"791a176b-d640-43d1-8c96-a7d9b2023624\" (UID: \"791a176b-d640-43d1-8c96-a7d9b2023624\") "
Sep 15 20:14:46.807774 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:46.807609    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ccd7ea5-c061-4a89-9885-117aa68abbe6-util\") pod \"6ccd7ea5-c061-4a89-9885-117aa68abbe6\" (UID: \"6ccd7ea5-c061-4a89-9885-117aa68abbe6\") "
Sep 15 20:14:46.807774 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:46.807630    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vnshw\" (UniqueName: \"kubernetes.io/projected/6ccd7ea5-c061-4a89-9885-117aa68abbe6-kube-api-access-vnshw\") pod \"6ccd7ea5-c061-4a89-9885-117aa68abbe6\" (UID: \"6ccd7ea5-c061-4a89-9885-117aa68abbe6\") "
Sep 15 20:14:46.808203 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:46.808155    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ccd7ea5-c061-4a89-9885-117aa68abbe6-bundle" (OuterVolumeSpecName: "bundle") pod "6ccd7ea5-c061-4a89-9885-117aa68abbe6" (UID: "6ccd7ea5-c061-4a89-9885-117aa68abbe6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:14:46.808203 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:46.808163    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/791a176b-d640-43d1-8c96-a7d9b2023624-bundle" (OuterVolumeSpecName: "bundle") pod "791a176b-d640-43d1-8c96-a7d9b2023624" (UID: "791a176b-d640-43d1-8c96-a7d9b2023624"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:14:46.808494 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:46.808303    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2d2bfef-d821-483a-995f-f3560ece2755-bundle" (OuterVolumeSpecName: "bundle") pod "d2d2bfef-d821-483a-995f-f3560ece2755" (UID: "d2d2bfef-d821-483a-995f-f3560ece2755"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:14:46.808604 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:46.808588    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/791a176b-d640-43d1-8c96-a7d9b2023624-bundle\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:14:46.808680 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:46.808608    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2d2bfef-d821-483a-995f-f3560ece2755-bundle\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:14:46.808680 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:46.808621    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ccd7ea5-c061-4a89-9885-117aa68abbe6-bundle\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:14:46.810085 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:46.810055    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2d2bfef-d821-483a-995f-f3560ece2755-kube-api-access-9nj9j" (OuterVolumeSpecName: "kube-api-access-9nj9j") pod "d2d2bfef-d821-483a-995f-f3560ece2755" (UID: "d2d2bfef-d821-483a-995f-f3560ece2755"). InnerVolumeSpecName "kube-api-access-9nj9j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:14:46.810206 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:46.810124    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/791a176b-d640-43d1-8c96-a7d9b2023624-kube-api-access-8lkcx" (OuterVolumeSpecName: "kube-api-access-8lkcx") pod "791a176b-d640-43d1-8c96-a7d9b2023624" (UID: "791a176b-d640-43d1-8c96-a7d9b2023624"). InnerVolumeSpecName "kube-api-access-8lkcx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:14:46.810633 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:46.810606    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ccd7ea5-c061-4a89-9885-117aa68abbe6-kube-api-access-vnshw" (OuterVolumeSpecName: "kube-api-access-vnshw") pod "6ccd7ea5-c061-4a89-9885-117aa68abbe6" (UID: "6ccd7ea5-c061-4a89-9885-117aa68abbe6"). InnerVolumeSpecName "kube-api-access-vnshw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:14:46.814036 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:46.814001    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2d2bfef-d821-483a-995f-f3560ece2755-util" (OuterVolumeSpecName: "util") pod "d2d2bfef-d821-483a-995f-f3560ece2755" (UID: "d2d2bfef-d821-483a-995f-f3560ece2755"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:14:46.814760 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:46.814736    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ccd7ea5-c061-4a89-9885-117aa68abbe6-util" (OuterVolumeSpecName: "util") pod "6ccd7ea5-c061-4a89-9885-117aa68abbe6" (UID: "6ccd7ea5-c061-4a89-9885-117aa68abbe6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:14:46.814811 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:46.814739    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/791a176b-d640-43d1-8c96-a7d9b2023624-util" (OuterVolumeSpecName: "util") pod "791a176b-d640-43d1-8c96-a7d9b2023624" (UID: "791a176b-d640-43d1-8c96-a7d9b2023624"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:14:46.909546 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:46.909511    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/791a176b-d640-43d1-8c96-a7d9b2023624-util\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:14:46.909546 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:46.909543    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ccd7ea5-c061-4a89-9885-117aa68abbe6-util\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:14:46.909772 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:46.909564    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vnshw\" (UniqueName: \"kubernetes.io/projected/6ccd7ea5-c061-4a89-9885-117aa68abbe6-kube-api-access-vnshw\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:14:46.909772 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:46.909574    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lkcx\" (UniqueName: \"kubernetes.io/projected/791a176b-d640-43d1-8c96-a7d9b2023624-kube-api-access-8lkcx\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:14:46.909772 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:46.909584    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2d2bfef-d821-483a-995f-f3560ece2755-util\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:14:46.909772 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:46.909592    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9nj9j\" (UniqueName: \"kubernetes.io/projected/d2d2bfef-d821-483a-995f-f3560ece2755-kube-api-access-9nj9j\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:14:47.542709 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:47.542677    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abvlzh"
Sep 15 20:14:47.542709 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:47.542688    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abvlzh" event={"ID":"d2d2bfef-d821-483a-995f-f3560ece2755","Type":"ContainerDied","Data":"67b59d28d5e09360c49f9a72c88cc682d8c0826586fa1bbc52d27426c17cd53b"}
Sep 15 20:14:47.543223 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:47.542756    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="67b59d28d5e09360c49f9a72c88cc682d8c0826586fa1bbc52d27426c17cd53b"
Sep 15 20:14:47.544715 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:47.544698    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dqxgd"
Sep 15 20:14:47.544853 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:47.544749    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6dqxgd" event={"ID":"791a176b-d640-43d1-8c96-a7d9b2023624","Type":"ContainerDied","Data":"1985cff09d7a218c32d10087bda7c538b13b658c57d63f58cbbb5367988c5f65"}
Sep 15 20:14:47.544853 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:47.544804    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1985cff09d7a218c32d10087bda7c538b13b658c57d63f58cbbb5367988c5f65"
Sep 15 20:14:47.546997 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:47.546971    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cp7nrn" event={"ID":"6ccd7ea5-c061-4a89-9885-117aa68abbe6","Type":"ContainerDied","Data":"401848b6b5a7ded3274a48f3556c2bfd8aa155c2474257336dcd6022efaa14cd"}
Sep 15 20:14:47.547103 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:47.547021    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="401848b6b5a7ded3274a48f3556c2bfd8aa155c2474257336dcd6022efaa14cd"
Sep 15 20:14:47.547103 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:47.546982    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cp7nrn"
Sep 15 20:14:47.802979 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:47.802889    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f89cb66dd-rhb7p"
Sep 15 20:14:47.803110 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:47.802980    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7f89cb66dd-rhb7p"
Sep 15 20:14:47.807603 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:47.807570    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7f89cb66dd-rhb7p"
Sep 15 20:14:48.556468 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:48.556438    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7f89cb66dd-rhb7p"
Sep 15 20:14:54.038445 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.038413    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-r227b_ecab5774-79a4-42eb-9bb5-a1f09b18f3fb/console-operator/1.log"
Sep 15 20:14:54.039231 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.039210    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-r227b_ecab5774-79a4-42eb-9bb5-a1f09b18f3fb/console-operator/1.log"
Sep 15 20:14:54.528331 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.528290    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-d67zs"]
Sep 15 20:14:54.528661 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.528650    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2d2bfef-d821-483a-995f-f3560ece2755" containerName="pull"
Sep 15 20:14:54.528771 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.528663    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2d2bfef-d821-483a-995f-f3560ece2755" containerName="pull"
Sep 15 20:14:54.528771 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.528672    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2d2bfef-d821-483a-995f-f3560ece2755" containerName="extract"
Sep 15 20:14:54.528771 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.528677    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2d2bfef-d821-483a-995f-f3560ece2755" containerName="extract"
Sep 15 20:14:54.528771 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.528688    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="791a176b-d640-43d1-8c96-a7d9b2023624" containerName="pull"
Sep 15 20:14:54.528771 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.528693    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="791a176b-d640-43d1-8c96-a7d9b2023624" containerName="pull"
Sep 15 20:14:54.528771 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.528707    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ccd7ea5-c061-4a89-9885-117aa68abbe6" containerName="util"
Sep 15 20:14:54.528771 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.528712    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ccd7ea5-c061-4a89-9885-117aa68abbe6" containerName="util"
Sep 15 20:14:54.528771 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.528719    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e99ab7d7-2145-4c7b-8138-6473d37226cb" containerName="extract"
Sep 15 20:14:54.528771 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.528739    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e99ab7d7-2145-4c7b-8138-6473d37226cb" containerName="extract"
Sep 15 20:14:54.528771 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.528746    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ccd7ea5-c061-4a89-9885-117aa68abbe6" containerName="extract"
Sep 15 20:14:54.528771 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.528752    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ccd7ea5-c061-4a89-9885-117aa68abbe6" containerName="extract"
Sep 15 20:14:54.528771 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.528759    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="791a176b-d640-43d1-8c96-a7d9b2023624" containerName="extract"
Sep 15 20:14:54.528771 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.528764    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="791a176b-d640-43d1-8c96-a7d9b2023624" containerName="extract"
Sep 15 20:14:54.528771 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.528770    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ccd7ea5-c061-4a89-9885-117aa68abbe6" containerName="pull"
Sep 15 20:14:54.528771 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.528775    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ccd7ea5-c061-4a89-9885-117aa68abbe6" containerName="pull"
Sep 15 20:14:54.529229 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.528784    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e99ab7d7-2145-4c7b-8138-6473d37226cb" containerName="pull"
Sep 15 20:14:54.529229 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.528789    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e99ab7d7-2145-4c7b-8138-6473d37226cb" containerName="pull"
Sep 15 20:14:54.529229 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.528796    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2d2bfef-d821-483a-995f-f3560ece2755" containerName="util"
Sep 15 20:14:54.529229 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.528803    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2d2bfef-d821-483a-995f-f3560ece2755" containerName="util"
Sep 15 20:14:54.529229 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.528814    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="791a176b-d640-43d1-8c96-a7d9b2023624" containerName="util"
Sep 15 20:14:54.529229 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.528819    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="791a176b-d640-43d1-8c96-a7d9b2023624" containerName="util"
Sep 15 20:14:54.529229 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.528826    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e99ab7d7-2145-4c7b-8138-6473d37226cb" containerName="util"
Sep 15 20:14:54.529229 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.528831    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e99ab7d7-2145-4c7b-8138-6473d37226cb" containerName="util"
Sep 15 20:14:54.529229 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.528883    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e99ab7d7-2145-4c7b-8138-6473d37226cb" containerName="extract"
Sep 15 20:14:54.529229 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.528890    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="791a176b-d640-43d1-8c96-a7d9b2023624" containerName="extract"
Sep 15 20:14:54.529229 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.528896    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2d2bfef-d821-483a-995f-f3560ece2755" containerName="extract"
Sep 15 20:14:54.529229 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.528901    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ccd7ea5-c061-4a89-9885-117aa68abbe6" containerName="extract"
Sep 15 20:14:54.531935 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.531917    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-d67zs"
Sep 15 20:14:54.534682 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.534656    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-fwbqn\""
Sep 15 20:14:54.534782 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.534699    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 15 20:14:54.535651 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.535632    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 15 20:14:54.545112 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.545067    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-d67zs"]
Sep 15 20:14:54.578897 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.578858    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwbq9\" (UniqueName: \"kubernetes.io/projected/f49caa1a-07d8-4a70-8260-bb16b54a7b03-kube-api-access-vwbq9\") pod \"authorino-operator-76b478bd78-d67zs\" (UID: \"f49caa1a-07d8-4a70-8260-bb16b54a7b03\") " pod="kuadrant-system/authorino-operator-76b478bd78-d67zs"
Sep 15 20:14:54.679938 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.679891    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwbq9\" (UniqueName: \"kubernetes.io/projected/f49caa1a-07d8-4a70-8260-bb16b54a7b03-kube-api-access-vwbq9\") pod \"authorino-operator-76b478bd78-d67zs\" (UID: \"f49caa1a-07d8-4a70-8260-bb16b54a7b03\") " pod="kuadrant-system/authorino-operator-76b478bd78-d67zs"
Sep 15 20:14:54.691114 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.691081    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwbq9\" (UniqueName: \"kubernetes.io/projected/f49caa1a-07d8-4a70-8260-bb16b54a7b03-kube-api-access-vwbq9\") pod \"authorino-operator-76b478bd78-d67zs\" (UID: \"f49caa1a-07d8-4a70-8260-bb16b54a7b03\") " pod="kuadrant-system/authorino-operator-76b478bd78-d67zs"
Sep 15 20:14:54.843260 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:54.843164    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-d67zs"
Sep 15 20:14:55.164446 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:55.164419    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-d67zs"]
Sep 15 20:14:55.166267 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:14:55.166240    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf49caa1a_07d8_4a70_8260_bb16b54a7b03.slice/crio-e8e786c21f9a4cfaf6eb9c8bb10360236e8abdc084e47dca72fe43fe5400066b WatchSource:0}: Error finding container e8e786c21f9a4cfaf6eb9c8bb10360236e8abdc084e47dca72fe43fe5400066b: Status 404 returned error can't find the container with id e8e786c21f9a4cfaf6eb9c8bb10360236e8abdc084e47dca72fe43fe5400066b
Sep 15 20:14:55.593265 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:55.593230    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-d67zs" event={"ID":"f49caa1a-07d8-4a70-8260-bb16b54a7b03","Type":"ContainerStarted","Data":"e8e786c21f9a4cfaf6eb9c8bb10360236e8abdc084e47dca72fe43fe5400066b"}
Sep 15 20:14:56.838127 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:56.838088    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-vrwfm"]
Sep 15 20:14:56.847434 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:56.847403    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-vrwfm"
Sep 15 20:14:56.849767 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:56.849739    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-jd9wt\""
Sep 15 20:14:56.850033 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:56.850007    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 15 20:14:56.854987 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:56.854828    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-vrwfm"]
Sep 15 20:14:56.899969 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:56.899917    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7bqk\" (UniqueName: \"kubernetes.io/projected/947328b0-ab34-4b57-b9ef-275efbc95ed4-kube-api-access-t7bqk\") pod \"dns-operator-controller-manager-7b84cd7c9c-vrwfm\" (UID: \"947328b0-ab34-4b57-b9ef-275efbc95ed4\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-vrwfm"
Sep 15 20:14:57.000954 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:57.000917    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7bqk\" (UniqueName: \"kubernetes.io/projected/947328b0-ab34-4b57-b9ef-275efbc95ed4-kube-api-access-t7bqk\") pod \"dns-operator-controller-manager-7b84cd7c9c-vrwfm\" (UID: \"947328b0-ab34-4b57-b9ef-275efbc95ed4\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-vrwfm"
Sep 15 20:14:57.026047 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:57.026009    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7bqk\" (UniqueName: \"kubernetes.io/projected/947328b0-ab34-4b57-b9ef-275efbc95ed4-kube-api-access-t7bqk\") pod \"dns-operator-controller-manager-7b84cd7c9c-vrwfm\" (UID: \"947328b0-ab34-4b57-b9ef-275efbc95ed4\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-vrwfm"
Sep 15 20:14:57.162062 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:57.161966    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-vrwfm"
Sep 15 20:14:57.875693 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:57.875664    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-vrwfm"]
Sep 15 20:14:57.877443 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:14:57.877409    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod947328b0_ab34_4b57_b9ef_275efbc95ed4.slice/crio-38176a3526e40e1aa0b99a90e59f870d25c9bf81fee9c200c0ce12a5b4f2d398 WatchSource:0}: Error finding container 38176a3526e40e1aa0b99a90e59f870d25c9bf81fee9c200c0ce12a5b4f2d398: Status 404 returned error can't find the container with id 38176a3526e40e1aa0b99a90e59f870d25c9bf81fee9c200c0ce12a5b4f2d398
Sep 15 20:14:58.615336 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:58.615300    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-d67zs" event={"ID":"f49caa1a-07d8-4a70-8260-bb16b54a7b03","Type":"ContainerStarted","Data":"2eb5451730d7b00bdd5434b10914a693a34f435bf549f7f1023bbe0bc4f37f15"}
Sep 15 20:14:58.615501 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:58.615400    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-d67zs"
Sep 15 20:14:58.616777 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:58.616752    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-vrwfm" event={"ID":"947328b0-ab34-4b57-b9ef-275efbc95ed4","Type":"ContainerStarted","Data":"38176a3526e40e1aa0b99a90e59f870d25c9bf81fee9c200c0ce12a5b4f2d398"}
Sep 15 20:14:58.661982 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:14:58.661924    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-d67zs" podStartSLOduration=2.160777559 podStartE2EDuration="4.661904808s" totalImagesPullingTime="2.501127249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:14:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:14:58.659483154 +0000 UTC m=+605.050436151" watchObservedRunningTime="2026-09-15 20:14:58.661904808 +0000 UTC m=+605.052857806"
Sep 15 20:15:00.639893 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:00.639855    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-vrwfm" event={"ID":"947328b0-ab34-4b57-b9ef-275efbc95ed4","Type":"ContainerStarted","Data":"126145696eb4ba1c7e7ff464f26cfd793d14a4cbd2d1f0a5bb6e6c937526be78"}
Sep 15 20:15:00.640297 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:00.639989    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-vrwfm"
Sep 15 20:15:00.670794 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:00.670732    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-vrwfm" podStartSLOduration=2.7865502319999997 podStartE2EDuration="4.670704687s" totalImagesPullingTime="1.884154455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:14:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:15:00.669208524 +0000 UTC m=+607.060161522" watchObservedRunningTime="2026-09-15 20:15:00.670704687 +0000 UTC m=+607.061657684"
Sep 15 20:15:01.144412 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:01.144305    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8r97z"]
Sep 15 20:15:01.152487 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:01.148818    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8r97z"
Sep 15 20:15:01.152487 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:01.151916    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-dbq9q\""
Sep 15 20:15:01.164436 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:01.164408    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8r97z"]
Sep 15 20:15:01.242428 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:01.242385    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bd803454-dcc5-4f86-923d-e1fc22ba6e35-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8r97z\" (UID: \"bd803454-dcc5-4f86-923d-e1fc22ba6e35\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8r97z"
Sep 15 20:15:01.242428 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:01.242434    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsc4p\" (UniqueName: \"kubernetes.io/projected/bd803454-dcc5-4f86-923d-e1fc22ba6e35-kube-api-access-gsc4p\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8r97z\" (UID: \"bd803454-dcc5-4f86-923d-e1fc22ba6e35\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8r97z"
Sep 15 20:15:01.343927 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:01.343883    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bd803454-dcc5-4f86-923d-e1fc22ba6e35-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8r97z\" (UID: \"bd803454-dcc5-4f86-923d-e1fc22ba6e35\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8r97z"
Sep 15 20:15:01.343927 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:01.343930    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsc4p\" (UniqueName: \"kubernetes.io/projected/bd803454-dcc5-4f86-923d-e1fc22ba6e35-kube-api-access-gsc4p\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8r97z\" (UID: \"bd803454-dcc5-4f86-923d-e1fc22ba6e35\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8r97z"
Sep 15 20:15:01.344334 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:01.344308    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bd803454-dcc5-4f86-923d-e1fc22ba6e35-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8r97z\" (UID: \"bd803454-dcc5-4f86-923d-e1fc22ba6e35\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8r97z"
Sep 15 20:15:01.352795 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:01.352763    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsc4p\" (UniqueName: \"kubernetes.io/projected/bd803454-dcc5-4f86-923d-e1fc22ba6e35-kube-api-access-gsc4p\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8r97z\" (UID: \"bd803454-dcc5-4f86-923d-e1fc22ba6e35\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8r97z"
Sep 15 20:15:01.458625 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:01.458581    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8r97z"
Sep 15 20:15:01.736527 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:01.736502    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8r97z"]
Sep 15 20:15:01.738458 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:15:01.738419    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd803454_dcc5_4f86_923d_e1fc22ba6e35.slice/crio-48258d9f518f994d7476e03848107ba4553c3af10f07a851b19be1dc54c79c29 WatchSource:0}: Error finding container 48258d9f518f994d7476e03848107ba4553c3af10f07a851b19be1dc54c79c29: Status 404 returned error can't find the container with id 48258d9f518f994d7476e03848107ba4553c3af10f07a851b19be1dc54c79c29
Sep 15 20:15:02.655951 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:02.655908    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8r97z" event={"ID":"bd803454-dcc5-4f86-923d-e1fc22ba6e35","Type":"ContainerStarted","Data":"48258d9f518f994d7476e03848107ba4553c3af10f07a851b19be1dc54c79c29"}
Sep 15 20:15:06.681884 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:06.681788    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8r97z" event={"ID":"bd803454-dcc5-4f86-923d-e1fc22ba6e35","Type":"ContainerStarted","Data":"80f06f7c4d25d4e30027ce590897a074ca60e277023376a18072e346469d9982"}
Sep 15 20:15:06.682273 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:06.681915    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8r97z"
Sep 15 20:15:06.704017 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:06.703953    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8r97z" podStartSLOduration=1.014588913 podStartE2EDuration="5.703934597s" totalImagesPullingTime="4.689345684s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:15:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:15:06.701708518 +0000 UTC m=+613.092661516" watchObservedRunningTime="2026-09-15 20:15:06.703934597 +0000 UTC m=+613.094887594"
Sep 15 20:15:09.095620 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:09.095575    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-6nkd2"]
Sep 15 20:15:09.105567 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:09.105538    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6nkd2"
Sep 15 20:15:09.108273 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:09.108242    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 15 20:15:09.108664 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:09.108242    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 15 20:15:09.108806 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:09.108362    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-zwnp4\""
Sep 15 20:15:09.109883 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:09.109860    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-6nkd2"]
Sep 15 20:15:09.217148 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:09.217109    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4f6ccaaa-6260-439a-8d7d-36dc24fe7ecc-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-6nkd2\" (UID: \"4f6ccaaa-6260-439a-8d7d-36dc24fe7ecc\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6nkd2"
Sep 15 20:15:09.217360 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:09.217332    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4grp\" (UniqueName: \"kubernetes.io/projected/4f6ccaaa-6260-439a-8d7d-36dc24fe7ecc-kube-api-access-s4grp\") pod \"kuadrant-console-plugin-fbdd4f445-6nkd2\" (UID: \"4f6ccaaa-6260-439a-8d7d-36dc24fe7ecc\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6nkd2"
Sep 15 20:15:09.217467 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:09.217446    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f6ccaaa-6260-439a-8d7d-36dc24fe7ecc-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-6nkd2\" (UID: \"4f6ccaaa-6260-439a-8d7d-36dc24fe7ecc\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6nkd2"
Sep 15 20:15:09.318129 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:09.318092    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f6ccaaa-6260-439a-8d7d-36dc24fe7ecc-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-6nkd2\" (UID: \"4f6ccaaa-6260-439a-8d7d-36dc24fe7ecc\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6nkd2"
Sep 15 20:15:09.318320 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:09.318178    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4f6ccaaa-6260-439a-8d7d-36dc24fe7ecc-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-6nkd2\" (UID: \"4f6ccaaa-6260-439a-8d7d-36dc24fe7ecc\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6nkd2"
Sep 15 20:15:09.318320 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:09.318212    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4grp\" (UniqueName: \"kubernetes.io/projected/4f6ccaaa-6260-439a-8d7d-36dc24fe7ecc-kube-api-access-s4grp\") pod \"kuadrant-console-plugin-fbdd4f445-6nkd2\" (UID: \"4f6ccaaa-6260-439a-8d7d-36dc24fe7ecc\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6nkd2"
Sep 15 20:15:09.318320 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:15:09.318263    2577 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Sep 15 20:15:09.318442 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:15:09.318352    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f6ccaaa-6260-439a-8d7d-36dc24fe7ecc-plugin-serving-cert podName:4f6ccaaa-6260-439a-8d7d-36dc24fe7ecc nodeName:}" failed. No retries permitted until 2026-09-15 20:15:09.818334307 +0000 UTC m=+616.209287283 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/4f6ccaaa-6260-439a-8d7d-36dc24fe7ecc-plugin-serving-cert") pod "kuadrant-console-plugin-fbdd4f445-6nkd2" (UID: "4f6ccaaa-6260-439a-8d7d-36dc24fe7ecc") : secret "plugin-serving-cert" not found
Sep 15 20:15:09.318837 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:09.318819    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4f6ccaaa-6260-439a-8d7d-36dc24fe7ecc-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-6nkd2\" (UID: \"4f6ccaaa-6260-439a-8d7d-36dc24fe7ecc\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6nkd2"
Sep 15 20:15:09.329712 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:09.329679    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4grp\" (UniqueName: \"kubernetes.io/projected/4f6ccaaa-6260-439a-8d7d-36dc24fe7ecc-kube-api-access-s4grp\") pod \"kuadrant-console-plugin-fbdd4f445-6nkd2\" (UID: \"4f6ccaaa-6260-439a-8d7d-36dc24fe7ecc\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6nkd2"
Sep 15 20:15:09.628449 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:09.628419    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-d67zs"
Sep 15 20:15:09.823303 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:09.823261    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f6ccaaa-6260-439a-8d7d-36dc24fe7ecc-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-6nkd2\" (UID: \"4f6ccaaa-6260-439a-8d7d-36dc24fe7ecc\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6nkd2"
Sep 15 20:15:09.825753 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:09.825719    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f6ccaaa-6260-439a-8d7d-36dc24fe7ecc-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-6nkd2\" (UID: \"4f6ccaaa-6260-439a-8d7d-36dc24fe7ecc\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6nkd2"
Sep 15 20:15:10.016600 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:10.016541    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6nkd2"
Sep 15 20:15:10.304871 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:10.304785    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-6nkd2"]
Sep 15 20:15:10.310329 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:15:10.310285    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4f6ccaaa_6260_439a_8d7d_36dc24fe7ecc.slice/crio-a897a97b289b8eaccf9c7e26c82b8c424ceeca90e4862ea304056e731af7b8db WatchSource:0}: Error finding container a897a97b289b8eaccf9c7e26c82b8c424ceeca90e4862ea304056e731af7b8db: Status 404 returned error can't find the container with id a897a97b289b8eaccf9c7e26c82b8c424ceeca90e4862ea304056e731af7b8db
Sep 15 20:15:10.710080 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:10.710047    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6nkd2" event={"ID":"4f6ccaaa-6260-439a-8d7d-36dc24fe7ecc","Type":"ContainerStarted","Data":"a897a97b289b8eaccf9c7e26c82b8c424ceeca90e4862ea304056e731af7b8db"}
Sep 15 20:15:11.648807 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:11.648765    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-vrwfm"
Sep 15 20:15:15.749028 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:15.748986    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6nkd2" event={"ID":"4f6ccaaa-6260-439a-8d7d-36dc24fe7ecc","Type":"ContainerStarted","Data":"42a8fcd499d049a992f5315d218046e1b68ae673d54ac8881946bfbe679f47eb"}
Sep 15 20:15:15.768252 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:15.768193    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6nkd2" podStartSLOduration=2.128320962 podStartE2EDuration="6.768175944s" totalImagesPullingTime="4.639854982s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:15:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:15:15.765954368 +0000 UTC m=+622.156907364" watchObservedRunningTime="2026-09-15 20:15:15.768175944 +0000 UTC m=+622.159128940"
Sep 15 20:15:17.689857 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:17.689825    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8r97z"
Sep 15 20:15:19.180325 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:19.180289    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-vcgd9"]
Sep 15 20:15:19.184049 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:19.184024    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-vcgd9"
Sep 15 20:15:19.195759 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:19.195720    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-vcgd9"]
Sep 15 20:15:19.320138 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:19.320103    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/41d2d589-9a27-441c-8f27-efa61e4e0616-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-vcgd9\" (UID: \"41d2d589-9a27-441c-8f27-efa61e4e0616\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-vcgd9"
Sep 15 20:15:19.320324 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:19.320183    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mn7c\" (UniqueName: \"kubernetes.io/projected/41d2d589-9a27-441c-8f27-efa61e4e0616-kube-api-access-2mn7c\") pod \"kuadrant-operator-controller-manager-5f685c496f-vcgd9\" (UID: \"41d2d589-9a27-441c-8f27-efa61e4e0616\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-vcgd9"
Sep 15 20:15:19.421290 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:19.421245    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mn7c\" (UniqueName: \"kubernetes.io/projected/41d2d589-9a27-441c-8f27-efa61e4e0616-kube-api-access-2mn7c\") pod \"kuadrant-operator-controller-manager-5f685c496f-vcgd9\" (UID: \"41d2d589-9a27-441c-8f27-efa61e4e0616\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-vcgd9"
Sep 15 20:15:19.421488 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:19.421338    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/41d2d589-9a27-441c-8f27-efa61e4e0616-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-vcgd9\" (UID: \"41d2d589-9a27-441c-8f27-efa61e4e0616\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-vcgd9"
Sep 15 20:15:19.421810 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:19.421788    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/41d2d589-9a27-441c-8f27-efa61e4e0616-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-vcgd9\" (UID: \"41d2d589-9a27-441c-8f27-efa61e4e0616\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-vcgd9"
Sep 15 20:15:19.432906 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:19.432834    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mn7c\" (UniqueName: \"kubernetes.io/projected/41d2d589-9a27-441c-8f27-efa61e4e0616-kube-api-access-2mn7c\") pod \"kuadrant-operator-controller-manager-5f685c496f-vcgd9\" (UID: \"41d2d589-9a27-441c-8f27-efa61e4e0616\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-vcgd9"
Sep 15 20:15:19.495906 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:19.495864    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-vcgd9"
Sep 15 20:15:19.783923 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:19.783889    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-vcgd9"]
Sep 15 20:15:19.789137 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:15:19.789101    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41d2d589_9a27_441c_8f27_efa61e4e0616.slice/crio-13ccf0104086394d6c8898ab911a3ab8b7f2fc2a625d5437848871708e8271aa WatchSource:0}: Error finding container 13ccf0104086394d6c8898ab911a3ab8b7f2fc2a625d5437848871708e8271aa: Status 404 returned error can't find the container with id 13ccf0104086394d6c8898ab911a3ab8b7f2fc2a625d5437848871708e8271aa
Sep 15 20:15:19.913714 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:19.913658    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-vcgd9"]
Sep 15 20:15:19.926132 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:19.926094    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-vcgd9"]
Sep 15 20:15:19.938210 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:19.938178    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8r97z"]
Sep 15 20:15:19.938405 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:19.938379    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8r97z" podUID="bd803454-dcc5-4f86-923d-e1fc22ba6e35" containerName="manager" containerID="cri-o://80f06f7c4d25d4e30027ce590897a074ca60e277023376a18072e346469d9982" gracePeriod=2
Sep 15 20:15:19.944088 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:19.944060    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-224ts"]
Sep 15 20:15:19.944660 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:19.944637    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41d2d589-9a27-441c-8f27-efa61e4e0616" containerName="manager"
Sep 15 20:15:19.944660 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:19.944658    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="41d2d589-9a27-441c-8f27-efa61e4e0616" containerName="manager"
Sep 15 20:15:19.944857 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:19.944805    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="41d2d589-9a27-441c-8f27-efa61e4e0616" containerName="manager"
Sep 15 20:15:19.948796 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:15:19.948747    2577 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"bd803454-dcc5-4f86-923d-e1fc22ba6e35\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-76f8f686bf-8r97z\": pods \"kuadrant-operator-controller-manager-76f8f686bf-8r97z\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8r97z"
Sep 15 20:15:19.949369 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:19.949346    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-224ts"
Sep 15 20:15:19.949883 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:19.949754    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8r97z"]
Sep 15 20:15:19.952333 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:15:19.952300    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-8r97z\" is forbidden: User \"system:node:ip-10-0-140-72.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-72.ec2.internal' and this object" podUID="bd803454-dcc5-4f86-923d-e1fc22ba6e35" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8r97z"
Sep 15 20:15:19.966752 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:19.966704    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-224ts"]
Sep 15 20:15:20.127471 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:20.127440    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f541707-b017-415e-b03e-4300726cb306-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-224ts\" (UID: \"0f541707-b017-415e-b03e-4300726cb306\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-224ts"
Sep 15 20:15:20.127842 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:20.127482    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44pzw\" (UniqueName: \"kubernetes.io/projected/0f541707-b017-415e-b03e-4300726cb306-kube-api-access-44pzw\") pod \"kuadrant-operator-controller-manager-5f685c496f-224ts\" (UID: \"0f541707-b017-415e-b03e-4300726cb306\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-224ts"
Sep 15 20:15:20.189473 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:20.189442    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8r97z"
Sep 15 20:15:20.228922 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:20.228754    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f541707-b017-415e-b03e-4300726cb306-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-224ts\" (UID: \"0f541707-b017-415e-b03e-4300726cb306\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-224ts"
Sep 15 20:15:20.228922 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:20.228812    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44pzw\" (UniqueName: \"kubernetes.io/projected/0f541707-b017-415e-b03e-4300726cb306-kube-api-access-44pzw\") pod \"kuadrant-operator-controller-manager-5f685c496f-224ts\" (UID: \"0f541707-b017-415e-b03e-4300726cb306\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-224ts"
Sep 15 20:15:20.229189 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:20.229164    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f541707-b017-415e-b03e-4300726cb306-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-224ts\" (UID: \"0f541707-b017-415e-b03e-4300726cb306\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-224ts"
Sep 15 20:15:20.252238 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:20.252202    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44pzw\" (UniqueName: \"kubernetes.io/projected/0f541707-b017-415e-b03e-4300726cb306-kube-api-access-44pzw\") pod \"kuadrant-operator-controller-manager-5f685c496f-224ts\" (UID: \"0f541707-b017-415e-b03e-4300726cb306\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-224ts"
Sep 15 20:15:20.305300 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:20.305264    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-224ts"
Sep 15 20:15:20.329921 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:20.329892    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gsc4p\" (UniqueName: \"kubernetes.io/projected/bd803454-dcc5-4f86-923d-e1fc22ba6e35-kube-api-access-gsc4p\") pod \"bd803454-dcc5-4f86-923d-e1fc22ba6e35\" (UID: \"bd803454-dcc5-4f86-923d-e1fc22ba6e35\") "
Sep 15 20:15:20.330072 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:20.329974    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bd803454-dcc5-4f86-923d-e1fc22ba6e35-extensions-socket-volume\") pod \"bd803454-dcc5-4f86-923d-e1fc22ba6e35\" (UID: \"bd803454-dcc5-4f86-923d-e1fc22ba6e35\") "
Sep 15 20:15:20.330491 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:20.330468    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd803454-dcc5-4f86-923d-e1fc22ba6e35-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "bd803454-dcc5-4f86-923d-e1fc22ba6e35" (UID: "bd803454-dcc5-4f86-923d-e1fc22ba6e35"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:15:20.332395 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:20.332355    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd803454-dcc5-4f86-923d-e1fc22ba6e35-kube-api-access-gsc4p" (OuterVolumeSpecName: "kube-api-access-gsc4p") pod "bd803454-dcc5-4f86-923d-e1fc22ba6e35" (UID: "bd803454-dcc5-4f86-923d-e1fc22ba6e35"). InnerVolumeSpecName "kube-api-access-gsc4p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:15:20.431698 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:20.431659    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bd803454-dcc5-4f86-923d-e1fc22ba6e35-extensions-socket-volume\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:15:20.431698 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:20.431690    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gsc4p\" (UniqueName: \"kubernetes.io/projected/bd803454-dcc5-4f86-923d-e1fc22ba6e35-kube-api-access-gsc4p\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:15:20.597671 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:20.597644    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-224ts"]
Sep 15 20:15:20.598707 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:15:20.598682    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f541707_b017_415e_b03e_4300726cb306.slice/crio-1af852dca27df1f070c437f4fadf4d6cc30497919023fd2d5272ad5b7016c4d4 WatchSource:0}: Error finding container 1af852dca27df1f070c437f4fadf4d6cc30497919023fd2d5272ad5b7016c4d4: Status 404 returned error can't find the container with id 1af852dca27df1f070c437f4fadf4d6cc30497919023fd2d5272ad5b7016c4d4
Sep 15 20:15:20.782098 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:15:20.782074    2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-vcgd9" container="manager"
Sep 15 20:15:20.783944 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:20.783909    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-224ts" event={"ID":"0f541707-b017-415e-b03e-4300726cb306","Type":"ContainerStarted","Data":"bd2dd367372e0efc92e83a77d6d243bbfa0ec6041af7329728b8984fce98903a"}
Sep 15 20:15:20.784095 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:20.783971    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-224ts" event={"ID":"0f541707-b017-415e-b03e-4300726cb306","Type":"ContainerStarted","Data":"1af852dca27df1f070c437f4fadf4d6cc30497919023fd2d5272ad5b7016c4d4"}
Sep 15 20:15:20.784095 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:20.784005    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-224ts"
Sep 15 20:15:20.785631 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:20.785605    2577 generic.go:356] "Generic (PLEG): container finished" podID="bd803454-dcc5-4f86-923d-e1fc22ba6e35" containerID="80f06f7c4d25d4e30027ce590897a074ca60e277023376a18072e346469d9982" exitCode=0
Sep 15 20:15:20.785788 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:20.785649    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8r97z"
Sep 15 20:15:20.785788 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:20.785648    2577 scope.go:117] "RemoveContainer" containerID="80f06f7c4d25d4e30027ce590897a074ca60e277023376a18072e346469d9982"
Sep 15 20:15:20.795305 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:20.795282    2577 scope.go:117] "RemoveContainer" containerID="80f06f7c4d25d4e30027ce590897a074ca60e277023376a18072e346469d9982"
Sep 15 20:15:20.795671 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:15:20.795651    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80f06f7c4d25d4e30027ce590897a074ca60e277023376a18072e346469d9982\": container with ID starting with 80f06f7c4d25d4e30027ce590897a074ca60e277023376a18072e346469d9982 not found: ID does not exist" containerID="80f06f7c4d25d4e30027ce590897a074ca60e277023376a18072e346469d9982"
Sep 15 20:15:20.795768 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:20.795687    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80f06f7c4d25d4e30027ce590897a074ca60e277023376a18072e346469d9982"} err="failed to get container status \"80f06f7c4d25d4e30027ce590897a074ca60e277023376a18072e346469d9982\": rpc error: code = NotFound desc = could not find container \"80f06f7c4d25d4e30027ce590897a074ca60e277023376a18072e346469d9982\": container with ID starting with 80f06f7c4d25d4e30027ce590897a074ca60e277023376a18072e346469d9982 not found: ID does not exist"
Sep 15 20:15:20.807626 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:20.807574    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-224ts" podStartSLOduration=1.807557842 podStartE2EDuration="1.807557842s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:15:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:15:20.805069793 +0000 UTC m=+627.196022794" watchObservedRunningTime="2026-09-15 20:15:20.807557842 +0000 UTC m=+627.198510839"
Sep 15 20:15:21.794656 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:21.794560    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-vcgd9" podUID="41d2d589-9a27-441c-8f27-efa61e4e0616" containerName="manager" containerID="cri-o://9c36f1897afae61c2dcb3883578228baa94c95a6f119fc451e224861e05476b0" gracePeriod=2
Sep 15 20:15:21.797361 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:15:21.797333    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-vcgd9\" is forbidden: User \"system:node:ip-10-0-140-72.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-72.ec2.internal' and this object" podUID="41d2d589-9a27-441c-8f27-efa61e4e0616" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-vcgd9"
Sep 15 20:15:22.014478 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:22.014453    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-vcgd9"
Sep 15 20:15:22.016804 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:15:22.016778    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-vcgd9\" is forbidden: User \"system:node:ip-10-0-140-72.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-72.ec2.internal' and this object" podUID="41d2d589-9a27-441c-8f27-efa61e4e0616" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-vcgd9"
Sep 15 20:15:22.060776 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:22.060680    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd803454-dcc5-4f86-923d-e1fc22ba6e35" path="/var/lib/kubelet/pods/bd803454-dcc5-4f86-923d-e1fc22ba6e35/volumes"
Sep 15 20:15:22.146655 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:22.146619    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2mn7c\" (UniqueName: \"kubernetes.io/projected/41d2d589-9a27-441c-8f27-efa61e4e0616-kube-api-access-2mn7c\") pod \"41d2d589-9a27-441c-8f27-efa61e4e0616\" (UID: \"41d2d589-9a27-441c-8f27-efa61e4e0616\") "
Sep 15 20:15:22.146873 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:22.146700    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/41d2d589-9a27-441c-8f27-efa61e4e0616-extensions-socket-volume\") pod \"41d2d589-9a27-441c-8f27-efa61e4e0616\" (UID: \"41d2d589-9a27-441c-8f27-efa61e4e0616\") "
Sep 15 20:15:22.146987 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:22.146962    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41d2d589-9a27-441c-8f27-efa61e4e0616-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "41d2d589-9a27-441c-8f27-efa61e4e0616" (UID: "41d2d589-9a27-441c-8f27-efa61e4e0616"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:15:22.148757 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:22.148715    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41d2d589-9a27-441c-8f27-efa61e4e0616-kube-api-access-2mn7c" (OuterVolumeSpecName: "kube-api-access-2mn7c") pod "41d2d589-9a27-441c-8f27-efa61e4e0616" (UID: "41d2d589-9a27-441c-8f27-efa61e4e0616"). InnerVolumeSpecName "kube-api-access-2mn7c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:15:22.248134 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:22.248093    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mn7c\" (UniqueName: \"kubernetes.io/projected/41d2d589-9a27-441c-8f27-efa61e4e0616-kube-api-access-2mn7c\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:15:22.248134 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:22.248129    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/41d2d589-9a27-441c-8f27-efa61e4e0616-extensions-socket-volume\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:15:22.802000 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:22.801964    2577 generic.go:356] "Generic (PLEG): container finished" podID="41d2d589-9a27-441c-8f27-efa61e4e0616" containerID="9c36f1897afae61c2dcb3883578228baa94c95a6f119fc451e224861e05476b0" exitCode=2
Sep 15 20:15:22.802424 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:22.802011    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-vcgd9"
Sep 15 20:15:22.802424 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:22.802063    2577 scope.go:117] "RemoveContainer" containerID="9c36f1897afae61c2dcb3883578228baa94c95a6f119fc451e224861e05476b0"
Sep 15 20:15:22.816572 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:22.816543    2577 scope.go:117] "RemoveContainer" containerID="9c36f1897afae61c2dcb3883578228baa94c95a6f119fc451e224861e05476b0"
Sep 15 20:15:22.817045 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:15:22.817014    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c36f1897afae61c2dcb3883578228baa94c95a6f119fc451e224861e05476b0\": container with ID starting with 9c36f1897afae61c2dcb3883578228baa94c95a6f119fc451e224861e05476b0 not found: ID does not exist" containerID="9c36f1897afae61c2dcb3883578228baa94c95a6f119fc451e224861e05476b0"
Sep 15 20:15:22.817160 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:22.817047    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c36f1897afae61c2dcb3883578228baa94c95a6f119fc451e224861e05476b0"} err="failed to get container status \"9c36f1897afae61c2dcb3883578228baa94c95a6f119fc451e224861e05476b0\": rpc error: code = NotFound desc = could not find container \"9c36f1897afae61c2dcb3883578228baa94c95a6f119fc451e224861e05476b0\": container with ID starting with 9c36f1897afae61c2dcb3883578228baa94c95a6f119fc451e224861e05476b0 not found: ID does not exist"
Sep 15 20:15:22.820561 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:15:22.820531    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-vcgd9\" is forbidden: User \"system:node:ip-10-0-140-72.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-72.ec2.internal' and this object" podUID="41d2d589-9a27-441c-8f27-efa61e4e0616" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-vcgd9"
Sep 15 20:15:24.059686 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:15:24.059646    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-vcgd9\" is forbidden: User \"system:node:ip-10-0-140-72.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-72.ec2.internal' and this object" podUID="41d2d589-9a27-441c-8f27-efa61e4e0616" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-vcgd9"
Sep 15 20:15:24.060682 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:24.060659    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41d2d589-9a27-441c-8f27-efa61e4e0616" path="/var/lib/kubelet/pods/41d2d589-9a27-441c-8f27-efa61e4e0616/volumes"
Sep 15 20:15:31.797543 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:31.797504    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-224ts"
Sep 15 20:15:32.152961 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:32.152927    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qrq84"]
Sep 15 20:15:32.153334 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:32.153321    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd803454-dcc5-4f86-923d-e1fc22ba6e35" containerName="manager"
Sep 15 20:15:32.153395 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:32.153335    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd803454-dcc5-4f86-923d-e1fc22ba6e35" containerName="manager"
Sep 15 20:15:32.153437 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:32.153414    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd803454-dcc5-4f86-923d-e1fc22ba6e35" containerName="manager"
Sep 15 20:15:32.157939 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:32.157918    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qrq84"
Sep 15 20:15:32.171601 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:32.171568    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qrq84"]
Sep 15 20:15:32.235873 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:32.235837    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6dlg\" (UniqueName: \"kubernetes.io/projected/c3c6f523-42a6-471f-ab50-f0fd4b821101-kube-api-access-m6dlg\") pod \"kuadrant-operator-controller-manager-5769f99d9f-qrq84\" (UID: \"c3c6f523-42a6-471f-ab50-f0fd4b821101\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qrq84"
Sep 15 20:15:32.236061 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:32.235897    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c3c6f523-42a6-471f-ab50-f0fd4b821101-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-qrq84\" (UID: \"c3c6f523-42a6-471f-ab50-f0fd4b821101\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qrq84"
Sep 15 20:15:32.337442 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:32.337345    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6dlg\" (UniqueName: \"kubernetes.io/projected/c3c6f523-42a6-471f-ab50-f0fd4b821101-kube-api-access-m6dlg\") pod \"kuadrant-operator-controller-manager-5769f99d9f-qrq84\" (UID: \"c3c6f523-42a6-471f-ab50-f0fd4b821101\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qrq84"
Sep 15 20:15:32.337442 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:32.337412    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c3c6f523-42a6-471f-ab50-f0fd4b821101-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-qrq84\" (UID: \"c3c6f523-42a6-471f-ab50-f0fd4b821101\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qrq84"
Sep 15 20:15:32.337830 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:32.337811    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c3c6f523-42a6-471f-ab50-f0fd4b821101-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-qrq84\" (UID: \"c3c6f523-42a6-471f-ab50-f0fd4b821101\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qrq84"
Sep 15 20:15:32.346228 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:32.346196    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6dlg\" (UniqueName: \"kubernetes.io/projected/c3c6f523-42a6-471f-ab50-f0fd4b821101-kube-api-access-m6dlg\") pod \"kuadrant-operator-controller-manager-5769f99d9f-qrq84\" (UID: \"c3c6f523-42a6-471f-ab50-f0fd4b821101\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qrq84"
Sep 15 20:15:32.468685 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:32.468629    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qrq84"
Sep 15 20:15:32.745647 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:32.745622    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qrq84"]
Sep 15 20:15:32.747821 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:15:32.747786    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3c6f523_42a6_471f_ab50_f0fd4b821101.slice/crio-fb31f1d9e16b11021715c20f3de69cbeb8f263327c5541a34bd829d4afd60d35 WatchSource:0}: Error finding container fb31f1d9e16b11021715c20f3de69cbeb8f263327c5541a34bd829d4afd60d35: Status 404 returned error can't find the container with id fb31f1d9e16b11021715c20f3de69cbeb8f263327c5541a34bd829d4afd60d35
Sep 15 20:15:32.867966 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:32.867929    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qrq84" event={"ID":"c3c6f523-42a6-471f-ab50-f0fd4b821101","Type":"ContainerStarted","Data":"20e757d2a4f7eabdac679565ba6a7a92ce457e8d86d132851ccaba02d3703ec5"}
Sep 15 20:15:32.868343 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:32.867979    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qrq84" event={"ID":"c3c6f523-42a6-471f-ab50-f0fd4b821101","Type":"ContainerStarted","Data":"fb31f1d9e16b11021715c20f3de69cbeb8f263327c5541a34bd829d4afd60d35"}
Sep 15 20:15:32.868343 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:32.868058    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qrq84"
Sep 15 20:15:32.887033 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:32.886962    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qrq84" podStartSLOduration=0.886912957 podStartE2EDuration="886.912957ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:15:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:15:32.885857756 +0000 UTC m=+639.276810863" watchObservedRunningTime="2026-09-15 20:15:32.886912957 +0000 UTC m=+639.277865954"
Sep 15 20:15:43.876381 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:43.876334    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qrq84"
Sep 15 20:15:43.933559 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:43.933525    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-224ts"]
Sep 15 20:15:43.934092 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:43.934057    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-224ts" podUID="0f541707-b017-415e-b03e-4300726cb306" containerName="manager" containerID="cri-o://bd2dd367372e0efc92e83a77d6d243bbfa0ec6041af7329728b8984fce98903a" gracePeriod=10
Sep 15 20:15:44.186851 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:44.186811    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-224ts"
Sep 15 20:15:44.251250 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:44.251211    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44pzw\" (UniqueName: \"kubernetes.io/projected/0f541707-b017-415e-b03e-4300726cb306-kube-api-access-44pzw\") pod \"0f541707-b017-415e-b03e-4300726cb306\" (UID: \"0f541707-b017-415e-b03e-4300726cb306\") "
Sep 15 20:15:44.251454 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:44.251307    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f541707-b017-415e-b03e-4300726cb306-extensions-socket-volume\") pod \"0f541707-b017-415e-b03e-4300726cb306\" (UID: \"0f541707-b017-415e-b03e-4300726cb306\") "
Sep 15 20:15:44.251781 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:44.251757    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f541707-b017-415e-b03e-4300726cb306-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0f541707-b017-415e-b03e-4300726cb306" (UID: "0f541707-b017-415e-b03e-4300726cb306"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:15:44.253363 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:44.253339    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f541707-b017-415e-b03e-4300726cb306-kube-api-access-44pzw" (OuterVolumeSpecName: "kube-api-access-44pzw") pod "0f541707-b017-415e-b03e-4300726cb306" (UID: "0f541707-b017-415e-b03e-4300726cb306"). InnerVolumeSpecName "kube-api-access-44pzw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:15:44.352992 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:44.352953    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44pzw\" (UniqueName: \"kubernetes.io/projected/0f541707-b017-415e-b03e-4300726cb306-kube-api-access-44pzw\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:15:44.352992 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:44.352983    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f541707-b017-415e-b03e-4300726cb306-extensions-socket-volume\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:15:44.942488 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:44.942449    2577 generic.go:356] "Generic (PLEG): container finished" podID="0f541707-b017-415e-b03e-4300726cb306" containerID="bd2dd367372e0efc92e83a77d6d243bbfa0ec6041af7329728b8984fce98903a" exitCode=0
Sep 15 20:15:44.942959 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:44.942498    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-224ts"
Sep 15 20:15:44.942959 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:44.942546    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-224ts" event={"ID":"0f541707-b017-415e-b03e-4300726cb306","Type":"ContainerDied","Data":"bd2dd367372e0efc92e83a77d6d243bbfa0ec6041af7329728b8984fce98903a"}
Sep 15 20:15:44.942959 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:44.942604    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-224ts" event={"ID":"0f541707-b017-415e-b03e-4300726cb306","Type":"ContainerDied","Data":"1af852dca27df1f070c437f4fadf4d6cc30497919023fd2d5272ad5b7016c4d4"}
Sep 15 20:15:44.942959 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:44.942634    2577 scope.go:117] "RemoveContainer" containerID="bd2dd367372e0efc92e83a77d6d243bbfa0ec6041af7329728b8984fce98903a"
Sep 15 20:15:44.952111 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:44.952091    2577 scope.go:117] "RemoveContainer" containerID="bd2dd367372e0efc92e83a77d6d243bbfa0ec6041af7329728b8984fce98903a"
Sep 15 20:15:44.952393 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:15:44.952373    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd2dd367372e0efc92e83a77d6d243bbfa0ec6041af7329728b8984fce98903a\": container with ID starting with bd2dd367372e0efc92e83a77d6d243bbfa0ec6041af7329728b8984fce98903a not found: ID does not exist" containerID="bd2dd367372e0efc92e83a77d6d243bbfa0ec6041af7329728b8984fce98903a"
Sep 15 20:15:44.952439 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:44.952402    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd2dd367372e0efc92e83a77d6d243bbfa0ec6041af7329728b8984fce98903a"} err="failed to get container status \"bd2dd367372e0efc92e83a77d6d243bbfa0ec6041af7329728b8984fce98903a\": rpc error: code = NotFound desc = could not find container \"bd2dd367372e0efc92e83a77d6d243bbfa0ec6041af7329728b8984fce98903a\": container with ID starting with bd2dd367372e0efc92e83a77d6d243bbfa0ec6041af7329728b8984fce98903a not found: ID does not exist"
Sep 15 20:15:44.976438 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:44.976397    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-224ts"]
Sep 15 20:15:44.983438 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:44.983410    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-224ts"]
Sep 15 20:15:46.060204 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:15:46.060167    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f541707-b017-415e-b03e-4300726cb306" path="/var/lib/kubelet/pods/0f541707-b017-415e-b03e-4300726cb306/volumes"
Sep 15 20:16:01.723966 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:01.723931    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-ccvvt"]
Sep 15 20:16:01.724411 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:01.724345    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f541707-b017-415e-b03e-4300726cb306" containerName="manager"
Sep 15 20:16:01.724411 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:01.724356    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f541707-b017-415e-b03e-4300726cb306" containerName="manager"
Sep 15 20:16:01.724484 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:01.724431    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f541707-b017-415e-b03e-4300726cb306" containerName="manager"
Sep 15 20:16:01.730604 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:01.730582    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-ccvvt"
Sep 15 20:16:01.733093 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:01.733071    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 15 20:16:01.735201 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:01.735174    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-ccvvt"]
Sep 15 20:16:01.823989 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:01.823951    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-ccvvt"]
Sep 15 20:16:01.921075 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:01.921044    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8ce2712f-6f20-488b-9d43-93e0eec5b769-config-file\") pod \"limitador-limitador-8588749977-ccvvt\" (UID: \"8ce2712f-6f20-488b-9d43-93e0eec5b769\") " pod="kuadrant-system/limitador-limitador-8588749977-ccvvt"
Sep 15 20:16:01.921284 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:01.921218    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4xct\" (UniqueName: \"kubernetes.io/projected/8ce2712f-6f20-488b-9d43-93e0eec5b769-kube-api-access-x4xct\") pod \"limitador-limitador-8588749977-ccvvt\" (UID: \"8ce2712f-6f20-488b-9d43-93e0eec5b769\") " pod="kuadrant-system/limitador-limitador-8588749977-ccvvt"
Sep 15 20:16:02.022465 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:02.022382    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4xct\" (UniqueName: \"kubernetes.io/projected/8ce2712f-6f20-488b-9d43-93e0eec5b769-kube-api-access-x4xct\") pod \"limitador-limitador-8588749977-ccvvt\" (UID: \"8ce2712f-6f20-488b-9d43-93e0eec5b769\") " pod="kuadrant-system/limitador-limitador-8588749977-ccvvt"
Sep 15 20:16:02.022465 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:02.022434    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8ce2712f-6f20-488b-9d43-93e0eec5b769-config-file\") pod \"limitador-limitador-8588749977-ccvvt\" (UID: \"8ce2712f-6f20-488b-9d43-93e0eec5b769\") " pod="kuadrant-system/limitador-limitador-8588749977-ccvvt"
Sep 15 20:16:02.023086 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:02.023068    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8ce2712f-6f20-488b-9d43-93e0eec5b769-config-file\") pod \"limitador-limitador-8588749977-ccvvt\" (UID: \"8ce2712f-6f20-488b-9d43-93e0eec5b769\") " pod="kuadrant-system/limitador-limitador-8588749977-ccvvt"
Sep 15 20:16:02.031259 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:02.031232    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4xct\" (UniqueName: \"kubernetes.io/projected/8ce2712f-6f20-488b-9d43-93e0eec5b769-kube-api-access-x4xct\") pod \"limitador-limitador-8588749977-ccvvt\" (UID: \"8ce2712f-6f20-488b-9d43-93e0eec5b769\") " pod="kuadrant-system/limitador-limitador-8588749977-ccvvt"
Sep 15 20:16:02.043283 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:02.043248    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-ccvvt"
Sep 15 20:16:02.319715 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:02.319689    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-ccvvt"]
Sep 15 20:16:02.321396 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:16:02.321366    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ce2712f_6f20_488b_9d43_93e0eec5b769.slice/crio-7e322adc868d58d9f44d5d02f1ecc000a3bf4455ee35905407c30988dfa328f5 WatchSource:0}: Error finding container 7e322adc868d58d9f44d5d02f1ecc000a3bf4455ee35905407c30988dfa328f5: Status 404 returned error can't find the container with id 7e322adc868d58d9f44d5d02f1ecc000a3bf4455ee35905407c30988dfa328f5
Sep 15 20:16:03.052055 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:03.052014    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-ccvvt" event={"ID":"8ce2712f-6f20-488b-9d43-93e0eec5b769","Type":"ContainerStarted","Data":"7e322adc868d58d9f44d5d02f1ecc000a3bf4455ee35905407c30988dfa328f5"}
Sep 15 20:16:05.067542 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:05.067497    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-ccvvt" event={"ID":"8ce2712f-6f20-488b-9d43-93e0eec5b769","Type":"ContainerStarted","Data":"7fbcfc6d5dee15144ab80614f4b342a46d28a61fb06b7e952ca7dbfa0566446f"}
Sep 15 20:16:05.068045 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:05.067676    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-ccvvt"
Sep 15 20:16:05.091141 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:05.091080    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-ccvvt" podStartSLOduration=2.195212248 podStartE2EDuration="4.091064706s" totalImagesPullingTime="1.895852458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:16:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:16:05.087756745 +0000 UTC m=+671.478709737" watchObservedRunningTime="2026-09-15 20:16:05.091064706 +0000 UTC m=+671.482017703"
Sep 15 20:16:16.074564 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:16.074524    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-ccvvt"
Sep 15 20:16:16.526351 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:16.526309    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-ccvvt"]
Sep 15 20:16:16.526606 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:16.526582    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-ccvvt" podUID="8ce2712f-6f20-488b-9d43-93e0eec5b769" containerName="limitador" containerID="cri-o://7fbcfc6d5dee15144ab80614f4b342a46d28a61fb06b7e952ca7dbfa0566446f" gracePeriod=30
Sep 15 20:16:17.081947 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:17.081916    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-ccvvt"
Sep 15 20:16:17.143770 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:17.143657    2577 generic.go:356] "Generic (PLEG): container finished" podID="8ce2712f-6f20-488b-9d43-93e0eec5b769" containerID="7fbcfc6d5dee15144ab80614f4b342a46d28a61fb06b7e952ca7dbfa0566446f" exitCode=0
Sep 15 20:16:17.143770 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:17.143709    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-ccvvt"
Sep 15 20:16:17.143770 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:17.143753    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-ccvvt" event={"ID":"8ce2712f-6f20-488b-9d43-93e0eec5b769","Type":"ContainerDied","Data":"7fbcfc6d5dee15144ab80614f4b342a46d28a61fb06b7e952ca7dbfa0566446f"}
Sep 15 20:16:17.144042 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:17.143807    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-ccvvt" event={"ID":"8ce2712f-6f20-488b-9d43-93e0eec5b769","Type":"ContainerDied","Data":"7e322adc868d58d9f44d5d02f1ecc000a3bf4455ee35905407c30988dfa328f5"}
Sep 15 20:16:17.144042 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:17.143829    2577 scope.go:117] "RemoveContainer" containerID="7fbcfc6d5dee15144ab80614f4b342a46d28a61fb06b7e952ca7dbfa0566446f"
Sep 15 20:16:17.155230 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:17.155204    2577 scope.go:117] "RemoveContainer" containerID="7fbcfc6d5dee15144ab80614f4b342a46d28a61fb06b7e952ca7dbfa0566446f"
Sep 15 20:16:17.155414 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:17.155393    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x4xct\" (UniqueName: \"kubernetes.io/projected/8ce2712f-6f20-488b-9d43-93e0eec5b769-kube-api-access-x4xct\") pod \"8ce2712f-6f20-488b-9d43-93e0eec5b769\" (UID: \"8ce2712f-6f20-488b-9d43-93e0eec5b769\") "
Sep 15 20:16:17.155517 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:17.155501    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8ce2712f-6f20-488b-9d43-93e0eec5b769-config-file\") pod \"8ce2712f-6f20-488b-9d43-93e0eec5b769\" (UID: \"8ce2712f-6f20-488b-9d43-93e0eec5b769\") "
Sep 15 20:16:17.155657 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:16:17.155559    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fbcfc6d5dee15144ab80614f4b342a46d28a61fb06b7e952ca7dbfa0566446f\": container with ID starting with 7fbcfc6d5dee15144ab80614f4b342a46d28a61fb06b7e952ca7dbfa0566446f not found: ID does not exist" containerID="7fbcfc6d5dee15144ab80614f4b342a46d28a61fb06b7e952ca7dbfa0566446f"
Sep 15 20:16:17.155657 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:17.155597    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fbcfc6d5dee15144ab80614f4b342a46d28a61fb06b7e952ca7dbfa0566446f"} err="failed to get container status \"7fbcfc6d5dee15144ab80614f4b342a46d28a61fb06b7e952ca7dbfa0566446f\": rpc error: code = NotFound desc = could not find container \"7fbcfc6d5dee15144ab80614f4b342a46d28a61fb06b7e952ca7dbfa0566446f\": container with ID starting with 7fbcfc6d5dee15144ab80614f4b342a46d28a61fb06b7e952ca7dbfa0566446f not found: ID does not exist"
Sep 15 20:16:17.156139 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:17.155899    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ce2712f-6f20-488b-9d43-93e0eec5b769-config-file" (OuterVolumeSpecName: "config-file") pod "8ce2712f-6f20-488b-9d43-93e0eec5b769" (UID: "8ce2712f-6f20-488b-9d43-93e0eec5b769"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:16:17.157498 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:17.157470    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ce2712f-6f20-488b-9d43-93e0eec5b769-kube-api-access-x4xct" (OuterVolumeSpecName: "kube-api-access-x4xct") pod "8ce2712f-6f20-488b-9d43-93e0eec5b769" (UID: "8ce2712f-6f20-488b-9d43-93e0eec5b769"). InnerVolumeSpecName "kube-api-access-x4xct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:16:17.257112 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:17.257079    2577 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8ce2712f-6f20-488b-9d43-93e0eec5b769-config-file\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:16:17.257112 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:17.257109    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4xct\" (UniqueName: \"kubernetes.io/projected/8ce2712f-6f20-488b-9d43-93e0eec5b769-kube-api-access-x4xct\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:16:17.465893 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:17.465860    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-ccvvt"]
Sep 15 20:16:17.470224 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:17.470196    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-ccvvt"]
Sep 15 20:16:18.060739 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:18.060698    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ce2712f-6f20-488b-9d43-93e0eec5b769" path="/var/lib/kubelet/pods/8ce2712f-6f20-488b-9d43-93e0eec5b769/volumes"
Sep 15 20:16:20.366091 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:20.366051    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-6w4pt"]
Sep 15 20:16:20.366673 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:20.366656    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ce2712f-6f20-488b-9d43-93e0eec5b769" containerName="limitador"
Sep 15 20:16:20.366768 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:20.366676    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ce2712f-6f20-488b-9d43-93e0eec5b769" containerName="limitador"
Sep 15 20:16:20.366815 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:20.366804    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ce2712f-6f20-488b-9d43-93e0eec5b769" containerName="limitador"
Sep 15 20:16:20.370087 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:20.370064    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6w4pt"
Sep 15 20:16:20.373635 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:20.373613    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-84945\""
Sep 15 20:16:20.374272 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:20.374250    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 15 20:16:20.374390 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:20.374250    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 15 20:16:20.374390 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:20.374292    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 20:16:20.386044 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:20.386016    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-6w4pt"]
Sep 15 20:16:20.487332 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:20.487288    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wsc7\" (UniqueName: \"kubernetes.io/projected/b60a69e7-cf8d-4848-8815-72eb0f3f89ae-kube-api-access-4wsc7\") pod \"postgres-69fb4594cf-6w4pt\" (UID: \"b60a69e7-cf8d-4848-8815-72eb0f3f89ae\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6w4pt"
Sep 15 20:16:20.487499 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:20.487365    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b60a69e7-cf8d-4848-8815-72eb0f3f89ae-data\") pod \"postgres-69fb4594cf-6w4pt\" (UID: \"b60a69e7-cf8d-4848-8815-72eb0f3f89ae\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6w4pt"
Sep 15 20:16:20.588432 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:20.588392    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b60a69e7-cf8d-4848-8815-72eb0f3f89ae-data\") pod \"postgres-69fb4594cf-6w4pt\" (UID: \"b60a69e7-cf8d-4848-8815-72eb0f3f89ae\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6w4pt"
Sep 15 20:16:20.588628 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:20.588481    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wsc7\" (UniqueName: \"kubernetes.io/projected/b60a69e7-cf8d-4848-8815-72eb0f3f89ae-kube-api-access-4wsc7\") pod \"postgres-69fb4594cf-6w4pt\" (UID: \"b60a69e7-cf8d-4848-8815-72eb0f3f89ae\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6w4pt"
Sep 15 20:16:20.588835 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:20.588813    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b60a69e7-cf8d-4848-8815-72eb0f3f89ae-data\") pod \"postgres-69fb4594cf-6w4pt\" (UID: \"b60a69e7-cf8d-4848-8815-72eb0f3f89ae\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6w4pt"
Sep 15 20:16:20.598097 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:20.598071    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wsc7\" (UniqueName: \"kubernetes.io/projected/b60a69e7-cf8d-4848-8815-72eb0f3f89ae-kube-api-access-4wsc7\") pod \"postgres-69fb4594cf-6w4pt\" (UID: \"b60a69e7-cf8d-4848-8815-72eb0f3f89ae\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6w4pt"
Sep 15 20:16:20.680714 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:20.680629    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6w4pt"
Sep 15 20:16:20.966342 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:20.966316    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-6w4pt"]
Sep 15 20:16:20.967874 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:16:20.967846    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb60a69e7_cf8d_4848_8815_72eb0f3f89ae.slice/crio-83579c0c6b2f7d12929d14024ba4f476d008fb38324e766debafb5cadb7cd9ac WatchSource:0}: Error finding container 83579c0c6b2f7d12929d14024ba4f476d008fb38324e766debafb5cadb7cd9ac: Status 404 returned error can't find the container with id 83579c0c6b2f7d12929d14024ba4f476d008fb38324e766debafb5cadb7cd9ac
Sep 15 20:16:20.969382 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:20.969363    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 20:16:21.170015 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:21.169972    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6w4pt" event={"ID":"b60a69e7-cf8d-4848-8815-72eb0f3f89ae","Type":"ContainerStarted","Data":"83579c0c6b2f7d12929d14024ba4f476d008fb38324e766debafb5cadb7cd9ac"}
Sep 15 20:16:24.190593 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:24.190551    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6w4pt" event={"ID":"b60a69e7-cf8d-4848-8815-72eb0f3f89ae","Type":"ContainerStarted","Data":"2c9982bcec0b71f83ba0caaa8008e9b6fcaaff61a21b49f413452964ad76bb20"}
Sep 15 20:16:24.191201 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:24.191177    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6w4pt"
Sep 15 20:16:24.210048 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:24.209996    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6w4pt" podStartSLOduration=2.045710275 podStartE2EDuration="4.209980691s" totalImagesPullingTime="2.164270416s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:16:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:16:24.207668179 +0000 UTC m=+690.598621177" watchObservedRunningTime="2026-09-15 20:16:24.209980691 +0000 UTC m=+690.600933687"
Sep 15 20:16:30.231265 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:30.231227    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6w4pt"
Sep 15 20:16:37.817540 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:37.817505    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7ff79c9c6c-8fcwk"]
Sep 15 20:16:37.827158 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:37.827133    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7ff79c9c6c-8fcwk"
Sep 15 20:16:37.829621 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:37.829596    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 15 20:16:37.830778 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:37.830752    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 15 20:16:37.830904 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:37.830781    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8wcdj\""
Sep 15 20:16:37.831048 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:37.831026    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 15 20:16:37.834701 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:37.834672    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7ff79c9c6c-8fcwk"]
Sep 15 20:16:37.953989 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:37.953953    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97664b01-dd26-4176-99d9-6bb0af1f9cd3-metrics-certs\") pod \"maas-controller-7ff79c9c6c-8fcwk\" (UID: \"97664b01-dd26-4176-99d9-6bb0af1f9cd3\") " pod="opendatahub/maas-controller-7ff79c9c6c-8fcwk"
Sep 15 20:16:37.954165 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:37.953995    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xhkk\" (UniqueName: \"kubernetes.io/projected/97664b01-dd26-4176-99d9-6bb0af1f9cd3-kube-api-access-6xhkk\") pod \"maas-controller-7ff79c9c6c-8fcwk\" (UID: \"97664b01-dd26-4176-99d9-6bb0af1f9cd3\") " pod="opendatahub/maas-controller-7ff79c9c6c-8fcwk"
Sep 15 20:16:37.954165 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:37.954100    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/97664b01-dd26-4176-99d9-6bb0af1f9cd3-webhook-cert\") pod \"maas-controller-7ff79c9c6c-8fcwk\" (UID: \"97664b01-dd26-4176-99d9-6bb0af1f9cd3\") " pod="opendatahub/maas-controller-7ff79c9c6c-8fcwk"
Sep 15 20:16:37.985996 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:37.985960    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5b579c7cb4-gg2pv"]
Sep 15 20:16:37.989709 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:37.989679    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b579c7cb4-gg2pv"
Sep 15 20:16:38.001339 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:38.001313    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b579c7cb4-gg2pv"]
Sep 15 20:16:38.055246 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:38.055203    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xhkk\" (UniqueName: \"kubernetes.io/projected/97664b01-dd26-4176-99d9-6bb0af1f9cd3-kube-api-access-6xhkk\") pod \"maas-controller-7ff79c9c6c-8fcwk\" (UID: \"97664b01-dd26-4176-99d9-6bb0af1f9cd3\") " pod="opendatahub/maas-controller-7ff79c9c6c-8fcwk"
Sep 15 20:16:38.055427 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:38.055282    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/97664b01-dd26-4176-99d9-6bb0af1f9cd3-webhook-cert\") pod \"maas-controller-7ff79c9c6c-8fcwk\" (UID: \"97664b01-dd26-4176-99d9-6bb0af1f9cd3\") " pod="opendatahub/maas-controller-7ff79c9c6c-8fcwk"
Sep 15 20:16:38.055427 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:38.055330    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97664b01-dd26-4176-99d9-6bb0af1f9cd3-metrics-certs\") pod \"maas-controller-7ff79c9c6c-8fcwk\" (UID: \"97664b01-dd26-4176-99d9-6bb0af1f9cd3\") " pod="opendatahub/maas-controller-7ff79c9c6c-8fcwk"
Sep 15 20:16:38.058061 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:38.058035    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/97664b01-dd26-4176-99d9-6bb0af1f9cd3-webhook-cert\") pod \"maas-controller-7ff79c9c6c-8fcwk\" (UID: \"97664b01-dd26-4176-99d9-6bb0af1f9cd3\") " pod="opendatahub/maas-controller-7ff79c9c6c-8fcwk"
Sep 15 20:16:38.058185 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:38.058035    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97664b01-dd26-4176-99d9-6bb0af1f9cd3-metrics-certs\") pod \"maas-controller-7ff79c9c6c-8fcwk\" (UID: \"97664b01-dd26-4176-99d9-6bb0af1f9cd3\") " pod="opendatahub/maas-controller-7ff79c9c6c-8fcwk"
Sep 15 20:16:38.063318 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:38.063293    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xhkk\" (UniqueName: \"kubernetes.io/projected/97664b01-dd26-4176-99d9-6bb0af1f9cd3-kube-api-access-6xhkk\") pod \"maas-controller-7ff79c9c6c-8fcwk\" (UID: \"97664b01-dd26-4176-99d9-6bb0af1f9cd3\") " pod="opendatahub/maas-controller-7ff79c9c6c-8fcwk"
Sep 15 20:16:38.139926 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:38.139836    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7ff79c9c6c-8fcwk"
Sep 15 20:16:38.155896 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:38.155866    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2k2k\" (UniqueName: \"kubernetes.io/projected/3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774-kube-api-access-w2k2k\") pod \"maas-controller-5b579c7cb4-gg2pv\" (UID: \"3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774\") " pod="opendatahub/maas-controller-5b579c7cb4-gg2pv"
Sep 15 20:16:38.156026 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:38.155911    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774-webhook-cert\") pod \"maas-controller-5b579c7cb4-gg2pv\" (UID: \"3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774\") " pod="opendatahub/maas-controller-5b579c7cb4-gg2pv"
Sep 15 20:16:38.156026 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:38.155938    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774-metrics-certs\") pod \"maas-controller-5b579c7cb4-gg2pv\" (UID: \"3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774\") " pod="opendatahub/maas-controller-5b579c7cb4-gg2pv"
Sep 15 20:16:38.256556 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:38.256518    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2k2k\" (UniqueName: \"kubernetes.io/projected/3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774-kube-api-access-w2k2k\") pod \"maas-controller-5b579c7cb4-gg2pv\" (UID: \"3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774\") " pod="opendatahub/maas-controller-5b579c7cb4-gg2pv"
Sep 15 20:16:38.256556 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:38.256560    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774-webhook-cert\") pod \"maas-controller-5b579c7cb4-gg2pv\" (UID: \"3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774\") " pod="opendatahub/maas-controller-5b579c7cb4-gg2pv"
Sep 15 20:16:38.256876 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:38.256577    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774-metrics-certs\") pod \"maas-controller-5b579c7cb4-gg2pv\" (UID: \"3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774\") " pod="opendatahub/maas-controller-5b579c7cb4-gg2pv"
Sep 15 20:16:38.259130 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:38.259102    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774-webhook-cert\") pod \"maas-controller-5b579c7cb4-gg2pv\" (UID: \"3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774\") " pod="opendatahub/maas-controller-5b579c7cb4-gg2pv"
Sep 15 20:16:38.259233 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:38.259211    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774-metrics-certs\") pod \"maas-controller-5b579c7cb4-gg2pv\" (UID: \"3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774\") " pod="opendatahub/maas-controller-5b579c7cb4-gg2pv"
Sep 15 20:16:38.266223 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:38.266195    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2k2k\" (UniqueName: \"kubernetes.io/projected/3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774-kube-api-access-w2k2k\") pod \"maas-controller-5b579c7cb4-gg2pv\" (UID: \"3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774\") " pod="opendatahub/maas-controller-5b579c7cb4-gg2pv"
Sep 15 20:16:38.300939 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:38.300902    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b579c7cb4-gg2pv"
Sep 15 20:16:38.418577 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:38.418545    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7ff79c9c6c-8fcwk"]
Sep 15 20:16:38.419987 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:16:38.419959    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97664b01_dd26_4176_99d9_6bb0af1f9cd3.slice/crio-58a029b2e629cfe72557eba4236738e7e70d0da45e47934491e38d5928be4b8c WatchSource:0}: Error finding container 58a029b2e629cfe72557eba4236738e7e70d0da45e47934491e38d5928be4b8c: Status 404 returned error can't find the container with id 58a029b2e629cfe72557eba4236738e7e70d0da45e47934491e38d5928be4b8c
Sep 15 20:16:38.576865 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:38.576839    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b579c7cb4-gg2pv"]
Sep 15 20:16:38.579480 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:16:38.579452    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f36b4cb_ff71_4d7d_a4e2_dd3b2d138774.slice/crio-1835036539be31494f67e56316771dc9813468f9a4aa9b0ed1e3762f00e7d0c4 WatchSource:0}: Error finding container 1835036539be31494f67e56316771dc9813468f9a4aa9b0ed1e3762f00e7d0c4: Status 404 returned error can't find the container with id 1835036539be31494f67e56316771dc9813468f9a4aa9b0ed1e3762f00e7d0c4
Sep 15 20:16:39.291744 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:39.291691    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7ff79c9c6c-8fcwk" event={"ID":"97664b01-dd26-4176-99d9-6bb0af1f9cd3","Type":"ContainerStarted","Data":"58a029b2e629cfe72557eba4236738e7e70d0da45e47934491e38d5928be4b8c"}
Sep 15 20:16:39.294122 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:39.294083    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b579c7cb4-gg2pv" event={"ID":"3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774","Type":"ContainerStarted","Data":"1835036539be31494f67e56316771dc9813468f9a4aa9b0ed1e3762f00e7d0c4"}
Sep 15 20:16:41.309602 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:41.309495    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7ff79c9c6c-8fcwk" event={"ID":"97664b01-dd26-4176-99d9-6bb0af1f9cd3","Type":"ContainerStarted","Data":"66627bcf62e51c31bf3ad37bc2fa830fa3f56d3ef293b928713e56c2752c36c6"}
Sep 15 20:16:41.309602 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:41.309562    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7ff79c9c6c-8fcwk"
Sep 15 20:16:41.311144 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:41.311115    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b579c7cb4-gg2pv" event={"ID":"3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774","Type":"ContainerStarted","Data":"53ced0205dd9e2362e41e709962695b259120bee5654617822c011ef43a7d905"}
Sep 15 20:16:41.311263 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:41.311230    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5b579c7cb4-gg2pv"
Sep 15 20:16:41.332032 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:41.331983    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7ff79c9c6c-8fcwk" podStartSLOduration=1.7335348210000001 podStartE2EDuration="4.331969012s" totalImagesPullingTime="2.598434191s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:16:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:16:41.329519841 +0000 UTC m=+707.720472833" watchObservedRunningTime="2026-09-15 20:16:41.331969012 +0000 UTC m=+707.722922017"
Sep 15 20:16:41.358261 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:41.358189    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5b579c7cb4-gg2pv" podStartSLOduration=1.9127512169999998 podStartE2EDuration="4.358166103s" totalImagesPullingTime="2.445414886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:16:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:16:41.354356184 +0000 UTC m=+707.745309181" watchObservedRunningTime="2026-09-15 20:16:41.358166103 +0000 UTC m=+707.749119113"
Sep 15 20:16:52.326786 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:52.326698    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5b579c7cb4-gg2pv"
Sep 15 20:16:52.327256 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:52.326804    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7ff79c9c6c-8fcwk"
Sep 15 20:16:52.396286 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:52.396245    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7ff79c9c6c-8fcwk"]
Sep 15 20:16:52.396538 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:52.396506    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7ff79c9c6c-8fcwk" podUID="97664b01-dd26-4176-99d9-6bb0af1f9cd3" containerName="manager" containerID="cri-o://66627bcf62e51c31bf3ad37bc2fa830fa3f56d3ef293b928713e56c2752c36c6" gracePeriod=30
Sep 15 20:16:52.642678 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:52.642651    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7ff79c9c6c-8fcwk"
Sep 15 20:16:52.687000 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:52.686965    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6xhkk\" (UniqueName: \"kubernetes.io/projected/97664b01-dd26-4176-99d9-6bb0af1f9cd3-kube-api-access-6xhkk\") pod \"97664b01-dd26-4176-99d9-6bb0af1f9cd3\" (UID: \"97664b01-dd26-4176-99d9-6bb0af1f9cd3\") "
Sep 15 20:16:52.687000 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:52.687010    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/97664b01-dd26-4176-99d9-6bb0af1f9cd3-webhook-cert\") pod \"97664b01-dd26-4176-99d9-6bb0af1f9cd3\" (UID: \"97664b01-dd26-4176-99d9-6bb0af1f9cd3\") "
Sep 15 20:16:52.687215 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:52.687080    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97664b01-dd26-4176-99d9-6bb0af1f9cd3-metrics-certs\") pod \"97664b01-dd26-4176-99d9-6bb0af1f9cd3\" (UID: \"97664b01-dd26-4176-99d9-6bb0af1f9cd3\") "
Sep 15 20:16:52.689259 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:52.689214    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97664b01-dd26-4176-99d9-6bb0af1f9cd3-kube-api-access-6xhkk" (OuterVolumeSpecName: "kube-api-access-6xhkk") pod "97664b01-dd26-4176-99d9-6bb0af1f9cd3" (UID: "97664b01-dd26-4176-99d9-6bb0af1f9cd3"). InnerVolumeSpecName "kube-api-access-6xhkk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:16:52.689392 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:52.689228    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97664b01-dd26-4176-99d9-6bb0af1f9cd3-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "97664b01-dd26-4176-99d9-6bb0af1f9cd3" (UID: "97664b01-dd26-4176-99d9-6bb0af1f9cd3"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:16:52.689392 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:52.689306    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97664b01-dd26-4176-99d9-6bb0af1f9cd3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "97664b01-dd26-4176-99d9-6bb0af1f9cd3" (UID: "97664b01-dd26-4176-99d9-6bb0af1f9cd3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:16:52.788230 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:52.788179    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xhkk\" (UniqueName: \"kubernetes.io/projected/97664b01-dd26-4176-99d9-6bb0af1f9cd3-kube-api-access-6xhkk\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:16:52.788230 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:52.788219    2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/97664b01-dd26-4176-99d9-6bb0af1f9cd3-webhook-cert\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:16:52.788230 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:52.788232    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97664b01-dd26-4176-99d9-6bb0af1f9cd3-metrics-certs\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:16:53.393656 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:53.393617    2577 generic.go:356] "Generic (PLEG): container finished" podID="97664b01-dd26-4176-99d9-6bb0af1f9cd3" containerID="66627bcf62e51c31bf3ad37bc2fa830fa3f56d3ef293b928713e56c2752c36c6" exitCode=0
Sep 15 20:16:53.394076 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:53.393669    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7ff79c9c6c-8fcwk"
Sep 15 20:16:53.394076 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:53.393687    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7ff79c9c6c-8fcwk" event={"ID":"97664b01-dd26-4176-99d9-6bb0af1f9cd3","Type":"ContainerDied","Data":"66627bcf62e51c31bf3ad37bc2fa830fa3f56d3ef293b928713e56c2752c36c6"}
Sep 15 20:16:53.394076 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:53.393761    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7ff79c9c6c-8fcwk" event={"ID":"97664b01-dd26-4176-99d9-6bb0af1f9cd3","Type":"ContainerDied","Data":"58a029b2e629cfe72557eba4236738e7e70d0da45e47934491e38d5928be4b8c"}
Sep 15 20:16:53.394076 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:53.393801    2577 scope.go:117] "RemoveContainer" containerID="66627bcf62e51c31bf3ad37bc2fa830fa3f56d3ef293b928713e56c2752c36c6"
Sep 15 20:16:53.405183 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:53.405019    2577 scope.go:117] "RemoveContainer" containerID="66627bcf62e51c31bf3ad37bc2fa830fa3f56d3ef293b928713e56c2752c36c6"
Sep 15 20:16:53.405375 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:16:53.405356    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66627bcf62e51c31bf3ad37bc2fa830fa3f56d3ef293b928713e56c2752c36c6\": container with ID starting with 66627bcf62e51c31bf3ad37bc2fa830fa3f56d3ef293b928713e56c2752c36c6 not found: ID does not exist" containerID="66627bcf62e51c31bf3ad37bc2fa830fa3f56d3ef293b928713e56c2752c36c6"
Sep 15 20:16:53.405429 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:53.405384    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66627bcf62e51c31bf3ad37bc2fa830fa3f56d3ef293b928713e56c2752c36c6"} err="failed to get container status \"66627bcf62e51c31bf3ad37bc2fa830fa3f56d3ef293b928713e56c2752c36c6\": rpc error: code = NotFound desc = could not find container \"66627bcf62e51c31bf3ad37bc2fa830fa3f56d3ef293b928713e56c2752c36c6\": container with ID starting with 66627bcf62e51c31bf3ad37bc2fa830fa3f56d3ef293b928713e56c2752c36c6 not found: ID does not exist"
Sep 15 20:16:53.418849 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:53.418819    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7ff79c9c6c-8fcwk"]
Sep 15 20:16:53.424466 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:53.424441    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7ff79c9c6c-8fcwk"]
Sep 15 20:16:54.060477 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:16:54.060442    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97664b01-dd26-4176-99d9-6bb0af1f9cd3" path="/var/lib/kubelet/pods/97664b01-dd26-4176-99d9-6bb0af1f9cd3/volumes"
Sep 15 20:17:12.938270 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:12.938233    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7d5d7c6468-hhdcz"]
Sep 15 20:17:12.938740 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:12.938648    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97664b01-dd26-4176-99d9-6bb0af1f9cd3" containerName="manager"
Sep 15 20:17:12.938740 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:12.938660    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="97664b01-dd26-4176-99d9-6bb0af1f9cd3" containerName="manager"
Sep 15 20:17:12.938824 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:12.938742    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="97664b01-dd26-4176-99d9-6bb0af1f9cd3" containerName="manager"
Sep 15 20:17:12.943321 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:12.943301    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7d5d7c6468-hhdcz"
Sep 15 20:17:12.947099 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:12.947075    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 15 20:17:12.947212 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:12.947108    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 15 20:17:12.947212 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:12.947110    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-76rfp\""
Sep 15 20:17:12.956704 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:12.956678    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7d5d7c6468-hhdcz"]
Sep 15 20:17:13.066693 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:13.066650    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be7ccdf9-7d15-4f0a-9c68-32a90c5031f1-openshift-service-ca\") pod \"maas-api-7d5d7c6468-hhdcz\" (UID: \"be7ccdf9-7d15-4f0a-9c68-32a90c5031f1\") " pod="odh-ai-gateway-infra/maas-api-7d5d7c6468-hhdcz"
Sep 15 20:17:13.066693 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:13.066697    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be7ccdf9-7d15-4f0a-9c68-32a90c5031f1-maas-api-tls\") pod \"maas-api-7d5d7c6468-hhdcz\" (UID: \"be7ccdf9-7d15-4f0a-9c68-32a90c5031f1\") " pod="odh-ai-gateway-infra/maas-api-7d5d7c6468-hhdcz"
Sep 15 20:17:13.066979 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:13.066779    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be7ccdf9-7d15-4f0a-9c68-32a90c5031f1-metrics-certs\") pod \"maas-api-7d5d7c6468-hhdcz\" (UID: \"be7ccdf9-7d15-4f0a-9c68-32a90c5031f1\") " pod="odh-ai-gateway-infra/maas-api-7d5d7c6468-hhdcz"
Sep 15 20:17:13.066979 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:13.066879    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftqc2\" (UniqueName: \"kubernetes.io/projected/be7ccdf9-7d15-4f0a-9c68-32a90c5031f1-kube-api-access-ftqc2\") pod \"maas-api-7d5d7c6468-hhdcz\" (UID: \"be7ccdf9-7d15-4f0a-9c68-32a90c5031f1\") " pod="odh-ai-gateway-infra/maas-api-7d5d7c6468-hhdcz"
Sep 15 20:17:13.167578 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:13.167542    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be7ccdf9-7d15-4f0a-9c68-32a90c5031f1-openshift-service-ca\") pod \"maas-api-7d5d7c6468-hhdcz\" (UID: \"be7ccdf9-7d15-4f0a-9c68-32a90c5031f1\") " pod="odh-ai-gateway-infra/maas-api-7d5d7c6468-hhdcz"
Sep 15 20:17:13.167803 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:13.167589    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be7ccdf9-7d15-4f0a-9c68-32a90c5031f1-maas-api-tls\") pod \"maas-api-7d5d7c6468-hhdcz\" (UID: \"be7ccdf9-7d15-4f0a-9c68-32a90c5031f1\") " pod="odh-ai-gateway-infra/maas-api-7d5d7c6468-hhdcz"
Sep 15 20:17:13.167803 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:13.167631    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be7ccdf9-7d15-4f0a-9c68-32a90c5031f1-metrics-certs\") pod \"maas-api-7d5d7c6468-hhdcz\" (UID: \"be7ccdf9-7d15-4f0a-9c68-32a90c5031f1\") " pod="odh-ai-gateway-infra/maas-api-7d5d7c6468-hhdcz"
Sep 15 20:17:13.167803 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:13.167706    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftqc2\" (UniqueName: \"kubernetes.io/projected/be7ccdf9-7d15-4f0a-9c68-32a90c5031f1-kube-api-access-ftqc2\") pod \"maas-api-7d5d7c6468-hhdcz\" (UID: \"be7ccdf9-7d15-4f0a-9c68-32a90c5031f1\") " pod="odh-ai-gateway-infra/maas-api-7d5d7c6468-hhdcz"
Sep 15 20:17:13.168259 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:13.168228    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be7ccdf9-7d15-4f0a-9c68-32a90c5031f1-openshift-service-ca\") pod \"maas-api-7d5d7c6468-hhdcz\" (UID: \"be7ccdf9-7d15-4f0a-9c68-32a90c5031f1\") " pod="odh-ai-gateway-infra/maas-api-7d5d7c6468-hhdcz"
Sep 15 20:17:13.170305 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:13.170264    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be7ccdf9-7d15-4f0a-9c68-32a90c5031f1-maas-api-tls\") pod \"maas-api-7d5d7c6468-hhdcz\" (UID: \"be7ccdf9-7d15-4f0a-9c68-32a90c5031f1\") " pod="odh-ai-gateway-infra/maas-api-7d5d7c6468-hhdcz"
Sep 15 20:17:13.170425 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:13.170320    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be7ccdf9-7d15-4f0a-9c68-32a90c5031f1-metrics-certs\") pod \"maas-api-7d5d7c6468-hhdcz\" (UID: \"be7ccdf9-7d15-4f0a-9c68-32a90c5031f1\") " pod="odh-ai-gateway-infra/maas-api-7d5d7c6468-hhdcz"
Sep 15 20:17:13.176089 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:13.176056    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftqc2\" (UniqueName: \"kubernetes.io/projected/be7ccdf9-7d15-4f0a-9c68-32a90c5031f1-kube-api-access-ftqc2\") pod \"maas-api-7d5d7c6468-hhdcz\" (UID: \"be7ccdf9-7d15-4f0a-9c68-32a90c5031f1\") " pod="odh-ai-gateway-infra/maas-api-7d5d7c6468-hhdcz"
Sep 15 20:17:13.254519 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:13.254483    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7d5d7c6468-hhdcz"
Sep 15 20:17:13.536849 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:13.536818    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7d5d7c6468-hhdcz"]
Sep 15 20:17:13.539137 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:17:13.539103    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe7ccdf9_7d15_4f0a_9c68_32a90c5031f1.slice/crio-60ed349f8a4e1404b6ce70aab5fb145c96136a2a3b522be0c25baec35062c17e WatchSource:0}: Error finding container 60ed349f8a4e1404b6ce70aab5fb145c96136a2a3b522be0c25baec35062c17e: Status 404 returned error can't find the container with id 60ed349f8a4e1404b6ce70aab5fb145c96136a2a3b522be0c25baec35062c17e
Sep 15 20:17:13.637885 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:13.637847    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-69vj4"]
Sep 15 20:17:13.643147 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:13.643121    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-69vj4"
Sep 15 20:17:13.646036 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:13.646002    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 15 20:17:13.646176 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:13.646003    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-hjmnc\""
Sep 15 20:17:13.654106 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:13.654080    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-69vj4"]
Sep 15 20:17:13.774303 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:13.774266    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2mwv\" (UniqueName: \"kubernetes.io/projected/8b20137b-9766-427f-b140-5c75d6f77f23-kube-api-access-n2mwv\") pod \"payload-processing-5b54b7fbb7-69vj4\" (UID: \"8b20137b-9766-427f-b140-5c75d6f77f23\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-69vj4"
Sep 15 20:17:13.774525 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:13.774337    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b20137b-9766-427f-b140-5c75d6f77f23-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-69vj4\" (UID: \"8b20137b-9766-427f-b140-5c75d6f77f23\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-69vj4"
Sep 15 20:17:13.875094 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:13.874998    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2mwv\" (UniqueName: \"kubernetes.io/projected/8b20137b-9766-427f-b140-5c75d6f77f23-kube-api-access-n2mwv\") pod \"payload-processing-5b54b7fbb7-69vj4\" (UID: \"8b20137b-9766-427f-b140-5c75d6f77f23\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-69vj4"
Sep 15 20:17:13.875242 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:13.875097    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b20137b-9766-427f-b140-5c75d6f77f23-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-69vj4\" (UID: \"8b20137b-9766-427f-b140-5c75d6f77f23\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-69vj4"
Sep 15 20:17:13.875853 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:13.875831    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b20137b-9766-427f-b140-5c75d6f77f23-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-69vj4\" (UID: \"8b20137b-9766-427f-b140-5c75d6f77f23\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-69vj4"
Sep 15 20:17:13.883837 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:13.883809    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2mwv\" (UniqueName: \"kubernetes.io/projected/8b20137b-9766-427f-b140-5c75d6f77f23-kube-api-access-n2mwv\") pod \"payload-processing-5b54b7fbb7-69vj4\" (UID: \"8b20137b-9766-427f-b140-5c75d6f77f23\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-69vj4"
Sep 15 20:17:13.955737 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:13.955683    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-69vj4"
Sep 15 20:17:14.236410 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:14.236377    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-69vj4"]
Sep 15 20:17:14.237807 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:17:14.237772    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b20137b_9766_427f_b140_5c75d6f77f23.slice/crio-39f4a00aec85558b273cdccf3baeacaae62886315d8def481a84b85ef713a31a WatchSource:0}: Error finding container 39f4a00aec85558b273cdccf3baeacaae62886315d8def481a84b85ef713a31a: Status 404 returned error can't find the container with id 39f4a00aec85558b273cdccf3baeacaae62886315d8def481a84b85ef713a31a
Sep 15 20:17:14.533793 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:14.533680    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7d5d7c6468-hhdcz" event={"ID":"be7ccdf9-7d15-4f0a-9c68-32a90c5031f1","Type":"ContainerStarted","Data":"60ed349f8a4e1404b6ce70aab5fb145c96136a2a3b522be0c25baec35062c17e"}
Sep 15 20:17:14.535408 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:14.535378    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-69vj4" event={"ID":"8b20137b-9766-427f-b140-5c75d6f77f23","Type":"ContainerStarted","Data":"39f4a00aec85558b273cdccf3baeacaae62886315d8def481a84b85ef713a31a"}
Sep 15 20:17:16.549721 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:16.549685    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7d5d7c6468-hhdcz" event={"ID":"be7ccdf9-7d15-4f0a-9c68-32a90c5031f1","Type":"ContainerStarted","Data":"b5e07feefe1a585a3f490ac73fe77aab35d28beac880c4501969318d08fa24eb"}
Sep 15 20:17:16.550181 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:16.549908    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7d5d7c6468-hhdcz"
Sep 15 20:17:16.551276 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:16.551243    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-69vj4" event={"ID":"8b20137b-9766-427f-b140-5c75d6f77f23","Type":"ContainerStarted","Data":"baa0228a0d28ceee73e6d42e34f5d37493e5f1ec7497e6e8706a9683e9010a7a"}
Sep 15 20:17:16.551397 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:16.551293    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-69vj4"
Sep 15 20:17:16.569121 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:16.569071    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7d5d7c6468-hhdcz" podStartSLOduration=2.347780656 podStartE2EDuration="4.569057267s" totalImagesPullingTime="2.221276611s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:17:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:17:16.567574494 +0000 UTC m=+742.958527492" watchObservedRunningTime="2026-09-15 20:17:16.569057267 +0000 UTC m=+742.960010263"
Sep 15 20:17:16.584366 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:16.584316    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-69vj4" podStartSLOduration=2.060973787 podStartE2EDuration="3.584299626s" totalImagesPullingTime="1.523325839s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:17:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:17:16.583164728 +0000 UTC m=+742.974117725" watchObservedRunningTime="2026-09-15 20:17:16.584299626 +0000 UTC m=+742.975252622"
Sep 15 20:17:22.563669 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:22.563639    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7d5d7c6468-hhdcz"
Sep 15 20:17:27.560616 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:27.560573    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-69vj4"
Sep 15 20:17:57.928169 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:57.928133    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-45wkt"]
Sep 15 20:17:57.931394 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:57.931364    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-45wkt"
Sep 15 20:17:57.935344 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:57.935190    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-mtfkw\""
Sep 15 20:17:57.935344 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:57.935209    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 15 20:17:57.935344 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:57.935236    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 15 20:17:57.935344 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:57.935190    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 15 20:17:57.944416 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:57.944390    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-45wkt"]
Sep 15 20:17:57.989769 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:57.989708    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ea3f838-a397-4de2-b56a-200b482a72c3-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-45wkt\" (UID: \"2ea3f838-a397-4de2-b56a-200b482a72c3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-45wkt"
Sep 15 20:17:57.989939 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:57.989815    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ea3f838-a397-4de2-b56a-200b482a72c3-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-45wkt\" (UID: \"2ea3f838-a397-4de2-b56a-200b482a72c3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-45wkt"
Sep 15 20:17:57.989939 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:57.989857    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ea3f838-a397-4de2-b56a-200b482a72c3-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-45wkt\" (UID: \"2ea3f838-a397-4de2-b56a-200b482a72c3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-45wkt"
Sep 15 20:17:57.989939 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:57.989884    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ds8x8\" (UniqueName: \"kubernetes.io/projected/2ea3f838-a397-4de2-b56a-200b482a72c3-kube-api-access-ds8x8\") pod \"e2e-distinct-simulated-kserve-59975d69f9-45wkt\" (UID: \"2ea3f838-a397-4de2-b56a-200b482a72c3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-45wkt"
Sep 15 20:17:57.990045 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:57.989955    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ea3f838-a397-4de2-b56a-200b482a72c3-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-45wkt\" (UID: \"2ea3f838-a397-4de2-b56a-200b482a72c3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-45wkt"
Sep 15 20:17:57.990045 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:57.989996    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ea3f838-a397-4de2-b56a-200b482a72c3-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-45wkt\" (UID: \"2ea3f838-a397-4de2-b56a-200b482a72c3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-45wkt"
Sep 15 20:17:58.090500 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:58.090458    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ea3f838-a397-4de2-b56a-200b482a72c3-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-45wkt\" (UID: \"2ea3f838-a397-4de2-b56a-200b482a72c3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-45wkt"
Sep 15 20:17:58.090500 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:58.090502    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ds8x8\" (UniqueName: \"kubernetes.io/projected/2ea3f838-a397-4de2-b56a-200b482a72c3-kube-api-access-ds8x8\") pod \"e2e-distinct-simulated-kserve-59975d69f9-45wkt\" (UID: \"2ea3f838-a397-4de2-b56a-200b482a72c3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-45wkt"
Sep 15 20:17:58.090861 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:58.090542    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ea3f838-a397-4de2-b56a-200b482a72c3-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-45wkt\" (UID: \"2ea3f838-a397-4de2-b56a-200b482a72c3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-45wkt"
Sep 15 20:17:58.090861 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:58.090567    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ea3f838-a397-4de2-b56a-200b482a72c3-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-45wkt\" (UID: \"2ea3f838-a397-4de2-b56a-200b482a72c3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-45wkt"
Sep 15 20:17:58.090861 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:58.090614    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ea3f838-a397-4de2-b56a-200b482a72c3-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-45wkt\" (UID: \"2ea3f838-a397-4de2-b56a-200b482a72c3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-45wkt"
Sep 15 20:17:58.090861 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:58.090784    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ea3f838-a397-4de2-b56a-200b482a72c3-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-45wkt\" (UID: \"2ea3f838-a397-4de2-b56a-200b482a72c3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-45wkt"
Sep 15 20:17:58.091089 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:58.091031    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ea3f838-a397-4de2-b56a-200b482a72c3-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-45wkt\" (UID: \"2ea3f838-a397-4de2-b56a-200b482a72c3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-45wkt"
Sep 15 20:17:58.091089 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:58.091042    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ea3f838-a397-4de2-b56a-200b482a72c3-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-45wkt\" (UID: \"2ea3f838-a397-4de2-b56a-200b482a72c3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-45wkt"
Sep 15 20:17:58.091177 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:58.091083    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ea3f838-a397-4de2-b56a-200b482a72c3-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-45wkt\" (UID: \"2ea3f838-a397-4de2-b56a-200b482a72c3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-45wkt"
Sep 15 20:17:58.093119 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:58.093090    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ea3f838-a397-4de2-b56a-200b482a72c3-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-45wkt\" (UID: \"2ea3f838-a397-4de2-b56a-200b482a72c3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-45wkt"
Sep 15 20:17:58.093259 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:58.093241    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ea3f838-a397-4de2-b56a-200b482a72c3-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-45wkt\" (UID: \"2ea3f838-a397-4de2-b56a-200b482a72c3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-45wkt"
Sep 15 20:17:58.098487 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:58.098461    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ds8x8\" (UniqueName: \"kubernetes.io/projected/2ea3f838-a397-4de2-b56a-200b482a72c3-kube-api-access-ds8x8\") pod \"e2e-distinct-simulated-kserve-59975d69f9-45wkt\" (UID: \"2ea3f838-a397-4de2-b56a-200b482a72c3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-45wkt"
Sep 15 20:17:58.242675 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:58.242629    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-45wkt"
Sep 15 20:17:58.524068 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:58.524038    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-45wkt"]
Sep 15 20:17:58.524946 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:17:58.524914    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ea3f838_a397_4de2_b56a_200b482a72c3.slice/crio-f6e013f7967204cf985bb34e5e9515e7b76ef19edf73ed6cfa2ad77172fd3757 WatchSource:0}: Error finding container f6e013f7967204cf985bb34e5e9515e7b76ef19edf73ed6cfa2ad77172fd3757: Status 404 returned error can't find the container with id f6e013f7967204cf985bb34e5e9515e7b76ef19edf73ed6cfa2ad77172fd3757
Sep 15 20:17:58.805927 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:58.805837    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-45wkt" event={"ID":"2ea3f838-a397-4de2-b56a-200b482a72c3","Type":"ContainerStarted","Data":"f6e013f7967204cf985bb34e5e9515e7b76ef19edf73ed6cfa2ad77172fd3757"}
Sep 15 20:17:59.816519 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:59.816424    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-45wkt" event={"ID":"2ea3f838-a397-4de2-b56a-200b482a72c3","Type":"ContainerStarted","Data":"3c26063945889bc4c93dfdfb9efd271f0a17103a3fa7b6c3946eec7878f897a1"}
Sep 15 20:17:59.836091 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:17:59.836036    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-45wkt" podStartSLOduration=1.898161808 podStartE2EDuration="2.836017807s" totalImagesPullingTime="937.855999ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:17:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:17:59.833378887 +0000 UTC m=+786.224331885" watchObservedRunningTime="2026-09-15 20:17:59.836017807 +0000 UTC m=+786.226970803"
Sep 15 20:18:08.242839 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:18:08.242794    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-45wkt"
Sep 15 20:18:08.243245 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:18:08.242863    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-45wkt"
Sep 15 20:18:08.255432 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:18:08.255401    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-45wkt"
Sep 15 20:18:08.880079 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:18:08.880050    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-45wkt"
Sep 15 20:19:54.083492 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:19:54.083458    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-r227b_ecab5774-79a4-42eb-9bb5-a1f09b18f3fb/console-operator/1.log"
Sep 15 20:19:54.084599 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:19:54.084578    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-r227b_ecab5774-79a4-42eb-9bb5-a1f09b18f3fb/console-operator/1.log"
Sep 15 20:21:20.495633 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:20.495546    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"]
Sep 15 20:21:20.499689 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:20.499658    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:21:20.502465 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:20.502441    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-725e67-openshift-default-dockercfg-tgg58\""
Sep 15 20:21:20.525587 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:20.525552    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"]
Sep 15 20:21:20.601211 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:20.601169    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wd292\" (UniqueName: \"kubernetes.io/projected/fa4b30cd-828e-4404-ab50-12cc8cf31f42-kube-api-access-wd292\") pod \"e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg\" (UID: \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\") " pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:21:20.601211 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:20.601209    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa4b30cd-828e-4404-ab50-12cc8cf31f42-credential-socket\") pod \"e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg\" (UID: \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\") " pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:21:20.601421 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:20.601239    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa4b30cd-828e-4404-ab50-12cc8cf31f42-istio-token\") pod \"e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg\" (UID: \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\") " pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:21:20.601421 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:20.601317    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa4b30cd-828e-4404-ab50-12cc8cf31f42-workload-certs\") pod \"e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg\" (UID: \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\") " pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:21:20.601421 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:20.601373    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa4b30cd-828e-4404-ab50-12cc8cf31f42-istio-podinfo\") pod \"e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg\" (UID: \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\") " pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:21:20.601525 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:20.601424    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa4b30cd-828e-4404-ab50-12cc8cf31f42-istio-data\") pod \"e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg\" (UID: \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\") " pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:21:20.601525 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:20.601452    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa4b30cd-828e-4404-ab50-12cc8cf31f42-istio-envoy\") pod \"e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg\" (UID: \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\") " pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:21:20.601593 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:20.601518    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa4b30cd-828e-4404-ab50-12cc8cf31f42-workload-socket\") pod \"e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg\" (UID: \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\") " pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:21:20.601593 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:20.601556    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa4b30cd-828e-4404-ab50-12cc8cf31f42-istiod-ca-cert\") pod \"e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg\" (UID: \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\") " pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:21:20.702451 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:20.702418    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa4b30cd-828e-4404-ab50-12cc8cf31f42-istio-podinfo\") pod \"e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg\" (UID: \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\") " pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:21:20.702629 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:20.702466    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa4b30cd-828e-4404-ab50-12cc8cf31f42-istio-data\") pod \"e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg\" (UID: \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\") " pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:21:20.702629 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:20.702488    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa4b30cd-828e-4404-ab50-12cc8cf31f42-istio-envoy\") pod \"e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg\" (UID: \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\") " pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:21:20.702629 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:20.702518    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa4b30cd-828e-4404-ab50-12cc8cf31f42-workload-socket\") pod \"e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg\" (UID: \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\") " pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:21:20.702629 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:20.702538    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa4b30cd-828e-4404-ab50-12cc8cf31f42-istiod-ca-cert\") pod \"e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg\" (UID: \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\") " pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:21:20.702629 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:20.702563    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd292\" (UniqueName: \"kubernetes.io/projected/fa4b30cd-828e-4404-ab50-12cc8cf31f42-kube-api-access-wd292\") pod \"e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg\" (UID: \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\") " pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:21:20.702629 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:20.702579    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa4b30cd-828e-4404-ab50-12cc8cf31f42-credential-socket\") pod \"e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg\" (UID: \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\") " pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:21:20.702629 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:20.702618    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa4b30cd-828e-4404-ab50-12cc8cf31f42-istio-token\") pod \"e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg\" (UID: \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\") " pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:21:20.703022 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:20.702912    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa4b30cd-828e-4404-ab50-12cc8cf31f42-istio-data\") pod \"e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg\" (UID: \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\") " pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:21:20.703022 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:20.702940    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa4b30cd-828e-4404-ab50-12cc8cf31f42-workload-socket\") pod \"e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg\" (UID: \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\") " pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:21:20.703022 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:20.702989    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa4b30cd-828e-4404-ab50-12cc8cf31f42-workload-certs\") pod \"e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg\" (UID: \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\") " pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:21:20.703182 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:20.703051    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa4b30cd-828e-4404-ab50-12cc8cf31f42-credential-socket\") pod \"e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg\" (UID: \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\") " pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:21:20.703395 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:20.703370    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa4b30cd-828e-4404-ab50-12cc8cf31f42-workload-certs\") pod \"e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg\" (UID: \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\") " pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:21:20.703669 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:20.703652    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa4b30cd-828e-4404-ab50-12cc8cf31f42-istiod-ca-cert\") pod \"e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg\" (UID: \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\") " pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:21:20.705103 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:20.705081    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa4b30cd-828e-4404-ab50-12cc8cf31f42-istio-envoy\") pod \"e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg\" (UID: \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\") " pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:21:20.705231 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:20.705214    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa4b30cd-828e-4404-ab50-12cc8cf31f42-istio-podinfo\") pod \"e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg\" (UID: \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\") " pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:21:20.711119 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:20.711090    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa4b30cd-828e-4404-ab50-12cc8cf31f42-istio-token\") pod \"e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg\" (UID: \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\") " pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:21:20.711229 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:20.711209    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wd292\" (UniqueName: \"kubernetes.io/projected/fa4b30cd-828e-4404-ab50-12cc8cf31f42-kube-api-access-wd292\") pod \"e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg\" (UID: \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\") " pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:21:20.812162 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:20.812057    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:21:21.095058 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:21.095026    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"]
Sep 15 20:21:21.096629 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:21:21.096597    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa4b30cd_828e_4404_ab50_12cc8cf31f42.slice/crio-2b2fc66ced5b5d204fb4984a687db91d657e5979fa76cd2afe9b6c721f8ec6b0 WatchSource:0}: Error finding container 2b2fc66ced5b5d204fb4984a687db91d657e5979fa76cd2afe9b6c721f8ec6b0: Status 404 returned error can't find the container with id 2b2fc66ced5b5d204fb4984a687db91d657e5979fa76cd2afe9b6c721f8ec6b0
Sep 15 20:21:21.098975 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:21.098936    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 20:21:21.099079 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:21.099003    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 20:21:21.099079 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:21.099035    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 20:21:22.041162 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:22.041110    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg" event={"ID":"fa4b30cd-828e-4404-ab50-12cc8cf31f42","Type":"ContainerStarted","Data":"15dde39be4fd6ef0b788d49d9afacb369a871c0aeb4c1123664ae1ec4de80353"}
Sep 15 20:21:22.041544 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:22.041177    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg" event={"ID":"fa4b30cd-828e-4404-ab50-12cc8cf31f42","Type":"ContainerStarted","Data":"2b2fc66ced5b5d204fb4984a687db91d657e5979fa76cd2afe9b6c721f8ec6b0"}
Sep 15 20:21:22.063995 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:22.063940    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg" podStartSLOduration=2.063925061 podStartE2EDuration="2.063925061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:21:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:21:22.062282254 +0000 UTC m=+988.453235253" watchObservedRunningTime="2026-09-15 20:21:22.063925061 +0000 UTC m=+988.454878058"
Sep 15 20:21:22.240031 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:22.239995    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7975-dcb00a"]
Sep 15 20:21:22.243625 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:22.243606    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7975-dcb00a"
Sep 15 20:21:22.246497 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:22.246470    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tppb5\""
Sep 15 20:21:22.250048 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:22.250023    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7975-dcb00a"]
Sep 15 20:21:22.319921 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:22.319821    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cssxn\" (UniqueName: \"kubernetes.io/projected/4e561a6d-19ac-497c-9506-4f904d0f3760-kube-api-access-cssxn\") pod \"test-curl-7975-dcb00a\" (UID: \"4e561a6d-19ac-497c-9506-4f904d0f3760\") " pod="opendatahub/test-curl-7975-dcb00a"
Sep 15 20:21:22.421228 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:22.421178    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cssxn\" (UniqueName: \"kubernetes.io/projected/4e561a6d-19ac-497c-9506-4f904d0f3760-kube-api-access-cssxn\") pod \"test-curl-7975-dcb00a\" (UID: \"4e561a6d-19ac-497c-9506-4f904d0f3760\") " pod="opendatahub/test-curl-7975-dcb00a"
Sep 15 20:21:22.429933 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:22.429897    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cssxn\" (UniqueName: \"kubernetes.io/projected/4e561a6d-19ac-497c-9506-4f904d0f3760-kube-api-access-cssxn\") pod \"test-curl-7975-dcb00a\" (UID: \"4e561a6d-19ac-497c-9506-4f904d0f3760\") " pod="opendatahub/test-curl-7975-dcb00a"
Sep 15 20:21:22.554662 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:22.554620    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7975-dcb00a"
Sep 15 20:21:22.812520 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:22.812483    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:21:22.818183 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:22.818154    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:21:22.835180 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:22.835149    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7975-dcb00a"]
Sep 15 20:21:22.836909 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:21:22.836881    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e561a6d_19ac_497c_9506_4f904d0f3760.slice/crio-63b55757414ee1c4091e49b509bbdbe9e11bbb32821d0d8f580b4545a2c32e5e WatchSource:0}: Error finding container 63b55757414ee1c4091e49b509bbdbe9e11bbb32821d0d8f580b4545a2c32e5e: Status 404 returned error can't find the container with id 63b55757414ee1c4091e49b509bbdbe9e11bbb32821d0d8f580b4545a2c32e5e
Sep 15 20:21:22.839373 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:22.839342    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 20:21:23.048212 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:23.048172    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7975-dcb00a" event={"ID":"4e561a6d-19ac-497c-9506-4f904d0f3760","Type":"ContainerStarted","Data":"63b55757414ee1c4091e49b509bbdbe9e11bbb32821d0d8f580b4545a2c32e5e"}
Sep 15 20:21:23.048639 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:23.048359    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:21:23.049454 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:23.049438    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:21:24.061227 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:24.061187    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7975-dcb00a" event={"ID":"4e561a6d-19ac-497c-9506-4f904d0f3760","Type":"ContainerStarted","Data":"e42239b2894146a2100934b529c0d847956878bfc97285663eb4f2ace0573a57"}
Sep 15 20:21:24.075204 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:24.075134    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7975-dcb00a" podStartSLOduration=1.539630453 podStartE2EDuration="2.075115222s" totalImagesPullingTime="535.484769ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:21:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:21:24.072752388 +0000 UTC m=+990.463705382" watchObservedRunningTime="2026-09-15 20:21:24.075115222 +0000 UTC m=+990.466068232"
Sep 15 20:21:24.680633 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:24.680595    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7975-dcb00a"]
Sep 15 20:21:24.695042 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:24.695014    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7975-dcb00a"]
Sep 15 20:21:24.822771 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:24.822664    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7975-2bd2ce"]
Sep 15 20:21:24.823206 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:24.823187    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e561a6d-19ac-497c-9506-4f904d0f3760" containerName="test-curl-7975-dcb00a"
Sep 15 20:21:24.823206 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:24.823205    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e561a6d-19ac-497c-9506-4f904d0f3760" containerName="test-curl-7975-dcb00a"
Sep 15 20:21:24.823327 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:24.823269    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e561a6d-19ac-497c-9506-4f904d0f3760" containerName="test-curl-7975-dcb00a"
Sep 15 20:21:24.826284 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:24.826267    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7975-2bd2ce"
Sep 15 20:21:24.834911 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:24.834885    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7975-2bd2ce"]
Sep 15 20:21:24.948349 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:24.948310    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtpxc\" (UniqueName: \"kubernetes.io/projected/b3e8b2ae-f862-4033-af64-2a88c6cc7188-kube-api-access-gtpxc\") pod \"test-curl-7975-2bd2ce\" (UID: \"b3e8b2ae-f862-4033-af64-2a88c6cc7188\") " pod="opendatahub/test-curl-7975-2bd2ce"
Sep 15 20:21:25.045247 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:25.045205    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86"]
Sep 15 20:21:25.049227 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:25.049194    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtpxc\" (UniqueName: \"kubernetes.io/projected/b3e8b2ae-f862-4033-af64-2a88c6cc7188-kube-api-access-gtpxc\") pod \"test-curl-7975-2bd2ce\" (UID: \"b3e8b2ae-f862-4033-af64-2a88c6cc7188\") " pod="opendatahub/test-curl-7975-2bd2ce"
Sep 15 20:21:25.049791 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:25.049771    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86"
Sep 15 20:21:25.052224 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:25.052201    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-7e3543-serving-cert\""
Sep 15 20:21:25.061839 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:25.061676    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86"]
Sep 15 20:21:25.064539 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:25.064513    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtpxc\" (UniqueName: \"kubernetes.io/projected/b3e8b2ae-f862-4033-af64-2a88c6cc7188-kube-api-access-gtpxc\") pod \"test-curl-7975-2bd2ce\" (UID: \"b3e8b2ae-f862-4033-af64-2a88c6cc7188\") " pod="opendatahub/test-curl-7975-2bd2ce"
Sep 15 20:21:25.065941 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:21:25.065403    2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7975-dcb00a" container="test-curl-7975-dcb00a"
Sep 15 20:21:25.138089 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:25.137995    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7975-2bd2ce"
Sep 15 20:21:25.150621 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:25.150590    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64cbef4e-b625-44f3-8acd-5b1ca3d4d154-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86\" (UID: \"64cbef4e-b625-44f3-8acd-5b1ca3d4d154\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86"
Sep 15 20:21:25.150832 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:25.150805    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64cbef4e-b625-44f3-8acd-5b1ca3d4d154-maas-api-tls\") pod \"maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86\" (UID: \"64cbef4e-b625-44f3-8acd-5b1ca3d4d154\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86"
Sep 15 20:21:25.150974 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:25.150954    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tfjz\" (UniqueName: \"kubernetes.io/projected/64cbef4e-b625-44f3-8acd-5b1ca3d4d154-kube-api-access-8tfjz\") pod \"maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86\" (UID: \"64cbef4e-b625-44f3-8acd-5b1ca3d4d154\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86"
Sep 15 20:21:25.151095 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:25.151075    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64cbef4e-b625-44f3-8acd-5b1ca3d4d154-metrics-certs\") pod \"maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86\" (UID: \"64cbef4e-b625-44f3-8acd-5b1ca3d4d154\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86"
Sep 15 20:21:25.251971 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:25.251932    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tfjz\" (UniqueName: \"kubernetes.io/projected/64cbef4e-b625-44f3-8acd-5b1ca3d4d154-kube-api-access-8tfjz\") pod \"maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86\" (UID: \"64cbef4e-b625-44f3-8acd-5b1ca3d4d154\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86"
Sep 15 20:21:25.252142 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:25.251990    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64cbef4e-b625-44f3-8acd-5b1ca3d4d154-metrics-certs\") pod \"maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86\" (UID: \"64cbef4e-b625-44f3-8acd-5b1ca3d4d154\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86"
Sep 15 20:21:25.252142 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:25.252027    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64cbef4e-b625-44f3-8acd-5b1ca3d4d154-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86\" (UID: \"64cbef4e-b625-44f3-8acd-5b1ca3d4d154\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86"
Sep 15 20:21:25.252142 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:25.252086    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64cbef4e-b625-44f3-8acd-5b1ca3d4d154-maas-api-tls\") pod \"maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86\" (UID: \"64cbef4e-b625-44f3-8acd-5b1ca3d4d154\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86"
Sep 15 20:21:25.252274 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:21:25.252183    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-7e3543-serving-cert: secret "maas-api-e2e-worker-w0-7e3543-serving-cert" not found
Sep 15 20:21:25.252274 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:21:25.252252    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64cbef4e-b625-44f3-8acd-5b1ca3d4d154-maas-api-tls podName:64cbef4e-b625-44f3-8acd-5b1ca3d4d154 nodeName:}" failed. No retries permitted until 2026-09-15 20:21:25.752232204 +0000 UTC m=+992.143185180 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/64cbef4e-b625-44f3-8acd-5b1ca3d4d154-maas-api-tls") pod "maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86" (UID: "64cbef4e-b625-44f3-8acd-5b1ca3d4d154") : secret "maas-api-e2e-worker-w0-7e3543-serving-cert" not found
Sep 15 20:21:25.252733 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:25.252702    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64cbef4e-b625-44f3-8acd-5b1ca3d4d154-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86\" (UID: \"64cbef4e-b625-44f3-8acd-5b1ca3d4d154\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86"
Sep 15 20:21:25.254586 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:25.254565    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64cbef4e-b625-44f3-8acd-5b1ca3d4d154-metrics-certs\") pod \"maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86\" (UID: \"64cbef4e-b625-44f3-8acd-5b1ca3d4d154\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86"
Sep 15 20:21:25.263947 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:25.263925    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tfjz\" (UniqueName: \"kubernetes.io/projected/64cbef4e-b625-44f3-8acd-5b1ca3d4d154-kube-api-access-8tfjz\") pod \"maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86\" (UID: \"64cbef4e-b625-44f3-8acd-5b1ca3d4d154\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86"
Sep 15 20:21:25.422513 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:25.422487    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7975-2bd2ce"]
Sep 15 20:21:25.423700 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:21:25.423672    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb3e8b2ae_f862_4033_af64_2a88c6cc7188.slice/crio-f2ae9c1d304fe1b6bf67fa260e54ec49705d21bb48cc5e402d2e029777bbcb06 WatchSource:0}: Error finding container f2ae9c1d304fe1b6bf67fa260e54ec49705d21bb48cc5e402d2e029777bbcb06: Status 404 returned error can't find the container with id f2ae9c1d304fe1b6bf67fa260e54ec49705d21bb48cc5e402d2e029777bbcb06
Sep 15 20:21:25.757284 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:25.757243    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64cbef4e-b625-44f3-8acd-5b1ca3d4d154-maas-api-tls\") pod \"maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86\" (UID: \"64cbef4e-b625-44f3-8acd-5b1ca3d4d154\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86"
Sep 15 20:21:25.759673 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:25.759652    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64cbef4e-b625-44f3-8acd-5b1ca3d4d154-maas-api-tls\") pod \"maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86\" (UID: \"64cbef4e-b625-44f3-8acd-5b1ca3d4d154\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86"
Sep 15 20:21:25.964800 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:25.964763    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86"
Sep 15 20:21:26.077438 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:26.076088    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7975-dcb00a" podUID="4e561a6d-19ac-497c-9506-4f904d0f3760" containerName="test-curl-7975-dcb00a" containerID="cri-o://e42239b2894146a2100934b529c0d847956878bfc97285663eb4f2ace0573a57" gracePeriod=2
Sep 15 20:21:26.077438 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:26.076888    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7975-2bd2ce" event={"ID":"b3e8b2ae-f862-4033-af64-2a88c6cc7188","Type":"ContainerStarted","Data":"ae21eb475bdc2226a1a60e38bb8798f14dfd7fd43bf523f36ec9790f1e1179fc"}
Sep 15 20:21:26.077438 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:26.076929    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7975-2bd2ce" event={"ID":"b3e8b2ae-f862-4033-af64-2a88c6cc7188","Type":"ContainerStarted","Data":"f2ae9c1d304fe1b6bf67fa260e54ec49705d21bb48cc5e402d2e029777bbcb06"}
Sep 15 20:21:26.093116 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:26.093049    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7975-2bd2ce" podStartSLOduration=2.093027753 podStartE2EDuration="2.093027753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:21:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:21:26.091661401 +0000 UTC m=+992.482614398" watchObservedRunningTime="2026-09-15 20:21:26.093027753 +0000 UTC m=+992.483980755"
Sep 15 20:21:26.155558 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:26.155514    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c"]
Sep 15 20:21:26.160285 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:26.160258    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c"
Sep 15 20:21:26.162906 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:26.162881    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-01ea3b-serving-cert\""
Sep 15 20:21:26.170548 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:26.170522    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c"]
Sep 15 20:21:26.253813 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:26.253786    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86"]
Sep 15 20:21:26.256496 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:21:26.256466    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64cbef4e_b625_44f3_8acd_5b1ca3d4d154.slice/crio-8a745c7c68267f628840f7f7c6f6ac8456b6af500736a51b6ec3ad79fccaeac3 WatchSource:0}: Error finding container 8a745c7c68267f628840f7f7c6f6ac8456b6af500736a51b6ec3ad79fccaeac3: Status 404 returned error can't find the container with id 8a745c7c68267f628840f7f7c6f6ac8456b6af500736a51b6ec3ad79fccaeac3
Sep 15 20:21:26.262897 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:26.262869    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/215a694b-de08-40fa-8049-858e834455c8-maas-api-tls\") pod \"maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c\" (UID: \"215a694b-de08-40fa-8049-858e834455c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c"
Sep 15 20:21:26.263043 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:26.263022    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/215a694b-de08-40fa-8049-858e834455c8-metrics-certs\") pod \"maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c\" (UID: \"215a694b-de08-40fa-8049-858e834455c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c"
Sep 15 20:21:26.263194 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:26.263103    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cplks\" (UniqueName: \"kubernetes.io/projected/215a694b-de08-40fa-8049-858e834455c8-kube-api-access-cplks\") pod \"maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c\" (UID: \"215a694b-de08-40fa-8049-858e834455c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c"
Sep 15 20:21:26.263194 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:26.263148    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/215a694b-de08-40fa-8049-858e834455c8-openshift-service-ca\") pod \"maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c\" (UID: \"215a694b-de08-40fa-8049-858e834455c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c"
Sep 15 20:21:26.364183 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:26.364085    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/215a694b-de08-40fa-8049-858e834455c8-metrics-certs\") pod \"maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c\" (UID: \"215a694b-de08-40fa-8049-858e834455c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c"
Sep 15 20:21:26.364183 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:26.364169    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cplks\" (UniqueName: \"kubernetes.io/projected/215a694b-de08-40fa-8049-858e834455c8-kube-api-access-cplks\") pod \"maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c\" (UID: \"215a694b-de08-40fa-8049-858e834455c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c"
Sep 15 20:21:26.364468 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:26.364205    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/215a694b-de08-40fa-8049-858e834455c8-openshift-service-ca\") pod \"maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c\" (UID: \"215a694b-de08-40fa-8049-858e834455c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c"
Sep 15 20:21:26.364468 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:26.364257    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/215a694b-de08-40fa-8049-858e834455c8-maas-api-tls\") pod \"maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c\" (UID: \"215a694b-de08-40fa-8049-858e834455c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c"
Sep 15 20:21:26.364468 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:21:26.364421    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-01ea3b-serving-cert: secret "maas-api-e2e-inf-a-01ea3b-serving-cert" not found
Sep 15 20:21:26.364621 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:21:26.364497    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/215a694b-de08-40fa-8049-858e834455c8-maas-api-tls podName:215a694b-de08-40fa-8049-858e834455c8 nodeName:}" failed. No retries permitted until 2026-09-15 20:21:26.864475932 +0000 UTC m=+993.255428909 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/215a694b-de08-40fa-8049-858e834455c8-maas-api-tls") pod "maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c" (UID: "215a694b-de08-40fa-8049-858e834455c8") : secret "maas-api-e2e-inf-a-01ea3b-serving-cert" not found
Sep 15 20:21:26.364935 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:26.364913    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/215a694b-de08-40fa-8049-858e834455c8-openshift-service-ca\") pod \"maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c\" (UID: \"215a694b-de08-40fa-8049-858e834455c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c"
Sep 15 20:21:26.366602 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:26.366585    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/215a694b-de08-40fa-8049-858e834455c8-metrics-certs\") pod \"maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c\" (UID: \"215a694b-de08-40fa-8049-858e834455c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c"
Sep 15 20:21:26.377878 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:26.377840    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cplks\" (UniqueName: \"kubernetes.io/projected/215a694b-de08-40fa-8049-858e834455c8-kube-api-access-cplks\") pod \"maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c\" (UID: \"215a694b-de08-40fa-8049-858e834455c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c"
Sep 15 20:21:26.695965 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:26.695865    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7975-2bd2ce"]
Sep 15 20:21:26.701056 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:26.701021    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7975-2bd2ce"]
Sep 15 20:21:26.869662 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:26.869622    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/215a694b-de08-40fa-8049-858e834455c8-maas-api-tls\") pod \"maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c\" (UID: \"215a694b-de08-40fa-8049-858e834455c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c"
Sep 15 20:21:26.872164 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:26.872136    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/215a694b-de08-40fa-8049-858e834455c8-maas-api-tls\") pod \"maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c\" (UID: \"215a694b-de08-40fa-8049-858e834455c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c"
Sep 15 20:21:27.074230 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:27.074194    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c"
Sep 15 20:21:27.084322 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:27.084289    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86" event={"ID":"64cbef4e-b625-44f3-8acd-5b1ca3d4d154","Type":"ContainerStarted","Data":"ad1fe5adddee1d826b6bae3e2c2654ed997af4bf70604c1a1743cd930ef5d035"}
Sep 15 20:21:27.084747 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:27.084380    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86" event={"ID":"64cbef4e-b625-44f3-8acd-5b1ca3d4d154","Type":"ContainerStarted","Data":"8a745c7c68267f628840f7f7c6f6ac8456b6af500736a51b6ec3ad79fccaeac3"}
Sep 15 20:21:27.084747 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:21:27.084511    2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7975-2bd2ce" container="test-curl-7975-2bd2ce"
Sep 15 20:21:27.084747 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:27.084549    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86"
Sep 15 20:21:27.104151 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:27.104091    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86" podStartSLOduration=1.514520359 podStartE2EDuration="2.104068411s" totalImagesPullingTime="589.548052ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:21:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:21:27.101070314 +0000 UTC m=+993.492023312" watchObservedRunningTime="2026-09-15 20:21:27.104068411 +0000 UTC m=+993.495021409"
Sep 15 20:21:27.256267 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:27.256232    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm"]
Sep 15 20:21:27.256971 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:27.256949    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3e8b2ae-f862-4033-af64-2a88c6cc7188" containerName="test-curl-7975-2bd2ce"
Sep 15 20:21:27.257038 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:27.256976    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3e8b2ae-f862-4033-af64-2a88c6cc7188" containerName="test-curl-7975-2bd2ce"
Sep 15 20:21:27.257138 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:27.257126    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3e8b2ae-f862-4033-af64-2a88c6cc7188" containerName="test-curl-7975-2bd2ce"
Sep 15 20:21:27.261764 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:27.261743    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm"
Sep 15 20:21:27.265102 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:27.264970    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-725e67-serving-cert\""
Sep 15 20:21:27.268271 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:27.268244    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm"]
Sep 15 20:21:27.360864 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:27.360832    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c"]
Sep 15 20:21:27.362081 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:21:27.362054    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod215a694b_de08_40fa_8049_858e834455c8.slice/crio-eb9cf862ef6c9da78e4b0505d9fcadb4833a0ce4261210c6c4641e53ec75ce3e WatchSource:0}: Error finding container eb9cf862ef6c9da78e4b0505d9fcadb4833a0ce4261210c6c4641e53ec75ce3e: Status 404 returned error can't find the container with id eb9cf862ef6c9da78e4b0505d9fcadb4833a0ce4261210c6c4641e53ec75ce3e
Sep 15 20:21:27.375472 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:27.375437    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjwhh\" (UniqueName: \"kubernetes.io/projected/091f856b-9ed9-43e3-a60f-71b61af8a89e-kube-api-access-qjwhh\") pod \"maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm\" (UID: \"091f856b-9ed9-43e3-a60f-71b61af8a89e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm"
Sep 15 20:21:27.375621 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:27.375524    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/091f856b-9ed9-43e3-a60f-71b61af8a89e-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm\" (UID: \"091f856b-9ed9-43e3-a60f-71b61af8a89e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm"
Sep 15 20:21:27.375621 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:27.375614    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/091f856b-9ed9-43e3-a60f-71b61af8a89e-metrics-certs\") pod \"maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm\" (UID: \"091f856b-9ed9-43e3-a60f-71b61af8a89e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm"
Sep 15 20:21:27.375784 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:27.375757    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/091f856b-9ed9-43e3-a60f-71b61af8a89e-maas-api-tls\") pod \"maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm\" (UID: \"091f856b-9ed9-43e3-a60f-71b61af8a89e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm"
Sep 15 20:21:27.477147 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:27.477107    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/091f856b-9ed9-43e3-a60f-71b61af8a89e-maas-api-tls\") pod \"maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm\" (UID: \"091f856b-9ed9-43e3-a60f-71b61af8a89e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm"
Sep 15 20:21:27.477325 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:27.477193    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjwhh\" (UniqueName: \"kubernetes.io/projected/091f856b-9ed9-43e3-a60f-71b61af8a89e-kube-api-access-qjwhh\") pod \"maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm\" (UID: \"091f856b-9ed9-43e3-a60f-71b61af8a89e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm"
Sep 15 20:21:27.477325 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:27.477247    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/091f856b-9ed9-43e3-a60f-71b61af8a89e-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm\" (UID: \"091f856b-9ed9-43e3-a60f-71b61af8a89e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm"
Sep 15 20:21:27.477325 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:27.477289    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/091f856b-9ed9-43e3-a60f-71b61af8a89e-metrics-certs\") pod \"maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm\" (UID: \"091f856b-9ed9-43e3-a60f-71b61af8a89e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm"
Sep 15 20:21:27.478046 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:27.478018    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/091f856b-9ed9-43e3-a60f-71b61af8a89e-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm\" (UID: \"091f856b-9ed9-43e3-a60f-71b61af8a89e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm"
Sep 15 20:21:27.480098 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:27.479973    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/091f856b-9ed9-43e3-a60f-71b61af8a89e-metrics-certs\") pod \"maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm\" (UID: \"091f856b-9ed9-43e3-a60f-71b61af8a89e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm"
Sep 15 20:21:27.480098 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:27.480041    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/091f856b-9ed9-43e3-a60f-71b61af8a89e-maas-api-tls\") pod \"maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm\" (UID: \"091f856b-9ed9-43e3-a60f-71b61af8a89e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm"
Sep 15 20:21:27.491515 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:27.491489    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjwhh\" (UniqueName: \"kubernetes.io/projected/091f856b-9ed9-43e3-a60f-71b61af8a89e-kube-api-access-qjwhh\") pod \"maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm\" (UID: \"091f856b-9ed9-43e3-a60f-71b61af8a89e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm"
Sep 15 20:21:27.580626 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:27.580582    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm"
Sep 15 20:21:27.872850 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:27.872814    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm"]
Sep 15 20:21:27.874226 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:21:27.874191    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod091f856b_9ed9_43e3_a60f_71b61af8a89e.slice/crio-3d8d8c97eff52ebd4f61b7c59920f0a858be49c7adf2e8776b9f5f89d58a8721 WatchSource:0}: Error finding container 3d8d8c97eff52ebd4f61b7c59920f0a858be49c7adf2e8776b9f5f89d58a8721: Status 404 returned error can't find the container with id 3d8d8c97eff52ebd4f61b7c59920f0a858be49c7adf2e8776b9f5f89d58a8721
Sep 15 20:21:28.093950 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:28.093914    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm" event={"ID":"091f856b-9ed9-43e3-a60f-71b61af8a89e","Type":"ContainerStarted","Data":"3d8d8c97eff52ebd4f61b7c59920f0a858be49c7adf2e8776b9f5f89d58a8721"}
Sep 15 20:21:28.096022 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:28.095992    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c" event={"ID":"215a694b-de08-40fa-8049-858e834455c8","Type":"ContainerStarted","Data":"7793011affcbc20d5e49fef33ada40da0cab069588993f240eeb0615b26aa2d9"}
Sep 15 20:21:28.096157 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:28.096048    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c" event={"ID":"215a694b-de08-40fa-8049-858e834455c8","Type":"ContainerStarted","Data":"eb9cf862ef6c9da78e4b0505d9fcadb4833a0ce4261210c6c4641e53ec75ce3e"}
Sep 15 20:21:28.096513 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:28.096417    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c"
Sep 15 20:21:28.096585 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:28.096519    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7975-2bd2ce" podUID="b3e8b2ae-f862-4033-af64-2a88c6cc7188" containerName="test-curl-7975-2bd2ce" containerID="cri-o://ae21eb475bdc2226a1a60e38bb8798f14dfd7fd43bf523f36ec9790f1e1179fc" gracePeriod=2
Sep 15 20:21:28.136538 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:28.136479    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c" podStartSLOduration=1.7548675459999998 podStartE2EDuration="2.136454409s" totalImagesPullingTime="381.586863ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:21:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:21:28.132760208 +0000 UTC m=+994.523713208" watchObservedRunningTime="2026-09-15 20:21:28.136454409 +0000 UTC m=+994.527407409"
Sep 15 20:21:28.287086 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:28.287056    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7975-dcb00a"
Sep 15 20:21:28.388238 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:28.388208    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cssxn\" (UniqueName: \"kubernetes.io/projected/4e561a6d-19ac-497c-9506-4f904d0f3760-kube-api-access-cssxn\") pod \"4e561a6d-19ac-497c-9506-4f904d0f3760\" (UID: \"4e561a6d-19ac-497c-9506-4f904d0f3760\") "
Sep 15 20:21:28.390985 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:28.390950    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e561a6d-19ac-497c-9506-4f904d0f3760-kube-api-access-cssxn" (OuterVolumeSpecName: "kube-api-access-cssxn") pod "4e561a6d-19ac-497c-9506-4f904d0f3760" (UID: "4e561a6d-19ac-497c-9506-4f904d0f3760"). InnerVolumeSpecName "kube-api-access-cssxn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:21:28.489108 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:28.489073    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cssxn\" (UniqueName: \"kubernetes.io/projected/4e561a6d-19ac-497c-9506-4f904d0f3760-kube-api-access-cssxn\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:21:29.104582 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:29.104540    2577 generic.go:356] "Generic (PLEG): container finished" podID="4e561a6d-19ac-497c-9506-4f904d0f3760" containerID="e42239b2894146a2100934b529c0d847956878bfc97285663eb4f2ace0573a57" exitCode=137
Sep 15 20:21:29.104582 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:29.104577    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7975-dcb00a"
Sep 15 20:21:29.105158 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:29.104644    2577 scope.go:117] "RemoveContainer" containerID="e42239b2894146a2100934b529c0d847956878bfc97285663eb4f2ace0573a57"
Sep 15 20:21:29.106690 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:29.106663    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm" event={"ID":"091f856b-9ed9-43e3-a60f-71b61af8a89e","Type":"ContainerStarted","Data":"48ae3b4bf7b72df345201165fbec611d891f458d7cc71659f53115458733aa6d"}
Sep 15 20:21:29.106945 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:29.106927    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm"
Sep 15 20:21:29.119138 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:29.119116    2577 scope.go:117] "RemoveContainer" containerID="e42239b2894146a2100934b529c0d847956878bfc97285663eb4f2ace0573a57"
Sep 15 20:21:29.119458 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:21:29.119438    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e42239b2894146a2100934b529c0d847956878bfc97285663eb4f2ace0573a57\": container with ID starting with e42239b2894146a2100934b529c0d847956878bfc97285663eb4f2ace0573a57 not found: ID does not exist" containerID="e42239b2894146a2100934b529c0d847956878bfc97285663eb4f2ace0573a57"
Sep 15 20:21:29.119540 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:29.119481    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e42239b2894146a2100934b529c0d847956878bfc97285663eb4f2ace0573a57"} err="failed to get container status \"e42239b2894146a2100934b529c0d847956878bfc97285663eb4f2ace0573a57\": rpc error: code = NotFound desc = could not find container \"e42239b2894146a2100934b529c0d847956878bfc97285663eb4f2ace0573a57\": container with ID starting with e42239b2894146a2100934b529c0d847956878bfc97285663eb4f2ace0573a57 not found: ID does not exist"
Sep 15 20:21:29.125663 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:29.125622    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm" podStartSLOduration=1.684413396 podStartE2EDuration="2.125608549s" totalImagesPullingTime="441.195153ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:21:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:21:29.124907683 +0000 UTC m=+995.515860681" watchObservedRunningTime="2026-09-15 20:21:29.125608549 +0000 UTC m=+995.516561577"
Sep 15 20:21:29.127083 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:21:29.127059    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7975-dcb00a\" is forbidden: User \"system:node:ip-10-0-140-72.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-72.ec2.internal' and this object" podUID="4e561a6d-19ac-497c-9506-4f904d0f3760" pod="opendatahub/test-curl-7975-dcb00a"
Sep 15 20:21:30.063119 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:30.063065    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e561a6d-19ac-497c-9506-4f904d0f3760" path="/var/lib/kubelet/pods/4e561a6d-19ac-497c-9506-4f904d0f3760/volumes"
Sep 15 20:21:30.301898 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:30.301869    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7975-2bd2ce"
Sep 15 20:21:30.410230 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:30.410143    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtpxc\" (UniqueName: \"kubernetes.io/projected/b3e8b2ae-f862-4033-af64-2a88c6cc7188-kube-api-access-gtpxc\") pod \"b3e8b2ae-f862-4033-af64-2a88c6cc7188\" (UID: \"b3e8b2ae-f862-4033-af64-2a88c6cc7188\") "
Sep 15 20:21:30.412518 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:30.412488    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3e8b2ae-f862-4033-af64-2a88c6cc7188-kube-api-access-gtpxc" (OuterVolumeSpecName: "kube-api-access-gtpxc") pod "b3e8b2ae-f862-4033-af64-2a88c6cc7188" (UID: "b3e8b2ae-f862-4033-af64-2a88c6cc7188"). InnerVolumeSpecName "kube-api-access-gtpxc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:21:30.511338 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:30.511298    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtpxc\" (UniqueName: \"kubernetes.io/projected/b3e8b2ae-f862-4033-af64-2a88c6cc7188-kube-api-access-gtpxc\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:21:31.123058 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:31.123023    2577 generic.go:356] "Generic (PLEG): container finished" podID="b3e8b2ae-f862-4033-af64-2a88c6cc7188" containerID="ae21eb475bdc2226a1a60e38bb8798f14dfd7fd43bf523f36ec9790f1e1179fc" exitCode=137
Sep 15 20:21:31.123274 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:31.123071    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7975-2bd2ce"
Sep 15 20:21:31.123274 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:31.123103    2577 scope.go:117] "RemoveContainer" containerID="ae21eb475bdc2226a1a60e38bb8798f14dfd7fd43bf523f36ec9790f1e1179fc"
Sep 15 20:21:31.133902 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:31.133880    2577 scope.go:117] "RemoveContainer" containerID="ae21eb475bdc2226a1a60e38bb8798f14dfd7fd43bf523f36ec9790f1e1179fc"
Sep 15 20:21:31.134205 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:21:31.134184    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae21eb475bdc2226a1a60e38bb8798f14dfd7fd43bf523f36ec9790f1e1179fc\": container with ID starting with ae21eb475bdc2226a1a60e38bb8798f14dfd7fd43bf523f36ec9790f1e1179fc not found: ID does not exist" containerID="ae21eb475bdc2226a1a60e38bb8798f14dfd7fd43bf523f36ec9790f1e1179fc"
Sep 15 20:21:31.134279 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:31.134213    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae21eb475bdc2226a1a60e38bb8798f14dfd7fd43bf523f36ec9790f1e1179fc"} err="failed to get container status \"ae21eb475bdc2226a1a60e38bb8798f14dfd7fd43bf523f36ec9790f1e1179fc\": rpc error: code = NotFound desc = could not find container \"ae21eb475bdc2226a1a60e38bb8798f14dfd7fd43bf523f36ec9790f1e1179fc\": container with ID starting with ae21eb475bdc2226a1a60e38bb8798f14dfd7fd43bf523f36ec9790f1e1179fc not found: ID does not exist"
Sep 15 20:21:31.134575 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:21:31.134552    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7975-2bd2ce\" is forbidden: User \"system:node:ip-10-0-140-72.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-72.ec2.internal' and this object" podUID="b3e8b2ae-f862-4033-af64-2a88c6cc7188" pod="opendatahub/test-curl-7975-2bd2ce"
Sep 15 20:21:32.063455 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:32.063414    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3e8b2ae-f862-4033-af64-2a88c6cc7188" path="/var/lib/kubelet/pods/b3e8b2ae-f862-4033-af64-2a88c6cc7188/volumes"
Sep 15 20:21:33.102613 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:33.102584    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86"
Sep 15 20:21:34.112866 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:34.112831    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c"
Sep 15 20:21:35.057624 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:35.057586    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx"]
Sep 15 20:21:35.062674 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:35.062648    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx"
Sep 15 20:21:35.065105 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:35.065081    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-30d5aa-serving-cert\""
Sep 15 20:21:35.079753 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:35.079711    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx"]
Sep 15 20:21:35.121903 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:35.121875    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm"
Sep 15 20:21:35.161021 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:35.160981    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwh9m\" (UniqueName: \"kubernetes.io/projected/e00d30ef-3f6c-4841-8f7c-1c64c01f29c5-kube-api-access-kwh9m\") pod \"maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx\" (UID: \"e00d30ef-3f6c-4841-8f7c-1c64c01f29c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx"
Sep 15 20:21:35.161316 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:35.161287    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e00d30ef-3f6c-4841-8f7c-1c64c01f29c5-openshift-service-ca\") pod \"maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx\" (UID: \"e00d30ef-3f6c-4841-8f7c-1c64c01f29c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx"
Sep 15 20:21:35.161442 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:35.161324    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e00d30ef-3f6c-4841-8f7c-1c64c01f29c5-metrics-certs\") pod \"maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx\" (UID: \"e00d30ef-3f6c-4841-8f7c-1c64c01f29c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx"
Sep 15 20:21:35.161442 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:35.161398    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e00d30ef-3f6c-4841-8f7c-1c64c01f29c5-maas-api-tls\") pod \"maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx\" (UID: \"e00d30ef-3f6c-4841-8f7c-1c64c01f29c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx"
Sep 15 20:21:35.262111 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:35.262073    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwh9m\" (UniqueName: \"kubernetes.io/projected/e00d30ef-3f6c-4841-8f7c-1c64c01f29c5-kube-api-access-kwh9m\") pod \"maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx\" (UID: \"e00d30ef-3f6c-4841-8f7c-1c64c01f29c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx"
Sep 15 20:21:35.262297 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:35.262163    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e00d30ef-3f6c-4841-8f7c-1c64c01f29c5-openshift-service-ca\") pod \"maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx\" (UID: \"e00d30ef-3f6c-4841-8f7c-1c64c01f29c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx"
Sep 15 20:21:35.262297 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:35.262187    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e00d30ef-3f6c-4841-8f7c-1c64c01f29c5-metrics-certs\") pod \"maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx\" (UID: \"e00d30ef-3f6c-4841-8f7c-1c64c01f29c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx"
Sep 15 20:21:35.262443 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:35.262378    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e00d30ef-3f6c-4841-8f7c-1c64c01f29c5-maas-api-tls\") pod \"maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx\" (UID: \"e00d30ef-3f6c-4841-8f7c-1c64c01f29c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx"
Sep 15 20:21:35.262534 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:21:35.262517    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-30d5aa-serving-cert: secret "maas-api-e2e-autores-30d5aa-serving-cert" not found
Sep 15 20:21:35.262589 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:21:35.262583    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e00d30ef-3f6c-4841-8f7c-1c64c01f29c5-maas-api-tls podName:e00d30ef-3f6c-4841-8f7c-1c64c01f29c5 nodeName:}" failed. No retries permitted until 2026-09-15 20:21:35.762563108 +0000 UTC m=+1002.153516087 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e00d30ef-3f6c-4841-8f7c-1c64c01f29c5-maas-api-tls") pod "maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx" (UID: "e00d30ef-3f6c-4841-8f7c-1c64c01f29c5") : secret "maas-api-e2e-autores-30d5aa-serving-cert" not found
Sep 15 20:21:35.263037 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:35.263005    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e00d30ef-3f6c-4841-8f7c-1c64c01f29c5-openshift-service-ca\") pod \"maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx\" (UID: \"e00d30ef-3f6c-4841-8f7c-1c64c01f29c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx"
Sep 15 20:21:35.265600 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:35.265579    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e00d30ef-3f6c-4841-8f7c-1c64c01f29c5-metrics-certs\") pod \"maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx\" (UID: \"e00d30ef-3f6c-4841-8f7c-1c64c01f29c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx"
Sep 15 20:21:35.273863 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:35.273834    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwh9m\" (UniqueName: \"kubernetes.io/projected/e00d30ef-3f6c-4841-8f7c-1c64c01f29c5-kube-api-access-kwh9m\") pod \"maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx\" (UID: \"e00d30ef-3f6c-4841-8f7c-1c64c01f29c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx"
Sep 15 20:21:35.768391 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:35.768324    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e00d30ef-3f6c-4841-8f7c-1c64c01f29c5-maas-api-tls\") pod \"maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx\" (UID: \"e00d30ef-3f6c-4841-8f7c-1c64c01f29c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx"
Sep 15 20:21:35.770833 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:35.770808    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e00d30ef-3f6c-4841-8f7c-1c64c01f29c5-maas-api-tls\") pod \"maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx\" (UID: \"e00d30ef-3f6c-4841-8f7c-1c64c01f29c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx"
Sep 15 20:21:35.981041 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:35.980997    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx"
Sep 15 20:21:36.259481 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:36.259451    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx"]
Sep 15 20:21:36.261051 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:21:36.261025    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode00d30ef_3f6c_4841_8f7c_1c64c01f29c5.slice/crio-bfab7b17c277840d432be33b53b5cb2b2da2dcb96aa20e012407742c60890b81 WatchSource:0}: Error finding container bfab7b17c277840d432be33b53b5cb2b2da2dcb96aa20e012407742c60890b81: Status 404 returned error can't find the container with id bfab7b17c277840d432be33b53b5cb2b2da2dcb96aa20e012407742c60890b81
Sep 15 20:21:37.053622 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:37.053583    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln"]
Sep 15 20:21:37.057890 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:37.057866    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln"
Sep 15 20:21:37.060185 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:37.060163    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-5005f4d9-serving-cert\""
Sep 15 20:21:37.067610 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:37.067589    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln"]
Sep 15 20:21:37.168401 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:37.168362    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx" event={"ID":"e00d30ef-3f6c-4841-8f7c-1c64c01f29c5","Type":"ContainerStarted","Data":"87b5ae2f0246ebafcd564c0fe30222ea619ea1bc570c60510e51e4264dcc57ec"}
Sep 15 20:21:37.168609 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:37.168430    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx" event={"ID":"e00d30ef-3f6c-4841-8f7c-1c64c01f29c5","Type":"ContainerStarted","Data":"bfab7b17c277840d432be33b53b5cb2b2da2dcb96aa20e012407742c60890b81"}
Sep 15 20:21:37.168609 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:37.168461    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx"
Sep 15 20:21:37.184650 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:37.184619    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k89ms\" (UniqueName: \"kubernetes.io/projected/192b10d0-2961-4736-895c-bc4837d1474f-kube-api-access-k89ms\") pod \"maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln\" (UID: \"192b10d0-2961-4736-895c-bc4837d1474f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln"
Sep 15 20:21:37.184809 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:37.184707    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/192b10d0-2961-4736-895c-bc4837d1474f-metrics-certs\") pod \"maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln\" (UID: \"192b10d0-2961-4736-895c-bc4837d1474f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln"
Sep 15 20:21:37.184809 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:37.184765    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/192b10d0-2961-4736-895c-bc4837d1474f-openshift-service-ca\") pod \"maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln\" (UID: \"192b10d0-2961-4736-895c-bc4837d1474f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln"
Sep 15 20:21:37.184809 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:37.184800    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/192b10d0-2961-4736-895c-bc4837d1474f-maas-api-tls\") pod \"maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln\" (UID: \"192b10d0-2961-4736-895c-bc4837d1474f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln"
Sep 15 20:21:37.191892 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:37.191841    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx" podStartSLOduration=1.8412807820000001 podStartE2EDuration="2.191822442s" totalImagesPullingTime="350.54166ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:21:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:21:37.190231295 +0000 UTC m=+1003.581184306" watchObservedRunningTime="2026-09-15 20:21:37.191822442 +0000 UTC m=+1003.582775439"
Sep 15 20:21:37.285896 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:37.285853    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k89ms\" (UniqueName: \"kubernetes.io/projected/192b10d0-2961-4736-895c-bc4837d1474f-kube-api-access-k89ms\") pod \"maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln\" (UID: \"192b10d0-2961-4736-895c-bc4837d1474f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln"
Sep 15 20:21:37.285896 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:37.285902    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/192b10d0-2961-4736-895c-bc4837d1474f-metrics-certs\") pod \"maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln\" (UID: \"192b10d0-2961-4736-895c-bc4837d1474f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln"
Sep 15 20:21:37.286434 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:37.285922    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/192b10d0-2961-4736-895c-bc4837d1474f-openshift-service-ca\") pod \"maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln\" (UID: \"192b10d0-2961-4736-895c-bc4837d1474f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln"
Sep 15 20:21:37.286434 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:37.285941    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/192b10d0-2961-4736-895c-bc4837d1474f-maas-api-tls\") pod \"maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln\" (UID: \"192b10d0-2961-4736-895c-bc4837d1474f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln"
Sep 15 20:21:37.286434 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:21:37.286059    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-5005f4d9-serving-cert: secret "maas-api-e2e-ait-5005f4d9-serving-cert" not found
Sep 15 20:21:37.286434 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:21:37.286127    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/192b10d0-2961-4736-895c-bc4837d1474f-maas-api-tls podName:192b10d0-2961-4736-895c-bc4837d1474f nodeName:}" failed. No retries permitted until 2026-09-15 20:21:37.786106898 +0000 UTC m=+1004.177059906 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/192b10d0-2961-4736-895c-bc4837d1474f-maas-api-tls") pod "maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln" (UID: "192b10d0-2961-4736-895c-bc4837d1474f") : secret "maas-api-e2e-ait-5005f4d9-serving-cert" not found
Sep 15 20:21:37.286761 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:37.286708    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/192b10d0-2961-4736-895c-bc4837d1474f-openshift-service-ca\") pod \"maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln\" (UID: \"192b10d0-2961-4736-895c-bc4837d1474f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln"
Sep 15 20:21:37.288549 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:37.288524    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/192b10d0-2961-4736-895c-bc4837d1474f-metrics-certs\") pod \"maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln\" (UID: \"192b10d0-2961-4736-895c-bc4837d1474f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln"
Sep 15 20:21:37.297172 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:37.297142    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k89ms\" (UniqueName: \"kubernetes.io/projected/192b10d0-2961-4736-895c-bc4837d1474f-kube-api-access-k89ms\") pod \"maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln\" (UID: \"192b10d0-2961-4736-895c-bc4837d1474f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln"
Sep 15 20:21:37.791547 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:37.791505    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/192b10d0-2961-4736-895c-bc4837d1474f-maas-api-tls\") pod \"maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln\" (UID: \"192b10d0-2961-4736-895c-bc4837d1474f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln"
Sep 15 20:21:37.794138 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:37.794113    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/192b10d0-2961-4736-895c-bc4837d1474f-maas-api-tls\") pod \"maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln\" (UID: \"192b10d0-2961-4736-895c-bc4837d1474f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln"
Sep 15 20:21:37.969950 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:37.969907    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln"
Sep 15 20:21:38.261134 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:38.261104    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln"]
Sep 15 20:21:38.262924 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:21:38.262894    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod192b10d0_2961_4736_895c_bc4837d1474f.slice/crio-28642d3b322bbd729753e9aeca107475d701ae9a5ce55335baf6e45cd6f9e21f WatchSource:0}: Error finding container 28642d3b322bbd729753e9aeca107475d701ae9a5ce55335baf6e45cd6f9e21f: Status 404 returned error can't find the container with id 28642d3b322bbd729753e9aeca107475d701ae9a5ce55335baf6e45cd6f9e21f
Sep 15 20:21:39.186042 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:39.186006    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln" event={"ID":"192b10d0-2961-4736-895c-bc4837d1474f","Type":"ContainerStarted","Data":"82dacd251cc961797a1b3df821eb77b14ab655b5703e08a75832cec7956566a9"}
Sep 15 20:21:39.186426 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:39.186056    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln" event={"ID":"192b10d0-2961-4736-895c-bc4837d1474f","Type":"ContainerStarted","Data":"28642d3b322bbd729753e9aeca107475d701ae9a5ce55335baf6e45cd6f9e21f"}
Sep 15 20:21:39.186426 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:39.186138    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln"
Sep 15 20:21:39.212857 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:39.212806    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln" podStartSLOduration=1.795373324 podStartE2EDuration="2.212788159s" totalImagesPullingTime="417.414835ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:21:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:21:39.212444547 +0000 UTC m=+1005.603397545" watchObservedRunningTime="2026-09-15 20:21:39.212788159 +0000 UTC m=+1005.603741155"
Sep 15 20:21:43.191099 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:43.191053    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx"
Sep 15 20:21:45.202229 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:45.202197    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln"
Sep 15 20:21:49.199092 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:49.199056    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn"]
Sep 15 20:21:49.201808 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:49.201788    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn"
Sep 15 20:21:49.204809 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:49.204790    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-39c7b0-serving-cert\""
Sep 15 20:21:49.220342 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:49.220313    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn"]
Sep 15 20:21:49.265362 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:49.265327    2577 generic.go:356] "Generic (PLEG): container finished" podID="fa4b30cd-828e-4404-ab50-12cc8cf31f42" containerID="15dde39be4fd6ef0b788d49d9afacb369a871c0aeb4c1123664ae1ec4de80353" exitCode=0
Sep 15 20:21:49.265536 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:49.265397    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg" event={"ID":"fa4b30cd-828e-4404-ab50-12cc8cf31f42","Type":"ContainerDied","Data":"15dde39be4fd6ef0b788d49d9afacb369a871c0aeb4c1123664ae1ec4de80353"}
Sep 15 20:21:49.266201 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:49.265876    2577 scope.go:117] "RemoveContainer" containerID="15dde39be4fd6ef0b788d49d9afacb369a871c0aeb4c1123664ae1ec4de80353"
Sep 15 20:21:49.267184 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:49.267149    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 20:21:49.267266 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:49.267228    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 20:21:49.267306 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:49.267263    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 20:21:49.307226 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:49.307005    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3b338f0-d987-483c-ba79-420e187989c4-maas-api-tls\") pod \"maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn\" (UID: \"c3b338f0-d987-483c-ba79-420e187989c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn"
Sep 15 20:21:49.307226 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:49.307070    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ff867\" (UniqueName: \"kubernetes.io/projected/c3b338f0-d987-483c-ba79-420e187989c4-kube-api-access-ff867\") pod \"maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn\" (UID: \"c3b338f0-d987-483c-ba79-420e187989c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn"
Sep 15 20:21:49.307226 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:49.307171    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3b338f0-d987-483c-ba79-420e187989c4-metrics-certs\") pod \"maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn\" (UID: \"c3b338f0-d987-483c-ba79-420e187989c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn"
Sep 15 20:21:49.307652 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:49.307303    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3b338f0-d987-483c-ba79-420e187989c4-openshift-service-ca\") pod \"maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn\" (UID: \"c3b338f0-d987-483c-ba79-420e187989c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn"
Sep 15 20:21:49.408109 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:49.408067    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3b338f0-d987-483c-ba79-420e187989c4-metrics-certs\") pod \"maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn\" (UID: \"c3b338f0-d987-483c-ba79-420e187989c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn"
Sep 15 20:21:49.408311 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:49.408201    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3b338f0-d987-483c-ba79-420e187989c4-openshift-service-ca\") pod \"maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn\" (UID: \"c3b338f0-d987-483c-ba79-420e187989c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn"
Sep 15 20:21:49.408311 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:49.408304    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3b338f0-d987-483c-ba79-420e187989c4-maas-api-tls\") pod \"maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn\" (UID: \"c3b338f0-d987-483c-ba79-420e187989c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn"
Sep 15 20:21:49.408446 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:49.408345    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ff867\" (UniqueName: \"kubernetes.io/projected/c3b338f0-d987-483c-ba79-420e187989c4-kube-api-access-ff867\") pod \"maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn\" (UID: \"c3b338f0-d987-483c-ba79-420e187989c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn"
Sep 15 20:21:49.408446 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:21:49.408416    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-39c7b0-serving-cert: secret "maas-api-e2e-inf-b-39c7b0-serving-cert" not found
Sep 15 20:21:49.408550 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:21:49.408479    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3b338f0-d987-483c-ba79-420e187989c4-maas-api-tls podName:c3b338f0-d987-483c-ba79-420e187989c4 nodeName:}" failed. No retries permitted until 2026-09-15 20:21:49.90845937 +0000 UTC m=+1016.299412357 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c3b338f0-d987-483c-ba79-420e187989c4-maas-api-tls") pod "maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn" (UID: "c3b338f0-d987-483c-ba79-420e187989c4") : secret "maas-api-e2e-inf-b-39c7b0-serving-cert" not found
Sep 15 20:21:49.409130 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:49.409105    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3b338f0-d987-483c-ba79-420e187989c4-openshift-service-ca\") pod \"maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn\" (UID: \"c3b338f0-d987-483c-ba79-420e187989c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn"
Sep 15 20:21:49.410697 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:49.410675    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3b338f0-d987-483c-ba79-420e187989c4-metrics-certs\") pod \"maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn\" (UID: \"c3b338f0-d987-483c-ba79-420e187989c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn"
Sep 15 20:21:49.419238 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:49.419213    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ff867\" (UniqueName: \"kubernetes.io/projected/c3b338f0-d987-483c-ba79-420e187989c4-kube-api-access-ff867\") pod \"maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn\" (UID: \"c3b338f0-d987-483c-ba79-420e187989c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn"
Sep 15 20:21:49.886034 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:49.885993    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-39c7b0-76f9c875cf-6km95"]
Sep 15 20:21:49.892100 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:49.890160    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-39c7b0-76f9c875cf-6km95"
Sep 15 20:21:49.895510 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:49.893643    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-39c7b0\""
Sep 15 20:21:49.895510 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:49.893952    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-39c7b0-dockercfg-v5gjt\""
Sep 15 20:21:49.908753 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:49.908695    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-39c7b0-76f9c875cf-6km95"]
Sep 15 20:21:49.913925 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:49.913894    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3b338f0-d987-483c-ba79-420e187989c4-maas-api-tls\") pod \"maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn\" (UID: \"c3b338f0-d987-483c-ba79-420e187989c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn"
Sep 15 20:21:49.916641 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:49.916610    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3b338f0-d987-483c-ba79-420e187989c4-maas-api-tls\") pod \"maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn\" (UID: \"c3b338f0-d987-483c-ba79-420e187989c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn"
Sep 15 20:21:50.015206 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:50.015150    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqz66\" (UniqueName: \"kubernetes.io/projected/93eee1a1-9cfd-436e-975d-e6ca9d3d722d-kube-api-access-xqz66\") pod \"payload-processing-e2e-inf-b-39c7b0-76f9c875cf-6km95\" (UID: \"93eee1a1-9cfd-436e-975d-e6ca9d3d722d\") " pod="openshift-ingress/payload-processing-e2e-inf-b-39c7b0-76f9c875cf-6km95"
Sep 15 20:21:50.015422 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:50.015269    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93eee1a1-9cfd-436e-975d-e6ca9d3d722d-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-39c7b0-76f9c875cf-6km95\" (UID: \"93eee1a1-9cfd-436e-975d-e6ca9d3d722d\") " pod="openshift-ingress/payload-processing-e2e-inf-b-39c7b0-76f9c875cf-6km95"
Sep 15 20:21:50.112524 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:50.112490    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn"
Sep 15 20:21:50.115931 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:50.115901    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqz66\" (UniqueName: \"kubernetes.io/projected/93eee1a1-9cfd-436e-975d-e6ca9d3d722d-kube-api-access-xqz66\") pod \"payload-processing-e2e-inf-b-39c7b0-76f9c875cf-6km95\" (UID: \"93eee1a1-9cfd-436e-975d-e6ca9d3d722d\") " pod="openshift-ingress/payload-processing-e2e-inf-b-39c7b0-76f9c875cf-6km95"
Sep 15 20:21:50.116068 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:50.116004    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93eee1a1-9cfd-436e-975d-e6ca9d3d722d-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-39c7b0-76f9c875cf-6km95\" (UID: \"93eee1a1-9cfd-436e-975d-e6ca9d3d722d\") " pod="openshift-ingress/payload-processing-e2e-inf-b-39c7b0-76f9c875cf-6km95"
Sep 15 20:21:50.116934 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:50.116910    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93eee1a1-9cfd-436e-975d-e6ca9d3d722d-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-39c7b0-76f9c875cf-6km95\" (UID: \"93eee1a1-9cfd-436e-975d-e6ca9d3d722d\") " pod="openshift-ingress/payload-processing-e2e-inf-b-39c7b0-76f9c875cf-6km95"
Sep 15 20:21:50.126968 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:50.126944    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqz66\" (UniqueName: \"kubernetes.io/projected/93eee1a1-9cfd-436e-975d-e6ca9d3d722d-kube-api-access-xqz66\") pod \"payload-processing-e2e-inf-b-39c7b0-76f9c875cf-6km95\" (UID: \"93eee1a1-9cfd-436e-975d-e6ca9d3d722d\") " pod="openshift-ingress/payload-processing-e2e-inf-b-39c7b0-76f9c875cf-6km95"
Sep 15 20:21:50.209788 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:50.209559    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-39c7b0-76f9c875cf-6km95"
Sep 15 20:21:50.275556 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:50.274313    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg" event={"ID":"fa4b30cd-828e-4404-ab50-12cc8cf31f42","Type":"ContainerStarted","Data":"53924c141a1783aa6f67bfbce1642f5a04163577e762e7de3dd4dd3d46e0e1ca"}
Sep 15 20:21:50.392708 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:50.392672    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn"]
Sep 15 20:21:50.393396 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:21:50.393374    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3b338f0_d987_483c_ba79_420e187989c4.slice/crio-16fc738d8e8ee53772170e662747d21e4f9a52f06fa5b663226ae33b986ec760 WatchSource:0}: Error finding container 16fc738d8e8ee53772170e662747d21e4f9a52f06fa5b663226ae33b986ec760: Status 404 returned error can't find the container with id 16fc738d8e8ee53772170e662747d21e4f9a52f06fa5b663226ae33b986ec760
Sep 15 20:21:50.511488 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:50.511458    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-39c7b0-76f9c875cf-6km95"]
Sep 15 20:21:50.513605 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:21:50.513575    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93eee1a1_9cfd_436e_975d_e6ca9d3d722d.slice/crio-1b94087590522e3e8fc44fa09cfbd0b5fca674901f46d35bce1897a86f95c644 WatchSource:0}: Error finding container 1b94087590522e3e8fc44fa09cfbd0b5fca674901f46d35bce1897a86f95c644: Status 404 returned error can't find the container with id 1b94087590522e3e8fc44fa09cfbd0b5fca674901f46d35bce1897a86f95c644
Sep 15 20:21:50.813279 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:50.813154    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:21:50.814533 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:50.814504    2577 patch_prober.go:28] interesting pod/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Sep 15 20:21:50.814652 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:50.814561    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg" podUID="fa4b30cd-828e-4404-ab50-12cc8cf31f42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Sep 15 20:21:51.290625 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:51.290574    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn" event={"ID":"c3b338f0-d987-483c-ba79-420e187989c4","Type":"ContainerStarted","Data":"9357fd089a4b2a57d7dcb4e31e1b40311b79efedb7631af1323cae56b6e69a38"}
Sep 15 20:21:51.291172 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:51.290663    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn" event={"ID":"c3b338f0-d987-483c-ba79-420e187989c4","Type":"ContainerStarted","Data":"16fc738d8e8ee53772170e662747d21e4f9a52f06fa5b663226ae33b986ec760"}
Sep 15 20:21:51.291698 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:51.291668    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn"
Sep 15 20:21:51.295139 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:51.295105    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-39c7b0-76f9c875cf-6km95" event={"ID":"93eee1a1-9cfd-436e-975d-e6ca9d3d722d","Type":"ContainerStarted","Data":"af8de42825456f74c4a058236d78f9fcb3fd415052b788bc99f1a9aacecd6ca2"}
Sep 15 20:21:51.295273 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:51.295163    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-39c7b0-76f9c875cf-6km95"
Sep 15 20:21:51.295273 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:51.295189    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-39c7b0-76f9c875cf-6km95" event={"ID":"93eee1a1-9cfd-436e-975d-e6ca9d3d722d","Type":"ContainerStarted","Data":"1b94087590522e3e8fc44fa09cfbd0b5fca674901f46d35bce1897a86f95c644"}
Sep 15 20:21:51.311517 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:51.311462    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn" podStartSLOduration=1.8868558979999999 podStartE2EDuration="2.311445054s" totalImagesPullingTime="424.589156ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:21:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:21:51.310745999 +0000 UTC m=+1017.701699016" watchObservedRunningTime="2026-09-15 20:21:51.311445054 +0000 UTC m=+1017.702398050"
Sep 15 20:21:51.330479 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:51.330419    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-39c7b0-76f9c875cf-6km95" podStartSLOduration=2.330404509 podStartE2EDuration="2.330404509s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:21:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:21:51.328084923 +0000 UTC m=+1017.719037921" watchObservedRunningTime="2026-09-15 20:21:51.330404509 +0000 UTC m=+1017.721357505"
Sep 15 20:21:51.812691 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:51.812658    2577 patch_prober.go:28] interesting pod/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Sep 15 20:21:51.812956 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:51.812713    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg" podUID="fa4b30cd-828e-4404-ab50-12cc8cf31f42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Sep 15 20:21:52.812786 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:52.812719    2577 patch_prober.go:28] interesting pod/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Sep 15 20:21:52.813192 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:52.812811    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg" podUID="fa4b30cd-828e-4404-ab50-12cc8cf31f42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Sep 15 20:21:53.048486 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:53.048429    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:21:53.813063 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:53.813023    2577 patch_prober.go:28] interesting pod/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Sep 15 20:21:53.813508 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:53.813088    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg" podUID="fa4b30cd-828e-4404-ab50-12cc8cf31f42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Sep 15 20:21:54.813405 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:54.813352    2577 patch_prober.go:28] interesting pod/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Sep 15 20:21:54.813919 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:54.813422    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg" podUID="fa4b30cd-828e-4404-ab50-12cc8cf31f42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Sep 15 20:21:55.813401 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:55.813362    2577 patch_prober.go:28] interesting pod/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Sep 15 20:21:55.813574 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:55.813425    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg" podUID="fa4b30cd-828e-4404-ab50-12cc8cf31f42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Sep 15 20:21:56.816988 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:56.816960    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:21:56.818294 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:56.818270    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:21:58.317107 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:21:58.317078    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn"
Sep 15 20:22:02.306578 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:02.306544    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-39c7b0-76f9c875cf-6km95"
Sep 15 20:22:04.211645 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:04.211605    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln"]
Sep 15 20:22:04.212126 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:04.211937    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln" podUID="192b10d0-2961-4736-895c-bc4837d1474f" containerName="maas-api" containerID="cri-o://82dacd251cc961797a1b3df821eb77b14ab655b5703e08a75832cec7956566a9" gracePeriod=30
Sep 15 20:22:04.391988 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:04.391949    2577 generic.go:356] "Generic (PLEG): container finished" podID="192b10d0-2961-4736-895c-bc4837d1474f" containerID="82dacd251cc961797a1b3df821eb77b14ab655b5703e08a75832cec7956566a9" exitCode=0
Sep 15 20:22:04.392138 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:04.392096    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln" event={"ID":"192b10d0-2961-4736-895c-bc4837d1474f","Type":"ContainerDied","Data":"82dacd251cc961797a1b3df821eb77b14ab655b5703e08a75832cec7956566a9"}
Sep 15 20:22:04.491719 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:04.491691    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln"
Sep 15 20:22:04.572705 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:04.572670    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/192b10d0-2961-4736-895c-bc4837d1474f-openshift-service-ca\") pod \"192b10d0-2961-4736-895c-bc4837d1474f\" (UID: \"192b10d0-2961-4736-895c-bc4837d1474f\") "
Sep 15 20:22:04.572914 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:04.572768    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/192b10d0-2961-4736-895c-bc4837d1474f-metrics-certs\") pod \"192b10d0-2961-4736-895c-bc4837d1474f\" (UID: \"192b10d0-2961-4736-895c-bc4837d1474f\") "
Sep 15 20:22:04.572914 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:04.572829    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/192b10d0-2961-4736-895c-bc4837d1474f-maas-api-tls\") pod \"192b10d0-2961-4736-895c-bc4837d1474f\" (UID: \"192b10d0-2961-4736-895c-bc4837d1474f\") "
Sep 15 20:22:04.572914 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:04.572908    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k89ms\" (UniqueName: \"kubernetes.io/projected/192b10d0-2961-4736-895c-bc4837d1474f-kube-api-access-k89ms\") pod \"192b10d0-2961-4736-895c-bc4837d1474f\" (UID: \"192b10d0-2961-4736-895c-bc4837d1474f\") "
Sep 15 20:22:04.581631 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:04.579135    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/192b10d0-2961-4736-895c-bc4837d1474f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "192b10d0-2961-4736-895c-bc4837d1474f" (UID: "192b10d0-2961-4736-895c-bc4837d1474f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:22:04.581954 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:04.581907    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/192b10d0-2961-4736-895c-bc4837d1474f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "192b10d0-2961-4736-895c-bc4837d1474f" (UID: "192b10d0-2961-4736-895c-bc4837d1474f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:22:04.582509 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:04.582479    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/192b10d0-2961-4736-895c-bc4837d1474f-kube-api-access-k89ms" (OuterVolumeSpecName: "kube-api-access-k89ms") pod "192b10d0-2961-4736-895c-bc4837d1474f" (UID: "192b10d0-2961-4736-895c-bc4837d1474f"). InnerVolumeSpecName "kube-api-access-k89ms". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:22:04.583717 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:04.583689    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/192b10d0-2961-4736-895c-bc4837d1474f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "192b10d0-2961-4736-895c-bc4837d1474f" (UID: "192b10d0-2961-4736-895c-bc4837d1474f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:22:04.675473 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:04.675380    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/192b10d0-2961-4736-895c-bc4837d1474f-maas-api-tls\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:22:04.675473 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:04.675417    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k89ms\" (UniqueName: \"kubernetes.io/projected/192b10d0-2961-4736-895c-bc4837d1474f-kube-api-access-k89ms\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:22:04.675473 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:04.675434    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/192b10d0-2961-4736-895c-bc4837d1474f-openshift-service-ca\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:22:04.675473 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:04.675447    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/192b10d0-2961-4736-895c-bc4837d1474f-metrics-certs\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:22:05.400085 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:05.400052    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln"
Sep 15 20:22:05.400085 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:05.400068    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln" event={"ID":"192b10d0-2961-4736-895c-bc4837d1474f","Type":"ContainerDied","Data":"28642d3b322bbd729753e9aeca107475d701ae9a5ce55335baf6e45cd6f9e21f"}
Sep 15 20:22:05.400633 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:05.400157    2577 scope.go:117] "RemoveContainer" containerID="82dacd251cc961797a1b3df821eb77b14ab655b5703e08a75832cec7956566a9"
Sep 15 20:22:05.422967 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:05.422933    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln"]
Sep 15 20:22:05.427433 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:05.427407    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5005f4d9-5d76c6d7b8-zprln"]
Sep 15 20:22:06.061499 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:06.061455    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="192b10d0-2961-4736-895c-bc4837d1474f" path="/var/lib/kubelet/pods/192b10d0-2961-4736-895c-bc4837d1474f/volumes"
Sep 15 20:22:10.236768 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:10.236736    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4"]
Sep 15 20:22:10.237446 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:10.237425    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="192b10d0-2961-4736-895c-bc4837d1474f" containerName="maas-api"
Sep 15 20:22:10.237545 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:10.237449    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="192b10d0-2961-4736-895c-bc4837d1474f" containerName="maas-api"
Sep 15 20:22:10.237627 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:10.237614    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="192b10d0-2961-4736-895c-bc4837d1474f" containerName="maas-api"
Sep 15 20:22:10.248666 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:10.248464    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4"
Sep 15 20:22:10.251690 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:10.251662    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-e0b5e00e-serving-cert\""
Sep 15 20:22:10.254774 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:10.254746    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4"]
Sep 15 20:22:10.334664 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:10.334625    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/981991fe-cddc-46f4-9136-6451b67ee559-metrics-certs\") pod \"maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4\" (UID: \"981991fe-cddc-46f4-9136-6451b67ee559\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4"
Sep 15 20:22:10.334891 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:10.334715    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gw8g9\" (UniqueName: \"kubernetes.io/projected/981991fe-cddc-46f4-9136-6451b67ee559-kube-api-access-gw8g9\") pod \"maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4\" (UID: \"981991fe-cddc-46f4-9136-6451b67ee559\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4"
Sep 15 20:22:10.334891 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:10.334836    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/981991fe-cddc-46f4-9136-6451b67ee559-openshift-service-ca\") pod \"maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4\" (UID: \"981991fe-cddc-46f4-9136-6451b67ee559\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4"
Sep 15 20:22:10.334891 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:10.334860    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/981991fe-cddc-46f4-9136-6451b67ee559-maas-api-tls\") pod \"maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4\" (UID: \"981991fe-cddc-46f4-9136-6451b67ee559\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4"
Sep 15 20:22:10.435899 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:10.435864    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/981991fe-cddc-46f4-9136-6451b67ee559-metrics-certs\") pod \"maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4\" (UID: \"981991fe-cddc-46f4-9136-6451b67ee559\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4"
Sep 15 20:22:10.435899 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:10.435903    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gw8g9\" (UniqueName: \"kubernetes.io/projected/981991fe-cddc-46f4-9136-6451b67ee559-kube-api-access-gw8g9\") pod \"maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4\" (UID: \"981991fe-cddc-46f4-9136-6451b67ee559\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4"
Sep 15 20:22:10.436166 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:10.435975    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/981991fe-cddc-46f4-9136-6451b67ee559-openshift-service-ca\") pod \"maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4\" (UID: \"981991fe-cddc-46f4-9136-6451b67ee559\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4"
Sep 15 20:22:10.436166 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:10.436002    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/981991fe-cddc-46f4-9136-6451b67ee559-maas-api-tls\") pod \"maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4\" (UID: \"981991fe-cddc-46f4-9136-6451b67ee559\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4"
Sep 15 20:22:10.436754 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:10.436700    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/981991fe-cddc-46f4-9136-6451b67ee559-openshift-service-ca\") pod \"maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4\" (UID: \"981991fe-cddc-46f4-9136-6451b67ee559\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4"
Sep 15 20:22:10.438374 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:10.438353    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/981991fe-cddc-46f4-9136-6451b67ee559-maas-api-tls\") pod \"maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4\" (UID: \"981991fe-cddc-46f4-9136-6451b67ee559\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4"
Sep 15 20:22:10.438574 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:10.438552    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/981991fe-cddc-46f4-9136-6451b67ee559-metrics-certs\") pod \"maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4\" (UID: \"981991fe-cddc-46f4-9136-6451b67ee559\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4"
Sep 15 20:22:10.444914 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:10.444889    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gw8g9\" (UniqueName: \"kubernetes.io/projected/981991fe-cddc-46f4-9136-6451b67ee559-kube-api-access-gw8g9\") pod \"maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4\" (UID: \"981991fe-cddc-46f4-9136-6451b67ee559\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4"
Sep 15 20:22:10.561821 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:10.561736    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4"
Sep 15 20:22:10.845979 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:10.845946    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4"]
Sep 15 20:22:10.846946 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:22:10.846920    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod981991fe_cddc_46f4_9136_6451b67ee559.slice/crio-d5fb4415f9bfbb18f2b40ff3fbee85aa5df68dc5651e71e0a3f5dd6836f22a89 WatchSource:0}: Error finding container d5fb4415f9bfbb18f2b40ff3fbee85aa5df68dc5651e71e0a3f5dd6836f22a89: Status 404 returned error can't find the container with id d5fb4415f9bfbb18f2b40ff3fbee85aa5df68dc5651e71e0a3f5dd6836f22a89
Sep 15 20:22:11.441852 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:11.441820    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4" event={"ID":"981991fe-cddc-46f4-9136-6451b67ee559","Type":"ContainerStarted","Data":"b798e8717e02cef941544323079c7f5d4b13c352689b7f554e06d10ffe8fb1c0"}
Sep 15 20:22:11.442254 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:11.441882    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4" event={"ID":"981991fe-cddc-46f4-9136-6451b67ee559","Type":"ContainerStarted","Data":"d5fb4415f9bfbb18f2b40ff3fbee85aa5df68dc5651e71e0a3f5dd6836f22a89"}
Sep 15 20:22:11.442254 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:11.441913    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4"
Sep 15 20:22:11.462958 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:11.462906    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4" podStartSLOduration=0.952537129 podStartE2EDuration="1.46288931s" totalImagesPullingTime="510.352181ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:22:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:22:11.459574456 +0000 UTC m=+1037.850527453" watchObservedRunningTime="2026-09-15 20:22:11.46288931 +0000 UTC m=+1037.853842304"
Sep 15 20:22:17.454372 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:17.454340    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4"
Sep 15 20:22:40.444000 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:40.443961    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4"]
Sep 15 20:22:40.444563 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:40.444241    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4" podUID="981991fe-cddc-46f4-9136-6451b67ee559" containerName="maas-api" containerID="cri-o://b798e8717e02cef941544323079c7f5d4b13c352689b7f554e06d10ffe8fb1c0" gracePeriod=30
Sep 15 20:22:40.640695 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:40.640561    2577 generic.go:356] "Generic (PLEG): container finished" podID="981991fe-cddc-46f4-9136-6451b67ee559" containerID="b798e8717e02cef941544323079c7f5d4b13c352689b7f554e06d10ffe8fb1c0" exitCode=0
Sep 15 20:22:40.640695 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:40.640633    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4" event={"ID":"981991fe-cddc-46f4-9136-6451b67ee559","Type":"ContainerDied","Data":"b798e8717e02cef941544323079c7f5d4b13c352689b7f554e06d10ffe8fb1c0"}
Sep 15 20:22:40.705283 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:40.705214    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4"
Sep 15 20:22:40.749437 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:40.749393    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/981991fe-cddc-46f4-9136-6451b67ee559-metrics-certs\") pod \"981991fe-cddc-46f4-9136-6451b67ee559\" (UID: \"981991fe-cddc-46f4-9136-6451b67ee559\") "
Sep 15 20:22:40.749602 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:40.749456    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/981991fe-cddc-46f4-9136-6451b67ee559-openshift-service-ca\") pod \"981991fe-cddc-46f4-9136-6451b67ee559\" (UID: \"981991fe-cddc-46f4-9136-6451b67ee559\") "
Sep 15 20:22:40.749602 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:40.749553    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gw8g9\" (UniqueName: \"kubernetes.io/projected/981991fe-cddc-46f4-9136-6451b67ee559-kube-api-access-gw8g9\") pod \"981991fe-cddc-46f4-9136-6451b67ee559\" (UID: \"981991fe-cddc-46f4-9136-6451b67ee559\") "
Sep 15 20:22:40.749702 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:40.749630    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/981991fe-cddc-46f4-9136-6451b67ee559-maas-api-tls\") pod \"981991fe-cddc-46f4-9136-6451b67ee559\" (UID: \"981991fe-cddc-46f4-9136-6451b67ee559\") "
Sep 15 20:22:40.749895 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:40.749868    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/981991fe-cddc-46f4-9136-6451b67ee559-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "981991fe-cddc-46f4-9136-6451b67ee559" (UID: "981991fe-cddc-46f4-9136-6451b67ee559"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:22:40.750064 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:40.750040    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/981991fe-cddc-46f4-9136-6451b67ee559-openshift-service-ca\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:22:40.751690 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:40.751651    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/981991fe-cddc-46f4-9136-6451b67ee559-kube-api-access-gw8g9" (OuterVolumeSpecName: "kube-api-access-gw8g9") pod "981991fe-cddc-46f4-9136-6451b67ee559" (UID: "981991fe-cddc-46f4-9136-6451b67ee559"). InnerVolumeSpecName "kube-api-access-gw8g9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:22:40.751690 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:40.751662    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/981991fe-cddc-46f4-9136-6451b67ee559-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "981991fe-cddc-46f4-9136-6451b67ee559" (UID: "981991fe-cddc-46f4-9136-6451b67ee559"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:22:40.751886 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:40.751765    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/981991fe-cddc-46f4-9136-6451b67ee559-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "981991fe-cddc-46f4-9136-6451b67ee559" (UID: "981991fe-cddc-46f4-9136-6451b67ee559"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:22:40.851741 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:40.851683    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/981991fe-cddc-46f4-9136-6451b67ee559-maas-api-tls\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:22:40.851935 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:40.851753    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/981991fe-cddc-46f4-9136-6451b67ee559-metrics-certs\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:22:40.851935 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:40.851768    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gw8g9\" (UniqueName: \"kubernetes.io/projected/981991fe-cddc-46f4-9136-6451b67ee559-kube-api-access-gw8g9\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:22:41.650658 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:41.650625    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4"
Sep 15 20:22:41.651168 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:41.650631    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4" event={"ID":"981991fe-cddc-46f4-9136-6451b67ee559","Type":"ContainerDied","Data":"d5fb4415f9bfbb18f2b40ff3fbee85aa5df68dc5651e71e0a3f5dd6836f22a89"}
Sep 15 20:22:41.651168 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:41.650776    2577 scope.go:117] "RemoveContainer" containerID="b798e8717e02cef941544323079c7f5d4b13c352689b7f554e06d10ffe8fb1c0"
Sep 15 20:22:41.679435 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:41.679402    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4"]
Sep 15 20:22:41.685105 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:41.685077    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e0b5e00e-744bd59b88-9plg4"]
Sep 15 20:22:42.064447 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:42.064412    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="981991fe-cddc-46f4-9136-6451b67ee559" path="/var/lib/kubelet/pods/981991fe-cddc-46f4-9136-6451b67ee559/volumes"
Sep 15 20:22:46.045156 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:46.045120    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5"]
Sep 15 20:22:46.045683 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:46.045669    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="981991fe-cddc-46f4-9136-6451b67ee559" containerName="maas-api"
Sep 15 20:22:46.045741 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:46.045685    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="981991fe-cddc-46f4-9136-6451b67ee559" containerName="maas-api"
Sep 15 20:22:46.045788 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:46.045778    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="981991fe-cddc-46f4-9136-6451b67ee559" containerName="maas-api"
Sep 15 20:22:46.050346 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:46.050324    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5"
Sep 15 20:22:46.052975 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:46.052954    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-9baa771e-serving-cert\""
Sep 15 20:22:46.065287 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:46.065257    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5"]
Sep 15 20:22:46.109218 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:46.109163    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3316d490-26f7-4e04-9d0e-2cbfacc65940-openshift-service-ca\") pod \"maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5\" (UID: \"3316d490-26f7-4e04-9d0e-2cbfacc65940\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5"
Sep 15 20:22:46.109429 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:46.109251    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3316d490-26f7-4e04-9d0e-2cbfacc65940-maas-api-tls\") pod \"maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5\" (UID: \"3316d490-26f7-4e04-9d0e-2cbfacc65940\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5"
Sep 15 20:22:46.109429 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:46.109319    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3316d490-26f7-4e04-9d0e-2cbfacc65940-metrics-certs\") pod \"maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5\" (UID: \"3316d490-26f7-4e04-9d0e-2cbfacc65940\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5"
Sep 15 20:22:46.109429 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:46.109346    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t5pj\" (UniqueName: \"kubernetes.io/projected/3316d490-26f7-4e04-9d0e-2cbfacc65940-kube-api-access-6t5pj\") pod \"maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5\" (UID: \"3316d490-26f7-4e04-9d0e-2cbfacc65940\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5"
Sep 15 20:22:46.210352 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:46.210266    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3316d490-26f7-4e04-9d0e-2cbfacc65940-metrics-certs\") pod \"maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5\" (UID: \"3316d490-26f7-4e04-9d0e-2cbfacc65940\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5"
Sep 15 20:22:46.210352 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:46.210311    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6t5pj\" (UniqueName: \"kubernetes.io/projected/3316d490-26f7-4e04-9d0e-2cbfacc65940-kube-api-access-6t5pj\") pod \"maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5\" (UID: \"3316d490-26f7-4e04-9d0e-2cbfacc65940\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5"
Sep 15 20:22:46.210600 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:46.210385    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3316d490-26f7-4e04-9d0e-2cbfacc65940-openshift-service-ca\") pod \"maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5\" (UID: \"3316d490-26f7-4e04-9d0e-2cbfacc65940\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5"
Sep 15 20:22:46.210600 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:46.210431    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3316d490-26f7-4e04-9d0e-2cbfacc65940-maas-api-tls\") pod \"maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5\" (UID: \"3316d490-26f7-4e04-9d0e-2cbfacc65940\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5"
Sep 15 20:22:46.210600 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:22:46.210554    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-9baa771e-serving-cert: secret "maas-api-e2e-ait-9baa771e-serving-cert" not found
Sep 15 20:22:46.210799 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:22:46.210631    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3316d490-26f7-4e04-9d0e-2cbfacc65940-maas-api-tls podName:3316d490-26f7-4e04-9d0e-2cbfacc65940 nodeName:}" failed. No retries permitted until 2026-09-15 20:22:46.710609284 +0000 UTC m=+1073.101562276 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3316d490-26f7-4e04-9d0e-2cbfacc65940-maas-api-tls") pod "maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5" (UID: "3316d490-26f7-4e04-9d0e-2cbfacc65940") : secret "maas-api-e2e-ait-9baa771e-serving-cert" not found
Sep 15 20:22:46.211164 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:46.211144    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3316d490-26f7-4e04-9d0e-2cbfacc65940-openshift-service-ca\") pod \"maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5\" (UID: \"3316d490-26f7-4e04-9d0e-2cbfacc65940\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5"
Sep 15 20:22:46.213019 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:46.212992    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3316d490-26f7-4e04-9d0e-2cbfacc65940-metrics-certs\") pod \"maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5\" (UID: \"3316d490-26f7-4e04-9d0e-2cbfacc65940\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5"
Sep 15 20:22:46.221340 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:46.221315    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t5pj\" (UniqueName: \"kubernetes.io/projected/3316d490-26f7-4e04-9d0e-2cbfacc65940-kube-api-access-6t5pj\") pod \"maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5\" (UID: \"3316d490-26f7-4e04-9d0e-2cbfacc65940\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5"
Sep 15 20:22:46.716317 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:46.716276    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3316d490-26f7-4e04-9d0e-2cbfacc65940-maas-api-tls\") pod \"maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5\" (UID: \"3316d490-26f7-4e04-9d0e-2cbfacc65940\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5"
Sep 15 20:22:46.718706 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:46.718685    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3316d490-26f7-4e04-9d0e-2cbfacc65940-maas-api-tls\") pod \"maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5\" (UID: \"3316d490-26f7-4e04-9d0e-2cbfacc65940\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5"
Sep 15 20:22:46.965667 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:46.965632    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5"
Sep 15 20:22:47.249868 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:47.249840    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5"]
Sep 15 20:22:47.251401 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:22:47.251375    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3316d490_26f7_4e04_9d0e_2cbfacc65940.slice/crio-5e7292406de1ec45264effa2b724bd152a2dd31c563197f9ad64eef446a7d6ed WatchSource:0}: Error finding container 5e7292406de1ec45264effa2b724bd152a2dd31c563197f9ad64eef446a7d6ed: Status 404 returned error can't find the container with id 5e7292406de1ec45264effa2b724bd152a2dd31c563197f9ad64eef446a7d6ed
Sep 15 20:22:47.694171 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:47.694135    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5" event={"ID":"3316d490-26f7-4e04-9d0e-2cbfacc65940","Type":"ContainerStarted","Data":"5e7292406de1ec45264effa2b724bd152a2dd31c563197f9ad64eef446a7d6ed"}
Sep 15 20:22:48.702548 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:48.702505    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5" event={"ID":"3316d490-26f7-4e04-9d0e-2cbfacc65940","Type":"ContainerStarted","Data":"2b5a7f0311733b1a0420f0d704417788aec824a9cda9392d75ebaeb2bd1ac1a7"}
Sep 15 20:22:48.702969 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:48.702578    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5"
Sep 15 20:22:48.722837 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:48.722761    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5" podStartSLOduration=2.309538986 podStartE2EDuration="2.722739772s" totalImagesPullingTime="413.200786ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:22:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:22:48.720700729 +0000 UTC m=+1075.111653726" watchObservedRunningTime="2026-09-15 20:22:48.722739772 +0000 UTC m=+1075.113692766"
Sep 15 20:22:54.725003 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:22:54.724971    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5"
Sep 15 20:23:15.939838 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:15.938759    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5"]
Sep 15 20:23:15.939838 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:15.939091    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5" podUID="3316d490-26f7-4e04-9d0e-2cbfacc65940" containerName="maas-api" containerID="cri-o://2b5a7f0311733b1a0420f0d704417788aec824a9cda9392d75ebaeb2bd1ac1a7" gracePeriod=30
Sep 15 20:23:16.210221 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:16.210193    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5"
Sep 15 20:23:16.219262 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:16.219225    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3316d490-26f7-4e04-9d0e-2cbfacc65940-metrics-certs\") pod \"3316d490-26f7-4e04-9d0e-2cbfacc65940\" (UID: \"3316d490-26f7-4e04-9d0e-2cbfacc65940\") "
Sep 15 20:23:16.219449 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:16.219274    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3316d490-26f7-4e04-9d0e-2cbfacc65940-maas-api-tls\") pod \"3316d490-26f7-4e04-9d0e-2cbfacc65940\" (UID: \"3316d490-26f7-4e04-9d0e-2cbfacc65940\") "
Sep 15 20:23:16.219449 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:16.219367    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6t5pj\" (UniqueName: \"kubernetes.io/projected/3316d490-26f7-4e04-9d0e-2cbfacc65940-kube-api-access-6t5pj\") pod \"3316d490-26f7-4e04-9d0e-2cbfacc65940\" (UID: \"3316d490-26f7-4e04-9d0e-2cbfacc65940\") "
Sep 15 20:23:16.219579 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:16.219472    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3316d490-26f7-4e04-9d0e-2cbfacc65940-openshift-service-ca\") pod \"3316d490-26f7-4e04-9d0e-2cbfacc65940\" (UID: \"3316d490-26f7-4e04-9d0e-2cbfacc65940\") "
Sep 15 20:23:16.219911 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:16.219880    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3316d490-26f7-4e04-9d0e-2cbfacc65940-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3316d490-26f7-4e04-9d0e-2cbfacc65940" (UID: "3316d490-26f7-4e04-9d0e-2cbfacc65940"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:23:16.221919 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:16.221885    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3316d490-26f7-4e04-9d0e-2cbfacc65940-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3316d490-26f7-4e04-9d0e-2cbfacc65940" (UID: "3316d490-26f7-4e04-9d0e-2cbfacc65940"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:23:16.222030 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:16.221962    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3316d490-26f7-4e04-9d0e-2cbfacc65940-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3316d490-26f7-4e04-9d0e-2cbfacc65940" (UID: "3316d490-26f7-4e04-9d0e-2cbfacc65940"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:23:16.222030 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:16.221992    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3316d490-26f7-4e04-9d0e-2cbfacc65940-kube-api-access-6t5pj" (OuterVolumeSpecName: "kube-api-access-6t5pj") pod "3316d490-26f7-4e04-9d0e-2cbfacc65940" (UID: "3316d490-26f7-4e04-9d0e-2cbfacc65940"). InnerVolumeSpecName "kube-api-access-6t5pj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:23:16.321020 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:16.320949    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3316d490-26f7-4e04-9d0e-2cbfacc65940-openshift-service-ca\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:23:16.321020 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:16.320978    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3316d490-26f7-4e04-9d0e-2cbfacc65940-metrics-certs\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:23:16.321020 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:16.320987    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3316d490-26f7-4e04-9d0e-2cbfacc65940-maas-api-tls\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:23:16.321365 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:16.321030    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6t5pj\" (UniqueName: \"kubernetes.io/projected/3316d490-26f7-4e04-9d0e-2cbfacc65940-kube-api-access-6t5pj\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:23:16.904996 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:16.904962    2577 generic.go:356] "Generic (PLEG): container finished" podID="3316d490-26f7-4e04-9d0e-2cbfacc65940" containerID="2b5a7f0311733b1a0420f0d704417788aec824a9cda9392d75ebaeb2bd1ac1a7" exitCode=0
Sep 15 20:23:16.905155 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:16.905008    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5"
Sep 15 20:23:16.905155 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:16.905043    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5" event={"ID":"3316d490-26f7-4e04-9d0e-2cbfacc65940","Type":"ContainerDied","Data":"2b5a7f0311733b1a0420f0d704417788aec824a9cda9392d75ebaeb2bd1ac1a7"}
Sep 15 20:23:16.905155 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:16.905094    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5" event={"ID":"3316d490-26f7-4e04-9d0e-2cbfacc65940","Type":"ContainerDied","Data":"5e7292406de1ec45264effa2b724bd152a2dd31c563197f9ad64eef446a7d6ed"}
Sep 15 20:23:16.905155 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:16.905120    2577 scope.go:117] "RemoveContainer" containerID="2b5a7f0311733b1a0420f0d704417788aec824a9cda9392d75ebaeb2bd1ac1a7"
Sep 15 20:23:16.918205 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:16.918179    2577 scope.go:117] "RemoveContainer" containerID="2b5a7f0311733b1a0420f0d704417788aec824a9cda9392d75ebaeb2bd1ac1a7"
Sep 15 20:23:16.918564 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:23:16.918542    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b5a7f0311733b1a0420f0d704417788aec824a9cda9392d75ebaeb2bd1ac1a7\": container with ID starting with 2b5a7f0311733b1a0420f0d704417788aec824a9cda9392d75ebaeb2bd1ac1a7 not found: ID does not exist" containerID="2b5a7f0311733b1a0420f0d704417788aec824a9cda9392d75ebaeb2bd1ac1a7"
Sep 15 20:23:16.918621 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:16.918575    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b5a7f0311733b1a0420f0d704417788aec824a9cda9392d75ebaeb2bd1ac1a7"} err="failed to get container status \"2b5a7f0311733b1a0420f0d704417788aec824a9cda9392d75ebaeb2bd1ac1a7\": rpc error: code = NotFound desc = could not find container \"2b5a7f0311733b1a0420f0d704417788aec824a9cda9392d75ebaeb2bd1ac1a7\": container with ID starting with 2b5a7f0311733b1a0420f0d704417788aec824a9cda9392d75ebaeb2bd1ac1a7 not found: ID does not exist"
Sep 15 20:23:16.932320 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:16.932285    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5"]
Sep 15 20:23:16.936740 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:16.936705    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9baa771e-79954dd4f8-5hrp5"]
Sep 15 20:23:18.061674 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:18.061637    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3316d490-26f7-4e04-9d0e-2cbfacc65940" path="/var/lib/kubelet/pods/3316d490-26f7-4e04-9d0e-2cbfacc65940/volumes"
Sep 15 20:23:25.933609 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:25.933575    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2"]
Sep 15 20:23:25.934329 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:25.934306    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3316d490-26f7-4e04-9d0e-2cbfacc65940" containerName="maas-api"
Sep 15 20:23:25.934427 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:25.934332    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3316d490-26f7-4e04-9d0e-2cbfacc65940" containerName="maas-api"
Sep 15 20:23:25.934484 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:25.934459    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3316d490-26f7-4e04-9d0e-2cbfacc65940" containerName="maas-api"
Sep 15 20:23:25.937034 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:25.937014    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2"
Sep 15 20:23:25.939353 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:25.939329    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-93ab770e-serving-cert\""
Sep 15 20:23:25.950409 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:25.950382    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2"]
Sep 15 20:23:26.018172 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:26.018136    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c85519c-2753-48a9-b089-ed2c3dd4864f-metrics-certs\") pod \"maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2\" (UID: \"4c85519c-2753-48a9-b089-ed2c3dd4864f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2"
Sep 15 20:23:26.018345 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:26.018239    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c85519c-2753-48a9-b089-ed2c3dd4864f-maas-api-tls\") pod \"maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2\" (UID: \"4c85519c-2753-48a9-b089-ed2c3dd4864f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2"
Sep 15 20:23:26.018345 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:26.018275    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qxv8\" (UniqueName: \"kubernetes.io/projected/4c85519c-2753-48a9-b089-ed2c3dd4864f-kube-api-access-6qxv8\") pod \"maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2\" (UID: \"4c85519c-2753-48a9-b089-ed2c3dd4864f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2"
Sep 15 20:23:26.018426 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:26.018346    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c85519c-2753-48a9-b089-ed2c3dd4864f-openshift-service-ca\") pod \"maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2\" (UID: \"4c85519c-2753-48a9-b089-ed2c3dd4864f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2"
Sep 15 20:23:26.119664 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:26.119619    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c85519c-2753-48a9-b089-ed2c3dd4864f-openshift-service-ca\") pod \"maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2\" (UID: \"4c85519c-2753-48a9-b089-ed2c3dd4864f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2"
Sep 15 20:23:26.119897 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:26.119832    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c85519c-2753-48a9-b089-ed2c3dd4864f-metrics-certs\") pod \"maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2\" (UID: \"4c85519c-2753-48a9-b089-ed2c3dd4864f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2"
Sep 15 20:23:26.119969 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:26.119934    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c85519c-2753-48a9-b089-ed2c3dd4864f-maas-api-tls\") pod \"maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2\" (UID: \"4c85519c-2753-48a9-b089-ed2c3dd4864f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2"
Sep 15 20:23:26.120029 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:26.119990    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qxv8\" (UniqueName: \"kubernetes.io/projected/4c85519c-2753-48a9-b089-ed2c3dd4864f-kube-api-access-6qxv8\") pod \"maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2\" (UID: \"4c85519c-2753-48a9-b089-ed2c3dd4864f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2"
Sep 15 20:23:26.120086 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:23:26.120042    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-93ab770e-serving-cert: secret "maas-api-e2e-derive-93ab770e-serving-cert" not found
Sep 15 20:23:26.120150 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:23:26.120131    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c85519c-2753-48a9-b089-ed2c3dd4864f-maas-api-tls podName:4c85519c-2753-48a9-b089-ed2c3dd4864f nodeName:}" failed. No retries permitted until 2026-09-15 20:23:26.620112325 +0000 UTC m=+1113.011065326 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4c85519c-2753-48a9-b089-ed2c3dd4864f-maas-api-tls") pod "maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2" (UID: "4c85519c-2753-48a9-b089-ed2c3dd4864f") : secret "maas-api-e2e-derive-93ab770e-serving-cert" not found
Sep 15 20:23:26.120383 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:26.120361    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c85519c-2753-48a9-b089-ed2c3dd4864f-openshift-service-ca\") pod \"maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2\" (UID: \"4c85519c-2753-48a9-b089-ed2c3dd4864f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2"
Sep 15 20:23:26.122285 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:26.122254    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c85519c-2753-48a9-b089-ed2c3dd4864f-metrics-certs\") pod \"maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2\" (UID: \"4c85519c-2753-48a9-b089-ed2c3dd4864f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2"
Sep 15 20:23:26.131250 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:26.131225    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qxv8\" (UniqueName: \"kubernetes.io/projected/4c85519c-2753-48a9-b089-ed2c3dd4864f-kube-api-access-6qxv8\") pod \"maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2\" (UID: \"4c85519c-2753-48a9-b089-ed2c3dd4864f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2"
Sep 15 20:23:26.626301 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:26.626261    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c85519c-2753-48a9-b089-ed2c3dd4864f-maas-api-tls\") pod \"maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2\" (UID: \"4c85519c-2753-48a9-b089-ed2c3dd4864f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2"
Sep 15 20:23:26.628896 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:26.628870    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c85519c-2753-48a9-b089-ed2c3dd4864f-maas-api-tls\") pod \"maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2\" (UID: \"4c85519c-2753-48a9-b089-ed2c3dd4864f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2"
Sep 15 20:23:26.848938 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:26.848895    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2"
Sep 15 20:23:27.126637 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:27.126601    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2"]
Sep 15 20:23:27.127610 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:23:27.127582    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c85519c_2753_48a9_b089_ed2c3dd4864f.slice/crio-2879ad6aacbdd72338a38e2a690660a049cc7bf8e37038ec6c1279a5d22a13fa WatchSource:0}: Error finding container 2879ad6aacbdd72338a38e2a690660a049cc7bf8e37038ec6c1279a5d22a13fa: Status 404 returned error can't find the container with id 2879ad6aacbdd72338a38e2a690660a049cc7bf8e37038ec6c1279a5d22a13fa
Sep 15 20:23:27.994584 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:27.994554    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2" event={"ID":"4c85519c-2753-48a9-b089-ed2c3dd4864f","Type":"ContainerStarted","Data":"2879ad6aacbdd72338a38e2a690660a049cc7bf8e37038ec6c1279a5d22a13fa"}
Sep 15 20:23:29.004318 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:29.004278    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2" event={"ID":"4c85519c-2753-48a9-b089-ed2c3dd4864f","Type":"ContainerStarted","Data":"2d995dbf4fe0edb6e0ddff49629462a80b75cc9ce0007fc0a7bd5ea8d3d93b72"}
Sep 15 20:23:29.004817 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:29.004505    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2"
Sep 15 20:23:29.034468 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:29.034414    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2" podStartSLOduration=3.232615539 podStartE2EDuration="4.034399201s" totalImagesPullingTime="801.783662ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:23:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:23:29.033951665 +0000 UTC m=+1115.424904663" watchObservedRunningTime="2026-09-15 20:23:29.034399201 +0000 UTC m=+1115.425352198"
Sep 15 20:23:35.017714 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:35.017669    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2"
Sep 15 20:23:39.372540 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:39.372497    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx"]
Sep 15 20:23:39.373063 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:39.372851    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx" podUID="e00d30ef-3f6c-4841-8f7c-1c64c01f29c5" containerName="maas-api" containerID="cri-o://87b5ae2f0246ebafcd564c0fe30222ea619ea1bc570c60510e51e4264dcc57ec" gracePeriod=30
Sep 15 20:23:39.663906 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:39.663880    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx"
Sep 15 20:23:39.765507 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:39.765468    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e00d30ef-3f6c-4841-8f7c-1c64c01f29c5-metrics-certs\") pod \"e00d30ef-3f6c-4841-8f7c-1c64c01f29c5\" (UID: \"e00d30ef-3f6c-4841-8f7c-1c64c01f29c5\") "
Sep 15 20:23:39.765699 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:39.765609    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e00d30ef-3f6c-4841-8f7c-1c64c01f29c5-openshift-service-ca\") pod \"e00d30ef-3f6c-4841-8f7c-1c64c01f29c5\" (UID: \"e00d30ef-3f6c-4841-8f7c-1c64c01f29c5\") "
Sep 15 20:23:39.765699 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:39.765640    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kwh9m\" (UniqueName: \"kubernetes.io/projected/e00d30ef-3f6c-4841-8f7c-1c64c01f29c5-kube-api-access-kwh9m\") pod \"e00d30ef-3f6c-4841-8f7c-1c64c01f29c5\" (UID: \"e00d30ef-3f6c-4841-8f7c-1c64c01f29c5\") "
Sep 15 20:23:39.765699 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:39.765672    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e00d30ef-3f6c-4841-8f7c-1c64c01f29c5-maas-api-tls\") pod \"e00d30ef-3f6c-4841-8f7c-1c64c01f29c5\" (UID: \"e00d30ef-3f6c-4841-8f7c-1c64c01f29c5\") "
Sep 15 20:23:39.766020 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:39.765993    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e00d30ef-3f6c-4841-8f7c-1c64c01f29c5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e00d30ef-3f6c-4841-8f7c-1c64c01f29c5" (UID: "e00d30ef-3f6c-4841-8f7c-1c64c01f29c5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:23:39.766104 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:39.766073    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e00d30ef-3f6c-4841-8f7c-1c64c01f29c5-openshift-service-ca\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:23:39.768035 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:39.767979    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e00d30ef-3f6c-4841-8f7c-1c64c01f29c5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e00d30ef-3f6c-4841-8f7c-1c64c01f29c5" (UID: "e00d30ef-3f6c-4841-8f7c-1c64c01f29c5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:23:39.768680 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:39.768645    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e00d30ef-3f6c-4841-8f7c-1c64c01f29c5-kube-api-access-kwh9m" (OuterVolumeSpecName: "kube-api-access-kwh9m") pod "e00d30ef-3f6c-4841-8f7c-1c64c01f29c5" (UID: "e00d30ef-3f6c-4841-8f7c-1c64c01f29c5"). InnerVolumeSpecName "kube-api-access-kwh9m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:23:39.768807 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:39.768681    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e00d30ef-3f6c-4841-8f7c-1c64c01f29c5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e00d30ef-3f6c-4841-8f7c-1c64c01f29c5" (UID: "e00d30ef-3f6c-4841-8f7c-1c64c01f29c5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:23:39.867628 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:39.867588    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwh9m\" (UniqueName: \"kubernetes.io/projected/e00d30ef-3f6c-4841-8f7c-1c64c01f29c5-kube-api-access-kwh9m\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:23:39.867628 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:39.867625    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e00d30ef-3f6c-4841-8f7c-1c64c01f29c5-maas-api-tls\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:23:39.867628 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:39.867638    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e00d30ef-3f6c-4841-8f7c-1c64c01f29c5-metrics-certs\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:23:40.080876 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:40.080840    2577 generic.go:356] "Generic (PLEG): container finished" podID="e00d30ef-3f6c-4841-8f7c-1c64c01f29c5" containerID="87b5ae2f0246ebafcd564c0fe30222ea619ea1bc570c60510e51e4264dcc57ec" exitCode=0
Sep 15 20:23:40.081022 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:40.080890    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx"
Sep 15 20:23:40.081022 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:40.080932    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx" event={"ID":"e00d30ef-3f6c-4841-8f7c-1c64c01f29c5","Type":"ContainerDied","Data":"87b5ae2f0246ebafcd564c0fe30222ea619ea1bc570c60510e51e4264dcc57ec"}
Sep 15 20:23:40.081022 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:40.080990    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx" event={"ID":"e00d30ef-3f6c-4841-8f7c-1c64c01f29c5","Type":"ContainerDied","Data":"bfab7b17c277840d432be33b53b5cb2b2da2dcb96aa20e012407742c60890b81"}
Sep 15 20:23:40.081022 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:40.081017    2577 scope.go:117] "RemoveContainer" containerID="87b5ae2f0246ebafcd564c0fe30222ea619ea1bc570c60510e51e4264dcc57ec"
Sep 15 20:23:40.091340 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:40.091320    2577 scope.go:117] "RemoveContainer" containerID="87b5ae2f0246ebafcd564c0fe30222ea619ea1bc570c60510e51e4264dcc57ec"
Sep 15 20:23:40.091667 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:23:40.091646    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87b5ae2f0246ebafcd564c0fe30222ea619ea1bc570c60510e51e4264dcc57ec\": container with ID starting with 87b5ae2f0246ebafcd564c0fe30222ea619ea1bc570c60510e51e4264dcc57ec not found: ID does not exist" containerID="87b5ae2f0246ebafcd564c0fe30222ea619ea1bc570c60510e51e4264dcc57ec"
Sep 15 20:23:40.091758 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:40.091681    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87b5ae2f0246ebafcd564c0fe30222ea619ea1bc570c60510e51e4264dcc57ec"} err="failed to get container status \"87b5ae2f0246ebafcd564c0fe30222ea619ea1bc570c60510e51e4264dcc57ec\": rpc error: code = NotFound desc = could not find container \"87b5ae2f0246ebafcd564c0fe30222ea619ea1bc570c60510e51e4264dcc57ec\": container with ID starting with 87b5ae2f0246ebafcd564c0fe30222ea619ea1bc570c60510e51e4264dcc57ec not found: ID does not exist"
Sep 15 20:23:40.102590 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:40.102557    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx"]
Sep 15 20:23:40.106469 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:40.106431    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-30d5aa-5dc58d7c4-sjlzx"]
Sep 15 20:23:42.062303 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:42.062266    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e00d30ef-3f6c-4841-8f7c-1c64c01f29c5" path="/var/lib/kubelet/pods/e00d30ef-3f6c-4841-8f7c-1c64c01f29c5/volumes"
Sep 15 20:23:52.615897 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:52.615861    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2"]
Sep 15 20:23:52.616409 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:52.616099    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2" podUID="4c85519c-2753-48a9-b089-ed2c3dd4864f" containerName="maas-api" containerID="cri-o://2d995dbf4fe0edb6e0ddff49629462a80b75cc9ce0007fc0a7bd5ea8d3d93b72" gracePeriod=30
Sep 15 20:23:52.889665 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:52.889635    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2"
Sep 15 20:23:52.898614 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:52.898587    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qxv8\" (UniqueName: \"kubernetes.io/projected/4c85519c-2753-48a9-b089-ed2c3dd4864f-kube-api-access-6qxv8\") pod \"4c85519c-2753-48a9-b089-ed2c3dd4864f\" (UID: \"4c85519c-2753-48a9-b089-ed2c3dd4864f\") "
Sep 15 20:23:52.898781 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:52.898640    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c85519c-2753-48a9-b089-ed2c3dd4864f-metrics-certs\") pod \"4c85519c-2753-48a9-b089-ed2c3dd4864f\" (UID: \"4c85519c-2753-48a9-b089-ed2c3dd4864f\") "
Sep 15 20:23:52.898833 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:52.898779    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c85519c-2753-48a9-b089-ed2c3dd4864f-openshift-service-ca\") pod \"4c85519c-2753-48a9-b089-ed2c3dd4864f\" (UID: \"4c85519c-2753-48a9-b089-ed2c3dd4864f\") "
Sep 15 20:23:52.898899 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:52.898859    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c85519c-2753-48a9-b089-ed2c3dd4864f-maas-api-tls\") pod \"4c85519c-2753-48a9-b089-ed2c3dd4864f\" (UID: \"4c85519c-2753-48a9-b089-ed2c3dd4864f\") "
Sep 15 20:23:52.899145 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:52.899109    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c85519c-2753-48a9-b089-ed2c3dd4864f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4c85519c-2753-48a9-b089-ed2c3dd4864f" (UID: "4c85519c-2753-48a9-b089-ed2c3dd4864f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:23:52.899282 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:52.899263    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c85519c-2753-48a9-b089-ed2c3dd4864f-openshift-service-ca\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:23:52.900900 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:52.900873    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c85519c-2753-48a9-b089-ed2c3dd4864f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4c85519c-2753-48a9-b089-ed2c3dd4864f" (UID: "4c85519c-2753-48a9-b089-ed2c3dd4864f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:23:52.901010 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:52.900917    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c85519c-2753-48a9-b089-ed2c3dd4864f-kube-api-access-6qxv8" (OuterVolumeSpecName: "kube-api-access-6qxv8") pod "4c85519c-2753-48a9-b089-ed2c3dd4864f" (UID: "4c85519c-2753-48a9-b089-ed2c3dd4864f"). InnerVolumeSpecName "kube-api-access-6qxv8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:23:52.901310 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:52.901280    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c85519c-2753-48a9-b089-ed2c3dd4864f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4c85519c-2753-48a9-b089-ed2c3dd4864f" (UID: "4c85519c-2753-48a9-b089-ed2c3dd4864f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:23:53.000843 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:53.000806    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c85519c-2753-48a9-b089-ed2c3dd4864f-maas-api-tls\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:23:53.000843 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:53.000838    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qxv8\" (UniqueName: \"kubernetes.io/projected/4c85519c-2753-48a9-b089-ed2c3dd4864f-kube-api-access-6qxv8\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:23:53.000843 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:53.000848    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c85519c-2753-48a9-b089-ed2c3dd4864f-metrics-certs\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:23:53.173753 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:53.173635    2577 generic.go:356] "Generic (PLEG): container finished" podID="4c85519c-2753-48a9-b089-ed2c3dd4864f" containerID="2d995dbf4fe0edb6e0ddff49629462a80b75cc9ce0007fc0a7bd5ea8d3d93b72" exitCode=0
Sep 15 20:23:53.173753 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:53.173690    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2"
Sep 15 20:23:53.173986 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:53.173897    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2" event={"ID":"4c85519c-2753-48a9-b089-ed2c3dd4864f","Type":"ContainerDied","Data":"2d995dbf4fe0edb6e0ddff49629462a80b75cc9ce0007fc0a7bd5ea8d3d93b72"}
Sep 15 20:23:53.173986 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:53.173953    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2" event={"ID":"4c85519c-2753-48a9-b089-ed2c3dd4864f","Type":"ContainerDied","Data":"2879ad6aacbdd72338a38e2a690660a049cc7bf8e37038ec6c1279a5d22a13fa"}
Sep 15 20:23:53.174091 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:53.173986    2577 scope.go:117] "RemoveContainer" containerID="2d995dbf4fe0edb6e0ddff49629462a80b75cc9ce0007fc0a7bd5ea8d3d93b72"
Sep 15 20:23:53.187880 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:53.187855    2577 scope.go:117] "RemoveContainer" containerID="2d995dbf4fe0edb6e0ddff49629462a80b75cc9ce0007fc0a7bd5ea8d3d93b72"
Sep 15 20:23:53.188236 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:23:53.188210    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d995dbf4fe0edb6e0ddff49629462a80b75cc9ce0007fc0a7bd5ea8d3d93b72\": container with ID starting with 2d995dbf4fe0edb6e0ddff49629462a80b75cc9ce0007fc0a7bd5ea8d3d93b72 not found: ID does not exist" containerID="2d995dbf4fe0edb6e0ddff49629462a80b75cc9ce0007fc0a7bd5ea8d3d93b72"
Sep 15 20:23:53.188322 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:53.188248    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d995dbf4fe0edb6e0ddff49629462a80b75cc9ce0007fc0a7bd5ea8d3d93b72"} err="failed to get container status \"2d995dbf4fe0edb6e0ddff49629462a80b75cc9ce0007fc0a7bd5ea8d3d93b72\": rpc error: code = NotFound desc = could not find container \"2d995dbf4fe0edb6e0ddff49629462a80b75cc9ce0007fc0a7bd5ea8d3d93b72\": container with ID starting with 2d995dbf4fe0edb6e0ddff49629462a80b75cc9ce0007fc0a7bd5ea8d3d93b72 not found: ID does not exist"
Sep 15 20:23:53.200028 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:53.200002    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2"]
Sep 15 20:23:53.203644 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:53.203616    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-93ab770e-795bd6cd74-x9xl2"]
Sep 15 20:23:54.062584 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:23:54.062537    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c85519c-2753-48a9-b089-ed2c3dd4864f" path="/var/lib/kubelet/pods/4c85519c-2753-48a9-b089-ed2c3dd4864f/volumes"
Sep 15 20:24:28.852654 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:24:28.852569    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc"]
Sep 15 20:24:28.853339 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:24:28.853317    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e00d30ef-3f6c-4841-8f7c-1c64c01f29c5" containerName="maas-api"
Sep 15 20:24:28.853426 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:24:28.853342    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e00d30ef-3f6c-4841-8f7c-1c64c01f29c5" containerName="maas-api"
Sep 15 20:24:28.853426 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:24:28.853378    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c85519c-2753-48a9-b089-ed2c3dd4864f" containerName="maas-api"
Sep 15 20:24:28.853426 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:24:28.853388    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c85519c-2753-48a9-b089-ed2c3dd4864f" containerName="maas-api"
Sep 15 20:24:28.853580 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:24:28.853494    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c85519c-2753-48a9-b089-ed2c3dd4864f" containerName="maas-api"
Sep 15 20:24:28.853580 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:24:28.853513    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e00d30ef-3f6c-4841-8f7c-1c64c01f29c5" containerName="maas-api"
Sep 15 20:24:28.856974 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:24:28.856949    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc"
Sep 15 20:24:28.859420 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:24:28.859393    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-5c37e0-serving-cert\""
Sep 15 20:24:28.859608 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:24:28.859586    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fe63e4b-01c8-480f-8bdc-f1d416b253ee-maas-api-tls\") pod \"maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc\" (UID: \"5fe63e4b-01c8-480f-8bdc-f1d416b253ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc"
Sep 15 20:24:28.859702 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:24:28.859675    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4rcc\" (UniqueName: \"kubernetes.io/projected/5fe63e4b-01c8-480f-8bdc-f1d416b253ee-kube-api-access-k4rcc\") pod \"maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc\" (UID: \"5fe63e4b-01c8-480f-8bdc-f1d416b253ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc"
Sep 15 20:24:28.859822 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:24:28.859716    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fe63e4b-01c8-480f-8bdc-f1d416b253ee-openshift-service-ca\") pod \"maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc\" (UID: \"5fe63e4b-01c8-480f-8bdc-f1d416b253ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc"
Sep 15 20:24:28.859887 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:24:28.859846    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fe63e4b-01c8-480f-8bdc-f1d416b253ee-metrics-certs\") pod \"maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc\" (UID: \"5fe63e4b-01c8-480f-8bdc-f1d416b253ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc"
Sep 15 20:24:28.868424 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:24:28.868388    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc"]
Sep 15 20:24:28.960606 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:24:28.960571    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fe63e4b-01c8-480f-8bdc-f1d416b253ee-metrics-certs\") pod \"maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc\" (UID: \"5fe63e4b-01c8-480f-8bdc-f1d416b253ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc"
Sep 15 20:24:28.960829 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:24:28.960618    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fe63e4b-01c8-480f-8bdc-f1d416b253ee-maas-api-tls\") pod \"maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc\" (UID: \"5fe63e4b-01c8-480f-8bdc-f1d416b253ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc"
Sep 15 20:24:28.960829 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:24:28.960663    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4rcc\" (UniqueName: \"kubernetes.io/projected/5fe63e4b-01c8-480f-8bdc-f1d416b253ee-kube-api-access-k4rcc\") pod \"maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc\" (UID: \"5fe63e4b-01c8-480f-8bdc-f1d416b253ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc"
Sep 15 20:24:28.960829 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:24:28.960688    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fe63e4b-01c8-480f-8bdc-f1d416b253ee-openshift-service-ca\") pod \"maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc\" (UID: \"5fe63e4b-01c8-480f-8bdc-f1d416b253ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc"
Sep 15 20:24:28.960829 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:24:28.960803    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-5c37e0-serving-cert: secret "maas-api-e2e-explicit-5c37e0-serving-cert" not found
Sep 15 20:24:28.961138 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:24:28.960899    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fe63e4b-01c8-480f-8bdc-f1d416b253ee-maas-api-tls podName:5fe63e4b-01c8-480f-8bdc-f1d416b253ee nodeName:}" failed. No retries permitted until 2026-09-15 20:24:29.460875427 +0000 UTC m=+1175.851828420 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5fe63e4b-01c8-480f-8bdc-f1d416b253ee-maas-api-tls") pod "maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc" (UID: "5fe63e4b-01c8-480f-8bdc-f1d416b253ee") : secret "maas-api-e2e-explicit-5c37e0-serving-cert" not found
Sep 15 20:24:28.961325 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:24:28.961305    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fe63e4b-01c8-480f-8bdc-f1d416b253ee-openshift-service-ca\") pod \"maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc\" (UID: \"5fe63e4b-01c8-480f-8bdc-f1d416b253ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc"
Sep 15 20:24:28.963301 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:24:28.963278    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fe63e4b-01c8-480f-8bdc-f1d416b253ee-metrics-certs\") pod \"maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc\" (UID: \"5fe63e4b-01c8-480f-8bdc-f1d416b253ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc"
Sep 15 20:24:28.969186 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:24:28.969163    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4rcc\" (UniqueName: \"kubernetes.io/projected/5fe63e4b-01c8-480f-8bdc-f1d416b253ee-kube-api-access-k4rcc\") pod \"maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc\" (UID: \"5fe63e4b-01c8-480f-8bdc-f1d416b253ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc"
Sep 15 20:24:29.465073 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:24:29.465038    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fe63e4b-01c8-480f-8bdc-f1d416b253ee-maas-api-tls\") pod \"maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc\" (UID: \"5fe63e4b-01c8-480f-8bdc-f1d416b253ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc"
Sep 15 20:24:29.467917 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:24:29.467890    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fe63e4b-01c8-480f-8bdc-f1d416b253ee-maas-api-tls\") pod \"maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc\" (UID: \"5fe63e4b-01c8-480f-8bdc-f1d416b253ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc"
Sep 15 20:24:29.469575 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:24:29.469551    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc"
Sep 15 20:24:29.762275 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:24:29.762251    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc"]
Sep 15 20:24:29.763635 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:24:29.763611    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fe63e4b_01c8_480f_8bdc_f1d416b253ee.slice/crio-9d315394835f948dfbfe2906ec46326a0c0201715c78179fd414524c22b324b0 WatchSource:0}: Error finding container 9d315394835f948dfbfe2906ec46326a0c0201715c78179fd414524c22b324b0: Status 404 returned error can't find the container with id 9d315394835f948dfbfe2906ec46326a0c0201715c78179fd414524c22b324b0
Sep 15 20:24:30.424980 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:24:30.424887    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc" event={"ID":"5fe63e4b-01c8-480f-8bdc-f1d416b253ee","Type":"ContainerStarted","Data":"d8958b22889e91ee0ab703531f6d74f9a7c18eb07450d07e9f6b2919e36f69cc"}
Sep 15 20:24:30.424980 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:24:30.424941    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc" event={"ID":"5fe63e4b-01c8-480f-8bdc-f1d416b253ee","Type":"ContainerStarted","Data":"9d315394835f948dfbfe2906ec46326a0c0201715c78179fd414524c22b324b0"}
Sep 15 20:24:30.424980 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:24:30.424961    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc"
Sep 15 20:24:30.443195 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:24:30.443144    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc" podStartSLOduration=2.044415503 podStartE2EDuration="2.443128087s" totalImagesPullingTime="398.712584ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:24:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:24:30.441590795 +0000 UTC m=+1176.832543783" watchObservedRunningTime="2026-09-15 20:24:30.443128087 +0000 UTC m=+1176.834081120"
Sep 15 20:24:36.436335 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:24:36.436306    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc"
Sep 15 20:24:54.134751 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:24:54.134707    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-r227b_ecab5774-79a4-42eb-9bb5-a1f09b18f3fb/console-operator/1.log"
Sep 15 20:24:54.137985 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:24:54.137958    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-r227b_ecab5774-79a4-42eb-9bb5-a1f09b18f3fb/console-operator/1.log"
Sep 15 20:25:40.046674 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:25:40.046632    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w"]
Sep 15 20:25:40.050759 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:25:40.050732    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w"
Sep 15 20:25:40.053999 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:25:40.053749    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-dad434fa-serving-cert\""
Sep 15 20:25:40.066459 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:25:40.066431    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w"]
Sep 15 20:25:40.197027 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:25:40.196989    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3abc9e19-0959-427b-9935-1a31a15c147d-openshift-service-ca\") pod \"maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w\" (UID: \"3abc9e19-0959-427b-9935-1a31a15c147d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w"
Sep 15 20:25:40.197027 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:25:40.197028    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pn5qc\" (UniqueName: \"kubernetes.io/projected/3abc9e19-0959-427b-9935-1a31a15c147d-kube-api-access-pn5qc\") pod \"maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w\" (UID: \"3abc9e19-0959-427b-9935-1a31a15c147d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w"
Sep 15 20:25:40.197277 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:25:40.197065    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3abc9e19-0959-427b-9935-1a31a15c147d-maas-api-tls\") pod \"maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w\" (UID: \"3abc9e19-0959-427b-9935-1a31a15c147d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w"
Sep 15 20:25:40.197277 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:25:40.197179    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3abc9e19-0959-427b-9935-1a31a15c147d-metrics-certs\") pod \"maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w\" (UID: \"3abc9e19-0959-427b-9935-1a31a15c147d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w"
Sep 15 20:25:40.298806 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:25:40.298695    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3abc9e19-0959-427b-9935-1a31a15c147d-openshift-service-ca\") pod \"maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w\" (UID: \"3abc9e19-0959-427b-9935-1a31a15c147d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w"
Sep 15 20:25:40.298806 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:25:40.298768    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pn5qc\" (UniqueName: \"kubernetes.io/projected/3abc9e19-0959-427b-9935-1a31a15c147d-kube-api-access-pn5qc\") pod \"maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w\" (UID: \"3abc9e19-0959-427b-9935-1a31a15c147d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w"
Sep 15 20:25:40.298806 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:25:40.298808    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3abc9e19-0959-427b-9935-1a31a15c147d-maas-api-tls\") pod \"maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w\" (UID: \"3abc9e19-0959-427b-9935-1a31a15c147d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w"
Sep 15 20:25:40.299118 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:25:40.298856    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3abc9e19-0959-427b-9935-1a31a15c147d-metrics-certs\") pod \"maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w\" (UID: \"3abc9e19-0959-427b-9935-1a31a15c147d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w"
Sep 15 20:25:40.299483 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:25:40.299453    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3abc9e19-0959-427b-9935-1a31a15c147d-openshift-service-ca\") pod \"maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w\" (UID: \"3abc9e19-0959-427b-9935-1a31a15c147d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w"
Sep 15 20:25:40.301446 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:25:40.301424    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3abc9e19-0959-427b-9935-1a31a15c147d-metrics-certs\") pod \"maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w\" (UID: \"3abc9e19-0959-427b-9935-1a31a15c147d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w"
Sep 15 20:25:40.301558 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:25:40.301534    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3abc9e19-0959-427b-9935-1a31a15c147d-maas-api-tls\") pod \"maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w\" (UID: \"3abc9e19-0959-427b-9935-1a31a15c147d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w"
Sep 15 20:25:40.309158 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:25:40.309115    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pn5qc\" (UniqueName: \"kubernetes.io/projected/3abc9e19-0959-427b-9935-1a31a15c147d-kube-api-access-pn5qc\") pod \"maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w\" (UID: \"3abc9e19-0959-427b-9935-1a31a15c147d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w"
Sep 15 20:25:40.366639 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:25:40.366601    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w"
Sep 15 20:25:40.650773 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:25:40.650741    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w"]
Sep 15 20:25:40.653317 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:25:40.653287    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3abc9e19_0959_427b_9935_1a31a15c147d.slice/crio-6b5b12d98d05e5b9103eed3d6081cc98e4ddf77bbe2f2896640c5282e072c0f0 WatchSource:0}: Error finding container 6b5b12d98d05e5b9103eed3d6081cc98e4ddf77bbe2f2896640c5282e072c0f0: Status 404 returned error can't find the container with id 6b5b12d98d05e5b9103eed3d6081cc98e4ddf77bbe2f2896640c5282e072c0f0
Sep 15 20:25:40.901232 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:25:40.901151    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w" event={"ID":"3abc9e19-0959-427b-9935-1a31a15c147d","Type":"ContainerStarted","Data":"6b5b12d98d05e5b9103eed3d6081cc98e4ddf77bbe2f2896640c5282e072c0f0"}
Sep 15 20:25:41.909948 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:25:41.909913    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w" event={"ID":"3abc9e19-0959-427b-9935-1a31a15c147d","Type":"ContainerStarted","Data":"7093a3948780c39c0423be39752a3423c83faa218bcae60ce9f26442217b23e6"}
Sep 15 20:25:41.910322 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:25:41.909974    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w"
Sep 15 20:25:41.934615 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:25:41.934562    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w" podStartSLOduration=1.438452969 podStartE2EDuration="1.93454523s" totalImagesPullingTime="496.092261ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:25:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:25:41.931965021 +0000 UTC m=+1248.322918017" watchObservedRunningTime="2026-09-15 20:25:41.93454523 +0000 UTC m=+1248.325498267"
Sep 15 20:25:47.921614 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:25:47.921532    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w"
Sep 15 20:26:24.405292 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:24.405253    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w"]
Sep 15 20:26:24.406061 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:24.405486    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w" podUID="3abc9e19-0959-427b-9935-1a31a15c147d" containerName="maas-api" containerID="cri-o://7093a3948780c39c0423be39752a3423c83faa218bcae60ce9f26442217b23e6" gracePeriod=30
Sep 15 20:26:24.677107 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:24.677075    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w"
Sep 15 20:26:24.747696 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:24.747658    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3abc9e19-0959-427b-9935-1a31a15c147d-maas-api-tls\") pod \"3abc9e19-0959-427b-9935-1a31a15c147d\" (UID: \"3abc9e19-0959-427b-9935-1a31a15c147d\") "
Sep 15 20:26:24.747884 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:24.747801    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3abc9e19-0959-427b-9935-1a31a15c147d-metrics-certs\") pod \"3abc9e19-0959-427b-9935-1a31a15c147d\" (UID: \"3abc9e19-0959-427b-9935-1a31a15c147d\") "
Sep 15 20:26:24.747884 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:24.747847    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3abc9e19-0959-427b-9935-1a31a15c147d-openshift-service-ca\") pod \"3abc9e19-0959-427b-9935-1a31a15c147d\" (UID: \"3abc9e19-0959-427b-9935-1a31a15c147d\") "
Sep 15 20:26:24.747960 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:24.747938    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pn5qc\" (UniqueName: \"kubernetes.io/projected/3abc9e19-0959-427b-9935-1a31a15c147d-kube-api-access-pn5qc\") pod \"3abc9e19-0959-427b-9935-1a31a15c147d\" (UID: \"3abc9e19-0959-427b-9935-1a31a15c147d\") "
Sep 15 20:26:24.748691 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:24.748662    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3abc9e19-0959-427b-9935-1a31a15c147d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3abc9e19-0959-427b-9935-1a31a15c147d" (UID: "3abc9e19-0959-427b-9935-1a31a15c147d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:26:24.750884 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:24.750850    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3abc9e19-0959-427b-9935-1a31a15c147d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3abc9e19-0959-427b-9935-1a31a15c147d" (UID: "3abc9e19-0959-427b-9935-1a31a15c147d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:26:24.750884 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:24.750868    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3abc9e19-0959-427b-9935-1a31a15c147d-kube-api-access-pn5qc" (OuterVolumeSpecName: "kube-api-access-pn5qc") pod "3abc9e19-0959-427b-9935-1a31a15c147d" (UID: "3abc9e19-0959-427b-9935-1a31a15c147d"). InnerVolumeSpecName "kube-api-access-pn5qc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:26:24.751015 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:24.750946    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3abc9e19-0959-427b-9935-1a31a15c147d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3abc9e19-0959-427b-9935-1a31a15c147d" (UID: "3abc9e19-0959-427b-9935-1a31a15c147d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:26:24.849302 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:24.849259    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3abc9e19-0959-427b-9935-1a31a15c147d-metrics-certs\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:26:24.849302 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:24.849296    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3abc9e19-0959-427b-9935-1a31a15c147d-openshift-service-ca\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:26:24.849302 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:24.849306    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pn5qc\" (UniqueName: \"kubernetes.io/projected/3abc9e19-0959-427b-9935-1a31a15c147d-kube-api-access-pn5qc\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:26:24.849556 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:24.849315    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3abc9e19-0959-427b-9935-1a31a15c147d-maas-api-tls\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:26:25.203576 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:25.203532    2577 generic.go:356] "Generic (PLEG): container finished" podID="3abc9e19-0959-427b-9935-1a31a15c147d" containerID="7093a3948780c39c0423be39752a3423c83faa218bcae60ce9f26442217b23e6" exitCode=0
Sep 15 20:26:25.203576 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:25.203583    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w"
Sep 15 20:26:25.203894 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:25.203617    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w" event={"ID":"3abc9e19-0959-427b-9935-1a31a15c147d","Type":"ContainerDied","Data":"7093a3948780c39c0423be39752a3423c83faa218bcae60ce9f26442217b23e6"}
Sep 15 20:26:25.203894 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:25.203674    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w" event={"ID":"3abc9e19-0959-427b-9935-1a31a15c147d","Type":"ContainerDied","Data":"6b5b12d98d05e5b9103eed3d6081cc98e4ddf77bbe2f2896640c5282e072c0f0"}
Sep 15 20:26:25.203894 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:25.203700    2577 scope.go:117] "RemoveContainer" containerID="7093a3948780c39c0423be39752a3423c83faa218bcae60ce9f26442217b23e6"
Sep 15 20:26:25.213942 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:25.213890    2577 scope.go:117] "RemoveContainer" containerID="7093a3948780c39c0423be39752a3423c83faa218bcae60ce9f26442217b23e6"
Sep 15 20:26:25.214227 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:26:25.214203    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7093a3948780c39c0423be39752a3423c83faa218bcae60ce9f26442217b23e6\": container with ID starting with 7093a3948780c39c0423be39752a3423c83faa218bcae60ce9f26442217b23e6 not found: ID does not exist" containerID="7093a3948780c39c0423be39752a3423c83faa218bcae60ce9f26442217b23e6"
Sep 15 20:26:25.214294 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:25.214240    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7093a3948780c39c0423be39752a3423c83faa218bcae60ce9f26442217b23e6"} err="failed to get container status \"7093a3948780c39c0423be39752a3423c83faa218bcae60ce9f26442217b23e6\": rpc error: code = NotFound desc = could not find container \"7093a3948780c39c0423be39752a3423c83faa218bcae60ce9f26442217b23e6\": container with ID starting with 7093a3948780c39c0423be39752a3423c83faa218bcae60ce9f26442217b23e6 not found: ID does not exist"
Sep 15 20:26:25.228107 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:25.228076    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w"]
Sep 15 20:26:25.237668 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:25.233490    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dad434fa-7d798798f7-7wh4w"]
Sep 15 20:26:26.061111 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:26.061074    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3abc9e19-0959-427b-9935-1a31a15c147d" path="/var/lib/kubelet/pods/3abc9e19-0959-427b-9935-1a31a15c147d/volumes"
Sep 15 20:26:34.360300 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:34.360262    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc"]
Sep 15 20:26:34.360819 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:34.360527    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc" podUID="5fe63e4b-01c8-480f-8bdc-f1d416b253ee" containerName="maas-api" containerID="cri-o://d8958b22889e91ee0ab703531f6d74f9a7c18eb07450d07e9f6b2919e36f69cc" gracePeriod=30
Sep 15 20:26:34.640507 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:34.640481    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc"
Sep 15 20:26:34.753203 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:34.753162    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fe63e4b-01c8-480f-8bdc-f1d416b253ee-maas-api-tls\") pod \"5fe63e4b-01c8-480f-8bdc-f1d416b253ee\" (UID: \"5fe63e4b-01c8-480f-8bdc-f1d416b253ee\") "
Sep 15 20:26:34.753203 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:34.753206    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fe63e4b-01c8-480f-8bdc-f1d416b253ee-metrics-certs\") pod \"5fe63e4b-01c8-480f-8bdc-f1d416b253ee\" (UID: \"5fe63e4b-01c8-480f-8bdc-f1d416b253ee\") "
Sep 15 20:26:34.753460 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:34.753290    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4rcc\" (UniqueName: \"kubernetes.io/projected/5fe63e4b-01c8-480f-8bdc-f1d416b253ee-kube-api-access-k4rcc\") pod \"5fe63e4b-01c8-480f-8bdc-f1d416b253ee\" (UID: \"5fe63e4b-01c8-480f-8bdc-f1d416b253ee\") "
Sep 15 20:26:34.753460 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:34.753321    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fe63e4b-01c8-480f-8bdc-f1d416b253ee-openshift-service-ca\") pod \"5fe63e4b-01c8-480f-8bdc-f1d416b253ee\" (UID: \"5fe63e4b-01c8-480f-8bdc-f1d416b253ee\") "
Sep 15 20:26:34.753803 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:34.753773    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5fe63e4b-01c8-480f-8bdc-f1d416b253ee-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5fe63e4b-01c8-480f-8bdc-f1d416b253ee" (UID: "5fe63e4b-01c8-480f-8bdc-f1d416b253ee"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:26:34.755448 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:34.755420    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fe63e4b-01c8-480f-8bdc-f1d416b253ee-kube-api-access-k4rcc" (OuterVolumeSpecName: "kube-api-access-k4rcc") pod "5fe63e4b-01c8-480f-8bdc-f1d416b253ee" (UID: "5fe63e4b-01c8-480f-8bdc-f1d416b253ee"). InnerVolumeSpecName "kube-api-access-k4rcc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:26:34.755561 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:34.755451    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5fe63e4b-01c8-480f-8bdc-f1d416b253ee-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5fe63e4b-01c8-480f-8bdc-f1d416b253ee" (UID: "5fe63e4b-01c8-480f-8bdc-f1d416b253ee"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:26:34.755700 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:34.755682    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5fe63e4b-01c8-480f-8bdc-f1d416b253ee-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5fe63e4b-01c8-480f-8bdc-f1d416b253ee" (UID: "5fe63e4b-01c8-480f-8bdc-f1d416b253ee"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:26:34.855210 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:34.855168    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fe63e4b-01c8-480f-8bdc-f1d416b253ee-maas-api-tls\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:26:34.855210 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:34.855203    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fe63e4b-01c8-480f-8bdc-f1d416b253ee-metrics-certs\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:26:34.855210 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:34.855216    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4rcc\" (UniqueName: \"kubernetes.io/projected/5fe63e4b-01c8-480f-8bdc-f1d416b253ee-kube-api-access-k4rcc\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:26:34.855457 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:34.855229    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fe63e4b-01c8-480f-8bdc-f1d416b253ee-openshift-service-ca\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:26:35.270783 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:35.270746    2577 generic.go:356] "Generic (PLEG): container finished" podID="5fe63e4b-01c8-480f-8bdc-f1d416b253ee" containerID="d8958b22889e91ee0ab703531f6d74f9a7c18eb07450d07e9f6b2919e36f69cc" exitCode=0
Sep 15 20:26:35.270783 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:35.270782    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc"
Sep 15 20:26:35.271052 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:35.270792    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc" event={"ID":"5fe63e4b-01c8-480f-8bdc-f1d416b253ee","Type":"ContainerDied","Data":"d8958b22889e91ee0ab703531f6d74f9a7c18eb07450d07e9f6b2919e36f69cc"}
Sep 15 20:26:35.271052 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:35.270846    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc" event={"ID":"5fe63e4b-01c8-480f-8bdc-f1d416b253ee","Type":"ContainerDied","Data":"9d315394835f948dfbfe2906ec46326a0c0201715c78179fd414524c22b324b0"}
Sep 15 20:26:35.271052 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:35.270879    2577 scope.go:117] "RemoveContainer" containerID="d8958b22889e91ee0ab703531f6d74f9a7c18eb07450d07e9f6b2919e36f69cc"
Sep 15 20:26:35.282060 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:35.282038    2577 scope.go:117] "RemoveContainer" containerID="d8958b22889e91ee0ab703531f6d74f9a7c18eb07450d07e9f6b2919e36f69cc"
Sep 15 20:26:35.282388 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:26:35.282366    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8958b22889e91ee0ab703531f6d74f9a7c18eb07450d07e9f6b2919e36f69cc\": container with ID starting with d8958b22889e91ee0ab703531f6d74f9a7c18eb07450d07e9f6b2919e36f69cc not found: ID does not exist" containerID="d8958b22889e91ee0ab703531f6d74f9a7c18eb07450d07e9f6b2919e36f69cc"
Sep 15 20:26:35.282453 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:35.282401    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8958b22889e91ee0ab703531f6d74f9a7c18eb07450d07e9f6b2919e36f69cc"} err="failed to get container status \"d8958b22889e91ee0ab703531f6d74f9a7c18eb07450d07e9f6b2919e36f69cc\": rpc error: code = NotFound desc = could not find container \"d8958b22889e91ee0ab703531f6d74f9a7c18eb07450d07e9f6b2919e36f69cc\": container with ID starting with d8958b22889e91ee0ab703531f6d74f9a7c18eb07450d07e9f6b2919e36f69cc not found: ID does not exist"
Sep 15 20:26:35.294257 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:35.294222    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc"]
Sep 15 20:26:35.298559 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:35.298531    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-5c37e0-7dcdf6576b-s6cmc"]
Sep 15 20:26:36.062272 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:36.062236    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fe63e4b-01c8-480f-8bdc-f1d416b253ee" path="/var/lib/kubelet/pods/5fe63e4b-01c8-480f-8bdc-f1d416b253ee/volumes"
Sep 15 20:26:51.225645 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:51.225608    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s"]
Sep 15 20:26:51.227914 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:51.226091    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3abc9e19-0959-427b-9935-1a31a15c147d" containerName="maas-api"
Sep 15 20:26:51.227914 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:51.226104    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3abc9e19-0959-427b-9935-1a31a15c147d" containerName="maas-api"
Sep 15 20:26:51.227914 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:51.226130    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fe63e4b-01c8-480f-8bdc-f1d416b253ee" containerName="maas-api"
Sep 15 20:26:51.227914 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:51.226135    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fe63e4b-01c8-480f-8bdc-f1d416b253ee" containerName="maas-api"
Sep 15 20:26:51.227914 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:51.226215    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fe63e4b-01c8-480f-8bdc-f1d416b253ee" containerName="maas-api"
Sep 15 20:26:51.227914 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:51.226222    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3abc9e19-0959-427b-9935-1a31a15c147d" containerName="maas-api"
Sep 15 20:26:51.230084 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:51.230059    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s"
Sep 15 20:26:51.232485 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:51.232465    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-dc12d7-serving-cert\""
Sep 15 20:26:51.241392 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:51.241359    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s"]
Sep 15 20:26:51.331042 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:51.331002    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/619bc5fd-649c-4990-b91a-f6ad01e1f035-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s\" (UID: \"619bc5fd-649c-4990-b91a-f6ad01e1f035\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s"
Sep 15 20:26:51.331231 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:51.331058    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/619bc5fd-649c-4990-b91a-f6ad01e1f035-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s\" (UID: \"619bc5fd-649c-4990-b91a-f6ad01e1f035\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s"
Sep 15 20:26:51.331276 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:51.331148    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/619bc5fd-649c-4990-b91a-f6ad01e1f035-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s\" (UID: \"619bc5fd-649c-4990-b91a-f6ad01e1f035\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s"
Sep 15 20:26:51.331337 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:51.331315    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jw4wc\" (UniqueName: \"kubernetes.io/projected/619bc5fd-649c-4990-b91a-f6ad01e1f035-kube-api-access-jw4wc\") pod \"maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s\" (UID: \"619bc5fd-649c-4990-b91a-f6ad01e1f035\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s"
Sep 15 20:26:51.432303 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:51.432261    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/619bc5fd-649c-4990-b91a-f6ad01e1f035-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s\" (UID: \"619bc5fd-649c-4990-b91a-f6ad01e1f035\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s"
Sep 15 20:26:51.432303 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:51.432309    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/619bc5fd-649c-4990-b91a-f6ad01e1f035-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s\" (UID: \"619bc5fd-649c-4990-b91a-f6ad01e1f035\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s"
Sep 15 20:26:51.432533 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:51.432344    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/619bc5fd-649c-4990-b91a-f6ad01e1f035-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s\" (UID: \"619bc5fd-649c-4990-b91a-f6ad01e1f035\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s"
Sep 15 20:26:51.432533 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:51.432397    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jw4wc\" (UniqueName: \"kubernetes.io/projected/619bc5fd-649c-4990-b91a-f6ad01e1f035-kube-api-access-jw4wc\") pod \"maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s\" (UID: \"619bc5fd-649c-4990-b91a-f6ad01e1f035\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s"
Sep 15 20:26:51.432533 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:26:51.432424    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-dc12d7-serving-cert: secret "maas-api-e2e-shared-a-w2-dc12d7-serving-cert" not found
Sep 15 20:26:51.432533 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:26:51.432527    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/619bc5fd-649c-4990-b91a-f6ad01e1f035-maas-api-tls podName:619bc5fd-649c-4990-b91a-f6ad01e1f035 nodeName:}" failed. No retries permitted until 2026-09-15 20:26:51.932502904 +0000 UTC m=+1318.323455894 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/619bc5fd-649c-4990-b91a-f6ad01e1f035-maas-api-tls") pod "maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s" (UID: "619bc5fd-649c-4990-b91a-f6ad01e1f035") : secret "maas-api-e2e-shared-a-w2-dc12d7-serving-cert" not found
Sep 15 20:26:51.433037 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:51.433016    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/619bc5fd-649c-4990-b91a-f6ad01e1f035-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s\" (UID: \"619bc5fd-649c-4990-b91a-f6ad01e1f035\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s"
Sep 15 20:26:51.434925 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:51.434900    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/619bc5fd-649c-4990-b91a-f6ad01e1f035-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s\" (UID: \"619bc5fd-649c-4990-b91a-f6ad01e1f035\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s"
Sep 15 20:26:51.440184 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:51.440164    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jw4wc\" (UniqueName: \"kubernetes.io/projected/619bc5fd-649c-4990-b91a-f6ad01e1f035-kube-api-access-jw4wc\") pod \"maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s\" (UID: \"619bc5fd-649c-4990-b91a-f6ad01e1f035\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s"
Sep 15 20:26:51.939451 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:51.939413    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/619bc5fd-649c-4990-b91a-f6ad01e1f035-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s\" (UID: \"619bc5fd-649c-4990-b91a-f6ad01e1f035\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s"
Sep 15 20:26:51.942193 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:51.942166    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/619bc5fd-649c-4990-b91a-f6ad01e1f035-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s\" (UID: \"619bc5fd-649c-4990-b91a-f6ad01e1f035\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s"
Sep 15 20:26:52.143899 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:52.143868    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s"
Sep 15 20:26:52.427253 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:52.427224    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s"]
Sep 15 20:26:52.429183 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:26:52.429157    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod619bc5fd_649c_4990_b91a_f6ad01e1f035.slice/crio-2b79295c2c90a05d8ffd078a94fc6150a51613ac211672a4ca1ec0fbb5d67396 WatchSource:0}: Error finding container 2b79295c2c90a05d8ffd078a94fc6150a51613ac211672a4ca1ec0fbb5d67396: Status 404 returned error can't find the container with id 2b79295c2c90a05d8ffd078a94fc6150a51613ac211672a4ca1ec0fbb5d67396
Sep 15 20:26:52.430514 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:52.430497    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 20:26:53.401489 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:53.401382    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s" event={"ID":"619bc5fd-649c-4990-b91a-f6ad01e1f035","Type":"ContainerStarted","Data":"6db7570f1ce9ba5fff2508690fe8d9ba54acdd31dcd1acd75e8dfe96703703d1"}
Sep 15 20:26:53.401489 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:53.401436    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s" event={"ID":"619bc5fd-649c-4990-b91a-f6ad01e1f035","Type":"ContainerStarted","Data":"2b79295c2c90a05d8ffd078a94fc6150a51613ac211672a4ca1ec0fbb5d67396"}
Sep 15 20:26:53.401489 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:53.401468    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s"
Sep 15 20:26:53.420795 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:53.420719    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s" podStartSLOduration=1.838167224 podStartE2EDuration="2.420702321s" totalImagesPullingTime="582.535097ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:26:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:26:53.419463455 +0000 UTC m=+1319.810416512" watchObservedRunningTime="2026-09-15 20:26:53.420702321 +0000 UTC m=+1319.811655318"
Sep 15 20:26:59.414444 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:26:59.414412    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s"
Sep 15 20:27:08.122635 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:08.122594    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-7e3543/distinct-w0-7e3543-kserve-777f86dd66-7clft"]
Sep 15 20:27:08.127860 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:08.127835    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-7e3543/distinct-w0-7e3543-kserve-777f86dd66-7clft"
Sep 15 20:27:08.131796 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:08.131762    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-7e3543\"/\"openshift-service-ca.crt\""
Sep 15 20:27:08.131950 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:08.131812    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-7e3543\"/\"kube-root-ca.crt\""
Sep 15 20:27:08.131950 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:08.131831    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-7e3543\"/\"distinct-w0-7e3543-kserve-self-signed-certs\""
Sep 15 20:27:08.131950 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:08.131771    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-7e3543\"/\"default-dockercfg-6p8m2\""
Sep 15 20:27:08.135804 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:08.135380    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-7e3543/distinct-w0-7e3543-kserve-777f86dd66-7clft"]
Sep 15 20:27:08.207352 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:08.207312    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmfjx\" (UniqueName: \"kubernetes.io/projected/b8401025-2132-4c14-a9a8-302aed11f8c4-kube-api-access-vmfjx\") pod \"distinct-w0-7e3543-kserve-777f86dd66-7clft\" (UID: \"b8401025-2132-4c14-a9a8-302aed11f8c4\") " pod="e2e-models-e2e-worker-w0-7e3543/distinct-w0-7e3543-kserve-777f86dd66-7clft"
Sep 15 20:27:08.207558 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:08.207368    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8401025-2132-4c14-a9a8-302aed11f8c4-model-cache\") pod \"distinct-w0-7e3543-kserve-777f86dd66-7clft\" (UID: \"b8401025-2132-4c14-a9a8-302aed11f8c4\") " pod="e2e-models-e2e-worker-w0-7e3543/distinct-w0-7e3543-kserve-777f86dd66-7clft"
Sep 15 20:27:08.207558 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:08.207402    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8401025-2132-4c14-a9a8-302aed11f8c4-dshm\") pod \"distinct-w0-7e3543-kserve-777f86dd66-7clft\" (UID: \"b8401025-2132-4c14-a9a8-302aed11f8c4\") " pod="e2e-models-e2e-worker-w0-7e3543/distinct-w0-7e3543-kserve-777f86dd66-7clft"
Sep 15 20:27:08.207558 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:08.207429    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8401025-2132-4c14-a9a8-302aed11f8c4-tmp-dir\") pod \"distinct-w0-7e3543-kserve-777f86dd66-7clft\" (UID: \"b8401025-2132-4c14-a9a8-302aed11f8c4\") " pod="e2e-models-e2e-worker-w0-7e3543/distinct-w0-7e3543-kserve-777f86dd66-7clft"
Sep 15 20:27:08.207680 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:08.207561    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8401025-2132-4c14-a9a8-302aed11f8c4-home\") pod \"distinct-w0-7e3543-kserve-777f86dd66-7clft\" (UID: \"b8401025-2132-4c14-a9a8-302aed11f8c4\") " pod="e2e-models-e2e-worker-w0-7e3543/distinct-w0-7e3543-kserve-777f86dd66-7clft"
Sep 15 20:27:08.207680 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:08.207602    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8401025-2132-4c14-a9a8-302aed11f8c4-tls-certs\") pod \"distinct-w0-7e3543-kserve-777f86dd66-7clft\" (UID: \"b8401025-2132-4c14-a9a8-302aed11f8c4\") " pod="e2e-models-e2e-worker-w0-7e3543/distinct-w0-7e3543-kserve-777f86dd66-7clft"
Sep 15 20:27:08.308841 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:08.308804    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8401025-2132-4c14-a9a8-302aed11f8c4-home\") pod \"distinct-w0-7e3543-kserve-777f86dd66-7clft\" (UID: \"b8401025-2132-4c14-a9a8-302aed11f8c4\") " pod="e2e-models-e2e-worker-w0-7e3543/distinct-w0-7e3543-kserve-777f86dd66-7clft"
Sep 15 20:27:08.308841 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:08.308845    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8401025-2132-4c14-a9a8-302aed11f8c4-tls-certs\") pod \"distinct-w0-7e3543-kserve-777f86dd66-7clft\" (UID: \"b8401025-2132-4c14-a9a8-302aed11f8c4\") " pod="e2e-models-e2e-worker-w0-7e3543/distinct-w0-7e3543-kserve-777f86dd66-7clft"
Sep 15 20:27:08.309114 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:08.308899    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmfjx\" (UniqueName: \"kubernetes.io/projected/b8401025-2132-4c14-a9a8-302aed11f8c4-kube-api-access-vmfjx\") pod \"distinct-w0-7e3543-kserve-777f86dd66-7clft\" (UID: \"b8401025-2132-4c14-a9a8-302aed11f8c4\") " pod="e2e-models-e2e-worker-w0-7e3543/distinct-w0-7e3543-kserve-777f86dd66-7clft"
Sep 15 20:27:08.309114 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:08.308932    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8401025-2132-4c14-a9a8-302aed11f8c4-model-cache\") pod \"distinct-w0-7e3543-kserve-777f86dd66-7clft\" (UID: \"b8401025-2132-4c14-a9a8-302aed11f8c4\") " pod="e2e-models-e2e-worker-w0-7e3543/distinct-w0-7e3543-kserve-777f86dd66-7clft"
Sep 15 20:27:08.309114 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:08.308997    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8401025-2132-4c14-a9a8-302aed11f8c4-dshm\") pod \"distinct-w0-7e3543-kserve-777f86dd66-7clft\" (UID: \"b8401025-2132-4c14-a9a8-302aed11f8c4\") " pod="e2e-models-e2e-worker-w0-7e3543/distinct-w0-7e3543-kserve-777f86dd66-7clft"
Sep 15 20:27:08.309114 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:08.309030    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8401025-2132-4c14-a9a8-302aed11f8c4-tmp-dir\") pod \"distinct-w0-7e3543-kserve-777f86dd66-7clft\" (UID: \"b8401025-2132-4c14-a9a8-302aed11f8c4\") " pod="e2e-models-e2e-worker-w0-7e3543/distinct-w0-7e3543-kserve-777f86dd66-7clft"
Sep 15 20:27:08.309317 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:08.309300    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8401025-2132-4c14-a9a8-302aed11f8c4-model-cache\") pod \"distinct-w0-7e3543-kserve-777f86dd66-7clft\" (UID: \"b8401025-2132-4c14-a9a8-302aed11f8c4\") " pod="e2e-models-e2e-worker-w0-7e3543/distinct-w0-7e3543-kserve-777f86dd66-7clft"
Sep 15 20:27:08.309851 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:08.309271    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8401025-2132-4c14-a9a8-302aed11f8c4-home\") pod \"distinct-w0-7e3543-kserve-777f86dd66-7clft\" (UID: \"b8401025-2132-4c14-a9a8-302aed11f8c4\") " pod="e2e-models-e2e-worker-w0-7e3543/distinct-w0-7e3543-kserve-777f86dd66-7clft"
Sep 15 20:27:08.310048 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:08.309861    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8401025-2132-4c14-a9a8-302aed11f8c4-tmp-dir\") pod \"distinct-w0-7e3543-kserve-777f86dd66-7clft\" (UID: \"b8401025-2132-4c14-a9a8-302aed11f8c4\") " pod="e2e-models-e2e-worker-w0-7e3543/distinct-w0-7e3543-kserve-777f86dd66-7clft"
Sep 15 20:27:08.317717 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:08.317686    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8401025-2132-4c14-a9a8-302aed11f8c4-dshm\") pod \"distinct-w0-7e3543-kserve-777f86dd66-7clft\" (UID: \"b8401025-2132-4c14-a9a8-302aed11f8c4\") " pod="e2e-models-e2e-worker-w0-7e3543/distinct-w0-7e3543-kserve-777f86dd66-7clft"
Sep 15 20:27:08.320649 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:08.320530    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8401025-2132-4c14-a9a8-302aed11f8c4-tls-certs\") pod \"distinct-w0-7e3543-kserve-777f86dd66-7clft\" (UID: \"b8401025-2132-4c14-a9a8-302aed11f8c4\") " pod="e2e-models-e2e-worker-w0-7e3543/distinct-w0-7e3543-kserve-777f86dd66-7clft"
Sep 15 20:27:08.322265 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:08.322236    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmfjx\" (UniqueName: \"kubernetes.io/projected/b8401025-2132-4c14-a9a8-302aed11f8c4-kube-api-access-vmfjx\") pod \"distinct-w0-7e3543-kserve-777f86dd66-7clft\" (UID: \"b8401025-2132-4c14-a9a8-302aed11f8c4\") " pod="e2e-models-e2e-worker-w0-7e3543/distinct-w0-7e3543-kserve-777f86dd66-7clft"
Sep 15 20:27:08.439391 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:08.439348    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-7e3543/distinct-w0-7e3543-kserve-777f86dd66-7clft"
Sep 15 20:27:08.725925 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:08.725896    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-7e3543/distinct-w0-7e3543-kserve-777f86dd66-7clft"]
Sep 15 20:27:08.727101 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:27:08.727073    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb8401025_2132_4c14_a9a8_302aed11f8c4.slice/crio-1e65789911418d177c7272ba6b694e812a57a804189c016e5bf9d0cf636104e5 WatchSource:0}: Error finding container 1e65789911418d177c7272ba6b694e812a57a804189c016e5bf9d0cf636104e5: Status 404 returned error can't find the container with id 1e65789911418d177c7272ba6b694e812a57a804189c016e5bf9d0cf636104e5
Sep 15 20:27:09.513597 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:09.513557    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-7e3543/distinct-w0-7e3543-kserve-777f86dd66-7clft" event={"ID":"b8401025-2132-4c14-a9a8-302aed11f8c4","Type":"ContainerStarted","Data":"f236c682e725930b368476d0f674a04a66d38781ab661561a43f123ec2e13ce8"}
Sep 15 20:27:09.514014 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:09.513609    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-7e3543/distinct-w0-7e3543-kserve-777f86dd66-7clft" event={"ID":"b8401025-2132-4c14-a9a8-302aed11f8c4","Type":"ContainerStarted","Data":"1e65789911418d177c7272ba6b694e812a57a804189c016e5bf9d0cf636104e5"}
Sep 15 20:27:09.530570 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:09.530520    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-7e3543/distinct-w0-7e3543-kserve-777f86dd66-7clft" podStartSLOduration=1.530504263 podStartE2EDuration="1.530504263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:27:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:27:09.529383218 +0000 UTC m=+1335.920336216" watchObservedRunningTime="2026-09-15 20:27:09.530504263 +0000 UTC m=+1335.921457261"
Sep 15 20:27:18.439938 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:18.439846    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-7e3543/distinct-w0-7e3543-kserve-777f86dd66-7clft"
Sep 15 20:27:18.439938 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:18.439915    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-7e3543/distinct-w0-7e3543-kserve-777f86dd66-7clft"
Sep 15 20:27:18.453114 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:18.453085    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-7e3543/distinct-w0-7e3543-kserve-777f86dd66-7clft"
Sep 15 20:27:18.588595 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:18.588563    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-7e3543/distinct-w0-7e3543-kserve-777f86dd66-7clft"
Sep 15 20:27:18.647936 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:18.647903    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"]
Sep 15 20:27:18.652176 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:18.652150    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"
Sep 15 20:27:18.654795 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:18.654768    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-6733e9-openshift-default-dockercfg-6lrm6\""
Sep 15 20:27:18.664802 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:18.664773    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"]
Sep 15 20:27:18.721372 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:18.721335    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-istio-envoy\") pod \"e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn\" (UID: \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\") " pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"
Sep 15 20:27:18.721372 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:18.721373    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-istio-podinfo\") pod \"e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn\" (UID: \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\") " pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"
Sep 15 20:27:18.721586 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:18.721407    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-workload-certs\") pod \"e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn\" (UID: \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\") " pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"
Sep 15 20:27:18.721586 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:18.721490    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-istiod-ca-cert\") pod \"e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn\" (UID: \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\") " pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"
Sep 15 20:27:18.721586 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:18.721524    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-workload-socket\") pod \"e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn\" (UID: \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\") " pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"
Sep 15 20:27:18.721782 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:18.721756    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nnzg\" (UniqueName: \"kubernetes.io/projected/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-kube-api-access-5nnzg\") pod \"e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn\" (UID: \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\") " pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"
Sep 15 20:27:18.721851 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:18.721834    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-istio-token\") pod \"e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn\" (UID: \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\") " pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"
Sep 15 20:27:18.721891 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:18.721869    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-credential-socket\") pod \"e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn\" (UID: \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\") " pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"
Sep 15 20:27:18.721939 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:18.721932    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-istio-data\") pod \"e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn\" (UID: \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\") " pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"
Sep 15 20:27:18.823290 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:18.823244    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nnzg\" (UniqueName: \"kubernetes.io/projected/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-kube-api-access-5nnzg\") pod \"e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn\" (UID: \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\") " pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"
Sep 15 20:27:18.823495 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:18.823309    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-istio-token\") pod \"e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn\" (UID: \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\") " pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"
Sep 15 20:27:18.823495 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:18.823337    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-credential-socket\") pod \"e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn\" (UID: \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\") " pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"
Sep 15 20:27:18.823495 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:18.823383    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-istio-data\") pod \"e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn\" (UID: \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\") " pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"
Sep 15 20:27:18.823690 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:18.823514    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-istio-envoy\") pod \"e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn\" (UID: \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\") " pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"
Sep 15 20:27:18.823690 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:18.823557    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-istio-podinfo\") pod \"e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn\" (UID: \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\") " pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"
Sep 15 20:27:18.823690 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:18.823607    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-workload-certs\") pod \"e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn\" (UID: \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\") " pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"
Sep 15 20:27:18.823886 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:18.823812    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-istiod-ca-cert\") pod \"e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn\" (UID: \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\") " pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"
Sep 15 20:27:18.823886 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:18.823821    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-istio-data\") pod \"e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn\" (UID: \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\") " pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"
Sep 15 20:27:18.823886 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:18.823854    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-credential-socket\") pod \"e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn\" (UID: \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\") " pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"
Sep 15 20:27:18.824014 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:18.823992    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-workload-certs\") pod \"e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn\" (UID: \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\") " pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"
Sep 15 20:27:18.824092 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:18.824075    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-workload-socket\") pod \"e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn\" (UID: \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\") " pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"
Sep 15 20:27:18.824361 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:18.824339    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-workload-socket\") pod \"e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn\" (UID: \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\") " pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"
Sep 15 20:27:18.824481 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:18.824442    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-istiod-ca-cert\") pod \"e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn\" (UID: \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\") " pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"
Sep 15 20:27:18.825955 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:18.825933    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-istio-envoy\") pod \"e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn\" (UID: \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\") " pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"
Sep 15 20:27:18.826217 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:18.826198    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-istio-podinfo\") pod \"e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn\" (UID: \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\") " pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"
Sep 15 20:27:18.831400 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:18.831378    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-istio-token\") pod \"e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn\" (UID: \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\") " pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"
Sep 15 20:27:18.831483 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:18.831444    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nnzg\" (UniqueName: \"kubernetes.io/projected/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-kube-api-access-5nnzg\") pod \"e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn\" (UID: \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\") " pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"
Sep 15 20:27:18.966438 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:18.966397    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"
Sep 15 20:27:19.250050 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:19.250006    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"]
Sep 15 20:27:19.250919 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:27:19.250896    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod791ba0be_2ec9_4bd6_a6c1_23c041733a5a.slice/crio-c0cbe9b89aa3dcf2ed3fb076634df24adfda51b44f4fc57cb4259d943ab9b573 WatchSource:0}: Error finding container c0cbe9b89aa3dcf2ed3fb076634df24adfda51b44f4fc57cb4259d943ab9b573: Status 404 returned error can't find the container with id c0cbe9b89aa3dcf2ed3fb076634df24adfda51b44f4fc57cb4259d943ab9b573
Sep 15 20:27:19.253372 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:19.253332    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 20:27:19.253494 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:19.253402    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 20:27:19.253494 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:19.253429    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 20:27:19.582125 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:19.582023    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn" event={"ID":"791ba0be-2ec9-4bd6-a6c1-23c041733a5a","Type":"ContainerStarted","Data":"5290df04b1bfc906056dac35e5e14a6441f37342a48493cb6014c9c1c0bbff83"}
Sep 15 20:27:19.582125 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:19.582121    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn" event={"ID":"791ba0be-2ec9-4bd6-a6c1-23c041733a5a","Type":"ContainerStarted","Data":"c0cbe9b89aa3dcf2ed3fb076634df24adfda51b44f4fc57cb4259d943ab9b573"}
Sep 15 20:27:19.601603 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:19.601547    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn" podStartSLOduration=1.601528809 podStartE2EDuration="1.601528809s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:27:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:27:19.600316392 +0000 UTC m=+1345.991269389" watchObservedRunningTime="2026-09-15 20:27:19.601528809 +0000 UTC m=+1345.992481806"
Sep 15 20:27:19.967263 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:19.967229    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"
Sep 15 20:27:20.972233 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:20.972198    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"
Sep 15 20:27:21.599271 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:21.599232    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"
Sep 15 20:27:21.600352 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:21.600328    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"
Sep 15 20:27:26.363141 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:26.363107    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68"]
Sep 15 20:27:26.366998 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:26.366976    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68"
Sep 15 20:27:26.370531 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:26.370509    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-f49968-serving-cert\""
Sep 15 20:27:26.397579 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:26.397554    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68"]
Sep 15 20:27:26.508781 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:26.508701    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6g4jq\" (UniqueName: \"kubernetes.io/projected/0ab26204-516b-4ffc-b982-a2aa5ce9690a-kube-api-access-6g4jq\") pod \"maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68\" (UID: \"0ab26204-516b-4ffc-b982-a2aa5ce9690a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68"
Sep 15 20:27:26.508781 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:26.508786    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ab26204-516b-4ffc-b982-a2aa5ce9690a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68\" (UID: \"0ab26204-516b-4ffc-b982-a2aa5ce9690a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68"
Sep 15 20:27:26.509022 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:26.508853    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ab26204-516b-4ffc-b982-a2aa5ce9690a-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68\" (UID: \"0ab26204-516b-4ffc-b982-a2aa5ce9690a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68"
Sep 15 20:27:26.509022 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:26.508955    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ab26204-516b-4ffc-b982-a2aa5ce9690a-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68\" (UID: \"0ab26204-516b-4ffc-b982-a2aa5ce9690a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68"
Sep 15 20:27:26.609788 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:26.609717    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g4jq\" (UniqueName: \"kubernetes.io/projected/0ab26204-516b-4ffc-b982-a2aa5ce9690a-kube-api-access-6g4jq\") pod \"maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68\" (UID: \"0ab26204-516b-4ffc-b982-a2aa5ce9690a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68"
Sep 15 20:27:26.609997 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:26.609797    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ab26204-516b-4ffc-b982-a2aa5ce9690a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68\" (UID: \"0ab26204-516b-4ffc-b982-a2aa5ce9690a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68"
Sep 15 20:27:26.609997 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:26.609878    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ab26204-516b-4ffc-b982-a2aa5ce9690a-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68\" (UID: \"0ab26204-516b-4ffc-b982-a2aa5ce9690a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68"
Sep 15 20:27:26.609997 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:26.609987    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ab26204-516b-4ffc-b982-a2aa5ce9690a-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68\" (UID: \"0ab26204-516b-4ffc-b982-a2aa5ce9690a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68"
Sep 15 20:27:26.610576 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:26.610545    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ab26204-516b-4ffc-b982-a2aa5ce9690a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68\" (UID: \"0ab26204-516b-4ffc-b982-a2aa5ce9690a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68"
Sep 15 20:27:26.612357 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:26.612336    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ab26204-516b-4ffc-b982-a2aa5ce9690a-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68\" (UID: \"0ab26204-516b-4ffc-b982-a2aa5ce9690a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68"
Sep 15 20:27:26.612461 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:26.612356    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ab26204-516b-4ffc-b982-a2aa5ce9690a-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68\" (UID: \"0ab26204-516b-4ffc-b982-a2aa5ce9690a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68"
Sep 15 20:27:26.621852 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:26.621789    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6g4jq\" (UniqueName: \"kubernetes.io/projected/0ab26204-516b-4ffc-b982-a2aa5ce9690a-kube-api-access-6g4jq\") pod \"maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68\" (UID: \"0ab26204-516b-4ffc-b982-a2aa5ce9690a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68"
Sep 15 20:27:26.679029 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:26.678988    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68"
Sep 15 20:27:26.965580 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:26.965545    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68"]
Sep 15 20:27:27.643069 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:27.642969    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68" event={"ID":"0ab26204-516b-4ffc-b982-a2aa5ce9690a","Type":"ContainerStarted","Data":"f529500852cb0292b94559e393236e5a353fbfe96d43304f1ae86931bfd1a78f"}
Sep 15 20:27:27.643069 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:27.643035    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68" event={"ID":"0ab26204-516b-4ffc-b982-a2aa5ce9690a","Type":"ContainerStarted","Data":"fef2a4106bcb24efcb9abe473df6ed2188486b5ede35e95fa6729a17f246b5e6"}
Sep 15 20:27:27.643495 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:27.643180    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68"
Sep 15 20:27:27.663021 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:27.662966    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68" podStartSLOduration=1.318412728 podStartE2EDuration="1.662946745s" totalImagesPullingTime="344.534017ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:27:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:27:27.660709878 +0000 UTC m=+1354.051662872" watchObservedRunningTime="2026-09-15 20:27:27.662946745 +0000 UTC m=+1354.053899834"
Sep 15 20:27:33.655454 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:33.655424    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68"
Sep 15 20:27:52.737552 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:52.737515    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7963-5e6381"]
Sep 15 20:27:52.742826 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:52.742793    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7963-5e6381"
Sep 15 20:27:52.745629 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:52.745602    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tppb5\""
Sep 15 20:27:52.746305 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:52.746274    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7963-5e6381"]
Sep 15 20:27:52.876641 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:52.876596    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgww2\" (UniqueName: \"kubernetes.io/projected/0be39111-f7f7-47be-a9e2-1129f4f6e73c-kube-api-access-xgww2\") pod \"test-curl-7963-5e6381\" (UID: \"0be39111-f7f7-47be-a9e2-1129f4f6e73c\") " pod="opendatahub/test-curl-7963-5e6381"
Sep 15 20:27:52.978088 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:52.978048    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgww2\" (UniqueName: \"kubernetes.io/projected/0be39111-f7f7-47be-a9e2-1129f4f6e73c-kube-api-access-xgww2\") pod \"test-curl-7963-5e6381\" (UID: \"0be39111-f7f7-47be-a9e2-1129f4f6e73c\") " pod="opendatahub/test-curl-7963-5e6381"
Sep 15 20:27:52.986574 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:52.986538    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgww2\" (UniqueName: \"kubernetes.io/projected/0be39111-f7f7-47be-a9e2-1129f4f6e73c-kube-api-access-xgww2\") pod \"test-curl-7963-5e6381\" (UID: \"0be39111-f7f7-47be-a9e2-1129f4f6e73c\") " pod="opendatahub/test-curl-7963-5e6381"
Sep 15 20:27:53.054933 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:53.054849    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7963-5e6381"
Sep 15 20:27:53.332836 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:53.332806    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7963-5e6381"]
Sep 15 20:27:53.333213 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:27:53.333189    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0be39111_f7f7_47be_a9e2_1129f4f6e73c.slice/crio-1a2b9ef2f85baf3286645a1a21d214994416fa6936b5c50004990cbe04a4788f WatchSource:0}: Error finding container 1a2b9ef2f85baf3286645a1a21d214994416fa6936b5c50004990cbe04a4788f: Status 404 returned error can't find the container with id 1a2b9ef2f85baf3286645a1a21d214994416fa6936b5c50004990cbe04a4788f
Sep 15 20:27:53.826941 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:53.826902    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7963-5e6381" event={"ID":"0be39111-f7f7-47be-a9e2-1129f4f6e73c","Type":"ContainerStarted","Data":"3b3a679f3c5ff760ea7ea923578758d031b932d091b1ad9d27937bdd25a21e14"}
Sep 15 20:27:53.826941 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:53.826948    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7963-5e6381" event={"ID":"0be39111-f7f7-47be-a9e2-1129f4f6e73c","Type":"ContainerStarted","Data":"1a2b9ef2f85baf3286645a1a21d214994416fa6936b5c50004990cbe04a4788f"}
Sep 15 20:27:53.846393 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:53.846342    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7963-5e6381" podStartSLOduration=1.846325872 podStartE2EDuration="1.846325872s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:27:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:27:53.843268522 +0000 UTC m=+1380.234221519" watchObservedRunningTime="2026-09-15 20:27:53.846325872 +0000 UTC m=+1380.237278869"
Sep 15 20:27:54.438355 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:54.438320    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7963-5e6381"]
Sep 15 20:27:54.443073 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:54.443039    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7963-5e6381"]
Sep 15 20:27:54.560084 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:54.559998    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7963-4c7780"]
Sep 15 20:27:54.560586 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:54.560570    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0be39111-f7f7-47be-a9e2-1129f4f6e73c" containerName="test-curl-7963-5e6381"
Sep 15 20:27:54.560634 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:54.560588    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0be39111-f7f7-47be-a9e2-1129f4f6e73c" containerName="test-curl-7963-5e6381"
Sep 15 20:27:54.560703 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:54.560693    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0be39111-f7f7-47be-a9e2-1129f4f6e73c" containerName="test-curl-7963-5e6381"
Sep 15 20:27:54.564083 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:54.564067    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7963-4c7780"
Sep 15 20:27:54.569082 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:54.569053    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7963-4c7780"]
Sep 15 20:27:54.699545 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:54.699495    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gltg5\" (UniqueName: \"kubernetes.io/projected/6915e3c0-2499-48b6-920e-1bbba07ff721-kube-api-access-gltg5\") pod \"test-curl-7963-4c7780\" (UID: \"6915e3c0-2499-48b6-920e-1bbba07ff721\") " pod="opendatahub/test-curl-7963-4c7780"
Sep 15 20:27:54.800883 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:54.800834    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gltg5\" (UniqueName: \"kubernetes.io/projected/6915e3c0-2499-48b6-920e-1bbba07ff721-kube-api-access-gltg5\") pod \"test-curl-7963-4c7780\" (UID: \"6915e3c0-2499-48b6-920e-1bbba07ff721\") " pod="opendatahub/test-curl-7963-4c7780"
Sep 15 20:27:54.811103 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:54.811029    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gltg5\" (UniqueName: \"kubernetes.io/projected/6915e3c0-2499-48b6-920e-1bbba07ff721-kube-api-access-gltg5\") pod \"test-curl-7963-4c7780\" (UID: \"6915e3c0-2499-48b6-920e-1bbba07ff721\") " pod="opendatahub/test-curl-7963-4c7780"
Sep 15 20:27:54.832908 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:27:54.832880    2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7963-5e6381" container="test-curl-7963-5e6381"
Sep 15 20:27:54.876177 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:54.876137    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7963-4c7780"
Sep 15 20:27:55.151189 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:55.151159    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7963-4c7780"]
Sep 15 20:27:55.152035 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:27:55.152006    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6915e3c0_2499_48b6_920e_1bbba07ff721.slice/crio-1592df0958283606bd7d1a4ecfd875a19237d1886d265f370f9c232dc518cff4 WatchSource:0}: Error finding container 1592df0958283606bd7d1a4ecfd875a19237d1886d265f370f9c232dc518cff4: Status 404 returned error can't find the container with id 1592df0958283606bd7d1a4ecfd875a19237d1886d265f370f9c232dc518cff4
Sep 15 20:27:55.840669 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:55.840631    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7963-4c7780" event={"ID":"6915e3c0-2499-48b6-920e-1bbba07ff721","Type":"ContainerStarted","Data":"9bac844ffa5ff03d5ff09729271967f4e7884f8179e14e3a412254232b19bae6"}
Sep 15 20:27:55.840669 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:55.840678    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7963-4c7780" event={"ID":"6915e3c0-2499-48b6-920e-1bbba07ff721","Type":"ContainerStarted","Data":"1592df0958283606bd7d1a4ecfd875a19237d1886d265f370f9c232dc518cff4"}
Sep 15 20:27:55.841198 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:55.840772    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7963-5e6381" podUID="0be39111-f7f7-47be-a9e2-1129f4f6e73c" containerName="test-curl-7963-5e6381" containerID="cri-o://3b3a679f3c5ff760ea7ea923578758d031b932d091b1ad9d27937bdd25a21e14" gracePeriod=2
Sep 15 20:27:55.843422 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:27:55.843397    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7963-5e6381\" is forbidden: User \"system:node:ip-10-0-140-72.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-72.ec2.internal' and this object" podUID="0be39111-f7f7-47be-a9e2-1129f4f6e73c" pod="opendatahub/test-curl-7963-5e6381"
Sep 15 20:27:55.868607 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:55.868555    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7963-4c7780" podStartSLOduration=1.868539321 podStartE2EDuration="1.868539321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:27:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:27:55.866430556 +0000 UTC m=+1382.257383553" watchObservedRunningTime="2026-09-15 20:27:55.868539321 +0000 UTC m=+1382.259492318"
Sep 15 20:27:55.869117 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:27:55.869087    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7963-5e6381\" is forbidden: User \"system:node:ip-10-0-140-72.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-72.ec2.internal' and this object" podUID="0be39111-f7f7-47be-a9e2-1129f4f6e73c" pod="opendatahub/test-curl-7963-5e6381"
Sep 15 20:27:56.497536 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:56.497496    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7963-4c7780"]
Sep 15 20:27:56.500973 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:56.500940    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7963-4c7780"]
Sep 15 20:27:56.618150 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:56.618070    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7963-d7d038"]
Sep 15 20:27:56.618599 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:56.618582    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6915e3c0-2499-48b6-920e-1bbba07ff721" containerName="test-curl-7963-4c7780"
Sep 15 20:27:56.618652 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:56.618601    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6915e3c0-2499-48b6-920e-1bbba07ff721" containerName="test-curl-7963-4c7780"
Sep 15 20:27:56.618714 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:56.618705    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6915e3c0-2499-48b6-920e-1bbba07ff721" containerName="test-curl-7963-4c7780"
Sep 15 20:27:56.622146 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:56.622128    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7963-d7d038"
Sep 15 20:27:56.627524 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:56.627490    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7963-d7d038"]
Sep 15 20:27:56.719300 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:56.719237    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9w9v\" (UniqueName: \"kubernetes.io/projected/e39bf764-5ef7-48e9-8cb7-58b0fe35cb3f-kube-api-access-j9w9v\") pod \"test-curl-7963-d7d038\" (UID: \"e39bf764-5ef7-48e9-8cb7-58b0fe35cb3f\") " pod="opendatahub/test-curl-7963-d7d038"
Sep 15 20:27:56.819994 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:56.819963    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9w9v\" (UniqueName: \"kubernetes.io/projected/e39bf764-5ef7-48e9-8cb7-58b0fe35cb3f-kube-api-access-j9w9v\") pod \"test-curl-7963-d7d038\" (UID: \"e39bf764-5ef7-48e9-8cb7-58b0fe35cb3f\") " pod="opendatahub/test-curl-7963-d7d038"
Sep 15 20:27:56.828289 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:56.828261    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9w9v\" (UniqueName: \"kubernetes.io/projected/e39bf764-5ef7-48e9-8cb7-58b0fe35cb3f-kube-api-access-j9w9v\") pod \"test-curl-7963-d7d038\" (UID: \"e39bf764-5ef7-48e9-8cb7-58b0fe35cb3f\") " pod="opendatahub/test-curl-7963-d7d038"
Sep 15 20:27:56.850412 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:27:56.850384    2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7963-4c7780" container="test-curl-7963-4c7780"
Sep 15 20:27:56.933961 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:56.933862    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7963-d7d038"
Sep 15 20:27:57.209876 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:57.209843    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7963-d7d038"]
Sep 15 20:27:57.210317 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:27:57.210289    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode39bf764_5ef7_48e9_8cb7_58b0fe35cb3f.slice/crio-e0e71a01590d275a3526834d91dd872c1d8707e160d711eaeca87642b3381d04 WatchSource:0}: Error finding container e0e71a01590d275a3526834d91dd872c1d8707e160d711eaeca87642b3381d04: Status 404 returned error can't find the container with id e0e71a01590d275a3526834d91dd872c1d8707e160d711eaeca87642b3381d04
Sep 15 20:27:57.860149 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:57.860116    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7963-d7d038" event={"ID":"e39bf764-5ef7-48e9-8cb7-58b0fe35cb3f","Type":"ContainerStarted","Data":"f1eb8bd289317ba9000c1507d14186e37903ecf90ddb7ff107a9ac8ccdba7110"}
Sep 15 20:27:57.860515 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:57.860163    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7963-d7d038" event={"ID":"e39bf764-5ef7-48e9-8cb7-58b0fe35cb3f","Type":"ContainerStarted","Data":"e0e71a01590d275a3526834d91dd872c1d8707e160d711eaeca87642b3381d04"}
Sep 15 20:27:57.860515 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:57.860217    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7963-4c7780" podUID="6915e3c0-2499-48b6-920e-1bbba07ff721" containerName="test-curl-7963-4c7780" containerID="cri-o://9bac844ffa5ff03d5ff09729271967f4e7884f8179e14e3a412254232b19bae6" gracePeriod=2
Sep 15 20:27:57.880265 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:27:57.880225    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7963-4c7780\" is forbidden: User \"system:node:ip-10-0-140-72.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-72.ec2.internal' and this object" podUID="6915e3c0-2499-48b6-920e-1bbba07ff721" pod="opendatahub/test-curl-7963-4c7780"
Sep 15 20:27:57.881246 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:57.881202    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7963-d7d038" podStartSLOduration=1.881188173 podStartE2EDuration="1.881188173s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:27:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:27:57.877944843 +0000 UTC m=+1384.268897841" watchObservedRunningTime="2026-09-15 20:27:57.881188173 +0000 UTC m=+1384.272141207"
Sep 15 20:27:57.882230 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:27:57.882202    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7963-4c7780\" is forbidden: User \"system:node:ip-10-0-140-72.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-72.ec2.internal' and this object" podUID="6915e3c0-2499-48b6-920e-1bbba07ff721" pod="opendatahub/test-curl-7963-4c7780"
Sep 15 20:27:58.001548 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:58.001523    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7963-5e6381"
Sep 15 20:27:58.003968 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:27:58.003936    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7963-4c7780\" is forbidden: User \"system:node:ip-10-0-140-72.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-72.ec2.internal' and this object" podUID="6915e3c0-2499-48b6-920e-1bbba07ff721" pod="opendatahub/test-curl-7963-4c7780"
Sep 15 20:27:58.135210 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:58.135124    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xgww2\" (UniqueName: \"kubernetes.io/projected/0be39111-f7f7-47be-a9e2-1129f4f6e73c-kube-api-access-xgww2\") pod \"0be39111-f7f7-47be-a9e2-1129f4f6e73c\" (UID: \"0be39111-f7f7-47be-a9e2-1129f4f6e73c\") "
Sep 15 20:27:58.137229 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:58.137196    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0be39111-f7f7-47be-a9e2-1129f4f6e73c-kube-api-access-xgww2" (OuterVolumeSpecName: "kube-api-access-xgww2") pod "0be39111-f7f7-47be-a9e2-1129f4f6e73c" (UID: "0be39111-f7f7-47be-a9e2-1129f4f6e73c"). InnerVolumeSpecName "kube-api-access-xgww2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:27:58.236294 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:58.236254    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgww2\" (UniqueName: \"kubernetes.io/projected/0be39111-f7f7-47be-a9e2-1129f4f6e73c-kube-api-access-xgww2\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:27:58.485423 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:58.485389    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7963-d7d038"]
Sep 15 20:27:58.489376 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:58.489350    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7963-d7d038"]
Sep 15 20:27:58.600528 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:58.600490    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7963-3fadc6"]
Sep 15 20:27:58.601071 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:58.601055    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e39bf764-5ef7-48e9-8cb7-58b0fe35cb3f" containerName="test-curl-7963-d7d038"
Sep 15 20:27:58.601129 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:58.601072    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e39bf764-5ef7-48e9-8cb7-58b0fe35cb3f" containerName="test-curl-7963-d7d038"
Sep 15 20:27:58.601192 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:58.601182    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e39bf764-5ef7-48e9-8cb7-58b0fe35cb3f" containerName="test-curl-7963-d7d038"
Sep 15 20:27:58.604768 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:58.604713    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7963-3fadc6"
Sep 15 20:27:58.609144 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:58.608957    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7963-3fadc6"]
Sep 15 20:27:58.741104 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:58.741002    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9l8kw\" (UniqueName: \"kubernetes.io/projected/a0cc1ef3-5e82-4808-aa1b-b705a04e48af-kube-api-access-9l8kw\") pod \"test-curl-7963-3fadc6\" (UID: \"a0cc1ef3-5e82-4808-aa1b-b705a04e48af\") " pod="opendatahub/test-curl-7963-3fadc6"
Sep 15 20:27:58.842506 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:58.842463    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l8kw\" (UniqueName: \"kubernetes.io/projected/a0cc1ef3-5e82-4808-aa1b-b705a04e48af-kube-api-access-9l8kw\") pod \"test-curl-7963-3fadc6\" (UID: \"a0cc1ef3-5e82-4808-aa1b-b705a04e48af\") " pod="opendatahub/test-curl-7963-3fadc6"
Sep 15 20:27:58.851560 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:58.851528    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9l8kw\" (UniqueName: \"kubernetes.io/projected/a0cc1ef3-5e82-4808-aa1b-b705a04e48af-kube-api-access-9l8kw\") pod \"test-curl-7963-3fadc6\" (UID: \"a0cc1ef3-5e82-4808-aa1b-b705a04e48af\") " pod="opendatahub/test-curl-7963-3fadc6"
Sep 15 20:27:58.867715 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:58.867683    2577 generic.go:356] "Generic (PLEG): container finished" podID="0be39111-f7f7-47be-a9e2-1129f4f6e73c" containerID="3b3a679f3c5ff760ea7ea923578758d031b932d091b1ad9d27937bdd25a21e14" exitCode=137
Sep 15 20:27:58.868109 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:58.867754    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7963-5e6381"
Sep 15 20:27:58.868109 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:58.867788    2577 scope.go:117] "RemoveContainer" containerID="3b3a679f3c5ff760ea7ea923578758d031b932d091b1ad9d27937bdd25a21e14"
Sep 15 20:27:58.868222 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:27:58.868127    2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7963-d7d038" container="test-curl-7963-d7d038"
Sep 15 20:27:58.880127 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:58.880108    2577 scope.go:117] "RemoveContainer" containerID="3b3a679f3c5ff760ea7ea923578758d031b932d091b1ad9d27937bdd25a21e14"
Sep 15 20:27:58.880442 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:27:58.880422    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b3a679f3c5ff760ea7ea923578758d031b932d091b1ad9d27937bdd25a21e14\": container with ID starting with 3b3a679f3c5ff760ea7ea923578758d031b932d091b1ad9d27937bdd25a21e14 not found: ID does not exist" containerID="3b3a679f3c5ff760ea7ea923578758d031b932d091b1ad9d27937bdd25a21e14"
Sep 15 20:27:58.880520 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:58.880451    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b3a679f3c5ff760ea7ea923578758d031b932d091b1ad9d27937bdd25a21e14"} err="failed to get container status \"3b3a679f3c5ff760ea7ea923578758d031b932d091b1ad9d27937bdd25a21e14\": rpc error: code = NotFound desc = could not find container \"3b3a679f3c5ff760ea7ea923578758d031b932d091b1ad9d27937bdd25a21e14\": container with ID starting with 3b3a679f3c5ff760ea7ea923578758d031b932d091b1ad9d27937bdd25a21e14 not found: ID does not exist"
Sep 15 20:27:58.916763 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:58.916706    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7963-3fadc6"
Sep 15 20:27:59.197075 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:59.197018    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7963-3fadc6"]
Sep 15 20:27:59.199601 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:27:59.199560    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda0cc1ef3_5e82_4808_aa1b_b705a04e48af.slice/crio-96b9854f3725deae68dfbc4afb1098d632b45a36ad83b07a1cf238a8955caedb WatchSource:0}: Error finding container 96b9854f3725deae68dfbc4afb1098d632b45a36ad83b07a1cf238a8955caedb: Status 404 returned error can't find the container with id 96b9854f3725deae68dfbc4afb1098d632b45a36ad83b07a1cf238a8955caedb
Sep 15 20:27:59.876780 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:59.876708    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7963-3fadc6" event={"ID":"a0cc1ef3-5e82-4808-aa1b-b705a04e48af","Type":"ContainerStarted","Data":"ec35282af0bc3b0c4065254800f828086e9f28fe75fd80094d3cbf137e05293d"}
Sep 15 20:27:59.877172 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:59.876790    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7963-3fadc6" event={"ID":"a0cc1ef3-5e82-4808-aa1b-b705a04e48af","Type":"ContainerStarted","Data":"96b9854f3725deae68dfbc4afb1098d632b45a36ad83b07a1cf238a8955caedb"}
Sep 15 20:27:59.878231 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:59.878196    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7963-d7d038" podUID="e39bf764-5ef7-48e9-8cb7-58b0fe35cb3f" containerName="test-curl-7963-d7d038" containerID="cri-o://f1eb8bd289317ba9000c1507d14186e37903ecf90ddb7ff107a9ac8ccdba7110" gracePeriod=2
Sep 15 20:27:59.894333 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:27:59.894293    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7963-d7d038\" is forbidden: User \"system:node:ip-10-0-140-72.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-72.ec2.internal' and this object" podUID="e39bf764-5ef7-48e9-8cb7-58b0fe35cb3f" pod="opendatahub/test-curl-7963-d7d038"
Sep 15 20:27:59.894897 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:27:59.894858    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7963-3fadc6" podStartSLOduration=1.8948425549999999 podStartE2EDuration="1.894842555s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:27:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:27:59.892079405 +0000 UTC m=+1386.283032402" watchObservedRunningTime="2026-09-15 20:27:59.894842555 +0000 UTC m=+1386.285795551"
Sep 15 20:28:00.053691 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:00.053658    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7963-4c7780"
Sep 15 20:28:00.057939 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:28:00.057904    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7963-d7d038\" is forbidden: User \"system:node:ip-10-0-140-72.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-72.ec2.internal' and this object" podUID="e39bf764-5ef7-48e9-8cb7-58b0fe35cb3f" pod="opendatahub/test-curl-7963-d7d038"
Sep 15 20:28:00.061138 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:00.061113    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0be39111-f7f7-47be-a9e2-1129f4f6e73c" path="/var/lib/kubelet/pods/0be39111-f7f7-47be-a9e2-1129f4f6e73c/volumes"
Sep 15 20:28:00.155312 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:00.155229    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gltg5\" (UniqueName: \"kubernetes.io/projected/6915e3c0-2499-48b6-920e-1bbba07ff721-kube-api-access-gltg5\") pod \"6915e3c0-2499-48b6-920e-1bbba07ff721\" (UID: \"6915e3c0-2499-48b6-920e-1bbba07ff721\") "
Sep 15 20:28:00.157324 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:00.157301    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6915e3c0-2499-48b6-920e-1bbba07ff721-kube-api-access-gltg5" (OuterVolumeSpecName: "kube-api-access-gltg5") pod "6915e3c0-2499-48b6-920e-1bbba07ff721" (UID: "6915e3c0-2499-48b6-920e-1bbba07ff721"). InnerVolumeSpecName "kube-api-access-gltg5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:28:00.256686 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:00.256650    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gltg5\" (UniqueName: \"kubernetes.io/projected/6915e3c0-2499-48b6-920e-1bbba07ff721-kube-api-access-gltg5\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:28:00.512635 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:00.512597    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7963-3fadc6"]
Sep 15 20:28:00.517772 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:00.517740    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7963-3fadc6"]
Sep 15 20:28:00.886599 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:00.886514    2577 generic.go:356] "Generic (PLEG): container finished" podID="6915e3c0-2499-48b6-920e-1bbba07ff721" containerID="9bac844ffa5ff03d5ff09729271967f4e7884f8179e14e3a412254232b19bae6" exitCode=137
Sep 15 20:28:00.886599 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:00.886557    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7963-4c7780"
Sep 15 20:28:00.887070 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:00.886613    2577 scope.go:117] "RemoveContainer" containerID="9bac844ffa5ff03d5ff09729271967f4e7884f8179e14e3a412254232b19bae6"
Sep 15 20:28:00.887070 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:28:00.886848    2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7963-3fadc6" container="test-curl-7963-3fadc6"
Sep 15 20:28:00.896953 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:00.896933    2577 scope.go:117] "RemoveContainer" containerID="9bac844ffa5ff03d5ff09729271967f4e7884f8179e14e3a412254232b19bae6"
Sep 15 20:28:00.897263 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:28:00.897243    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bac844ffa5ff03d5ff09729271967f4e7884f8179e14e3a412254232b19bae6\": container with ID starting with 9bac844ffa5ff03d5ff09729271967f4e7884f8179e14e3a412254232b19bae6 not found: ID does not exist" containerID="9bac844ffa5ff03d5ff09729271967f4e7884f8179e14e3a412254232b19bae6"
Sep 15 20:28:00.897332 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:00.897270    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bac844ffa5ff03d5ff09729271967f4e7884f8179e14e3a412254232b19bae6"} err="failed to get container status \"9bac844ffa5ff03d5ff09729271967f4e7884f8179e14e3a412254232b19bae6\": rpc error: code = NotFound desc = could not find container \"9bac844ffa5ff03d5ff09729271967f4e7884f8179e14e3a412254232b19bae6\": container with ID starting with 9bac844ffa5ff03d5ff09729271967f4e7884f8179e14e3a412254232b19bae6 not found: ID does not exist"
Sep 15 20:28:01.894591 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:01.894561    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7963-3fadc6" podUID="a0cc1ef3-5e82-4808-aa1b-b705a04e48af" containerName="test-curl-7963-3fadc6" containerID="cri-o://ec35282af0bc3b0c4065254800f828086e9f28fe75fd80094d3cbf137e05293d" gracePeriod=2
Sep 15 20:28:01.897330 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:28:01.897304    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7963-3fadc6\" is forbidden: User \"system:node:ip-10-0-140-72.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-72.ec2.internal' and this object" podUID="a0cc1ef3-5e82-4808-aa1b-b705a04e48af" pod="opendatahub/test-curl-7963-3fadc6"
Sep 15 20:28:02.071214 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:02.071177    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6915e3c0-2499-48b6-920e-1bbba07ff721" path="/var/lib/kubelet/pods/6915e3c0-2499-48b6-920e-1bbba07ff721/volumes"
Sep 15 20:28:02.088393 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:02.088369    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7963-d7d038"
Sep 15 20:28:02.279708 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:02.279671    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j9w9v\" (UniqueName: \"kubernetes.io/projected/e39bf764-5ef7-48e9-8cb7-58b0fe35cb3f-kube-api-access-j9w9v\") pod \"e39bf764-5ef7-48e9-8cb7-58b0fe35cb3f\" (UID: \"e39bf764-5ef7-48e9-8cb7-58b0fe35cb3f\") "
Sep 15 20:28:02.281883 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:02.281849    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e39bf764-5ef7-48e9-8cb7-58b0fe35cb3f-kube-api-access-j9w9v" (OuterVolumeSpecName: "kube-api-access-j9w9v") pod "e39bf764-5ef7-48e9-8cb7-58b0fe35cb3f" (UID: "e39bf764-5ef7-48e9-8cb7-58b0fe35cb3f"). InnerVolumeSpecName "kube-api-access-j9w9v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:28:02.380685 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:02.380651    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9w9v\" (UniqueName: \"kubernetes.io/projected/e39bf764-5ef7-48e9-8cb7-58b0fe35cb3f-kube-api-access-j9w9v\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:28:02.902849 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:02.902812    2577 generic.go:356] "Generic (PLEG): container finished" podID="e39bf764-5ef7-48e9-8cb7-58b0fe35cb3f" containerID="f1eb8bd289317ba9000c1507d14186e37903ecf90ddb7ff107a9ac8ccdba7110" exitCode=137
Sep 15 20:28:02.903305 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:02.902862    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7963-d7d038"
Sep 15 20:28:02.903305 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:02.902906    2577 scope.go:117] "RemoveContainer" containerID="f1eb8bd289317ba9000c1507d14186e37903ecf90ddb7ff107a9ac8ccdba7110"
Sep 15 20:28:02.913802 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:02.913775    2577 scope.go:117] "RemoveContainer" containerID="f1eb8bd289317ba9000c1507d14186e37903ecf90ddb7ff107a9ac8ccdba7110"
Sep 15 20:28:02.914106 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:28:02.914084    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1eb8bd289317ba9000c1507d14186e37903ecf90ddb7ff107a9ac8ccdba7110\": container with ID starting with f1eb8bd289317ba9000c1507d14186e37903ecf90ddb7ff107a9ac8ccdba7110 not found: ID does not exist" containerID="f1eb8bd289317ba9000c1507d14186e37903ecf90ddb7ff107a9ac8ccdba7110"
Sep 15 20:28:02.914180 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:02.914115    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1eb8bd289317ba9000c1507d14186e37903ecf90ddb7ff107a9ac8ccdba7110"} err="failed to get container status \"f1eb8bd289317ba9000c1507d14186e37903ecf90ddb7ff107a9ac8ccdba7110\": rpc error: code = NotFound desc = could not find container \"f1eb8bd289317ba9000c1507d14186e37903ecf90ddb7ff107a9ac8ccdba7110\": container with ID starting with f1eb8bd289317ba9000c1507d14186e37903ecf90ddb7ff107a9ac8ccdba7110 not found: ID does not exist"
Sep 15 20:28:02.914545 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:28:02.914520    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7963-d7d038\" is forbidden: User \"system:node:ip-10-0-140-72.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-72.ec2.internal' and this object" podUID="e39bf764-5ef7-48e9-8cb7-58b0fe35cb3f" pod="opendatahub/test-curl-7963-d7d038"
Sep 15 20:28:04.061104 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:28:04.061070    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7963-d7d038\" is forbidden: User \"system:node:ip-10-0-140-72.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-72.ec2.internal' and this object" podUID="e39bf764-5ef7-48e9-8cb7-58b0fe35cb3f" pod="opendatahub/test-curl-7963-d7d038"
Sep 15 20:28:04.062916 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:04.062889    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e39bf764-5ef7-48e9-8cb7-58b0fe35cb3f" path="/var/lib/kubelet/pods/e39bf764-5ef7-48e9-8cb7-58b0fe35cb3f/volumes"
Sep 15 20:28:04.089020 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:04.088991    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7963-3fadc6"
Sep 15 20:28:04.091077 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:28:04.091049    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7963-3fadc6\" is forbidden: User \"system:node:ip-10-0-140-72.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-72.ec2.internal' and this object" podUID="a0cc1ef3-5e82-4808-aa1b-b705a04e48af" pod="opendatahub/test-curl-7963-3fadc6"
Sep 15 20:28:04.096787 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:04.096763    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9l8kw\" (UniqueName: \"kubernetes.io/projected/a0cc1ef3-5e82-4808-aa1b-b705a04e48af-kube-api-access-9l8kw\") pod \"a0cc1ef3-5e82-4808-aa1b-b705a04e48af\" (UID: \"a0cc1ef3-5e82-4808-aa1b-b705a04e48af\") "
Sep 15 20:28:04.098910 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:04.098880    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0cc1ef3-5e82-4808-aa1b-b705a04e48af-kube-api-access-9l8kw" (OuterVolumeSpecName: "kube-api-access-9l8kw") pod "a0cc1ef3-5e82-4808-aa1b-b705a04e48af" (UID: "a0cc1ef3-5e82-4808-aa1b-b705a04e48af"). InnerVolumeSpecName "kube-api-access-9l8kw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:28:04.197769 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:04.197656    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9l8kw\" (UniqueName: \"kubernetes.io/projected/a0cc1ef3-5e82-4808-aa1b-b705a04e48af-kube-api-access-9l8kw\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:28:04.919575 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:04.919538    2577 generic.go:356] "Generic (PLEG): container finished" podID="a0cc1ef3-5e82-4808-aa1b-b705a04e48af" containerID="ec35282af0bc3b0c4065254800f828086e9f28fe75fd80094d3cbf137e05293d" exitCode=137
Sep 15 20:28:04.919813 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:04.919582    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7963-3fadc6"
Sep 15 20:28:04.919813 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:04.919659    2577 scope.go:117] "RemoveContainer" containerID="ec35282af0bc3b0c4065254800f828086e9f28fe75fd80094d3cbf137e05293d"
Sep 15 20:28:04.921987 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:28:04.921960    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7963-3fadc6\" is forbidden: User \"system:node:ip-10-0-140-72.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-72.ec2.internal' and this object" podUID="a0cc1ef3-5e82-4808-aa1b-b705a04e48af" pod="opendatahub/test-curl-7963-3fadc6"
Sep 15 20:28:04.930025 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:04.930005    2577 scope.go:117] "RemoveContainer" containerID="ec35282af0bc3b0c4065254800f828086e9f28fe75fd80094d3cbf137e05293d"
Sep 15 20:28:04.930332 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:28:04.930310    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec35282af0bc3b0c4065254800f828086e9f28fe75fd80094d3cbf137e05293d\": container with ID starting with ec35282af0bc3b0c4065254800f828086e9f28fe75fd80094d3cbf137e05293d not found: ID does not exist" containerID="ec35282af0bc3b0c4065254800f828086e9f28fe75fd80094d3cbf137e05293d"
Sep 15 20:28:04.930402 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:04.930339    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec35282af0bc3b0c4065254800f828086e9f28fe75fd80094d3cbf137e05293d"} err="failed to get container status \"ec35282af0bc3b0c4065254800f828086e9f28fe75fd80094d3cbf137e05293d\": rpc error: code = NotFound desc = could not find container \"ec35282af0bc3b0c4065254800f828086e9f28fe75fd80094d3cbf137e05293d\": container with ID starting with ec35282af0bc3b0c4065254800f828086e9f28fe75fd80094d3cbf137e05293d not found: ID does not exist"
Sep 15 20:28:04.931814 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:28:04.931789    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7963-3fadc6\" is forbidden: User \"system:node:ip-10-0-140-72.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-72.ec2.internal' and this object" podUID="a0cc1ef3-5e82-4808-aa1b-b705a04e48af" pod="opendatahub/test-curl-7963-3fadc6"
Sep 15 20:28:06.061957 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:06.061923    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0cc1ef3-5e82-4808-aa1b-b705a04e48af" path="/var/lib/kubelet/pods/a0cc1ef3-5e82-4808-aa1b-b705a04e48af/volumes"
Sep 15 20:28:09.132319 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:09.132283    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-025acc06-696df6544b-jpg2q"]
Sep 15 20:28:09.133054 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:09.133032    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0cc1ef3-5e82-4808-aa1b-b705a04e48af" containerName="test-curl-7963-3fadc6"
Sep 15 20:28:09.133163 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:09.133056    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0cc1ef3-5e82-4808-aa1b-b705a04e48af" containerName="test-curl-7963-3fadc6"
Sep 15 20:28:09.133256 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:09.133242    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0cc1ef3-5e82-4808-aa1b-b705a04e48af" containerName="test-curl-7963-3fadc6"
Sep 15 20:28:09.142931 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:09.142897    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-025acc06-696df6544b-jpg2q"
Sep 15 20:28:09.149986 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:09.149775    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-025acc06-serving-cert\""
Sep 15 20:28:09.152224 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:09.152194    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-025acc06-696df6544b-jpg2q"]
Sep 15 20:28:09.243276 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:09.243240    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/717a748e-0243-47ef-80c3-3c7dbc97b8c7-maas-api-tls\") pod \"maas-api-e2e-mt-025acc06-696df6544b-jpg2q\" (UID: \"717a748e-0243-47ef-80c3-3c7dbc97b8c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-025acc06-696df6544b-jpg2q"
Sep 15 20:28:09.243455 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:09.243317    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/717a748e-0243-47ef-80c3-3c7dbc97b8c7-openshift-service-ca\") pod \"maas-api-e2e-mt-025acc06-696df6544b-jpg2q\" (UID: \"717a748e-0243-47ef-80c3-3c7dbc97b8c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-025acc06-696df6544b-jpg2q"
Sep 15 20:28:09.243455 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:09.243385    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/717a748e-0243-47ef-80c3-3c7dbc97b8c7-metrics-certs\") pod \"maas-api-e2e-mt-025acc06-696df6544b-jpg2q\" (UID: \"717a748e-0243-47ef-80c3-3c7dbc97b8c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-025acc06-696df6544b-jpg2q"
Sep 15 20:28:09.243455 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:09.243408    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbbz9\" (UniqueName: \"kubernetes.io/projected/717a748e-0243-47ef-80c3-3c7dbc97b8c7-kube-api-access-lbbz9\") pod \"maas-api-e2e-mt-025acc06-696df6544b-jpg2q\" (UID: \"717a748e-0243-47ef-80c3-3c7dbc97b8c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-025acc06-696df6544b-jpg2q"
Sep 15 20:28:09.344134 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:09.344096    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/717a748e-0243-47ef-80c3-3c7dbc97b8c7-maas-api-tls\") pod \"maas-api-e2e-mt-025acc06-696df6544b-jpg2q\" (UID: \"717a748e-0243-47ef-80c3-3c7dbc97b8c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-025acc06-696df6544b-jpg2q"
Sep 15 20:28:09.344325 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:09.344184    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/717a748e-0243-47ef-80c3-3c7dbc97b8c7-openshift-service-ca\") pod \"maas-api-e2e-mt-025acc06-696df6544b-jpg2q\" (UID: \"717a748e-0243-47ef-80c3-3c7dbc97b8c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-025acc06-696df6544b-jpg2q"
Sep 15 20:28:09.344325 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:09.344211    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/717a748e-0243-47ef-80c3-3c7dbc97b8c7-metrics-certs\") pod \"maas-api-e2e-mt-025acc06-696df6544b-jpg2q\" (UID: \"717a748e-0243-47ef-80c3-3c7dbc97b8c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-025acc06-696df6544b-jpg2q"
Sep 15 20:28:09.344325 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:09.344227    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbbz9\" (UniqueName: \"kubernetes.io/projected/717a748e-0243-47ef-80c3-3c7dbc97b8c7-kube-api-access-lbbz9\") pod \"maas-api-e2e-mt-025acc06-696df6544b-jpg2q\" (UID: \"717a748e-0243-47ef-80c3-3c7dbc97b8c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-025acc06-696df6544b-jpg2q"
Sep 15 20:28:09.344325 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:28:09.344267    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-025acc06-serving-cert: secret "maas-api-e2e-mt-025acc06-serving-cert" not found
Sep 15 20:28:09.344524 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:28:09.344334    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/717a748e-0243-47ef-80c3-3c7dbc97b8c7-maas-api-tls podName:717a748e-0243-47ef-80c3-3c7dbc97b8c7 nodeName:}" failed. No retries permitted until 2026-09-15 20:28:09.84431524 +0000 UTC m=+1396.235268238 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/717a748e-0243-47ef-80c3-3c7dbc97b8c7-maas-api-tls") pod "maas-api-e2e-mt-025acc06-696df6544b-jpg2q" (UID: "717a748e-0243-47ef-80c3-3c7dbc97b8c7") : secret "maas-api-e2e-mt-025acc06-serving-cert" not found
Sep 15 20:28:09.344920 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:09.344902    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/717a748e-0243-47ef-80c3-3c7dbc97b8c7-openshift-service-ca\") pod \"maas-api-e2e-mt-025acc06-696df6544b-jpg2q\" (UID: \"717a748e-0243-47ef-80c3-3c7dbc97b8c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-025acc06-696df6544b-jpg2q"
Sep 15 20:28:09.346810 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:09.346786    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/717a748e-0243-47ef-80c3-3c7dbc97b8c7-metrics-certs\") pod \"maas-api-e2e-mt-025acc06-696df6544b-jpg2q\" (UID: \"717a748e-0243-47ef-80c3-3c7dbc97b8c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-025acc06-696df6544b-jpg2q"
Sep 15 20:28:09.353131 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:09.353109    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbbz9\" (UniqueName: \"kubernetes.io/projected/717a748e-0243-47ef-80c3-3c7dbc97b8c7-kube-api-access-lbbz9\") pod \"maas-api-e2e-mt-025acc06-696df6544b-jpg2q\" (UID: \"717a748e-0243-47ef-80c3-3c7dbc97b8c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-025acc06-696df6544b-jpg2q"
Sep 15 20:28:09.849357 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:09.849318    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/717a748e-0243-47ef-80c3-3c7dbc97b8c7-maas-api-tls\") pod \"maas-api-e2e-mt-025acc06-696df6544b-jpg2q\" (UID: \"717a748e-0243-47ef-80c3-3c7dbc97b8c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-025acc06-696df6544b-jpg2q"
Sep 15 20:28:09.851964 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:09.851931    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/717a748e-0243-47ef-80c3-3c7dbc97b8c7-maas-api-tls\") pod \"maas-api-e2e-mt-025acc06-696df6544b-jpg2q\" (UID: \"717a748e-0243-47ef-80c3-3c7dbc97b8c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-025acc06-696df6544b-jpg2q"
Sep 15 20:28:10.062654 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:10.062611    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-025acc06-696df6544b-jpg2q"
Sep 15 20:28:10.339454 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:10.339421    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-025acc06-696df6544b-jpg2q"]
Sep 15 20:28:10.340710 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:28:10.340687    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod717a748e_0243_47ef_80c3_3c7dbc97b8c7.slice/crio-909041176d3c85a2ea9fc58a89b17c1985ff4e1291dda5ae7d4ebc4fd3d7b2f2 WatchSource:0}: Error finding container 909041176d3c85a2ea9fc58a89b17c1985ff4e1291dda5ae7d4ebc4fd3d7b2f2: Status 404 returned error can't find the container with id 909041176d3c85a2ea9fc58a89b17c1985ff4e1291dda5ae7d4ebc4fd3d7b2f2
Sep 15 20:28:10.965571 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:10.965537    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-025acc06-696df6544b-jpg2q" event={"ID":"717a748e-0243-47ef-80c3-3c7dbc97b8c7","Type":"ContainerStarted","Data":"97054d47a3ec21b983d0e9c865f20b932f0d91e37fea51e1ed3cb85d08d7d6f5"}
Sep 15 20:28:10.965795 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:10.965598    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-025acc06-696df6544b-jpg2q" event={"ID":"717a748e-0243-47ef-80c3-3c7dbc97b8c7","Type":"ContainerStarted","Data":"909041176d3c85a2ea9fc58a89b17c1985ff4e1291dda5ae7d4ebc4fd3d7b2f2"}
Sep 15 20:28:10.965795 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:10.965749    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-025acc06-696df6544b-jpg2q"
Sep 15 20:28:10.986806 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:10.986748    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-025acc06-696df6544b-jpg2q" podStartSLOduration=1.626050719 podStartE2EDuration="1.986713577s" totalImagesPullingTime="360.662858ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:28:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:28:10.984885706 +0000 UTC m=+1397.375838705" watchObservedRunningTime="2026-09-15 20:28:10.986713577 +0000 UTC m=+1397.377666574"
Sep 15 20:28:16.980089 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:16.980054    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-025acc06-696df6544b-jpg2q"
Sep 15 20:28:50.866005 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:50.865895    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c"]
Sep 15 20:28:50.866589 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:50.866147    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c" podUID="215a694b-de08-40fa-8049-858e834455c8" containerName="maas-api" containerID="cri-o://7793011affcbc20d5e49fef33ada40da0cab069588993f240eeb0615b26aa2d9" gracePeriod=30
Sep 15 20:28:51.133585 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:51.133516    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c"
Sep 15 20:28:51.192025 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:51.191984    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/215a694b-de08-40fa-8049-858e834455c8-openshift-service-ca\") pod \"215a694b-de08-40fa-8049-858e834455c8\" (UID: \"215a694b-de08-40fa-8049-858e834455c8\") "
Sep 15 20:28:51.192220 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:51.192092    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/215a694b-de08-40fa-8049-858e834455c8-maas-api-tls\") pod \"215a694b-de08-40fa-8049-858e834455c8\" (UID: \"215a694b-de08-40fa-8049-858e834455c8\") "
Sep 15 20:28:51.192220 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:51.192214    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/215a694b-de08-40fa-8049-858e834455c8-metrics-certs\") pod \"215a694b-de08-40fa-8049-858e834455c8\" (UID: \"215a694b-de08-40fa-8049-858e834455c8\") "
Sep 15 20:28:51.192347 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:51.192261    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cplks\" (UniqueName: \"kubernetes.io/projected/215a694b-de08-40fa-8049-858e834455c8-kube-api-access-cplks\") pod \"215a694b-de08-40fa-8049-858e834455c8\" (UID: \"215a694b-de08-40fa-8049-858e834455c8\") "
Sep 15 20:28:51.193027 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:51.192782    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/215a694b-de08-40fa-8049-858e834455c8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "215a694b-de08-40fa-8049-858e834455c8" (UID: "215a694b-de08-40fa-8049-858e834455c8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:28:51.195363 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:51.195329    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/215a694b-de08-40fa-8049-858e834455c8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "215a694b-de08-40fa-8049-858e834455c8" (UID: "215a694b-de08-40fa-8049-858e834455c8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:28:51.195806 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:51.195774    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/215a694b-de08-40fa-8049-858e834455c8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "215a694b-de08-40fa-8049-858e834455c8" (UID: "215a694b-de08-40fa-8049-858e834455c8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:28:51.196106 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:51.196085    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/215a694b-de08-40fa-8049-858e834455c8-kube-api-access-cplks" (OuterVolumeSpecName: "kube-api-access-cplks") pod "215a694b-de08-40fa-8049-858e834455c8" (UID: "215a694b-de08-40fa-8049-858e834455c8"). InnerVolumeSpecName "kube-api-access-cplks". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:28:51.246865 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:51.246823    2577 generic.go:356] "Generic (PLEG): container finished" podID="215a694b-de08-40fa-8049-858e834455c8" containerID="7793011affcbc20d5e49fef33ada40da0cab069588993f240eeb0615b26aa2d9" exitCode=0
Sep 15 20:28:51.246865 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:51.246867    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c"
Sep 15 20:28:51.247122 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:51.246900    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c" event={"ID":"215a694b-de08-40fa-8049-858e834455c8","Type":"ContainerDied","Data":"7793011affcbc20d5e49fef33ada40da0cab069588993f240eeb0615b26aa2d9"}
Sep 15 20:28:51.247122 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:51.246967    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c" event={"ID":"215a694b-de08-40fa-8049-858e834455c8","Type":"ContainerDied","Data":"eb9cf862ef6c9da78e4b0505d9fcadb4833a0ce4261210c6c4641e53ec75ce3e"}
Sep 15 20:28:51.247122 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:51.246999    2577 scope.go:117] "RemoveContainer" containerID="7793011affcbc20d5e49fef33ada40da0cab069588993f240eeb0615b26aa2d9"
Sep 15 20:28:51.257871 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:51.257850    2577 scope.go:117] "RemoveContainer" containerID="7793011affcbc20d5e49fef33ada40da0cab069588993f240eeb0615b26aa2d9"
Sep 15 20:28:51.258165 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:28:51.258143    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7793011affcbc20d5e49fef33ada40da0cab069588993f240eeb0615b26aa2d9\": container with ID starting with 7793011affcbc20d5e49fef33ada40da0cab069588993f240eeb0615b26aa2d9 not found: ID does not exist" containerID="7793011affcbc20d5e49fef33ada40da0cab069588993f240eeb0615b26aa2d9"
Sep 15 20:28:51.258239 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:51.258174    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7793011affcbc20d5e49fef33ada40da0cab069588993f240eeb0615b26aa2d9"} err="failed to get container status \"7793011affcbc20d5e49fef33ada40da0cab069588993f240eeb0615b26aa2d9\": rpc error: code = NotFound desc = could not find container \"7793011affcbc20d5e49fef33ada40da0cab069588993f240eeb0615b26aa2d9\": container with ID starting with 7793011affcbc20d5e49fef33ada40da0cab069588993f240eeb0615b26aa2d9 not found: ID does not exist"
Sep 15 20:28:51.270237 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:51.270211    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c"]
Sep 15 20:28:51.274146 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:51.274121    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-01ea3b-9f9b996d6-zgw2c"]
Sep 15 20:28:51.293360 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:51.293331    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/215a694b-de08-40fa-8049-858e834455c8-maas-api-tls\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:28:51.293360 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:51.293356    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/215a694b-de08-40fa-8049-858e834455c8-metrics-certs\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:28:51.293360 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:51.293366    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cplks\" (UniqueName: \"kubernetes.io/projected/215a694b-de08-40fa-8049-858e834455c8-kube-api-access-cplks\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:28:51.293585 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:51.293376    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/215a694b-de08-40fa-8049-858e834455c8-openshift-service-ca\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:28:52.062949 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:28:52.062915    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="215a694b-de08-40fa-8049-858e834455c8" path="/var/lib/kubelet/pods/215a694b-de08-40fa-8049-858e834455c8/volumes"
Sep 15 20:29:03.455181 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:03.455143    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"]
Sep 15 20:29:03.455860 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:03.455348    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn" podUID="791ba0be-2ec9-4bd6-a6c1-23c041733a5a" containerName="istio-proxy" containerID="cri-o://5290df04b1bfc906056dac35e5e14a6441f37342a48493cb6014c9c1c0bbff83" gracePeriod=30
Sep 15 20:29:06.601735 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:06.601696    2577 patch_prober.go:28] interesting pod/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 20:29:06.602119 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:06.601777    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn" podUID="791ba0be-2ec9-4bd6-a6c1-23c041733a5a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 20:29:08.510139 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.510104    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn"]
Sep 15 20:29:08.510594 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.510335    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn" podUID="c3b338f0-d987-483c-ba79-420e187989c4" containerName="maas-api" containerID="cri-o://9357fd089a4b2a57d7dcb4e31e1b40311b79efedb7631af1323cae56b6e69a38" gracePeriod=30
Sep 15 20:29:08.566979 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.566942    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-39c7b0-76f9c875cf-6km95"]
Sep 15 20:29:08.567536 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.567484    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-39c7b0-76f9c875cf-6km95" podUID="93eee1a1-9cfd-436e-975d-e6ca9d3d722d" containerName="payload-processing" containerID="cri-o://af8de42825456f74c4a058236d78f9fcb3fd415052b788bc99f1a9aacecd6ca2" gracePeriod=30
Sep 15 20:29:08.655311 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:29:08.655270    2577 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93eee1a1_9cfd_436e_975d_e6ca9d3d722d.slice/crio-conmon-af8de42825456f74c4a058236d78f9fcb3fd415052b788bc99f1a9aacecd6ca2.scope\": RecentStats: unable to find data in memory cache]"
Sep 15 20:29:08.804575 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.804542    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"
Sep 15 20:29:08.844313 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.843980    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn"
Sep 15 20:29:08.886948 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.886883    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-istio-data\") pod \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\" (UID: \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\") "
Sep 15 20:29:08.886948 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.886924    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-workload-socket\") pod \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\" (UID: \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\") "
Sep 15 20:29:08.887187 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.886991    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-istiod-ca-cert\") pod \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\" (UID: \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\") "
Sep 15 20:29:08.887187 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.887054    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-istio-envoy\") pod \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\" (UID: \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\") "
Sep 15 20:29:08.887187 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.887077    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-workload-certs\") pod \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\" (UID: \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\") "
Sep 15 20:29:08.887187 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.887124    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-istio-podinfo\") pod \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\" (UID: \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\") "
Sep 15 20:29:08.887187 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.887153    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-istio-token\") pod \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\" (UID: \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\") "
Sep 15 20:29:08.887445 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.887215    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-credential-socket\") pod \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\" (UID: \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\") "
Sep 15 20:29:08.887445 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.887261    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5nnzg\" (UniqueName: \"kubernetes.io/projected/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-kube-api-access-5nnzg\") pod \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\" (UID: \"791ba0be-2ec9-4bd6-a6c1-23c041733a5a\") "
Sep 15 20:29:08.887925 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.887898    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "791ba0be-2ec9-4bd6-a6c1-23c041733a5a" (UID: "791ba0be-2ec9-4bd6-a6c1-23c041733a5a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:29:08.888214 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.888155    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-istio-data" (OuterVolumeSpecName: "istio-data") pod "791ba0be-2ec9-4bd6-a6c1-23c041733a5a" (UID: "791ba0be-2ec9-4bd6-a6c1-23c041733a5a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:29:08.888645 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.888348    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "791ba0be-2ec9-4bd6-a6c1-23c041733a5a" (UID: "791ba0be-2ec9-4bd6-a6c1-23c041733a5a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:29:08.888645 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.888464    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "791ba0be-2ec9-4bd6-a6c1-23c041733a5a" (UID: "791ba0be-2ec9-4bd6-a6c1-23c041733a5a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:29:08.888645 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.888585    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "791ba0be-2ec9-4bd6-a6c1-23c041733a5a" (UID: "791ba0be-2ec9-4bd6-a6c1-23c041733a5a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:29:08.891065 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.891030    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-kube-api-access-5nnzg" (OuterVolumeSpecName: "kube-api-access-5nnzg") pod "791ba0be-2ec9-4bd6-a6c1-23c041733a5a" (UID: "791ba0be-2ec9-4bd6-a6c1-23c041733a5a"). InnerVolumeSpecName "kube-api-access-5nnzg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:29:08.891170 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.891108    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "791ba0be-2ec9-4bd6-a6c1-23c041733a5a" (UID: "791ba0be-2ec9-4bd6-a6c1-23c041733a5a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 20:29:08.891170 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.891124    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "791ba0be-2ec9-4bd6-a6c1-23c041733a5a" (UID: "791ba0be-2ec9-4bd6-a6c1-23c041733a5a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:29:08.892123 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.892097    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-istio-token" (OuterVolumeSpecName: "istio-token") pod "791ba0be-2ec9-4bd6-a6c1-23c041733a5a" (UID: "791ba0be-2ec9-4bd6-a6c1-23c041733a5a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:29:08.894625 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.894601    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-39c7b0-76f9c875cf-6km95"
Sep 15 20:29:08.988546 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.988502    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3b338f0-d987-483c-ba79-420e187989c4-openshift-service-ca\") pod \"c3b338f0-d987-483c-ba79-420e187989c4\" (UID: \"c3b338f0-d987-483c-ba79-420e187989c4\") "
Sep 15 20:29:08.988790 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.988558    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3b338f0-d987-483c-ba79-420e187989c4-metrics-certs\") pod \"c3b338f0-d987-483c-ba79-420e187989c4\" (UID: \"c3b338f0-d987-483c-ba79-420e187989c4\") "
Sep 15 20:29:08.988790 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.988592    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3b338f0-d987-483c-ba79-420e187989c4-maas-api-tls\") pod \"c3b338f0-d987-483c-ba79-420e187989c4\" (UID: \"c3b338f0-d987-483c-ba79-420e187989c4\") "
Sep 15 20:29:08.988790 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.988616    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93eee1a1-9cfd-436e-975d-e6ca9d3d722d-plugins-config-volume\") pod \"93eee1a1-9cfd-436e-975d-e6ca9d3d722d\" (UID: \"93eee1a1-9cfd-436e-975d-e6ca9d3d722d\") "
Sep 15 20:29:08.988790 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.988690    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ff867\" (UniqueName: \"kubernetes.io/projected/c3b338f0-d987-483c-ba79-420e187989c4-kube-api-access-ff867\") pod \"c3b338f0-d987-483c-ba79-420e187989c4\" (UID: \"c3b338f0-d987-483c-ba79-420e187989c4\") "
Sep 15 20:29:08.988790 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.988781    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xqz66\" (UniqueName: \"kubernetes.io/projected/93eee1a1-9cfd-436e-975d-e6ca9d3d722d-kube-api-access-xqz66\") pod \"93eee1a1-9cfd-436e-975d-e6ca9d3d722d\" (UID: \"93eee1a1-9cfd-436e-975d-e6ca9d3d722d\") "
Sep 15 20:29:08.989324 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.989160    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93eee1a1-9cfd-436e-975d-e6ca9d3d722d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "93eee1a1-9cfd-436e-975d-e6ca9d3d722d" (UID: "93eee1a1-9cfd-436e-975d-e6ca9d3d722d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:29:08.989324 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.989188    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-istio-envoy\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:29:08.989324 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.989212    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-workload-certs\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:29:08.989324 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.989225    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-istio-podinfo\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:29:08.989324 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.989237    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-istio-token\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:29:08.989324 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.989250    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-credential-socket\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:29:08.989324 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.989263    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5nnzg\" (UniqueName: \"kubernetes.io/projected/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-kube-api-access-5nnzg\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:29:08.989324 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.989276    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-istio-data\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:29:08.989324 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.989289    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-workload-socket\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:29:08.989324 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.989300    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/791ba0be-2ec9-4bd6-a6c1-23c041733a5a-istiod-ca-cert\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:29:08.990016 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.989805    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3b338f0-d987-483c-ba79-420e187989c4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c3b338f0-d987-483c-ba79-420e187989c4" (UID: "c3b338f0-d987-483c-ba79-420e187989c4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:29:08.991473 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.991444    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3b338f0-d987-483c-ba79-420e187989c4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c3b338f0-d987-483c-ba79-420e187989c4" (UID: "c3b338f0-d987-483c-ba79-420e187989c4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:29:08.991834 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.991802    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3b338f0-d987-483c-ba79-420e187989c4-kube-api-access-ff867" (OuterVolumeSpecName: "kube-api-access-ff867") pod "c3b338f0-d987-483c-ba79-420e187989c4" (UID: "c3b338f0-d987-483c-ba79-420e187989c4"). InnerVolumeSpecName "kube-api-access-ff867". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:29:08.991935 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.991859    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93eee1a1-9cfd-436e-975d-e6ca9d3d722d-kube-api-access-xqz66" (OuterVolumeSpecName: "kube-api-access-xqz66") pod "93eee1a1-9cfd-436e-975d-e6ca9d3d722d" (UID: "93eee1a1-9cfd-436e-975d-e6ca9d3d722d"). InnerVolumeSpecName "kube-api-access-xqz66". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:29:08.992288 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:08.992266    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3b338f0-d987-483c-ba79-420e187989c4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c3b338f0-d987-483c-ba79-420e187989c4" (UID: "c3b338f0-d987-483c-ba79-420e187989c4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:29:09.089999 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:09.089951    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3b338f0-d987-483c-ba79-420e187989c4-openshift-service-ca\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:29:09.089999 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:09.089998    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3b338f0-d987-483c-ba79-420e187989c4-metrics-certs\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:29:09.089999 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:09.090011    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3b338f0-d987-483c-ba79-420e187989c4-maas-api-tls\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:29:09.090314 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:09.090024    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93eee1a1-9cfd-436e-975d-e6ca9d3d722d-plugins-config-volume\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:29:09.090314 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:09.090036    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ff867\" (UniqueName: \"kubernetes.io/projected/c3b338f0-d987-483c-ba79-420e187989c4-kube-api-access-ff867\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:29:09.090314 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:09.090047    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xqz66\" (UniqueName: \"kubernetes.io/projected/93eee1a1-9cfd-436e-975d-e6ca9d3d722d-kube-api-access-xqz66\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:29:09.374910 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:09.374811    2577 generic.go:356] "Generic (PLEG): container finished" podID="93eee1a1-9cfd-436e-975d-e6ca9d3d722d" containerID="af8de42825456f74c4a058236d78f9fcb3fd415052b788bc99f1a9aacecd6ca2" exitCode=0
Sep 15 20:29:09.374910 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:09.374870    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-39c7b0-76f9c875cf-6km95"
Sep 15 20:29:09.374910 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:09.374892    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-39c7b0-76f9c875cf-6km95" event={"ID":"93eee1a1-9cfd-436e-975d-e6ca9d3d722d","Type":"ContainerDied","Data":"af8de42825456f74c4a058236d78f9fcb3fd415052b788bc99f1a9aacecd6ca2"}
Sep 15 20:29:09.375200 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:09.374951    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-39c7b0-76f9c875cf-6km95" event={"ID":"93eee1a1-9cfd-436e-975d-e6ca9d3d722d","Type":"ContainerDied","Data":"1b94087590522e3e8fc44fa09cfbd0b5fca674901f46d35bce1897a86f95c644"}
Sep 15 20:29:09.375200 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:09.374981    2577 scope.go:117] "RemoveContainer" containerID="af8de42825456f74c4a058236d78f9fcb3fd415052b788bc99f1a9aacecd6ca2"
Sep 15 20:29:09.376632 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:09.376607    2577 generic.go:356] "Generic (PLEG): container finished" podID="791ba0be-2ec9-4bd6-a6c1-23c041733a5a" containerID="5290df04b1bfc906056dac35e5e14a6441f37342a48493cb6014c9c1c0bbff83" exitCode=0
Sep 15 20:29:09.376771 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:09.376669    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"
Sep 15 20:29:09.376771 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:09.376712    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn" event={"ID":"791ba0be-2ec9-4bd6-a6c1-23c041733a5a","Type":"ContainerDied","Data":"5290df04b1bfc906056dac35e5e14a6441f37342a48493cb6014c9c1c0bbff83"}
Sep 15 20:29:09.377154 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:09.376796    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn" event={"ID":"791ba0be-2ec9-4bd6-a6c1-23c041733a5a","Type":"ContainerDied","Data":"c0cbe9b89aa3dcf2ed3fb076634df24adfda51b44f4fc57cb4259d943ab9b573"}
Sep 15 20:29:09.378567 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:09.378541    2577 generic.go:356] "Generic (PLEG): container finished" podID="c3b338f0-d987-483c-ba79-420e187989c4" containerID="9357fd089a4b2a57d7dcb4e31e1b40311b79efedb7631af1323cae56b6e69a38" exitCode=0
Sep 15 20:29:09.378677 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:09.378629    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn" event={"ID":"c3b338f0-d987-483c-ba79-420e187989c4","Type":"ContainerDied","Data":"9357fd089a4b2a57d7dcb4e31e1b40311b79efedb7631af1323cae56b6e69a38"}
Sep 15 20:29:09.378677 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:09.378669    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn" event={"ID":"c3b338f0-d987-483c-ba79-420e187989c4","Type":"ContainerDied","Data":"16fc738d8e8ee53772170e662747d21e4f9a52f06fa5b663226ae33b986ec760"}
Sep 15 20:29:09.378840 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:09.378754    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn"
Sep 15 20:29:09.391621 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:09.391595    2577 scope.go:117] "RemoveContainer" containerID="af8de42825456f74c4a058236d78f9fcb3fd415052b788bc99f1a9aacecd6ca2"
Sep 15 20:29:09.391978 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:29:09.391958    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af8de42825456f74c4a058236d78f9fcb3fd415052b788bc99f1a9aacecd6ca2\": container with ID starting with af8de42825456f74c4a058236d78f9fcb3fd415052b788bc99f1a9aacecd6ca2 not found: ID does not exist" containerID="af8de42825456f74c4a058236d78f9fcb3fd415052b788bc99f1a9aacecd6ca2"
Sep 15 20:29:09.392045 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:09.391987    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af8de42825456f74c4a058236d78f9fcb3fd415052b788bc99f1a9aacecd6ca2"} err="failed to get container status \"af8de42825456f74c4a058236d78f9fcb3fd415052b788bc99f1a9aacecd6ca2\": rpc error: code = NotFound desc = could not find container \"af8de42825456f74c4a058236d78f9fcb3fd415052b788bc99f1a9aacecd6ca2\": container with ID starting with af8de42825456f74c4a058236d78f9fcb3fd415052b788bc99f1a9aacecd6ca2 not found: ID does not exist"
Sep 15 20:29:09.392045 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:09.392003    2577 scope.go:117] "RemoveContainer" containerID="5290df04b1bfc906056dac35e5e14a6441f37342a48493cb6014c9c1c0bbff83"
Sep 15 20:29:09.401939 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:09.401917    2577 scope.go:117] "RemoveContainer" containerID="5290df04b1bfc906056dac35e5e14a6441f37342a48493cb6014c9c1c0bbff83"
Sep 15 20:29:09.402225 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:29:09.402207    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5290df04b1bfc906056dac35e5e14a6441f37342a48493cb6014c9c1c0bbff83\": container with ID starting with 5290df04b1bfc906056dac35e5e14a6441f37342a48493cb6014c9c1c0bbff83 not found: ID does not exist" containerID="5290df04b1bfc906056dac35e5e14a6441f37342a48493cb6014c9c1c0bbff83"
Sep 15 20:29:09.402296 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:09.402231    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5290df04b1bfc906056dac35e5e14a6441f37342a48493cb6014c9c1c0bbff83"} err="failed to get container status \"5290df04b1bfc906056dac35e5e14a6441f37342a48493cb6014c9c1c0bbff83\": rpc error: code = NotFound desc = could not find container \"5290df04b1bfc906056dac35e5e14a6441f37342a48493cb6014c9c1c0bbff83\": container with ID starting with 5290df04b1bfc906056dac35e5e14a6441f37342a48493cb6014c9c1c0bbff83 not found: ID does not exist"
Sep 15 20:29:09.402296 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:09.402247    2577 scope.go:117] "RemoveContainer" containerID="9357fd089a4b2a57d7dcb4e31e1b40311b79efedb7631af1323cae56b6e69a38"
Sep 15 20:29:09.406949 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:09.406927    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-39c7b0-76f9c875cf-6km95"]
Sep 15 20:29:09.412799 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:09.412768    2577 scope.go:117] "RemoveContainer" containerID="9357fd089a4b2a57d7dcb4e31e1b40311b79efedb7631af1323cae56b6e69a38"
Sep 15 20:29:09.413140 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:29:09.413117    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9357fd089a4b2a57d7dcb4e31e1b40311b79efedb7631af1323cae56b6e69a38\": container with ID starting with 9357fd089a4b2a57d7dcb4e31e1b40311b79efedb7631af1323cae56b6e69a38 not found: ID does not exist" containerID="9357fd089a4b2a57d7dcb4e31e1b40311b79efedb7631af1323cae56b6e69a38"
Sep 15 20:29:09.413238 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:09.413148    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9357fd089a4b2a57d7dcb4e31e1b40311b79efedb7631af1323cae56b6e69a38"} err="failed to get container status \"9357fd089a4b2a57d7dcb4e31e1b40311b79efedb7631af1323cae56b6e69a38\": rpc error: code = NotFound desc = could not find container \"9357fd089a4b2a57d7dcb4e31e1b40311b79efedb7631af1323cae56b6e69a38\": container with ID starting with 9357fd089a4b2a57d7dcb4e31e1b40311b79efedb7631af1323cae56b6e69a38 not found: ID does not exist"
Sep 15 20:29:09.414116 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:09.414098    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-39c7b0-76f9c875cf-6km95"]
Sep 15 20:29:09.436154 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:09.436126    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"]
Sep 15 20:29:09.441980 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:09.441955    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-6733e9-openshift-default-75fb85fc8-f46hn"]
Sep 15 20:29:09.453934 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:09.453908    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn"]
Sep 15 20:29:09.458076 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:09.458050    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-39c7b0-79b5dc7b79-gmpfn"]
Sep 15 20:29:10.061709 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:10.061673    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="791ba0be-2ec9-4bd6-a6c1-23c041733a5a" path="/var/lib/kubelet/pods/791ba0be-2ec9-4bd6-a6c1-23c041733a5a/volumes"
Sep 15 20:29:10.062146 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:10.062129    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93eee1a1-9cfd-436e-975d-e6ca9d3d722d" path="/var/lib/kubelet/pods/93eee1a1-9cfd-436e-975d-e6ca9d3d722d/volumes"
Sep 15 20:29:10.062499 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:10.062486    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3b338f0-d987-483c-ba79-420e187989c4" path="/var/lib/kubelet/pods/c3b338f0-d987-483c-ba79-420e187989c4/volumes"
Sep 15 20:29:16.359489 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.359443    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"]
Sep 15 20:29:16.360218 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.360195    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="215a694b-de08-40fa-8049-858e834455c8" containerName="maas-api"
Sep 15 20:29:16.360218 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.360218    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="215a694b-de08-40fa-8049-858e834455c8" containerName="maas-api"
Sep 15 20:29:16.360401 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.360230    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3b338f0-d987-483c-ba79-420e187989c4" containerName="maas-api"
Sep 15 20:29:16.360401 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.360238    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3b338f0-d987-483c-ba79-420e187989c4" containerName="maas-api"
Sep 15 20:29:16.360401 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.360263    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93eee1a1-9cfd-436e-975d-e6ca9d3d722d" containerName="payload-processing"
Sep 15 20:29:16.360401 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.360270    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="93eee1a1-9cfd-436e-975d-e6ca9d3d722d" containerName="payload-processing"
Sep 15 20:29:16.360401 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.360282    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="791ba0be-2ec9-4bd6-a6c1-23c041733a5a" containerName="istio-proxy"
Sep 15 20:29:16.360401 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.360287    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="791ba0be-2ec9-4bd6-a6c1-23c041733a5a" containerName="istio-proxy"
Sep 15 20:29:16.360401 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.360356    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="93eee1a1-9cfd-436e-975d-e6ca9d3d722d" containerName="payload-processing"
Sep 15 20:29:16.360401 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.360366    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="215a694b-de08-40fa-8049-858e834455c8" containerName="maas-api"
Sep 15 20:29:16.360401 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.360374    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3b338f0-d987-483c-ba79-420e187989c4" containerName="maas-api"
Sep 15 20:29:16.360401 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.360381    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="791ba0be-2ec9-4bd6-a6c1-23c041733a5a" containerName="istio-proxy"
Sep 15 20:29:16.365340 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.365323    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:29:16.368413 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.368380    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-92702a-openshift-default-dockercfg-njv2f\""
Sep 15 20:29:16.370241 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.369281    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d311d46a-a8a2-4f83-9559-39fd8630e3f3-credential-socket\") pod \"e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk\" (UID: \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\") " pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:29:16.370241 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.369358    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d311d46a-a8a2-4f83-9559-39fd8630e3f3-workload-certs\") pod \"e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk\" (UID: \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\") " pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:29:16.370241 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.369396    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d311d46a-a8a2-4f83-9559-39fd8630e3f3-istiod-ca-cert\") pod \"e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk\" (UID: \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\") " pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:29:16.370241 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.369429    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d311d46a-a8a2-4f83-9559-39fd8630e3f3-istio-podinfo\") pod \"e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk\" (UID: \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\") " pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:29:16.370241 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.369486    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d311d46a-a8a2-4f83-9559-39fd8630e3f3-workload-socket\") pod \"e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk\" (UID: \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\") " pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:29:16.370241 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.369521    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d311d46a-a8a2-4f83-9559-39fd8630e3f3-istio-envoy\") pod \"e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk\" (UID: \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\") " pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:29:16.370241 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.369551    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d311d46a-a8a2-4f83-9559-39fd8630e3f3-istio-data\") pod \"e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk\" (UID: \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\") " pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:29:16.370241 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.369584    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d311d46a-a8a2-4f83-9559-39fd8630e3f3-istio-token\") pod \"e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk\" (UID: \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\") " pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:29:16.370241 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.369633    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvppj\" (UniqueName: \"kubernetes.io/projected/d311d46a-a8a2-4f83-9559-39fd8630e3f3-kube-api-access-wvppj\") pod \"e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk\" (UID: \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\") " pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:29:16.382424 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.378217    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"]
Sep 15 20:29:16.470428 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.470383    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvppj\" (UniqueName: \"kubernetes.io/projected/d311d46a-a8a2-4f83-9559-39fd8630e3f3-kube-api-access-wvppj\") pod \"e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk\" (UID: \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\") " pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:29:16.470692 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.470537    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d311d46a-a8a2-4f83-9559-39fd8630e3f3-credential-socket\") pod \"e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk\" (UID: \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\") " pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:29:16.470692 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.470588    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d311d46a-a8a2-4f83-9559-39fd8630e3f3-workload-certs\") pod \"e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk\" (UID: \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\") " pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:29:16.470692 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.470624    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d311d46a-a8a2-4f83-9559-39fd8630e3f3-istiod-ca-cert\") pod \"e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk\" (UID: \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\") " pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:29:16.471072 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.471044    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d311d46a-a8a2-4f83-9559-39fd8630e3f3-istio-podinfo\") pod \"e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk\" (UID: \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\") " pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:29:16.471270 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.471251    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d311d46a-a8a2-4f83-9559-39fd8630e3f3-workload-socket\") pod \"e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk\" (UID: \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\") " pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:29:16.471406 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.471389    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d311d46a-a8a2-4f83-9559-39fd8630e3f3-istio-envoy\") pod \"e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk\" (UID: \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\") " pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:29:16.471514 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.471500    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d311d46a-a8a2-4f83-9559-39fd8630e3f3-istio-data\") pod \"e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk\" (UID: \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\") " pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:29:16.472377 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.471956    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d311d46a-a8a2-4f83-9559-39fd8630e3f3-istio-token\") pod \"e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk\" (UID: \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\") " pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:29:16.472377 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.471110    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d311d46a-a8a2-4f83-9559-39fd8630e3f3-workload-certs\") pod \"e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk\" (UID: \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\") " pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:29:16.472377 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.471759    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d311d46a-a8a2-4f83-9559-39fd8630e3f3-credential-socket\") pod \"e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk\" (UID: \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\") " pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:29:16.472377 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.471846    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d311d46a-a8a2-4f83-9559-39fd8630e3f3-workload-socket\") pod \"e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk\" (UID: \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\") " pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:29:16.472377 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.471563    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d311d46a-a8a2-4f83-9559-39fd8630e3f3-istiod-ca-cert\") pod \"e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk\" (UID: \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\") " pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:29:16.472789 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.472485    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d311d46a-a8a2-4f83-9559-39fd8630e3f3-istio-data\") pod \"e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk\" (UID: \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\") " pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:29:16.474897 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.474873    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d311d46a-a8a2-4f83-9559-39fd8630e3f3-istio-envoy\") pod \"e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk\" (UID: \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\") " pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:29:16.479198 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.479175    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d311d46a-a8a2-4f83-9559-39fd8630e3f3-istio-podinfo\") pod \"e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk\" (UID: \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\") " pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:29:16.479429 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.479405    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvppj\" (UniqueName: \"kubernetes.io/projected/d311d46a-a8a2-4f83-9559-39fd8630e3f3-kube-api-access-wvppj\") pod \"e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk\" (UID: \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\") " pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:29:16.480884 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.480864    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d311d46a-a8a2-4f83-9559-39fd8630e3f3-istio-token\") pod \"e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk\" (UID: \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\") " pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:29:16.682787 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.682680    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:29:16.966173 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.966140    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"]
Sep 15 20:29:16.967621 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:29:16.967593    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd311d46a_a8a2_4f83_9559_39fd8630e3f3.slice/crio-0a000a196bc66d6bcd5c69a6f151c0e0d758da1860be15515a5e8d596f3a9546 WatchSource:0}: Error finding container 0a000a196bc66d6bcd5c69a6f151c0e0d758da1860be15515a5e8d596f3a9546: Status 404 returned error can't find the container with id 0a000a196bc66d6bcd5c69a6f151c0e0d758da1860be15515a5e8d596f3a9546
Sep 15 20:29:16.969987 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.969941    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 20:29:16.970068 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.970042    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 20:29:16.970110 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:16.970090    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 20:29:17.436572 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:17.436463    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk" event={"ID":"d311d46a-a8a2-4f83-9559-39fd8630e3f3","Type":"ContainerStarted","Data":"bcc8b5607ad60d7265250701e59d15acb1ace8814b4e754dcf8eca193b6ee040"}
Sep 15 20:29:17.436572 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:17.436525    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk" event={"ID":"d311d46a-a8a2-4f83-9559-39fd8630e3f3","Type":"ContainerStarted","Data":"0a000a196bc66d6bcd5c69a6f151c0e0d758da1860be15515a5e8d596f3a9546"}
Sep 15 20:29:17.460404 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:17.460339    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk" podStartSLOduration=1.460318888 podStartE2EDuration="1.460318888s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:29:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:29:17.455466877 +0000 UTC m=+1463.846419875" watchObservedRunningTime="2026-09-15 20:29:17.460318888 +0000 UTC m=+1463.851271888"
Sep 15 20:29:17.683418 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:17.683380    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:29:18.688602 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:18.688569    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:29:19.451928 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:19.451893    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:29:19.452984 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:19.452962    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:29:23.939742 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:23.939690    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf"]
Sep 15 20:29:23.943762 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:23.943742    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf"
Sep 15 20:29:23.946186 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:23.946166    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-92702a-serving-cert\""
Sep 15 20:29:23.954929 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:23.954903    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf"]
Sep 15 20:29:24.052741 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:24.052691    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxlgn\" (UniqueName: \"kubernetes.io/projected/fdcedef1-0099-4a2d-99cc-b76ec9074ca1-kube-api-access-jxlgn\") pod \"maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf\" (UID: \"fdcedef1-0099-4a2d-99cc-b76ec9074ca1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf"
Sep 15 20:29:24.052909 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:24.052789    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdcedef1-0099-4a2d-99cc-b76ec9074ca1-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf\" (UID: \"fdcedef1-0099-4a2d-99cc-b76ec9074ca1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf"
Sep 15 20:29:24.052909 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:24.052826    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdcedef1-0099-4a2d-99cc-b76ec9074ca1-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf\" (UID: \"fdcedef1-0099-4a2d-99cc-b76ec9074ca1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf"
Sep 15 20:29:24.052983 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:24.052900    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdcedef1-0099-4a2d-99cc-b76ec9074ca1-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf\" (UID: \"fdcedef1-0099-4a2d-99cc-b76ec9074ca1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf"
Sep 15 20:29:24.154238 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:24.154182    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdcedef1-0099-4a2d-99cc-b76ec9074ca1-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf\" (UID: \"fdcedef1-0099-4a2d-99cc-b76ec9074ca1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf"
Sep 15 20:29:24.154424 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:24.154337    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxlgn\" (UniqueName: \"kubernetes.io/projected/fdcedef1-0099-4a2d-99cc-b76ec9074ca1-kube-api-access-jxlgn\") pod \"maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf\" (UID: \"fdcedef1-0099-4a2d-99cc-b76ec9074ca1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf"
Sep 15 20:29:24.154424 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:24.154369    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdcedef1-0099-4a2d-99cc-b76ec9074ca1-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf\" (UID: \"fdcedef1-0099-4a2d-99cc-b76ec9074ca1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf"
Sep 15 20:29:24.154424 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:24.154400    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdcedef1-0099-4a2d-99cc-b76ec9074ca1-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf\" (UID: \"fdcedef1-0099-4a2d-99cc-b76ec9074ca1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf"
Sep 15 20:29:24.154597 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:29:24.154516    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-92702a-serving-cert: secret "maas-api-e2e-shared-a-w3-92702a-serving-cert" not found
Sep 15 20:29:24.154668 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:29:24.154604    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdcedef1-0099-4a2d-99cc-b76ec9074ca1-maas-api-tls podName:fdcedef1-0099-4a2d-99cc-b76ec9074ca1 nodeName:}" failed. No retries permitted until 2026-09-15 20:29:24.654584655 +0000 UTC m=+1471.045537633 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fdcedef1-0099-4a2d-99cc-b76ec9074ca1-maas-api-tls") pod "maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf" (UID: "fdcedef1-0099-4a2d-99cc-b76ec9074ca1") : secret "maas-api-e2e-shared-a-w3-92702a-serving-cert" not found
Sep 15 20:29:24.155104 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:24.155079    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdcedef1-0099-4a2d-99cc-b76ec9074ca1-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf\" (UID: \"fdcedef1-0099-4a2d-99cc-b76ec9074ca1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf"
Sep 15 20:29:24.156874 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:24.156854    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdcedef1-0099-4a2d-99cc-b76ec9074ca1-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf\" (UID: \"fdcedef1-0099-4a2d-99cc-b76ec9074ca1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf"
Sep 15 20:29:24.162968 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:24.162948    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxlgn\" (UniqueName: \"kubernetes.io/projected/fdcedef1-0099-4a2d-99cc-b76ec9074ca1-kube-api-access-jxlgn\") pod \"maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf\" (UID: \"fdcedef1-0099-4a2d-99cc-b76ec9074ca1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf"
Sep 15 20:29:24.660008 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:24.659967    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdcedef1-0099-4a2d-99cc-b76ec9074ca1-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf\" (UID: \"fdcedef1-0099-4a2d-99cc-b76ec9074ca1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf"
Sep 15 20:29:24.662591 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:24.662554    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdcedef1-0099-4a2d-99cc-b76ec9074ca1-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf\" (UID: \"fdcedef1-0099-4a2d-99cc-b76ec9074ca1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf"
Sep 15 20:29:24.856253 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:24.856210    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf"
Sep 15 20:29:25.135620 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:25.135592    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf"]
Sep 15 20:29:25.137776 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:29:25.137744    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdcedef1_0099_4a2d_99cc_b76ec9074ca1.slice/crio-4edeeac597fa3813e8a1de38cf47cd3eafbac6447ccb4318b5663e6ca65f163f WatchSource:0}: Error finding container 4edeeac597fa3813e8a1de38cf47cd3eafbac6447ccb4318b5663e6ca65f163f: Status 404 returned error can't find the container with id 4edeeac597fa3813e8a1de38cf47cd3eafbac6447ccb4318b5663e6ca65f163f
Sep 15 20:29:25.496123 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:25.496086    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf" event={"ID":"fdcedef1-0099-4a2d-99cc-b76ec9074ca1","Type":"ContainerStarted","Data":"4edeeac597fa3813e8a1de38cf47cd3eafbac6447ccb4318b5663e6ca65f163f"}
Sep 15 20:29:26.504990 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:26.504946    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf" event={"ID":"fdcedef1-0099-4a2d-99cc-b76ec9074ca1","Type":"ContainerStarted","Data":"b7150d533c89d3f2da8df4cf2e66d7635f3fe8b3f0a30d6444556b29401e9e22"}
Sep 15 20:29:26.505379 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:26.505025    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf"
Sep 15 20:29:26.526202 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:26.526142    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf" podStartSLOduration=3.143543812 podStartE2EDuration="3.526127169s" totalImagesPullingTime="382.583357ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:29:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:29:26.522408429 +0000 UTC m=+1472.913361439" watchObservedRunningTime="2026-09-15 20:29:26.526127169 +0000 UTC m=+1472.917080166"
Sep 15 20:29:32.516884 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:32.516753    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf"
Sep 15 20:29:49.952771 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:49.952715    2577 patch_prober.go:28] interesting pod/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": read tcp 10.133.0.2:39192->10.133.0.81:15021: read: connection reset by peer" start-of-body=
Sep 15 20:29:49.953153 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:49.952811    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk" podUID="d311d46a-a8a2-4f83-9559-39fd8630e3f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": read tcp 10.133.0.2:39192->10.133.0.81:15021: read: connection reset by peer"
Sep 15 20:29:50.113503 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.113429    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"]
Sep 15 20:29:50.116155 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.116133    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:29:50.118717 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.118696    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-e80b38-openshift-default-dockercfg-vm8b9\""
Sep 15 20:29:50.127508 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.127481    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"]
Sep 15 20:29:50.144655 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.144619    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-istio-podinfo\") pod \"e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf\" (UID: \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\") " pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:29:50.144655 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.144658    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-istio-envoy\") pod \"e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf\" (UID: \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\") " pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:29:50.144974 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.144702    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-workload-socket\") pod \"e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf\" (UID: \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\") " pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:29:50.144974 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.144767    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-credential-socket\") pod \"e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf\" (UID: \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\") " pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:29:50.144974 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.144794    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-istiod-ca-cert\") pod \"e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf\" (UID: \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\") " pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:29:50.144974 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.144821    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-workload-certs\") pod \"e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf\" (UID: \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\") " pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:29:50.144974 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.144846    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggbsm\" (UniqueName: \"kubernetes.io/projected/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-kube-api-access-ggbsm\") pod \"e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf\" (UID: \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\") " pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:29:50.144974 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.144908    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-istio-token\") pod \"e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf\" (UID: \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\") " pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:29:50.144974 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.144932    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-istio-data\") pod \"e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf\" (UID: \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\") " pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:29:50.245488 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.245450    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-istio-podinfo\") pod \"e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf\" (UID: \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\") " pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:29:50.245691 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.245509    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-istio-envoy\") pod \"e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf\" (UID: \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\") " pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:29:50.245691 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.245566    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-workload-socket\") pod \"e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf\" (UID: \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\") " pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:29:50.245691 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.245584    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-credential-socket\") pod \"e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf\" (UID: \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\") " pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:29:50.245691 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.245625    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-istiod-ca-cert\") pod \"e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf\" (UID: \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\") " pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:29:50.245691 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.245663    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-workload-certs\") pod \"e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf\" (UID: \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\") " pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:29:50.245691 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.245681    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggbsm\" (UniqueName: \"kubernetes.io/projected/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-kube-api-access-ggbsm\") pod \"e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf\" (UID: \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\") " pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:29:50.246091 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.245802    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-istio-token\") pod \"e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf\" (UID: \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\") " pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:29:50.246091 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.245828    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-istio-data\") pod \"e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf\" (UID: \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\") " pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:29:50.246091 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.245988    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-workload-socket\") pod \"e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf\" (UID: \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\") " pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:29:50.246091 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.246048    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-credential-socket\") pod \"e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf\" (UID: \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\") " pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:29:50.246417 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.246389    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-workload-certs\") pod \"e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf\" (UID: \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\") " pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:29:50.246536 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.246493    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-istio-data\") pod \"e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf\" (UID: \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\") " pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:29:50.246782 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.246712    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-istiod-ca-cert\") pod \"e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf\" (UID: \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\") " pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:29:50.247988 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.247963    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-istio-envoy\") pod \"e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf\" (UID: \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\") " pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:29:50.248562 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.248538    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-istio-podinfo\") pod \"e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf\" (UID: \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\") " pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:29:50.253888 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.253868    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-istio-token\") pod \"e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf\" (UID: \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\") " pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:29:50.254063 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.254043    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggbsm\" (UniqueName: \"kubernetes.io/projected/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-kube-api-access-ggbsm\") pod \"e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf\" (UID: \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\") " pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:29:50.428658 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.428622    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:29:50.690905 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.690713    2577 generic.go:356] "Generic (PLEG): container finished" podID="d311d46a-a8a2-4f83-9559-39fd8630e3f3" containerID="bcc8b5607ad60d7265250701e59d15acb1ace8814b4e754dcf8eca193b6ee040" exitCode=0
Sep 15 20:29:50.690905 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.690767    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk" event={"ID":"d311d46a-a8a2-4f83-9559-39fd8630e3f3","Type":"ContainerDied","Data":"bcc8b5607ad60d7265250701e59d15acb1ace8814b4e754dcf8eca193b6ee040"}
Sep 15 20:29:50.691322 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.691302    2577 scope.go:117] "RemoveContainer" containerID="bcc8b5607ad60d7265250701e59d15acb1ace8814b4e754dcf8eca193b6ee040"
Sep 15 20:29:50.692786 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.692745    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 20:29:50.692909 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.692842    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 20:29:50.692909 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.692887    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 20:29:50.714304 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.714276    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"]
Sep 15 20:29:50.730390 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:29:50.730358    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1c8d5ae_098d_47d7_9476_afc4bb8ec4cb.slice/crio-2c517683f2209ff1d554e133a4ded480c6b1237d4bffad258e98b1525b42f1e6 WatchSource:0}: Error finding container 2c517683f2209ff1d554e133a4ded480c6b1237d4bffad258e98b1525b42f1e6: Status 404 returned error can't find the container with id 2c517683f2209ff1d554e133a4ded480c6b1237d4bffad258e98b1525b42f1e6
Sep 15 20:29:50.733214 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.733164    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 20:29:50.733316 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.733262    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 20:29:50.733316 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:50.733306    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 20:29:51.699235 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:51.699195    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf" event={"ID":"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb","Type":"ContainerStarted","Data":"d6d50268b388edf5d86f5e12cdce13731a938248b16c80597195253d7e000d5a"}
Sep 15 20:29:51.699672 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:51.699257    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf" event={"ID":"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb","Type":"ContainerStarted","Data":"2c517683f2209ff1d554e133a4ded480c6b1237d4bffad258e98b1525b42f1e6"}
Sep 15 20:29:51.701339 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:51.701314    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk" event={"ID":"d311d46a-a8a2-4f83-9559-39fd8630e3f3","Type":"ContainerStarted","Data":"afd25d786d2c8f7ed36039050404f061f828bab2b638aaffa5c7ed789a5b3ca7"}
Sep 15 20:29:51.721740 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:51.721675    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf" podStartSLOduration=1.721658491 podStartE2EDuration="1.721658491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:29:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:29:51.719251886 +0000 UTC m=+1498.110204883" watchObservedRunningTime="2026-09-15 20:29:51.721658491 +0000 UTC m=+1498.112611488"
Sep 15 20:29:52.429389 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:52.429346    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:29:52.435040 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:52.435013    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:29:52.683944 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:52.683851    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:29:52.684292 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:52.684270    2577 patch_prober.go:28] interesting pod/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Sep 15 20:29:52.684433 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:52.684405    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk" podUID="d311d46a-a8a2-4f83-9559-39fd8630e3f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Sep 15 20:29:52.711925 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:52.711893    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:29:52.712926 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:52.712901    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:29:53.683587 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:53.683549    2577 patch_prober.go:28] interesting pod/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Sep 15 20:29:53.683904 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:53.683875    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk" podUID="d311d46a-a8a2-4f83-9559-39fd8630e3f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Sep 15 20:29:54.206865 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:54.206830    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-r227b_ecab5774-79a4-42eb-9bb5-a1f09b18f3fb/console-operator/1.log"
Sep 15 20:29:54.216490 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:54.216466    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-r227b_ecab5774-79a4-42eb-9bb5-a1f09b18f3fb/console-operator/1.log"
Sep 15 20:29:54.683870 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:54.683770    2577 patch_prober.go:28] interesting pod/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Sep 15 20:29:54.684132 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:54.684105    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk" podUID="d311d46a-a8a2-4f83-9559-39fd8630e3f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Sep 15 20:29:55.683439 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:55.683392    2577 patch_prober.go:28] interesting pod/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Sep 15 20:29:55.683948 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:55.683464    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk" podUID="d311d46a-a8a2-4f83-9559-39fd8630e3f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Sep 15 20:29:56.683426 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:56.683387    2577 patch_prober.go:28] interesting pod/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Sep 15 20:29:56.683710 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:56.683676    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk" podUID="d311d46a-a8a2-4f83-9559-39fd8630e3f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Sep 15 20:29:57.733567 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:57.733530    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:29:57.734084 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:57.733944    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:29:57.736551 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:57.736528    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:29:58.750685 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:58.750652    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk"]
Sep 15 20:29:58.756369 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:58.756340    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk"
Sep 15 20:29:58.758910 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:58.758882    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-e80b38-serving-cert\""
Sep 15 20:29:58.764511 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:58.764479    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk"]
Sep 15 20:29:58.855689 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:58.855654    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/730a8f9a-0c38-453f-8cd0-56321c4592e0-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk\" (UID: \"730a8f9a-0c38-453f-8cd0-56321c4592e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk"
Sep 15 20:29:58.855910 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:58.855769    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/730a8f9a-0c38-453f-8cd0-56321c4592e0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk\" (UID: \"730a8f9a-0c38-453f-8cd0-56321c4592e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk"
Sep 15 20:29:58.855910 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:58.855803    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5ggg\" (UniqueName: \"kubernetes.io/projected/730a8f9a-0c38-453f-8cd0-56321c4592e0-kube-api-access-x5ggg\") pod \"maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk\" (UID: \"730a8f9a-0c38-453f-8cd0-56321c4592e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk"
Sep 15 20:29:58.856118 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:58.856100    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/730a8f9a-0c38-453f-8cd0-56321c4592e0-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk\" (UID: \"730a8f9a-0c38-453f-8cd0-56321c4592e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk"
Sep 15 20:29:58.956973 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:58.956933    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/730a8f9a-0c38-453f-8cd0-56321c4592e0-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk\" (UID: \"730a8f9a-0c38-453f-8cd0-56321c4592e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk"
Sep 15 20:29:58.957192 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:58.956997    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/730a8f9a-0c38-453f-8cd0-56321c4592e0-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk\" (UID: \"730a8f9a-0c38-453f-8cd0-56321c4592e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk"
Sep 15 20:29:58.957192 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:58.957059    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/730a8f9a-0c38-453f-8cd0-56321c4592e0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk\" (UID: \"730a8f9a-0c38-453f-8cd0-56321c4592e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk"
Sep 15 20:29:58.957192 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:58.957084    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5ggg\" (UniqueName: \"kubernetes.io/projected/730a8f9a-0c38-453f-8cd0-56321c4592e0-kube-api-access-x5ggg\") pod \"maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk\" (UID: \"730a8f9a-0c38-453f-8cd0-56321c4592e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk"
Sep 15 20:29:58.957715 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:58.957695    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/730a8f9a-0c38-453f-8cd0-56321c4592e0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk\" (UID: \"730a8f9a-0c38-453f-8cd0-56321c4592e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk"
Sep 15 20:29:58.959532 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:58.959500    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/730a8f9a-0c38-453f-8cd0-56321c4592e0-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk\" (UID: \"730a8f9a-0c38-453f-8cd0-56321c4592e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk"
Sep 15 20:29:58.959680 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:58.959659    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/730a8f9a-0c38-453f-8cd0-56321c4592e0-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk\" (UID: \"730a8f9a-0c38-453f-8cd0-56321c4592e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk"
Sep 15 20:29:58.965773 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:58.965745    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5ggg\" (UniqueName: \"kubernetes.io/projected/730a8f9a-0c38-453f-8cd0-56321c4592e0-kube-api-access-x5ggg\") pod \"maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk\" (UID: \"730a8f9a-0c38-453f-8cd0-56321c4592e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk"
Sep 15 20:29:59.072516 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:59.072439    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk"
Sep 15 20:29:59.355996 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:59.355962    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk"]
Sep 15 20:29:59.357037 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:29:59.357007    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod730a8f9a_0c38_453f_8cd0_56321c4592e0.slice/crio-03a1205d83e03b61364999c2c9c0f1c820654dcd904b4046c6489d054c75685c WatchSource:0}: Error finding container 03a1205d83e03b61364999c2c9c0f1c820654dcd904b4046c6489d054c75685c: Status 404 returned error can't find the container with id 03a1205d83e03b61364999c2c9c0f1c820654dcd904b4046c6489d054c75685c
Sep 15 20:29:59.762436 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:29:59.762404    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk" event={"ID":"730a8f9a-0c38-453f-8cd0-56321c4592e0","Type":"ContainerStarted","Data":"03a1205d83e03b61364999c2c9c0f1c820654dcd904b4046c6489d054c75685c"}
Sep 15 20:30:00.771537 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:00.771487    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk" event={"ID":"730a8f9a-0c38-453f-8cd0-56321c4592e0","Type":"ContainerStarted","Data":"1f63f3dd69168d106f800b838ffe9e9c3ad26f75469b2c18ca2a0e2a3b545f5a"}
Sep 15 20:30:00.771944 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:00.771570    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk"
Sep 15 20:30:00.791497 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:00.791436    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk" podStartSLOduration=2.415378804 podStartE2EDuration="2.791418065s" totalImagesPullingTime="376.039261ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:29:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:30:00.789360434 +0000 UTC m=+1507.180313432" watchObservedRunningTime="2026-09-15 20:30:00.791418065 +0000 UTC m=+1507.182371061"
Sep 15 20:30:06.784082 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:06.784047    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk"
Sep 15 20:30:21.922788 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:21.922720    2577 generic.go:356] "Generic (PLEG): container finished" podID="e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb" containerID="d6d50268b388edf5d86f5e12cdce13731a938248b16c80597195253d7e000d5a" exitCode=0
Sep 15 20:30:21.923333 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:21.922797    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf" event={"ID":"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb","Type":"ContainerDied","Data":"d6d50268b388edf5d86f5e12cdce13731a938248b16c80597195253d7e000d5a"}
Sep 15 20:30:21.923448 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:21.923429    2577 scope.go:117] "RemoveContainer" containerID="d6d50268b388edf5d86f5e12cdce13731a938248b16c80597195253d7e000d5a"
Sep 15 20:30:21.924770 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:21.924739    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 20:30:21.924901 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:21.924824    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 20:30:21.924901 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:21.924874    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 20:30:22.711410 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:22.711376    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:30:22.933059 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:22.933020    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf" event={"ID":"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb","Type":"ContainerStarted","Data":"3d9b9dd0cbb281481cdec6de5289cf87ba55eab04b3eee93b935865a41a6df11"}
Sep 15 20:30:23.428782 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:23.428737    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:30:23.430010 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:23.429980    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Sep 15 20:30:23.430188 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:23.430042    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf" podUID="e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Sep 15 20:30:24.429142 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:24.429104    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Sep 15 20:30:24.429578 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:24.429158    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf" podUID="e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Sep 15 20:30:25.429420 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:25.429376    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Sep 15 20:30:25.429923 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:25.429440    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf" podUID="e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Sep 15 20:30:26.429440 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:26.429398    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Sep 15 20:30:26.429947 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:26.429467    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf" podUID="e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Sep 15 20:30:27.429315 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:27.429280    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Sep 15 20:30:27.429518 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:27.429345    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf" podUID="e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Sep 15 20:30:28.434047 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:28.434013    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:30:28.434422 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:28.434399    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:30:28.435203 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:28.435180    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:30:57.650911 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:57.650866    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-025acc06-696df6544b-jpg2q"]
Sep 15 20:30:57.651546 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:57.651101    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-025acc06-696df6544b-jpg2q" podUID="717a748e-0243-47ef-80c3-3c7dbc97b8c7" containerName="maas-api" containerID="cri-o://97054d47a3ec21b983d0e9c865f20b932f0d91e37fea51e1ed3cb85d08d7d6f5" gracePeriod=30
Sep 15 20:30:57.932019 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:57.931993    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-025acc06-696df6544b-jpg2q"
Sep 15 20:30:58.037848 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:58.037806    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/717a748e-0243-47ef-80c3-3c7dbc97b8c7-metrics-certs\") pod \"717a748e-0243-47ef-80c3-3c7dbc97b8c7\" (UID: \"717a748e-0243-47ef-80c3-3c7dbc97b8c7\") "
Sep 15 20:30:58.038042 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:58.037912    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/717a748e-0243-47ef-80c3-3c7dbc97b8c7-maas-api-tls\") pod \"717a748e-0243-47ef-80c3-3c7dbc97b8c7\" (UID: \"717a748e-0243-47ef-80c3-3c7dbc97b8c7\") "
Sep 15 20:30:58.038042 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:58.038001    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/717a748e-0243-47ef-80c3-3c7dbc97b8c7-openshift-service-ca\") pod \"717a748e-0243-47ef-80c3-3c7dbc97b8c7\" (UID: \"717a748e-0243-47ef-80c3-3c7dbc97b8c7\") "
Sep 15 20:30:58.038177 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:58.038116    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lbbz9\" (UniqueName: \"kubernetes.io/projected/717a748e-0243-47ef-80c3-3c7dbc97b8c7-kube-api-access-lbbz9\") pod \"717a748e-0243-47ef-80c3-3c7dbc97b8c7\" (UID: \"717a748e-0243-47ef-80c3-3c7dbc97b8c7\") "
Sep 15 20:30:58.038435 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:58.038404    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/717a748e-0243-47ef-80c3-3c7dbc97b8c7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "717a748e-0243-47ef-80c3-3c7dbc97b8c7" (UID: "717a748e-0243-47ef-80c3-3c7dbc97b8c7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:30:58.038585 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:58.038554    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/717a748e-0243-47ef-80c3-3c7dbc97b8c7-openshift-service-ca\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:30:58.040178 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:58.040152    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/717a748e-0243-47ef-80c3-3c7dbc97b8c7-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "717a748e-0243-47ef-80c3-3c7dbc97b8c7" (UID: "717a748e-0243-47ef-80c3-3c7dbc97b8c7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:30:58.040475 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:58.040457    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/717a748e-0243-47ef-80c3-3c7dbc97b8c7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "717a748e-0243-47ef-80c3-3c7dbc97b8c7" (UID: "717a748e-0243-47ef-80c3-3c7dbc97b8c7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:30:58.040660 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:58.040643    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/717a748e-0243-47ef-80c3-3c7dbc97b8c7-kube-api-access-lbbz9" (OuterVolumeSpecName: "kube-api-access-lbbz9") pod "717a748e-0243-47ef-80c3-3c7dbc97b8c7" (UID: "717a748e-0243-47ef-80c3-3c7dbc97b8c7"). InnerVolumeSpecName "kube-api-access-lbbz9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:30:58.140066 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:58.140023    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lbbz9\" (UniqueName: \"kubernetes.io/projected/717a748e-0243-47ef-80c3-3c7dbc97b8c7-kube-api-access-lbbz9\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:30:58.140242 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:58.140141    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/717a748e-0243-47ef-80c3-3c7dbc97b8c7-metrics-certs\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:30:58.140242 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:58.140159    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/717a748e-0243-47ef-80c3-3c7dbc97b8c7-maas-api-tls\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:30:58.172405 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:58.172292    2577 generic.go:356] "Generic (PLEG): container finished" podID="717a748e-0243-47ef-80c3-3c7dbc97b8c7" containerID="97054d47a3ec21b983d0e9c865f20b932f0d91e37fea51e1ed3cb85d08d7d6f5" exitCode=0
Sep 15 20:30:58.172405 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:58.172338    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-025acc06-696df6544b-jpg2q"
Sep 15 20:30:58.172639 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:58.172330    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-025acc06-696df6544b-jpg2q" event={"ID":"717a748e-0243-47ef-80c3-3c7dbc97b8c7","Type":"ContainerDied","Data":"97054d47a3ec21b983d0e9c865f20b932f0d91e37fea51e1ed3cb85d08d7d6f5"}
Sep 15 20:30:58.172639 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:58.172463    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-025acc06-696df6544b-jpg2q" event={"ID":"717a748e-0243-47ef-80c3-3c7dbc97b8c7","Type":"ContainerDied","Data":"909041176d3c85a2ea9fc58a89b17c1985ff4e1291dda5ae7d4ebc4fd3d7b2f2"}
Sep 15 20:30:58.172639 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:58.172491    2577 scope.go:117] "RemoveContainer" containerID="97054d47a3ec21b983d0e9c865f20b932f0d91e37fea51e1ed3cb85d08d7d6f5"
Sep 15 20:30:58.184533 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:58.184511    2577 scope.go:117] "RemoveContainer" containerID="97054d47a3ec21b983d0e9c865f20b932f0d91e37fea51e1ed3cb85d08d7d6f5"
Sep 15 20:30:58.184915 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:30:58.184893    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97054d47a3ec21b983d0e9c865f20b932f0d91e37fea51e1ed3cb85d08d7d6f5\": container with ID starting with 97054d47a3ec21b983d0e9c865f20b932f0d91e37fea51e1ed3cb85d08d7d6f5 not found: ID does not exist" containerID="97054d47a3ec21b983d0e9c865f20b932f0d91e37fea51e1ed3cb85d08d7d6f5"
Sep 15 20:30:58.185018 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:58.184924    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97054d47a3ec21b983d0e9c865f20b932f0d91e37fea51e1ed3cb85d08d7d6f5"} err="failed to get container status \"97054d47a3ec21b983d0e9c865f20b932f0d91e37fea51e1ed3cb85d08d7d6f5\": rpc error: code = NotFound desc = could not find container \"97054d47a3ec21b983d0e9c865f20b932f0d91e37fea51e1ed3cb85d08d7d6f5\": container with ID starting with 97054d47a3ec21b983d0e9c865f20b932f0d91e37fea51e1ed3cb85d08d7d6f5 not found: ID does not exist"
Sep 15 20:30:58.192512 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:58.192480    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-025acc06-696df6544b-jpg2q"]
Sep 15 20:30:58.199927 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:30:58.199896    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-025acc06-696df6544b-jpg2q"]
Sep 15 20:31:00.069262 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:31:00.069225    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="717a748e-0243-47ef-80c3-3c7dbc97b8c7" path="/var/lib/kubelet/pods/717a748e-0243-47ef-80c3-3c7dbc97b8c7/volumes"
Sep 15 20:32:52.443428 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:32:52.443391    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj"]
Sep 15 20:32:52.444179 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:32:52.444159    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="717a748e-0243-47ef-80c3-3c7dbc97b8c7" containerName="maas-api"
Sep 15 20:32:52.444272 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:32:52.444183    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="717a748e-0243-47ef-80c3-3c7dbc97b8c7" containerName="maas-api"
Sep 15 20:32:52.444359 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:32:52.444344    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="717a748e-0243-47ef-80c3-3c7dbc97b8c7" containerName="maas-api"
Sep 15 20:32:52.447690 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:32:52.447670    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj"
Sep 15 20:32:52.450312 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:32:52.450288    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-a8e967-serving-cert\""
Sep 15 20:32:52.470312 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:32:52.463601    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj"]
Sep 15 20:32:52.582885 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:32:52.582831    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d5c7423-7b7f-4f95-9447-f4cfb49617da-maas-api-tls\") pod \"maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj\" (UID: \"3d5c7423-7b7f-4f95-9447-f4cfb49617da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj"
Sep 15 20:32:52.582885 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:32:52.582891    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d5c7423-7b7f-4f95-9447-f4cfb49617da-openshift-service-ca\") pod \"maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj\" (UID: \"3d5c7423-7b7f-4f95-9447-f4cfb49617da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj"
Sep 15 20:32:52.583128 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:32:52.582917    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgz6n\" (UniqueName: \"kubernetes.io/projected/3d5c7423-7b7f-4f95-9447-f4cfb49617da-kube-api-access-dgz6n\") pod \"maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj\" (UID: \"3d5c7423-7b7f-4f95-9447-f4cfb49617da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj"
Sep 15 20:32:52.583128 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:32:52.582966    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d5c7423-7b7f-4f95-9447-f4cfb49617da-metrics-certs\") pod \"maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj\" (UID: \"3d5c7423-7b7f-4f95-9447-f4cfb49617da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj"
Sep 15 20:32:52.684128 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:32:52.684091    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d5c7423-7b7f-4f95-9447-f4cfb49617da-metrics-certs\") pod \"maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj\" (UID: \"3d5c7423-7b7f-4f95-9447-f4cfb49617da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj"
Sep 15 20:32:52.684341 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:32:52.684256    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d5c7423-7b7f-4f95-9447-f4cfb49617da-maas-api-tls\") pod \"maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj\" (UID: \"3d5c7423-7b7f-4f95-9447-f4cfb49617da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj"
Sep 15 20:32:52.684341 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:32:52.684292    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d5c7423-7b7f-4f95-9447-f4cfb49617da-openshift-service-ca\") pod \"maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj\" (UID: \"3d5c7423-7b7f-4f95-9447-f4cfb49617da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj"
Sep 15 20:32:52.684341 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:32:52.684323    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgz6n\" (UniqueName: \"kubernetes.io/projected/3d5c7423-7b7f-4f95-9447-f4cfb49617da-kube-api-access-dgz6n\") pod \"maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj\" (UID: \"3d5c7423-7b7f-4f95-9447-f4cfb49617da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj"
Sep 15 20:32:52.685109 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:32:52.685081    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d5c7423-7b7f-4f95-9447-f4cfb49617da-openshift-service-ca\") pod \"maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj\" (UID: \"3d5c7423-7b7f-4f95-9447-f4cfb49617da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj"
Sep 15 20:32:52.686702 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:32:52.686673    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d5c7423-7b7f-4f95-9447-f4cfb49617da-maas-api-tls\") pod \"maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj\" (UID: \"3d5c7423-7b7f-4f95-9447-f4cfb49617da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj"
Sep 15 20:32:52.686824 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:32:52.686761    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d5c7423-7b7f-4f95-9447-f4cfb49617da-metrics-certs\") pod \"maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj\" (UID: \"3d5c7423-7b7f-4f95-9447-f4cfb49617da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj"
Sep 15 20:32:52.694936 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:32:52.694874    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgz6n\" (UniqueName: \"kubernetes.io/projected/3d5c7423-7b7f-4f95-9447-f4cfb49617da-kube-api-access-dgz6n\") pod \"maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj\" (UID: \"3d5c7423-7b7f-4f95-9447-f4cfb49617da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj"
Sep 15 20:32:52.763185 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:32:52.763149    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj"
Sep 15 20:32:53.048523 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:32:53.048478    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj"]
Sep 15 20:32:53.050845 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:32:53.050813    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d5c7423_7b7f_4f95_9447_f4cfb49617da.slice/crio-732bcf43ede1843896b6d6e5e519aeeecf9aed1789bb95de0d1c6993ae6179d2 WatchSource:0}: Error finding container 732bcf43ede1843896b6d6e5e519aeeecf9aed1789bb95de0d1c6993ae6179d2: Status 404 returned error can't find the container with id 732bcf43ede1843896b6d6e5e519aeeecf9aed1789bb95de0d1c6993ae6179d2
Sep 15 20:32:53.052309 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:32:53.052288    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 20:32:53.990912 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:32:53.990868    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj" event={"ID":"3d5c7423-7b7f-4f95-9447-f4cfb49617da","Type":"ContainerStarted","Data":"8911f4d74eeb4adeb322cea31026e934edfd595d4c3c9d77d75f67d75ae1ba16"}
Sep 15 20:32:53.991394 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:32:53.990944    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj" event={"ID":"3d5c7423-7b7f-4f95-9447-f4cfb49617da","Type":"ContainerStarted","Data":"732bcf43ede1843896b6d6e5e519aeeecf9aed1789bb95de0d1c6993ae6179d2"}
Sep 15 20:32:53.991394 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:32:53.990975    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj"
Sep 15 20:32:54.008066 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:32:54.008014    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj" podStartSLOduration=1.5248374120000001 podStartE2EDuration="2.007997144s" totalImagesPullingTime="483.159732ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:32:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:32:54.006666208 +0000 UTC m=+1680.397619206" watchObservedRunningTime="2026-09-15 20:32:54.007997144 +0000 UTC m=+1680.398950170"
Sep 15 20:33:00.003307 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:00.003273    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj"
Sep 15 20:33:19.172128 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:19.172037    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"]
Sep 15 20:33:19.176978 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:19.176950    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:33:19.180546 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:19.180521    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-df0188-openshift-default-dockercfg-gxtjx\""
Sep 15 20:33:19.187967 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:19.187930    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"]
Sep 15 20:33:19.272432 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:19.272299    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-istio-envoy\") pod \"e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg\" (UID: \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\") " pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:33:19.272432 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:19.272440    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-istiod-ca-cert\") pod \"e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg\" (UID: \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\") " pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:33:19.272705 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:19.272500    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-workload-certs\") pod \"e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg\" (UID: \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\") " pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:33:19.272705 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:19.272543    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-istio-data\") pod \"e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg\" (UID: \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\") " pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:33:19.272705 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:19.272682    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-workload-socket\") pod \"e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg\" (UID: \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\") " pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:33:19.272901 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:19.272781    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lc5t\" (UniqueName: \"kubernetes.io/projected/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-kube-api-access-6lc5t\") pod \"e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg\" (UID: \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\") " pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:33:19.272901 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:19.272844    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-istio-token\") pod \"e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg\" (UID: \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\") " pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:33:19.272901 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:19.272869    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-istio-podinfo\") pod \"e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg\" (UID: \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\") " pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:33:19.273067 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:19.272899    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-credential-socket\") pod \"e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg\" (UID: \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\") " pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:33:19.373847 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:19.373808    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-istio-envoy\") pod \"e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg\" (UID: \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\") " pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:33:19.374053 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:19.373880    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-istiod-ca-cert\") pod \"e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg\" (UID: \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\") " pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:33:19.374053 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:19.373920    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-workload-certs\") pod \"e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg\" (UID: \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\") " pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:33:19.374053 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:19.373950    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-istio-data\") pod \"e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg\" (UID: \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\") " pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:33:19.374236 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:19.374066    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-workload-socket\") pod \"e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg\" (UID: \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\") " pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:33:19.374236 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:19.374122    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lc5t\" (UniqueName: \"kubernetes.io/projected/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-kube-api-access-6lc5t\") pod \"e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg\" (UID: \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\") " pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:33:19.374236 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:19.374175    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-istio-token\") pod \"e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg\" (UID: \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\") " pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:33:19.374236 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:19.374203    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-istio-podinfo\") pod \"e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg\" (UID: \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\") " pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:33:19.374236 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:19.374231    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-credential-socket\") pod \"e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg\" (UID: \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\") " pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:33:19.374629 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:19.374601    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-credential-socket\") pod \"e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg\" (UID: \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\") " pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:33:19.374828 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:19.374614    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-workload-socket\") pod \"e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg\" (UID: \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\") " pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:33:19.374943 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:19.374675    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-istiod-ca-cert\") pod \"e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg\" (UID: \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\") " pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:33:19.375035 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:19.374865    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-workload-certs\") pod \"e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg\" (UID: \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\") " pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:33:19.375121 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:19.374929    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-istio-data\") pod \"e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg\" (UID: \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\") " pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:33:19.377081 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:19.377053    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-istio-envoy\") pod \"e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg\" (UID: \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\") " pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:33:19.377961 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:19.377934    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-istio-podinfo\") pod \"e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg\" (UID: \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\") " pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:33:19.382604 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:19.382578    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lc5t\" (UniqueName: \"kubernetes.io/projected/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-kube-api-access-6lc5t\") pod \"e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg\" (UID: \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\") " pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:33:19.382785 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:19.382766    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-istio-token\") pod \"e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg\" (UID: \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\") " pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:33:19.496398 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:19.496364    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:33:19.807399 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:19.807373    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"]
Sep 15 20:33:19.808453 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:33:19.808427    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dae1622_aa93_4ccf_9c98_a09fbbc6a74d.slice/crio-3d43f680c8f75c1d48dbf0a6e8fd2a1491e7e14fe06c9ca4e82cfd7f82dbb23e WatchSource:0}: Error finding container 3d43f680c8f75c1d48dbf0a6e8fd2a1491e7e14fe06c9ca4e82cfd7f82dbb23e: Status 404 returned error can't find the container with id 3d43f680c8f75c1d48dbf0a6e8fd2a1491e7e14fe06c9ca4e82cfd7f82dbb23e
Sep 15 20:33:19.811265 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:19.811232    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 20:33:19.811351 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:19.811296    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 20:33:19.811351 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:19.811327    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 20:33:20.189144 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:20.189041    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg" event={"ID":"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d","Type":"ContainerStarted","Data":"e8af4f246885d11d06271ab55e87706afce49857416d88c2072189e837f4dd4f"}
Sep 15 20:33:20.189144 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:20.189114    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg" event={"ID":"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d","Type":"ContainerStarted","Data":"3d43f680c8f75c1d48dbf0a6e8fd2a1491e7e14fe06c9ca4e82cfd7f82dbb23e"}
Sep 15 20:33:20.212248 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:20.212182    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg" podStartSLOduration=1.212162048 podStartE2EDuration="1.212162048s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:33:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:33:20.210301343 +0000 UTC m=+1706.601254341" watchObservedRunningTime="2026-09-15 20:33:20.212162048 +0000 UTC m=+1706.603115047"
Sep 15 20:33:20.497575 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:20.497533    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:33:20.503270 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:20.503241    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:33:21.196552 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:21.196521    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:33:21.197577 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:21.197552    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:33:27.542016 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:27.541974    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-df0188-7f66f69447-8vb6b"]
Sep 15 20:33:27.546200 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:27.546178    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-df0188-7f66f69447-8vb6b"
Sep 15 20:33:27.548459 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:27.548439    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-df0188-serving-cert\""
Sep 15 20:33:27.558424 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:27.558397    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-df0188-7f66f69447-8vb6b"]
Sep 15 20:33:27.676869 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:27.676821    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c68080ad-f647-4e15-a1be-e6ab2030092e-openshift-service-ca\") pod \"maas-api-e2e-api-b-df0188-7f66f69447-8vb6b\" (UID: \"c68080ad-f647-4e15-a1be-e6ab2030092e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-df0188-7f66f69447-8vb6b"
Sep 15 20:33:27.677072 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:27.676931    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5dg9\" (UniqueName: \"kubernetes.io/projected/c68080ad-f647-4e15-a1be-e6ab2030092e-kube-api-access-b5dg9\") pod \"maas-api-e2e-api-b-df0188-7f66f69447-8vb6b\" (UID: \"c68080ad-f647-4e15-a1be-e6ab2030092e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-df0188-7f66f69447-8vb6b"
Sep 15 20:33:27.677072 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:27.676979    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c68080ad-f647-4e15-a1be-e6ab2030092e-maas-api-tls\") pod \"maas-api-e2e-api-b-df0188-7f66f69447-8vb6b\" (UID: \"c68080ad-f647-4e15-a1be-e6ab2030092e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-df0188-7f66f69447-8vb6b"
Sep 15 20:33:27.677192 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:27.677151    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c68080ad-f647-4e15-a1be-e6ab2030092e-metrics-certs\") pod \"maas-api-e2e-api-b-df0188-7f66f69447-8vb6b\" (UID: \"c68080ad-f647-4e15-a1be-e6ab2030092e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-df0188-7f66f69447-8vb6b"
Sep 15 20:33:27.778414 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:27.778369    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5dg9\" (UniqueName: \"kubernetes.io/projected/c68080ad-f647-4e15-a1be-e6ab2030092e-kube-api-access-b5dg9\") pod \"maas-api-e2e-api-b-df0188-7f66f69447-8vb6b\" (UID: \"c68080ad-f647-4e15-a1be-e6ab2030092e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-df0188-7f66f69447-8vb6b"
Sep 15 20:33:27.778622 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:27.778423    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c68080ad-f647-4e15-a1be-e6ab2030092e-maas-api-tls\") pod \"maas-api-e2e-api-b-df0188-7f66f69447-8vb6b\" (UID: \"c68080ad-f647-4e15-a1be-e6ab2030092e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-df0188-7f66f69447-8vb6b"
Sep 15 20:33:27.778622 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:27.778502    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c68080ad-f647-4e15-a1be-e6ab2030092e-metrics-certs\") pod \"maas-api-e2e-api-b-df0188-7f66f69447-8vb6b\" (UID: \"c68080ad-f647-4e15-a1be-e6ab2030092e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-df0188-7f66f69447-8vb6b"
Sep 15 20:33:27.778622 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:27.778526    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c68080ad-f647-4e15-a1be-e6ab2030092e-openshift-service-ca\") pod \"maas-api-e2e-api-b-df0188-7f66f69447-8vb6b\" (UID: \"c68080ad-f647-4e15-a1be-e6ab2030092e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-df0188-7f66f69447-8vb6b"
Sep 15 20:33:27.778830 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:33:27.778628    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-df0188-serving-cert: secret "maas-api-e2e-api-b-df0188-serving-cert" not found
Sep 15 20:33:27.778830 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:33:27.778736    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c68080ad-f647-4e15-a1be-e6ab2030092e-maas-api-tls podName:c68080ad-f647-4e15-a1be-e6ab2030092e nodeName:}" failed. No retries permitted until 2026-09-15 20:33:28.278700676 +0000 UTC m=+1714.669653677 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c68080ad-f647-4e15-a1be-e6ab2030092e-maas-api-tls") pod "maas-api-e2e-api-b-df0188-7f66f69447-8vb6b" (UID: "c68080ad-f647-4e15-a1be-e6ab2030092e") : secret "maas-api-e2e-api-b-df0188-serving-cert" not found
Sep 15 20:33:27.779218 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:27.779199    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c68080ad-f647-4e15-a1be-e6ab2030092e-openshift-service-ca\") pod \"maas-api-e2e-api-b-df0188-7f66f69447-8vb6b\" (UID: \"c68080ad-f647-4e15-a1be-e6ab2030092e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-df0188-7f66f69447-8vb6b"
Sep 15 20:33:27.781032 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:27.781004    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c68080ad-f647-4e15-a1be-e6ab2030092e-metrics-certs\") pod \"maas-api-e2e-api-b-df0188-7f66f69447-8vb6b\" (UID: \"c68080ad-f647-4e15-a1be-e6ab2030092e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-df0188-7f66f69447-8vb6b"
Sep 15 20:33:27.788841 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:27.788820    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5dg9\" (UniqueName: \"kubernetes.io/projected/c68080ad-f647-4e15-a1be-e6ab2030092e-kube-api-access-b5dg9\") pod \"maas-api-e2e-api-b-df0188-7f66f69447-8vb6b\" (UID: \"c68080ad-f647-4e15-a1be-e6ab2030092e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-df0188-7f66f69447-8vb6b"
Sep 15 20:33:28.284916 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:28.284882    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c68080ad-f647-4e15-a1be-e6ab2030092e-maas-api-tls\") pod \"maas-api-e2e-api-b-df0188-7f66f69447-8vb6b\" (UID: \"c68080ad-f647-4e15-a1be-e6ab2030092e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-df0188-7f66f69447-8vb6b"
Sep 15 20:33:28.287427 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:28.287402    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c68080ad-f647-4e15-a1be-e6ab2030092e-maas-api-tls\") pod \"maas-api-e2e-api-b-df0188-7f66f69447-8vb6b\" (UID: \"c68080ad-f647-4e15-a1be-e6ab2030092e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-df0188-7f66f69447-8vb6b"
Sep 15 20:33:28.461679 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:28.461638    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-df0188-7f66f69447-8vb6b"
Sep 15 20:33:28.743990 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:28.743957    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-df0188-7f66f69447-8vb6b"]
Sep 15 20:33:28.745323 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:33:28.745292    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc68080ad_f647_4e15_a1be_e6ab2030092e.slice/crio-6ee3dca12c65fcd220e81b281d7d3f110791de103466b3320bdf29e560e6b46f WatchSource:0}: Error finding container 6ee3dca12c65fcd220e81b281d7d3f110791de103466b3320bdf29e560e6b46f: Status 404 returned error can't find the container with id 6ee3dca12c65fcd220e81b281d7d3f110791de103466b3320bdf29e560e6b46f
Sep 15 20:33:29.266944 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:29.266901    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-df0188-7f66f69447-8vb6b" event={"ID":"c68080ad-f647-4e15-a1be-e6ab2030092e","Type":"ContainerStarted","Data":"910eebfe4e704a0d6587f2f3417946f77a840db4fffdfdaa6f916456dca9cebe"}
Sep 15 20:33:29.267123 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:29.266973    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-df0188-7f66f69447-8vb6b" event={"ID":"c68080ad-f647-4e15-a1be-e6ab2030092e","Type":"ContainerStarted","Data":"6ee3dca12c65fcd220e81b281d7d3f110791de103466b3320bdf29e560e6b46f"}
Sep 15 20:33:29.267123 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:29.267004    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-df0188-7f66f69447-8vb6b"
Sep 15 20:33:29.284684 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:29.284630    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-df0188-7f66f69447-8vb6b" podStartSLOduration=1.8571895920000001 podStartE2EDuration="2.284610948s" totalImagesPullingTime="427.421356ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:33:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:33:29.281805141 +0000 UTC m=+1715.672758137" watchObservedRunningTime="2026-09-15 20:33:29.284610948 +0000 UTC m=+1715.675563958"
Sep 15 20:33:35.279549 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:35.279507    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-df0188-7f66f69447-8vb6b"
Sep 15 20:33:51.341184 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:51.341145    2577 patch_prober.go:28] interesting pod/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": read tcp 10.133.0.2:57520->10.133.0.86:15021: read: connection reset by peer" start-of-body=
Sep 15 20:33:51.342046 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:51.341239    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg" podUID="3dae1622-aa93-4ccf-9c98-a09fbbc6a74d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": read tcp 10.133.0.2:57520->10.133.0.86:15021: read: connection reset by peer"
Sep 15 20:33:51.432561 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:51.432525    2577 generic.go:356] "Generic (PLEG): container finished" podID="3dae1622-aa93-4ccf-9c98-a09fbbc6a74d" containerID="e8af4f246885d11d06271ab55e87706afce49857416d88c2072189e837f4dd4f" exitCode=0
Sep 15 20:33:51.432787 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:51.432606    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg" event={"ID":"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d","Type":"ContainerDied","Data":"e8af4f246885d11d06271ab55e87706afce49857416d88c2072189e837f4dd4f"}
Sep 15 20:33:51.433169 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:51.433150    2577 scope.go:117] "RemoveContainer" containerID="e8af4f246885d11d06271ab55e87706afce49857416d88c2072189e837f4dd4f"
Sep 15 20:33:51.434404 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:51.434377    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 20:33:51.434459 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:51.434446    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 20:33:51.434496 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:51.434478    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 20:33:52.441849 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:52.441810    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg" event={"ID":"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d","Type":"ContainerStarted","Data":"3c6e57f9a569f5ab4162bd71750cff65916a212aae36cde6f1f58f395d28b3d1"}
Sep 15 20:33:52.497522 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:52.497488    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:33:52.497940 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:52.497909    2577 patch_prober.go:28] interesting pod/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Sep 15 20:33:52.498127 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:52.498102    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg" podUID="3dae1622-aa93-4ccf-9c98-a09fbbc6a74d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Sep 15 20:33:53.497719 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:53.497673    2577 patch_prober.go:28] interesting pod/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Sep 15 20:33:53.498245 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:53.497768    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg" podUID="3dae1622-aa93-4ccf-9c98-a09fbbc6a74d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Sep 15 20:33:54.497004 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:54.496955    2577 patch_prober.go:28] interesting pod/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Sep 15 20:33:54.497231 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:54.497031    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg" podUID="3dae1622-aa93-4ccf-9c98-a09fbbc6a74d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Sep 15 20:33:55.497206 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:55.497155    2577 patch_prober.go:28] interesting pod/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Sep 15 20:33:55.497743 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:55.497232    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg" podUID="3dae1622-aa93-4ccf-9c98-a09fbbc6a74d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Sep 15 20:33:56.497233 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:56.497184    2577 patch_prober.go:28] interesting pod/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Sep 15 20:33:56.497765 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:56.497258    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg" podUID="3dae1622-aa93-4ccf-9c98-a09fbbc6a74d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Sep 15 20:33:57.497228 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:57.497174    2577 patch_prober.go:28] interesting pod/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Sep 15 20:33:57.497450 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:57.497255    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg" podUID="3dae1622-aa93-4ccf-9c98-a09fbbc6a74d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Sep 15 20:33:58.501229 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:58.501191    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:33:58.501784 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:58.501572    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:33:58.502453 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:33:58.502432    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:34:07.547862 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:07.547826    2577 generic.go:356] "Generic (PLEG): container finished" podID="c3c6f523-42a6-471f-ab50-f0fd4b821101" containerID="20e757d2a4f7eabdac679565ba6a7a92ce457e8d86d132851ccaba02d3703ec5" exitCode=137
Sep 15 20:34:07.548281 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:07.547901    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qrq84" event={"ID":"c3c6f523-42a6-471f-ab50-f0fd4b821101","Type":"ContainerDied","Data":"20e757d2a4f7eabdac679565ba6a7a92ce457e8d86d132851ccaba02d3703ec5"}
Sep 15 20:34:07.548281 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:07.548220    2577 scope.go:117] "RemoveContainer" containerID="20e757d2a4f7eabdac679565ba6a7a92ce457e8d86d132851ccaba02d3703ec5"
Sep 15 20:34:08.557910 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:08.557874    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qrq84" event={"ID":"c3c6f523-42a6-471f-ab50-f0fd4b821101","Type":"ContainerStarted","Data":"f002a1493ea8b0c470d1341d6248f314642a40b40768c1eab56c8c048acf05ea"}
Sep 15 20:34:08.558337 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:08.558077    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qrq84"
Sep 15 20:34:12.070849 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:12.070813    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj"]
Sep 15 20:34:12.072045 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:12.071980    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj" podUID="3d5c7423-7b7f-4f95-9447-f4cfb49617da" containerName="maas-api" containerID="cri-o://8911f4d74eeb4adeb322cea31026e934edfd595d4c3c9d77d75f67d75ae1ba16" gracePeriod=30
Sep 15 20:34:12.342690 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:12.342600    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj"
Sep 15 20:34:12.504328 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:12.504294    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgz6n\" (UniqueName: \"kubernetes.io/projected/3d5c7423-7b7f-4f95-9447-f4cfb49617da-kube-api-access-dgz6n\") pod \"3d5c7423-7b7f-4f95-9447-f4cfb49617da\" (UID: \"3d5c7423-7b7f-4f95-9447-f4cfb49617da\") "
Sep 15 20:34:12.504521 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:12.504342    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d5c7423-7b7f-4f95-9447-f4cfb49617da-metrics-certs\") pod \"3d5c7423-7b7f-4f95-9447-f4cfb49617da\" (UID: \"3d5c7423-7b7f-4f95-9447-f4cfb49617da\") "
Sep 15 20:34:12.504521 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:12.504387    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d5c7423-7b7f-4f95-9447-f4cfb49617da-openshift-service-ca\") pod \"3d5c7423-7b7f-4f95-9447-f4cfb49617da\" (UID: \"3d5c7423-7b7f-4f95-9447-f4cfb49617da\") "
Sep 15 20:34:12.504521 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:12.504513    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d5c7423-7b7f-4f95-9447-f4cfb49617da-maas-api-tls\") pod \"3d5c7423-7b7f-4f95-9447-f4cfb49617da\" (UID: \"3d5c7423-7b7f-4f95-9447-f4cfb49617da\") "
Sep 15 20:34:12.504886 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:12.504857    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d5c7423-7b7f-4f95-9447-f4cfb49617da-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3d5c7423-7b7f-4f95-9447-f4cfb49617da" (UID: "3d5c7423-7b7f-4f95-9447-f4cfb49617da"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:34:12.505035 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:12.504982    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d5c7423-7b7f-4f95-9447-f4cfb49617da-openshift-service-ca\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:34:12.506642 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:12.506617    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d5c7423-7b7f-4f95-9447-f4cfb49617da-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3d5c7423-7b7f-4f95-9447-f4cfb49617da" (UID: "3d5c7423-7b7f-4f95-9447-f4cfb49617da"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:34:12.507097 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:12.507073    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d5c7423-7b7f-4f95-9447-f4cfb49617da-kube-api-access-dgz6n" (OuterVolumeSpecName: "kube-api-access-dgz6n") pod "3d5c7423-7b7f-4f95-9447-f4cfb49617da" (UID: "3d5c7423-7b7f-4f95-9447-f4cfb49617da"). InnerVolumeSpecName "kube-api-access-dgz6n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:34:12.507187 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:12.507116    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d5c7423-7b7f-4f95-9447-f4cfb49617da-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3d5c7423-7b7f-4f95-9447-f4cfb49617da" (UID: "3d5c7423-7b7f-4f95-9447-f4cfb49617da"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:34:12.587170 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:12.587132    2577 generic.go:356] "Generic (PLEG): container finished" podID="3d5c7423-7b7f-4f95-9447-f4cfb49617da" containerID="8911f4d74eeb4adeb322cea31026e934edfd595d4c3c9d77d75f67d75ae1ba16" exitCode=0
Sep 15 20:34:12.587360 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:12.587186    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj"
Sep 15 20:34:12.587360 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:12.587226    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj" event={"ID":"3d5c7423-7b7f-4f95-9447-f4cfb49617da","Type":"ContainerDied","Data":"8911f4d74eeb4adeb322cea31026e934edfd595d4c3c9d77d75f67d75ae1ba16"}
Sep 15 20:34:12.587360 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:12.587297    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj" event={"ID":"3d5c7423-7b7f-4f95-9447-f4cfb49617da","Type":"ContainerDied","Data":"732bcf43ede1843896b6d6e5e519aeeecf9aed1789bb95de0d1c6993ae6179d2"}
Sep 15 20:34:12.587360 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:12.587329    2577 scope.go:117] "RemoveContainer" containerID="8911f4d74eeb4adeb322cea31026e934edfd595d4c3c9d77d75f67d75ae1ba16"
Sep 15 20:34:12.598327 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:12.598305    2577 scope.go:117] "RemoveContainer" containerID="8911f4d74eeb4adeb322cea31026e934edfd595d4c3c9d77d75f67d75ae1ba16"
Sep 15 20:34:12.598667 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:34:12.598648    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8911f4d74eeb4adeb322cea31026e934edfd595d4c3c9d77d75f67d75ae1ba16\": container with ID starting with 8911f4d74eeb4adeb322cea31026e934edfd595d4c3c9d77d75f67d75ae1ba16 not found: ID does not exist" containerID="8911f4d74eeb4adeb322cea31026e934edfd595d4c3c9d77d75f67d75ae1ba16"
Sep 15 20:34:12.598759 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:12.598675    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8911f4d74eeb4adeb322cea31026e934edfd595d4c3c9d77d75f67d75ae1ba16"} err="failed to get container status \"8911f4d74eeb4adeb322cea31026e934edfd595d4c3c9d77d75f67d75ae1ba16\": rpc error: code = NotFound desc = could not find container \"8911f4d74eeb4adeb322cea31026e934edfd595d4c3c9d77d75f67d75ae1ba16\": container with ID starting with 8911f4d74eeb4adeb322cea31026e934edfd595d4c3c9d77d75f67d75ae1ba16 not found: ID does not exist"
Sep 15 20:34:12.606958 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:12.606851    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgz6n\" (UniqueName: \"kubernetes.io/projected/3d5c7423-7b7f-4f95-9447-f4cfb49617da-kube-api-access-dgz6n\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:34:12.606958 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:12.606881    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d5c7423-7b7f-4f95-9447-f4cfb49617da-metrics-certs\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:34:12.606958 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:12.606893    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d5c7423-7b7f-4f95-9447-f4cfb49617da-maas-api-tls\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:34:12.610849 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:12.610824    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj"]
Sep 15 20:34:12.613809 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:12.613789    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a8e967-b96855fd5-bfvcj"]
Sep 15 20:34:14.062487 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:14.062456    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d5c7423-7b7f-4f95-9447-f4cfb49617da" path="/var/lib/kubelet/pods/3d5c7423-7b7f-4f95-9447-f4cfb49617da/volumes"
Sep 15 20:34:19.566384 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:19.566348    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qrq84"
Sep 15 20:34:35.233385 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:35.233348    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-df0188-7f66f69447-8vb6b"]
Sep 15 20:34:35.233961 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:35.233833    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-df0188-7f66f69447-8vb6b" podUID="c68080ad-f647-4e15-a1be-e6ab2030092e" containerName="maas-api" containerID="cri-o://910eebfe4e704a0d6587f2f3417946f77a840db4fffdfdaa6f916456dca9cebe" gracePeriod=30
Sep 15 20:34:35.275111 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:35.275066    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-df0188-7f66f69447-8vb6b" podUID="c68080ad-f647-4e15-a1be-e6ab2030092e" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.87:8443/health\": dial tcp 10.133.0.87:8443: connect: connection refused"
Sep 15 20:34:35.521703 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:35.521674    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-df0188-7f66f69447-8vb6b"
Sep 15 20:34:35.668888 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:35.668836    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c68080ad-f647-4e15-a1be-e6ab2030092e-metrics-certs\") pod \"c68080ad-f647-4e15-a1be-e6ab2030092e\" (UID: \"c68080ad-f647-4e15-a1be-e6ab2030092e\") "
Sep 15 20:34:35.668888 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:35.668886    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c68080ad-f647-4e15-a1be-e6ab2030092e-maas-api-tls\") pod \"c68080ad-f647-4e15-a1be-e6ab2030092e\" (UID: \"c68080ad-f647-4e15-a1be-e6ab2030092e\") "
Sep 15 20:34:35.669170 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:35.668930    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c68080ad-f647-4e15-a1be-e6ab2030092e-openshift-service-ca\") pod \"c68080ad-f647-4e15-a1be-e6ab2030092e\" (UID: \"c68080ad-f647-4e15-a1be-e6ab2030092e\") "
Sep 15 20:34:35.669170 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:35.668953    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5dg9\" (UniqueName: \"kubernetes.io/projected/c68080ad-f647-4e15-a1be-e6ab2030092e-kube-api-access-b5dg9\") pod \"c68080ad-f647-4e15-a1be-e6ab2030092e\" (UID: \"c68080ad-f647-4e15-a1be-e6ab2030092e\") "
Sep 15 20:34:35.669393 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:35.669365    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c68080ad-f647-4e15-a1be-e6ab2030092e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c68080ad-f647-4e15-a1be-e6ab2030092e" (UID: "c68080ad-f647-4e15-a1be-e6ab2030092e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:34:35.671267 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:35.671226    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c68080ad-f647-4e15-a1be-e6ab2030092e-kube-api-access-b5dg9" (OuterVolumeSpecName: "kube-api-access-b5dg9") pod "c68080ad-f647-4e15-a1be-e6ab2030092e" (UID: "c68080ad-f647-4e15-a1be-e6ab2030092e"). InnerVolumeSpecName "kube-api-access-b5dg9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:34:35.671597 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:35.671571    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c68080ad-f647-4e15-a1be-e6ab2030092e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c68080ad-f647-4e15-a1be-e6ab2030092e" (UID: "c68080ad-f647-4e15-a1be-e6ab2030092e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:34:35.671739 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:35.671653    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c68080ad-f647-4e15-a1be-e6ab2030092e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c68080ad-f647-4e15-a1be-e6ab2030092e" (UID: "c68080ad-f647-4e15-a1be-e6ab2030092e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:34:35.765671 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:35.765582    2577 generic.go:356] "Generic (PLEG): container finished" podID="c68080ad-f647-4e15-a1be-e6ab2030092e" containerID="910eebfe4e704a0d6587f2f3417946f77a840db4fffdfdaa6f916456dca9cebe" exitCode=0
Sep 15 20:34:35.765671 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:35.765631    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-df0188-7f66f69447-8vb6b"
Sep 15 20:34:35.765878 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:35.765673    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-df0188-7f66f69447-8vb6b" event={"ID":"c68080ad-f647-4e15-a1be-e6ab2030092e","Type":"ContainerDied","Data":"910eebfe4e704a0d6587f2f3417946f77a840db4fffdfdaa6f916456dca9cebe"}
Sep 15 20:34:35.765878 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:35.765738    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-df0188-7f66f69447-8vb6b" event={"ID":"c68080ad-f647-4e15-a1be-e6ab2030092e","Type":"ContainerDied","Data":"6ee3dca12c65fcd220e81b281d7d3f110791de103466b3320bdf29e560e6b46f"}
Sep 15 20:34:35.765878 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:35.765762    2577 scope.go:117] "RemoveContainer" containerID="910eebfe4e704a0d6587f2f3417946f77a840db4fffdfdaa6f916456dca9cebe"
Sep 15 20:34:35.769910 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:35.769884    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c68080ad-f647-4e15-a1be-e6ab2030092e-openshift-service-ca\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:34:35.769910 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:35.769908    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5dg9\" (UniqueName: \"kubernetes.io/projected/c68080ad-f647-4e15-a1be-e6ab2030092e-kube-api-access-b5dg9\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:34:35.770108 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:35.769917    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c68080ad-f647-4e15-a1be-e6ab2030092e-metrics-certs\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:34:35.770108 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:35.769928    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c68080ad-f647-4e15-a1be-e6ab2030092e-maas-api-tls\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:34:35.778222 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:35.778203    2577 scope.go:117] "RemoveContainer" containerID="910eebfe4e704a0d6587f2f3417946f77a840db4fffdfdaa6f916456dca9cebe"
Sep 15 20:34:35.778592 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:34:35.778568    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"910eebfe4e704a0d6587f2f3417946f77a840db4fffdfdaa6f916456dca9cebe\": container with ID starting with 910eebfe4e704a0d6587f2f3417946f77a840db4fffdfdaa6f916456dca9cebe not found: ID does not exist" containerID="910eebfe4e704a0d6587f2f3417946f77a840db4fffdfdaa6f916456dca9cebe"
Sep 15 20:34:35.778705 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:35.778598    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"910eebfe4e704a0d6587f2f3417946f77a840db4fffdfdaa6f916456dca9cebe"} err="failed to get container status \"910eebfe4e704a0d6587f2f3417946f77a840db4fffdfdaa6f916456dca9cebe\": rpc error: code = NotFound desc = could not find container \"910eebfe4e704a0d6587f2f3417946f77a840db4fffdfdaa6f916456dca9cebe\": container with ID starting with 910eebfe4e704a0d6587f2f3417946f77a840db4fffdfdaa6f916456dca9cebe not found: ID does not exist"
Sep 15 20:34:35.790315 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:35.790279    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-df0188-7f66f69447-8vb6b"]
Sep 15 20:34:35.793527 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:35.793497    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-df0188-7f66f69447-8vb6b"]
Sep 15 20:34:36.061885 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:36.061806    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c68080ad-f647-4e15-a1be-e6ab2030092e" path="/var/lib/kubelet/pods/c68080ad-f647-4e15-a1be-e6ab2030092e/volumes"
Sep 15 20:34:41.973455 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:41.973417    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"]
Sep 15 20:34:41.974323 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:41.974290    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg" podUID="3dae1622-aa93-4ccf-9c98-a09fbbc6a74d" containerName="istio-proxy" containerID="cri-o://3c6e57f9a569f5ab4162bd71750cff65916a212aae36cde6f1f58f395d28b3d1" gracePeriod=30
Sep 15 20:34:43.503852 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:43.503800    2577 patch_prober.go:28] interesting pod/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 20:34:43.504230 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:43.503893    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg" podUID="3dae1622-aa93-4ccf-9c98-a09fbbc6a74d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 20:34:47.257081 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.257003    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:34:47.404839 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.404804    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-credential-socket\") pod \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\" (UID: \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\") "
Sep 15 20:34:47.405019 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.404886    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-istio-envoy\") pod \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\" (UID: \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\") "
Sep 15 20:34:47.405019 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.404920    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-workload-certs\") pod \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\" (UID: \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\") "
Sep 15 20:34:47.405019 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.404942    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-istio-podinfo\") pod \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\" (UID: \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\") "
Sep 15 20:34:47.405019 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.405002    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-istiod-ca-cert\") pod \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\" (UID: \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\") "
Sep 15 20:34:47.405233 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.405037    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6lc5t\" (UniqueName: \"kubernetes.io/projected/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-kube-api-access-6lc5t\") pod \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\" (UID: \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\") "
Sep 15 20:34:47.405233 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.405085    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-workload-socket\") pod \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\" (UID: \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\") "
Sep 15 20:34:47.405233 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.405106    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3dae1622-aa93-4ccf-9c98-a09fbbc6a74d" (UID: "3dae1622-aa93-4ccf-9c98-a09fbbc6a74d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:34:47.405233 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.405119    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-istio-data\") pod \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\" (UID: \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\") "
Sep 15 20:34:47.405233 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.405182    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-istio-token\") pod \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\" (UID: \"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d\") "
Sep 15 20:34:47.405504 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.405311    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3dae1622-aa93-4ccf-9c98-a09fbbc6a74d" (UID: "3dae1622-aa93-4ccf-9c98-a09fbbc6a74d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:34:47.405566 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.405494    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3dae1622-aa93-4ccf-9c98-a09fbbc6a74d" (UID: "3dae1622-aa93-4ccf-9c98-a09fbbc6a74d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:34:47.405566 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.405527    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3dae1622-aa93-4ccf-9c98-a09fbbc6a74d" (UID: "3dae1622-aa93-4ccf-9c98-a09fbbc6a74d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:34:47.405677 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.405655    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-istio-data" (OuterVolumeSpecName: "istio-data") pod "3dae1622-aa93-4ccf-9c98-a09fbbc6a74d" (UID: "3dae1622-aa93-4ccf-9c98-a09fbbc6a74d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:34:47.405824 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.405806    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-credential-socket\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:34:47.405895 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.405827    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-workload-certs\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:34:47.405895 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.405843    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-istiod-ca-cert\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:34:47.405895 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.405855    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-workload-socket\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:34:47.405895 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.405867    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-istio-data\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:34:47.407459 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.407430    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-kube-api-access-6lc5t" (OuterVolumeSpecName: "kube-api-access-6lc5t") pod "3dae1622-aa93-4ccf-9c98-a09fbbc6a74d" (UID: "3dae1622-aa93-4ccf-9c98-a09fbbc6a74d"). InnerVolumeSpecName "kube-api-access-6lc5t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:34:47.407570 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.407453    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3dae1622-aa93-4ccf-9c98-a09fbbc6a74d" (UID: "3dae1622-aa93-4ccf-9c98-a09fbbc6a74d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 20:34:47.407624 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.407606    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3dae1622-aa93-4ccf-9c98-a09fbbc6a74d" (UID: "3dae1622-aa93-4ccf-9c98-a09fbbc6a74d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:34:47.407739 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.407701    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-istio-token" (OuterVolumeSpecName: "istio-token") pod "3dae1622-aa93-4ccf-9c98-a09fbbc6a74d" (UID: "3dae1622-aa93-4ccf-9c98-a09fbbc6a74d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:34:47.463229 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.463192    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-7e3543/distinct-w0-7e3543-kserve-777f86dd66-7clft"]
Sep 15 20:34:47.507447 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.507366    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-istio-token\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:34:47.507447 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.507395    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-istio-envoy\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:34:47.507447 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.507405    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-istio-podinfo\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:34:47.507447 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.507414    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6lc5t\" (UniqueName: \"kubernetes.io/projected/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d-kube-api-access-6lc5t\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:34:47.865518 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.865419    2577 generic.go:356] "Generic (PLEG): container finished" podID="3dae1622-aa93-4ccf-9c98-a09fbbc6a74d" containerID="3c6e57f9a569f5ab4162bd71750cff65916a212aae36cde6f1f58f395d28b3d1" exitCode=0
Sep 15 20:34:47.865518 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.865486    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"
Sep 15 20:34:47.865793 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.865527    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg" event={"ID":"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d","Type":"ContainerDied","Data":"3c6e57f9a569f5ab4162bd71750cff65916a212aae36cde6f1f58f395d28b3d1"}
Sep 15 20:34:47.865793 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.865616    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg" event={"ID":"3dae1622-aa93-4ccf-9c98-a09fbbc6a74d","Type":"ContainerDied","Data":"3d43f680c8f75c1d48dbf0a6e8fd2a1491e7e14fe06c9ca4e82cfd7f82dbb23e"}
Sep 15 20:34:47.865793 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.865639    2577 scope.go:117] "RemoveContainer" containerID="3c6e57f9a569f5ab4162bd71750cff65916a212aae36cde6f1f58f395d28b3d1"
Sep 15 20:34:47.877215 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.877190    2577 scope.go:117] "RemoveContainer" containerID="e8af4f246885d11d06271ab55e87706afce49857416d88c2072189e837f4dd4f"
Sep 15 20:34:47.887976 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.887955    2577 scope.go:117] "RemoveContainer" containerID="3c6e57f9a569f5ab4162bd71750cff65916a212aae36cde6f1f58f395d28b3d1"
Sep 15 20:34:47.888308 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:34:47.888281    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c6e57f9a569f5ab4162bd71750cff65916a212aae36cde6f1f58f395d28b3d1\": container with ID starting with 3c6e57f9a569f5ab4162bd71750cff65916a212aae36cde6f1f58f395d28b3d1 not found: ID does not exist" containerID="3c6e57f9a569f5ab4162bd71750cff65916a212aae36cde6f1f58f395d28b3d1"
Sep 15 20:34:47.888387 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.888321    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c6e57f9a569f5ab4162bd71750cff65916a212aae36cde6f1f58f395d28b3d1"} err="failed to get container status \"3c6e57f9a569f5ab4162bd71750cff65916a212aae36cde6f1f58f395d28b3d1\": rpc error: code = NotFound desc = could not find container \"3c6e57f9a569f5ab4162bd71750cff65916a212aae36cde6f1f58f395d28b3d1\": container with ID starting with 3c6e57f9a569f5ab4162bd71750cff65916a212aae36cde6f1f58f395d28b3d1 not found: ID does not exist"
Sep 15 20:34:47.888387 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.888347    2577 scope.go:117] "RemoveContainer" containerID="e8af4f246885d11d06271ab55e87706afce49857416d88c2072189e837f4dd4f"
Sep 15 20:34:47.888632 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:34:47.888606    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8af4f246885d11d06271ab55e87706afce49857416d88c2072189e837f4dd4f\": container with ID starting with e8af4f246885d11d06271ab55e87706afce49857416d88c2072189e837f4dd4f not found: ID does not exist" containerID="e8af4f246885d11d06271ab55e87706afce49857416d88c2072189e837f4dd4f"
Sep 15 20:34:47.888750 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.888640    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8af4f246885d11d06271ab55e87706afce49857416d88c2072189e837f4dd4f"} err="failed to get container status \"e8af4f246885d11d06271ab55e87706afce49857416d88c2072189e837f4dd4f\": rpc error: code = NotFound desc = could not find container \"e8af4f246885d11d06271ab55e87706afce49857416d88c2072189e837f4dd4f\": container with ID starting with e8af4f246885d11d06271ab55e87706afce49857416d88c2072189e837f4dd4f not found: ID does not exist"
Sep 15 20:34:47.889140 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.889106    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"]
Sep 15 20:34:47.892492 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:47.892470    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-df0188-openshift-default-6b7b66f884-9hfzg"]
Sep 15 20:34:48.061638 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:48.061603    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3dae1622-aa93-4ccf-9c98-a09fbbc6a74d" path="/var/lib/kubelet/pods/3dae1622-aa93-4ccf-9c98-a09fbbc6a74d/volumes"
Sep 15 20:34:53.317758 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:53.317705    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s"]
Sep 15 20:34:53.318302 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:53.317977    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s" podUID="619bc5fd-649c-4990-b91a-f6ad01e1f035" containerName="maas-api" containerID="cri-o://6db7570f1ce9ba5fff2508690fe8d9ba54acdd31dcd1acd75e8dfe96703703d1" gracePeriod=30
Sep 15 20:34:53.592640 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:53.592607    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s"
Sep 15 20:34:53.668290 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:53.668248    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/619bc5fd-649c-4990-b91a-f6ad01e1f035-maas-api-tls\") pod \"619bc5fd-649c-4990-b91a-f6ad01e1f035\" (UID: \"619bc5fd-649c-4990-b91a-f6ad01e1f035\") "
Sep 15 20:34:53.668494 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:53.668340    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/619bc5fd-649c-4990-b91a-f6ad01e1f035-openshift-service-ca\") pod \"619bc5fd-649c-4990-b91a-f6ad01e1f035\" (UID: \"619bc5fd-649c-4990-b91a-f6ad01e1f035\") "
Sep 15 20:34:53.668494 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:53.668366    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jw4wc\" (UniqueName: \"kubernetes.io/projected/619bc5fd-649c-4990-b91a-f6ad01e1f035-kube-api-access-jw4wc\") pod \"619bc5fd-649c-4990-b91a-f6ad01e1f035\" (UID: \"619bc5fd-649c-4990-b91a-f6ad01e1f035\") "
Sep 15 20:34:53.668494 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:53.668434    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/619bc5fd-649c-4990-b91a-f6ad01e1f035-metrics-certs\") pod \"619bc5fd-649c-4990-b91a-f6ad01e1f035\" (UID: \"619bc5fd-649c-4990-b91a-f6ad01e1f035\") "
Sep 15 20:34:53.668835 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:53.668806    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/619bc5fd-649c-4990-b91a-f6ad01e1f035-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "619bc5fd-649c-4990-b91a-f6ad01e1f035" (UID: "619bc5fd-649c-4990-b91a-f6ad01e1f035"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:34:53.670514 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:53.670486    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/619bc5fd-649c-4990-b91a-f6ad01e1f035-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "619bc5fd-649c-4990-b91a-f6ad01e1f035" (UID: "619bc5fd-649c-4990-b91a-f6ad01e1f035"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:34:53.670660 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:53.670635    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/619bc5fd-649c-4990-b91a-f6ad01e1f035-kube-api-access-jw4wc" (OuterVolumeSpecName: "kube-api-access-jw4wc") pod "619bc5fd-649c-4990-b91a-f6ad01e1f035" (UID: "619bc5fd-649c-4990-b91a-f6ad01e1f035"). InnerVolumeSpecName "kube-api-access-jw4wc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:34:53.670710 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:53.670636    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/619bc5fd-649c-4990-b91a-f6ad01e1f035-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "619bc5fd-649c-4990-b91a-f6ad01e1f035" (UID: "619bc5fd-649c-4990-b91a-f6ad01e1f035"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:34:53.770021 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:53.769981    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/619bc5fd-649c-4990-b91a-f6ad01e1f035-openshift-service-ca\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:34:53.770021 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:53.770016    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jw4wc\" (UniqueName: \"kubernetes.io/projected/619bc5fd-649c-4990-b91a-f6ad01e1f035-kube-api-access-jw4wc\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:34:53.770021 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:53.770027    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/619bc5fd-649c-4990-b91a-f6ad01e1f035-metrics-certs\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:34:53.770276 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:53.770035    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/619bc5fd-649c-4990-b91a-f6ad01e1f035-maas-api-tls\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:34:53.913751 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:53.913639    2577 generic.go:356] "Generic (PLEG): container finished" podID="619bc5fd-649c-4990-b91a-f6ad01e1f035" containerID="6db7570f1ce9ba5fff2508690fe8d9ba54acdd31dcd1acd75e8dfe96703703d1" exitCode=0
Sep 15 20:34:53.913751 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:53.913708    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s"
Sep 15 20:34:53.913751 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:53.913716    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s" event={"ID":"619bc5fd-649c-4990-b91a-f6ad01e1f035","Type":"ContainerDied","Data":"6db7570f1ce9ba5fff2508690fe8d9ba54acdd31dcd1acd75e8dfe96703703d1"}
Sep 15 20:34:53.914049 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:53.913830    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s" event={"ID":"619bc5fd-649c-4990-b91a-f6ad01e1f035","Type":"ContainerDied","Data":"2b79295c2c90a05d8ffd078a94fc6150a51613ac211672a4ca1ec0fbb5d67396"}
Sep 15 20:34:53.914049 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:53.913863    2577 scope.go:117] "RemoveContainer" containerID="6db7570f1ce9ba5fff2508690fe8d9ba54acdd31dcd1acd75e8dfe96703703d1"
Sep 15 20:34:53.927596 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:53.927568    2577 scope.go:117] "RemoveContainer" containerID="6db7570f1ce9ba5fff2508690fe8d9ba54acdd31dcd1acd75e8dfe96703703d1"
Sep 15 20:34:53.928067 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:34:53.928040    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6db7570f1ce9ba5fff2508690fe8d9ba54acdd31dcd1acd75e8dfe96703703d1\": container with ID starting with 6db7570f1ce9ba5fff2508690fe8d9ba54acdd31dcd1acd75e8dfe96703703d1 not found: ID does not exist" containerID="6db7570f1ce9ba5fff2508690fe8d9ba54acdd31dcd1acd75e8dfe96703703d1"
Sep 15 20:34:53.928239 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:53.928078    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6db7570f1ce9ba5fff2508690fe8d9ba54acdd31dcd1acd75e8dfe96703703d1"} err="failed to get container status \"6db7570f1ce9ba5fff2508690fe8d9ba54acdd31dcd1acd75e8dfe96703703d1\": rpc error: code = NotFound desc = could not find container \"6db7570f1ce9ba5fff2508690fe8d9ba54acdd31dcd1acd75e8dfe96703703d1\": container with ID starting with 6db7570f1ce9ba5fff2508690fe8d9ba54acdd31dcd1acd75e8dfe96703703d1 not found: ID does not exist"
Sep 15 20:34:53.943521 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:53.943477    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s"]
Sep 15 20:34:53.948753 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:53.948691    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-dc12d7-5b66c766fc-5qs7s"]
Sep 15 20:34:54.064136 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:54.064091    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="619bc5fd-649c-4990-b91a-f6ad01e1f035" path="/var/lib/kubelet/pods/619bc5fd-649c-4990-b91a-f6ad01e1f035/volumes"
Sep 15 20:34:54.261383 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:54.261355    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-r227b_ecab5774-79a4-42eb-9bb5-a1f09b18f3fb/console-operator/1.log"
Sep 15 20:34:54.270111 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:54.270082    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-r227b_ecab5774-79a4-42eb-9bb5-a1f09b18f3fb/console-operator/1.log"
Sep 15 20:34:57.573360 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:57.573322    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86"]
Sep 15 20:34:57.574408 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:57.574370    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86" podUID="64cbef4e-b625-44f3-8acd-5b1ca3d4d154" containerName="maas-api" containerID="cri-o://ad1fe5adddee1d826b6bae3e2c2654ed997af4bf70604c1a1743cd930ef5d035" gracePeriod=30
Sep 15 20:34:57.852600 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:57.852561    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86"
Sep 15 20:34:57.923014 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:57.922977    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64cbef4e-b625-44f3-8acd-5b1ca3d4d154-openshift-service-ca\") pod \"64cbef4e-b625-44f3-8acd-5b1ca3d4d154\" (UID: \"64cbef4e-b625-44f3-8acd-5b1ca3d4d154\") "
Sep 15 20:34:57.923220 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:57.923210    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8tfjz\" (UniqueName: \"kubernetes.io/projected/64cbef4e-b625-44f3-8acd-5b1ca3d4d154-kube-api-access-8tfjz\") pod \"64cbef4e-b625-44f3-8acd-5b1ca3d4d154\" (UID: \"64cbef4e-b625-44f3-8acd-5b1ca3d4d154\") "
Sep 15 20:34:57.923297 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:57.923265    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64cbef4e-b625-44f3-8acd-5b1ca3d4d154-metrics-certs\") pod \"64cbef4e-b625-44f3-8acd-5b1ca3d4d154\" (UID: \"64cbef4e-b625-44f3-8acd-5b1ca3d4d154\") "
Sep 15 20:34:57.923361 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:57.923313    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64cbef4e-b625-44f3-8acd-5b1ca3d4d154-maas-api-tls\") pod \"64cbef4e-b625-44f3-8acd-5b1ca3d4d154\" (UID: \"64cbef4e-b625-44f3-8acd-5b1ca3d4d154\") "
Sep 15 20:34:57.923569 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:57.923426    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64cbef4e-b625-44f3-8acd-5b1ca3d4d154-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "64cbef4e-b625-44f3-8acd-5b1ca3d4d154" (UID: "64cbef4e-b625-44f3-8acd-5b1ca3d4d154"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:34:57.923738 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:57.923709    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64cbef4e-b625-44f3-8acd-5b1ca3d4d154-openshift-service-ca\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:34:57.926119 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:57.926078    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64cbef4e-b625-44f3-8acd-5b1ca3d4d154-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "64cbef4e-b625-44f3-8acd-5b1ca3d4d154" (UID: "64cbef4e-b625-44f3-8acd-5b1ca3d4d154"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:34:57.926331 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:57.926306    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64cbef4e-b625-44f3-8acd-5b1ca3d4d154-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "64cbef4e-b625-44f3-8acd-5b1ca3d4d154" (UID: "64cbef4e-b625-44f3-8acd-5b1ca3d4d154"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:34:57.926392 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:57.926344    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64cbef4e-b625-44f3-8acd-5b1ca3d4d154-kube-api-access-8tfjz" (OuterVolumeSpecName: "kube-api-access-8tfjz") pod "64cbef4e-b625-44f3-8acd-5b1ca3d4d154" (UID: "64cbef4e-b625-44f3-8acd-5b1ca3d4d154"). InnerVolumeSpecName "kube-api-access-8tfjz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:34:57.945423 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:57.945387    2577 generic.go:356] "Generic (PLEG): container finished" podID="64cbef4e-b625-44f3-8acd-5b1ca3d4d154" containerID="ad1fe5adddee1d826b6bae3e2c2654ed997af4bf70604c1a1743cd930ef5d035" exitCode=0
Sep 15 20:34:57.945658 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:57.945444    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86"
Sep 15 20:34:57.945658 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:57.945550    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86" event={"ID":"64cbef4e-b625-44f3-8acd-5b1ca3d4d154","Type":"ContainerDied","Data":"ad1fe5adddee1d826b6bae3e2c2654ed997af4bf70604c1a1743cd930ef5d035"}
Sep 15 20:34:57.945658 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:57.945596    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86" event={"ID":"64cbef4e-b625-44f3-8acd-5b1ca3d4d154","Type":"ContainerDied","Data":"8a745c7c68267f628840f7f7c6f6ac8456b6af500736a51b6ec3ad79fccaeac3"}
Sep 15 20:34:57.945658 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:57.945620    2577 scope.go:117] "RemoveContainer" containerID="ad1fe5adddee1d826b6bae3e2c2654ed997af4bf70604c1a1743cd930ef5d035"
Sep 15 20:34:57.960026 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:57.959995    2577 scope.go:117] "RemoveContainer" containerID="ad1fe5adddee1d826b6bae3e2c2654ed997af4bf70604c1a1743cd930ef5d035"
Sep 15 20:34:57.960436 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:34:57.960410    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad1fe5adddee1d826b6bae3e2c2654ed997af4bf70604c1a1743cd930ef5d035\": container with ID starting with ad1fe5adddee1d826b6bae3e2c2654ed997af4bf70604c1a1743cd930ef5d035 not found: ID does not exist" containerID="ad1fe5adddee1d826b6bae3e2c2654ed997af4bf70604c1a1743cd930ef5d035"
Sep 15 20:34:57.960551 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:57.960444    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad1fe5adddee1d826b6bae3e2c2654ed997af4bf70604c1a1743cd930ef5d035"} err="failed to get container status \"ad1fe5adddee1d826b6bae3e2c2654ed997af4bf70604c1a1743cd930ef5d035\": rpc error: code = NotFound desc = could not find container \"ad1fe5adddee1d826b6bae3e2c2654ed997af4bf70604c1a1743cd930ef5d035\": container with ID starting with ad1fe5adddee1d826b6bae3e2c2654ed997af4bf70604c1a1743cd930ef5d035 not found: ID does not exist"
Sep 15 20:34:57.973272 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:57.973220    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86"]
Sep 15 20:34:57.978835 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:57.978806    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-7e3543-f468f9478-b4s86"]
Sep 15 20:34:58.024563 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:58.024520    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8tfjz\" (UniqueName: \"kubernetes.io/projected/64cbef4e-b625-44f3-8acd-5b1ca3d4d154-kube-api-access-8tfjz\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:34:58.024563 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:58.024560    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64cbef4e-b625-44f3-8acd-5b1ca3d4d154-metrics-certs\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:34:58.024563 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:58.024570    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64cbef4e-b625-44f3-8acd-5b1ca3d4d154-maas-api-tls\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:34:58.062272 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:34:58.062236    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64cbef4e-b625-44f3-8acd-5b1ca3d4d154" path="/var/lib/kubelet/pods/64cbef4e-b625-44f3-8acd-5b1ca3d4d154/volumes"
Sep 15 20:35:02.488752 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:02.488671    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-7e3543/distinct-w0-7e3543-kserve-777f86dd66-7clft" podUID="b8401025-2132-4c14-a9a8-302aed11f8c4" containerName="main" containerID="cri-o://f236c682e725930b368476d0f674a04a66d38781ab661561a43f123ec2e13ce8" gracePeriod=45
Sep 15 20:35:02.784794 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:02.784719    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-7e3543/distinct-w0-7e3543-kserve-777f86dd66-7clft"
Sep 15 20:35:02.880704 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:02.880670    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8401025-2132-4c14-a9a8-302aed11f8c4-tmp-dir\") pod \"b8401025-2132-4c14-a9a8-302aed11f8c4\" (UID: \"b8401025-2132-4c14-a9a8-302aed11f8c4\") "
Sep 15 20:35:02.880930 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:02.880745    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8401025-2132-4c14-a9a8-302aed11f8c4-home\") pod \"b8401025-2132-4c14-a9a8-302aed11f8c4\" (UID: \"b8401025-2132-4c14-a9a8-302aed11f8c4\") "
Sep 15 20:35:02.880930 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:02.880787    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vmfjx\" (UniqueName: \"kubernetes.io/projected/b8401025-2132-4c14-a9a8-302aed11f8c4-kube-api-access-vmfjx\") pod \"b8401025-2132-4c14-a9a8-302aed11f8c4\" (UID: \"b8401025-2132-4c14-a9a8-302aed11f8c4\") "
Sep 15 20:35:02.880930 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:02.880839    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8401025-2132-4c14-a9a8-302aed11f8c4-model-cache\") pod \"b8401025-2132-4c14-a9a8-302aed11f8c4\" (UID: \"b8401025-2132-4c14-a9a8-302aed11f8c4\") "
Sep 15 20:35:02.881121 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:02.881029    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8401025-2132-4c14-a9a8-302aed11f8c4-home" (OuterVolumeSpecName: "home") pod "b8401025-2132-4c14-a9a8-302aed11f8c4" (UID: "b8401025-2132-4c14-a9a8-302aed11f8c4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:35:02.881269 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:02.881251    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8401025-2132-4c14-a9a8-302aed11f8c4-dshm\") pod \"b8401025-2132-4c14-a9a8-302aed11f8c4\" (UID: \"b8401025-2132-4c14-a9a8-302aed11f8c4\") "
Sep 15 20:35:02.881447 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:02.881431    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8401025-2132-4c14-a9a8-302aed11f8c4-tls-certs\") pod \"b8401025-2132-4c14-a9a8-302aed11f8c4\" (UID: \"b8401025-2132-4c14-a9a8-302aed11f8c4\") "
Sep 15 20:35:02.881842 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:02.881319    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8401025-2132-4c14-a9a8-302aed11f8c4-model-cache" (OuterVolumeSpecName: "model-cache") pod "b8401025-2132-4c14-a9a8-302aed11f8c4" (UID: "b8401025-2132-4c14-a9a8-302aed11f8c4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:35:02.881842 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:02.881705    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8401025-2132-4c14-a9a8-302aed11f8c4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b8401025-2132-4c14-a9a8-302aed11f8c4" (UID: "b8401025-2132-4c14-a9a8-302aed11f8c4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:35:02.882156 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:02.882136    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8401025-2132-4c14-a9a8-302aed11f8c4-tmp-dir\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:35:02.882260 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:02.882247    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8401025-2132-4c14-a9a8-302aed11f8c4-home\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:35:02.882362 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:02.882349    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8401025-2132-4c14-a9a8-302aed11f8c4-model-cache\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:35:02.884385 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:02.884342    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8401025-2132-4c14-a9a8-302aed11f8c4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b8401025-2132-4c14-a9a8-302aed11f8c4" (UID: "b8401025-2132-4c14-a9a8-302aed11f8c4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:35:02.884670 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:02.884634    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8401025-2132-4c14-a9a8-302aed11f8c4-kube-api-access-vmfjx" (OuterVolumeSpecName: "kube-api-access-vmfjx") pod "b8401025-2132-4c14-a9a8-302aed11f8c4" (UID: "b8401025-2132-4c14-a9a8-302aed11f8c4"). InnerVolumeSpecName "kube-api-access-vmfjx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:35:02.884670 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:02.884652    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8401025-2132-4c14-a9a8-302aed11f8c4-dshm" (OuterVolumeSpecName: "dshm") pod "b8401025-2132-4c14-a9a8-302aed11f8c4" (UID: "b8401025-2132-4c14-a9a8-302aed11f8c4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:35:02.983937 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:02.983903    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8401025-2132-4c14-a9a8-302aed11f8c4-tls-certs\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:35:02.983937 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:02.983933    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmfjx\" (UniqueName: \"kubernetes.io/projected/b8401025-2132-4c14-a9a8-302aed11f8c4-kube-api-access-vmfjx\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:35:02.984150 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:02.983948    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8401025-2132-4c14-a9a8-302aed11f8c4-dshm\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:35:02.985786 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:02.985756    2577 generic.go:356] "Generic (PLEG): container finished" podID="b8401025-2132-4c14-a9a8-302aed11f8c4" containerID="f236c682e725930b368476d0f674a04a66d38781ab661561a43f123ec2e13ce8" exitCode=0
Sep 15 20:35:02.985928 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:02.985807    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-7e3543/distinct-w0-7e3543-kserve-777f86dd66-7clft"
Sep 15 20:35:02.985928 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:02.985852    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-7e3543/distinct-w0-7e3543-kserve-777f86dd66-7clft" event={"ID":"b8401025-2132-4c14-a9a8-302aed11f8c4","Type":"ContainerDied","Data":"f236c682e725930b368476d0f674a04a66d38781ab661561a43f123ec2e13ce8"}
Sep 15 20:35:02.985928 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:02.985905    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-7e3543/distinct-w0-7e3543-kserve-777f86dd66-7clft" event={"ID":"b8401025-2132-4c14-a9a8-302aed11f8c4","Type":"ContainerDied","Data":"1e65789911418d177c7272ba6b694e812a57a804189c016e5bf9d0cf636104e5"}
Sep 15 20:35:02.986100 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:02.985930    2577 scope.go:117] "RemoveContainer" containerID="f236c682e725930b368476d0f674a04a66d38781ab661561a43f123ec2e13ce8"
Sep 15 20:35:02.997596 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:02.997574    2577 scope.go:117] "RemoveContainer" containerID="f236c682e725930b368476d0f674a04a66d38781ab661561a43f123ec2e13ce8"
Sep 15 20:35:02.997962 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:35:02.997941    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f236c682e725930b368476d0f674a04a66d38781ab661561a43f123ec2e13ce8\": container with ID starting with f236c682e725930b368476d0f674a04a66d38781ab661561a43f123ec2e13ce8 not found: ID does not exist" containerID="f236c682e725930b368476d0f674a04a66d38781ab661561a43f123ec2e13ce8"
Sep 15 20:35:02.998035 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:02.997972    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f236c682e725930b368476d0f674a04a66d38781ab661561a43f123ec2e13ce8"} err="failed to get container status \"f236c682e725930b368476d0f674a04a66d38781ab661561a43f123ec2e13ce8\": rpc error: code = NotFound desc = could not find container \"f236c682e725930b368476d0f674a04a66d38781ab661561a43f123ec2e13ce8\": container with ID starting with f236c682e725930b368476d0f674a04a66d38781ab661561a43f123ec2e13ce8 not found: ID does not exist"
Sep 15 20:35:03.009820 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:03.009791    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-7e3543/distinct-w0-7e3543-kserve-777f86dd66-7clft"]
Sep 15 20:35:03.012690 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:03.012669    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-7e3543/distinct-w0-7e3543-kserve-777f86dd66-7clft"]
Sep 15 20:35:04.062517 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:04.062484    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8401025-2132-4c14-a9a8-302aed11f8c4" path="/var/lib/kubelet/pods/b8401025-2132-4c14-a9a8-302aed11f8c4/volumes"
Sep 15 20:35:11.849455 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:11.849411    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68"]
Sep 15 20:35:11.849998 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:11.849645    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68" podUID="0ab26204-516b-4ffc-b982-a2aa5ce9690a" containerName="maas-api" containerID="cri-o://f529500852cb0292b94559e393236e5a353fbfe96d43304f1ae86931bfd1a78f" gracePeriod=30
Sep 15 20:35:12.049558 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:12.049524    2577 generic.go:356] "Generic (PLEG): container finished" podID="0ab26204-516b-4ffc-b982-a2aa5ce9690a" containerID="f529500852cb0292b94559e393236e5a353fbfe96d43304f1ae86931bfd1a78f" exitCode=0
Sep 15 20:35:12.049766 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:12.049581    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68" event={"ID":"0ab26204-516b-4ffc-b982-a2aa5ce9690a","Type":"ContainerDied","Data":"f529500852cb0292b94559e393236e5a353fbfe96d43304f1ae86931bfd1a78f"}
Sep 15 20:35:12.119435 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:12.119392    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68"
Sep 15 20:35:12.283647 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:12.283606    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ab26204-516b-4ffc-b982-a2aa5ce9690a-openshift-service-ca\") pod \"0ab26204-516b-4ffc-b982-a2aa5ce9690a\" (UID: \"0ab26204-516b-4ffc-b982-a2aa5ce9690a\") "
Sep 15 20:35:12.283872 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:12.283683    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ab26204-516b-4ffc-b982-a2aa5ce9690a-metrics-certs\") pod \"0ab26204-516b-4ffc-b982-a2aa5ce9690a\" (UID: \"0ab26204-516b-4ffc-b982-a2aa5ce9690a\") "
Sep 15 20:35:12.283872 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:12.283742    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ab26204-516b-4ffc-b982-a2aa5ce9690a-maas-api-tls\") pod \"0ab26204-516b-4ffc-b982-a2aa5ce9690a\" (UID: \"0ab26204-516b-4ffc-b982-a2aa5ce9690a\") "
Sep 15 20:35:12.283872 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:12.283821    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6g4jq\" (UniqueName: \"kubernetes.io/projected/0ab26204-516b-4ffc-b982-a2aa5ce9690a-kube-api-access-6g4jq\") pod \"0ab26204-516b-4ffc-b982-a2aa5ce9690a\" (UID: \"0ab26204-516b-4ffc-b982-a2aa5ce9690a\") "
Sep 15 20:35:12.284111 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:12.284085    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ab26204-516b-4ffc-b982-a2aa5ce9690a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0ab26204-516b-4ffc-b982-a2aa5ce9690a" (UID: "0ab26204-516b-4ffc-b982-a2aa5ce9690a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:35:12.284302 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:12.284283    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ab26204-516b-4ffc-b982-a2aa5ce9690a-openshift-service-ca\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:35:12.286338 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:12.286309    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ab26204-516b-4ffc-b982-a2aa5ce9690a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0ab26204-516b-4ffc-b982-a2aa5ce9690a" (UID: "0ab26204-516b-4ffc-b982-a2aa5ce9690a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:35:12.286471 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:12.286382    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ab26204-516b-4ffc-b982-a2aa5ce9690a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0ab26204-516b-4ffc-b982-a2aa5ce9690a" (UID: "0ab26204-516b-4ffc-b982-a2aa5ce9690a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:35:12.286471 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:12.286407    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ab26204-516b-4ffc-b982-a2aa5ce9690a-kube-api-access-6g4jq" (OuterVolumeSpecName: "kube-api-access-6g4jq") pod "0ab26204-516b-4ffc-b982-a2aa5ce9690a" (UID: "0ab26204-516b-4ffc-b982-a2aa5ce9690a"). InnerVolumeSpecName "kube-api-access-6g4jq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:35:12.386140 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:12.386050    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ab26204-516b-4ffc-b982-a2aa5ce9690a-metrics-certs\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:35:12.386140 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:12.386093    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ab26204-516b-4ffc-b982-a2aa5ce9690a-maas-api-tls\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:35:12.386140 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:12.386108    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6g4jq\" (UniqueName: \"kubernetes.io/projected/0ab26204-516b-4ffc-b982-a2aa5ce9690a-kube-api-access-6g4jq\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:35:13.057592 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:13.057561    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68"
Sep 15 20:35:13.058038 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:13.057597    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68" event={"ID":"0ab26204-516b-4ffc-b982-a2aa5ce9690a","Type":"ContainerDied","Data":"fef2a4106bcb24efcb9abe473df6ed2188486b5ede35e95fa6729a17f246b5e6"}
Sep 15 20:35:13.058038 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:13.057648    2577 scope.go:117] "RemoveContainer" containerID="f529500852cb0292b94559e393236e5a353fbfe96d43304f1ae86931bfd1a78f"
Sep 15 20:35:13.080868 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:13.080839    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68"]
Sep 15 20:35:13.083915 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:13.083890    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f49968-5cb6c66f6f-lrg68"]
Sep 15 20:35:14.063872 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:14.063832    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ab26204-516b-4ffc-b982-a2aa5ce9690a" path="/var/lib/kubelet/pods/0ab26204-516b-4ffc-b982-a2aa5ce9690a/volumes"
Sep 15 20:35:57.206368 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:57.206331    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7966-270820"]
Sep 15 20:35:57.206841 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:57.206826    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8401025-2132-4c14-a9a8-302aed11f8c4" containerName="main"
Sep 15 20:35:57.206841 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:57.206842    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8401025-2132-4c14-a9a8-302aed11f8c4" containerName="main"
Sep 15 20:35:57.206923 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:57.206858    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3dae1622-aa93-4ccf-9c98-a09fbbc6a74d" containerName="istio-proxy"
Sep 15 20:35:57.206923 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:57.206865    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3dae1622-aa93-4ccf-9c98-a09fbbc6a74d" containerName="istio-proxy"
Sep 15 20:35:57.206923 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:57.206873    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c68080ad-f647-4e15-a1be-e6ab2030092e" containerName="maas-api"
Sep 15 20:35:57.206923 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:57.206879    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c68080ad-f647-4e15-a1be-e6ab2030092e" containerName="maas-api"
Sep 15 20:35:57.206923 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:57.206887    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d5c7423-7b7f-4f95-9447-f4cfb49617da" containerName="maas-api"
Sep 15 20:35:57.206923 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:57.206892    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d5c7423-7b7f-4f95-9447-f4cfb49617da" containerName="maas-api"
Sep 15 20:35:57.206923 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:57.206900    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64cbef4e-b625-44f3-8acd-5b1ca3d4d154" containerName="maas-api"
Sep 15 20:35:57.206923 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:57.206905    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="64cbef4e-b625-44f3-8acd-5b1ca3d4d154" containerName="maas-api"
Sep 15 20:35:57.206923 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:57.206912    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3dae1622-aa93-4ccf-9c98-a09fbbc6a74d" containerName="istio-proxy"
Sep 15 20:35:57.206923 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:57.206917    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3dae1622-aa93-4ccf-9c98-a09fbbc6a74d" containerName="istio-proxy"
Sep 15 20:35:57.207222 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:57.206933    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ab26204-516b-4ffc-b982-a2aa5ce9690a" containerName="maas-api"
Sep 15 20:35:57.207222 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:57.206939    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ab26204-516b-4ffc-b982-a2aa5ce9690a" containerName="maas-api"
Sep 15 20:35:57.207222 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:57.206951    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="619bc5fd-649c-4990-b91a-f6ad01e1f035" containerName="maas-api"
Sep 15 20:35:57.207222 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:57.206956    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="619bc5fd-649c-4990-b91a-f6ad01e1f035" containerName="maas-api"
Sep 15 20:35:57.207222 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:57.207020    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="64cbef4e-b625-44f3-8acd-5b1ca3d4d154" containerName="maas-api"
Sep 15 20:35:57.207222 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:57.207027    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3dae1622-aa93-4ccf-9c98-a09fbbc6a74d" containerName="istio-proxy"
Sep 15 20:35:57.207222 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:57.207033    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="619bc5fd-649c-4990-b91a-f6ad01e1f035" containerName="maas-api"
Sep 15 20:35:57.207222 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:57.207037    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c68080ad-f647-4e15-a1be-e6ab2030092e" containerName="maas-api"
Sep 15 20:35:57.207222 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:57.207048    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ab26204-516b-4ffc-b982-a2aa5ce9690a" containerName="maas-api"
Sep 15 20:35:57.207222 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:57.207053    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8401025-2132-4c14-a9a8-302aed11f8c4" containerName="main"
Sep 15 20:35:57.207222 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:57.207060    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d5c7423-7b7f-4f95-9447-f4cfb49617da" containerName="maas-api"
Sep 15 20:35:57.209369 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:57.209347    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7966-270820"
Sep 15 20:35:57.212302 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:57.212282    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tppb5\""
Sep 15 20:35:57.215306 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:57.215278    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7966-270820"]
Sep 15 20:35:57.259075 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:57.259038    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbb4b\" (UniqueName: \"kubernetes.io/projected/51d9392d-564c-4997-838e-11167858b45d-kube-api-access-qbb4b\") pod \"test-curl-7966-270820\" (UID: \"51d9392d-564c-4997-838e-11167858b45d\") " pod="opendatahub/test-curl-7966-270820"
Sep 15 20:35:57.359735 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:57.359698    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbb4b\" (UniqueName: \"kubernetes.io/projected/51d9392d-564c-4997-838e-11167858b45d-kube-api-access-qbb4b\") pod \"test-curl-7966-270820\" (UID: \"51d9392d-564c-4997-838e-11167858b45d\") " pod="opendatahub/test-curl-7966-270820"
Sep 15 20:35:57.367851 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:57.367827    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbb4b\" (UniqueName: \"kubernetes.io/projected/51d9392d-564c-4997-838e-11167858b45d-kube-api-access-qbb4b\") pod \"test-curl-7966-270820\" (UID: \"51d9392d-564c-4997-838e-11167858b45d\") " pod="opendatahub/test-curl-7966-270820"
Sep 15 20:35:57.521406 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:57.521326    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7966-270820"
Sep 15 20:35:57.796853 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:57.796819    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7966-270820"]
Sep 15 20:35:57.798347 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:35:57.798317    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod51d9392d_564c_4997_838e_11167858b45d.slice/crio-bf05f63919388d887a798ed30f11ab93348873006590cb0191deafbf5c3bdcb4 WatchSource:0}: Error finding container bf05f63919388d887a798ed30f11ab93348873006590cb0191deafbf5c3bdcb4: Status 404 returned error can't find the container with id bf05f63919388d887a798ed30f11ab93348873006590cb0191deafbf5c3bdcb4
Sep 15 20:35:58.369776 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:58.369700    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7966-270820" event={"ID":"51d9392d-564c-4997-838e-11167858b45d","Type":"ContainerStarted","Data":"abf9a26a6dfa9598444d5f2709fb2503b8565190263fa72eaecbab5bff0c259a"}
Sep 15 20:35:58.369776 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:58.369781    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7966-270820" event={"ID":"51d9392d-564c-4997-838e-11167858b45d","Type":"ContainerStarted","Data":"bf05f63919388d887a798ed30f11ab93348873006590cb0191deafbf5c3bdcb4"}
Sep 15 20:35:58.386381 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:58.386314    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7966-270820" podStartSLOduration=1.386296868 podStartE2EDuration="1.386296868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:35:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:35:58.383137857 +0000 UTC m=+1864.774090853" watchObservedRunningTime="2026-09-15 20:35:58.386296868 +0000 UTC m=+1864.777249865"
Sep 15 20:35:59.046328 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:59.046280    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7966-270820"]
Sep 15 20:35:59.049521 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:35:59.049492    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7966-270820"]
Sep 15 20:35:59.376652 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:35:59.376618    2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7966-270820" container="test-curl-7966-270820"
Sep 15 20:36:00.384084 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:00.384044    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7966-270820" podUID="51d9392d-564c-4997-838e-11167858b45d" containerName="test-curl-7966-270820" containerID="cri-o://abf9a26a6dfa9598444d5f2709fb2503b8565190263fa72eaecbab5bff0c259a" gracePeriod=2
Sep 15 20:36:02.538455 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:02.538427    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7966-270820"
Sep 15 20:36:02.610640 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:02.610600    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qbb4b\" (UniqueName: \"kubernetes.io/projected/51d9392d-564c-4997-838e-11167858b45d-kube-api-access-qbb4b\") pod \"51d9392d-564c-4997-838e-11167858b45d\" (UID: \"51d9392d-564c-4997-838e-11167858b45d\") "
Sep 15 20:36:02.612738 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:02.612696    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51d9392d-564c-4997-838e-11167858b45d-kube-api-access-qbb4b" (OuterVolumeSpecName: "kube-api-access-qbb4b") pod "51d9392d-564c-4997-838e-11167858b45d" (UID: "51d9392d-564c-4997-838e-11167858b45d"). InnerVolumeSpecName "kube-api-access-qbb4b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:36:02.712133 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:02.712099    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbb4b\" (UniqueName: \"kubernetes.io/projected/51d9392d-564c-4997-838e-11167858b45d-kube-api-access-qbb4b\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:36:03.406046 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:03.406009    2577 generic.go:356] "Generic (PLEG): container finished" podID="51d9392d-564c-4997-838e-11167858b45d" containerID="abf9a26a6dfa9598444d5f2709fb2503b8565190263fa72eaecbab5bff0c259a" exitCode=137
Sep 15 20:36:03.406221 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:03.406070    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7966-270820"
Sep 15 20:36:03.406221 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:03.406115    2577 scope.go:117] "RemoveContainer" containerID="abf9a26a6dfa9598444d5f2709fb2503b8565190263fa72eaecbab5bff0c259a"
Sep 15 20:36:03.415631 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:03.415610    2577 scope.go:117] "RemoveContainer" containerID="abf9a26a6dfa9598444d5f2709fb2503b8565190263fa72eaecbab5bff0c259a"
Sep 15 20:36:03.415952 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:36:03.415931    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abf9a26a6dfa9598444d5f2709fb2503b8565190263fa72eaecbab5bff0c259a\": container with ID starting with abf9a26a6dfa9598444d5f2709fb2503b8565190263fa72eaecbab5bff0c259a not found: ID does not exist" containerID="abf9a26a6dfa9598444d5f2709fb2503b8565190263fa72eaecbab5bff0c259a"
Sep 15 20:36:03.416017 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:03.415960    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abf9a26a6dfa9598444d5f2709fb2503b8565190263fa72eaecbab5bff0c259a"} err="failed to get container status \"abf9a26a6dfa9598444d5f2709fb2503b8565190263fa72eaecbab5bff0c259a\": rpc error: code = NotFound desc = could not find container \"abf9a26a6dfa9598444d5f2709fb2503b8565190263fa72eaecbab5bff0c259a\": container with ID starting with abf9a26a6dfa9598444d5f2709fb2503b8565190263fa72eaecbab5bff0c259a not found: ID does not exist"
Sep 15 20:36:03.416410 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:36:03.416389    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7966-270820\" is forbidden: User \"system:node:ip-10-0-140-72.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-72.ec2.internal' and this object" podUID="51d9392d-564c-4997-838e-11167858b45d" pod="opendatahub/test-curl-7966-270820"
Sep 15 20:36:04.059322 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:36:04.059282    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7966-270820\" is forbidden: User \"system:node:ip-10-0-140-72.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-72.ec2.internal' and this object" podUID="51d9392d-564c-4997-838e-11167858b45d" pod="opendatahub/test-curl-7966-270820"
Sep 15 20:36:04.061102 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:04.061080    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51d9392d-564c-4997-838e-11167858b45d" path="/var/lib/kubelet/pods/51d9392d-564c-4997-838e-11167858b45d/volumes"
Sep 15 20:36:29.357405 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:29.357184    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm"]
Sep 15 20:36:29.358310 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:29.358271    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm" podUID="091f856b-9ed9-43e3-a60f-71b61af8a89e" containerName="maas-api" containerID="cri-o://48ae3b4bf7b72df345201165fbec611d891f458d7cc71659f53115458733aa6d" gracePeriod=30
Sep 15 20:36:29.580755 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:29.580409    2577 generic.go:356] "Generic (PLEG): container finished" podID="091f856b-9ed9-43e3-a60f-71b61af8a89e" containerID="48ae3b4bf7b72df345201165fbec611d891f458d7cc71659f53115458733aa6d" exitCode=0
Sep 15 20:36:29.580755 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:29.580573    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm" event={"ID":"091f856b-9ed9-43e3-a60f-71b61af8a89e","Type":"ContainerDied","Data":"48ae3b4bf7b72df345201165fbec611d891f458d7cc71659f53115458733aa6d"}
Sep 15 20:36:29.632606 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:29.632574    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm"
Sep 15 20:36:29.699066 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:29.699034    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/091f856b-9ed9-43e3-a60f-71b61af8a89e-openshift-service-ca\") pod \"091f856b-9ed9-43e3-a60f-71b61af8a89e\" (UID: \"091f856b-9ed9-43e3-a60f-71b61af8a89e\") "
Sep 15 20:36:29.699273 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:29.699149    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/091f856b-9ed9-43e3-a60f-71b61af8a89e-maas-api-tls\") pod \"091f856b-9ed9-43e3-a60f-71b61af8a89e\" (UID: \"091f856b-9ed9-43e3-a60f-71b61af8a89e\") "
Sep 15 20:36:29.699273 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:29.699193    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/091f856b-9ed9-43e3-a60f-71b61af8a89e-metrics-certs\") pod \"091f856b-9ed9-43e3-a60f-71b61af8a89e\" (UID: \"091f856b-9ed9-43e3-a60f-71b61af8a89e\") "
Sep 15 20:36:29.699273 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:29.699258    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qjwhh\" (UniqueName: \"kubernetes.io/projected/091f856b-9ed9-43e3-a60f-71b61af8a89e-kube-api-access-qjwhh\") pod \"091f856b-9ed9-43e3-a60f-71b61af8a89e\" (UID: \"091f856b-9ed9-43e3-a60f-71b61af8a89e\") "
Sep 15 20:36:29.699569 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:29.699540    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/091f856b-9ed9-43e3-a60f-71b61af8a89e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "091f856b-9ed9-43e3-a60f-71b61af8a89e" (UID: "091f856b-9ed9-43e3-a60f-71b61af8a89e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:36:29.702141 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:29.702108    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/091f856b-9ed9-43e3-a60f-71b61af8a89e-kube-api-access-qjwhh" (OuterVolumeSpecName: "kube-api-access-qjwhh") pod "091f856b-9ed9-43e3-a60f-71b61af8a89e" (UID: "091f856b-9ed9-43e3-a60f-71b61af8a89e"). InnerVolumeSpecName "kube-api-access-qjwhh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:36:29.702348 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:29.702164    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/091f856b-9ed9-43e3-a60f-71b61af8a89e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "091f856b-9ed9-43e3-a60f-71b61af8a89e" (UID: "091f856b-9ed9-43e3-a60f-71b61af8a89e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:36:29.702532 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:29.702503    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/091f856b-9ed9-43e3-a60f-71b61af8a89e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "091f856b-9ed9-43e3-a60f-71b61af8a89e" (UID: "091f856b-9ed9-43e3-a60f-71b61af8a89e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:36:29.800131 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:29.800091    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjwhh\" (UniqueName: \"kubernetes.io/projected/091f856b-9ed9-43e3-a60f-71b61af8a89e-kube-api-access-qjwhh\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:36:29.800131 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:29.800121    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/091f856b-9ed9-43e3-a60f-71b61af8a89e-openshift-service-ca\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:36:29.800131 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:29.800134    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/091f856b-9ed9-43e3-a60f-71b61af8a89e-maas-api-tls\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:36:29.800131 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:29.800144    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/091f856b-9ed9-43e3-a60f-71b61af8a89e-metrics-certs\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:36:30.589742 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:30.589700    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm"
Sep 15 20:36:30.590219 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:30.589745    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm" event={"ID":"091f856b-9ed9-43e3-a60f-71b61af8a89e","Type":"ContainerDied","Data":"3d8d8c97eff52ebd4f61b7c59920f0a858be49c7adf2e8776b9f5f89d58a8721"}
Sep 15 20:36:30.590219 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:30.589817    2577 scope.go:117] "RemoveContainer" containerID="48ae3b4bf7b72df345201165fbec611d891f458d7cc71659f53115458733aa6d"
Sep 15 20:36:30.611115 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:30.611083    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm"]
Sep 15 20:36:30.618660 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:30.618613    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-725e67-6d5cbc6b5c-hkjjm"]
Sep 15 20:36:32.061784 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:32.061748    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="091f856b-9ed9-43e3-a60f-71b61af8a89e" path="/var/lib/kubelet/pods/091f856b-9ed9-43e3-a60f-71b61af8a89e/volumes"
Sep 15 20:36:36.346539 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:36.346505    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"]
Sep 15 20:36:36.346970 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:36.346774    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg" podUID="fa4b30cd-828e-4404-ab50-12cc8cf31f42" containerName="istio-proxy" containerID="cri-o://53924c141a1783aa6f67bfbce1642f5a04163577e762e7de3dd4dd3d46e0e1ca" gracePeriod=30
Sep 15 20:36:41.608369 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.608341    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:36:41.663347 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.663260    2577 generic.go:356] "Generic (PLEG): container finished" podID="fa4b30cd-828e-4404-ab50-12cc8cf31f42" containerID="53924c141a1783aa6f67bfbce1642f5a04163577e762e7de3dd4dd3d46e0e1ca" exitCode=0
Sep 15 20:36:41.663347 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.663312    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"
Sep 15 20:36:41.663347 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.663339    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg" event={"ID":"fa4b30cd-828e-4404-ab50-12cc8cf31f42","Type":"ContainerDied","Data":"53924c141a1783aa6f67bfbce1642f5a04163577e762e7de3dd4dd3d46e0e1ca"}
Sep 15 20:36:41.663587 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.663394    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg" event={"ID":"fa4b30cd-828e-4404-ab50-12cc8cf31f42","Type":"ContainerDied","Data":"2b2fc66ced5b5d204fb4984a687db91d657e5979fa76cd2afe9b6c721f8ec6b0"}
Sep 15 20:36:41.663587 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.663417    2577 scope.go:117] "RemoveContainer" containerID="53924c141a1783aa6f67bfbce1642f5a04163577e762e7de3dd4dd3d46e0e1ca"
Sep 15 20:36:41.673193 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.673173    2577 scope.go:117] "RemoveContainer" containerID="15dde39be4fd6ef0b788d49d9afacb369a871c0aeb4c1123664ae1ec4de80353"
Sep 15 20:36:41.682844 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.682820    2577 scope.go:117] "RemoveContainer" containerID="53924c141a1783aa6f67bfbce1642f5a04163577e762e7de3dd4dd3d46e0e1ca"
Sep 15 20:36:41.683155 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:36:41.683136    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53924c141a1783aa6f67bfbce1642f5a04163577e762e7de3dd4dd3d46e0e1ca\": container with ID starting with 53924c141a1783aa6f67bfbce1642f5a04163577e762e7de3dd4dd3d46e0e1ca not found: ID does not exist" containerID="53924c141a1783aa6f67bfbce1642f5a04163577e762e7de3dd4dd3d46e0e1ca"
Sep 15 20:36:41.683220 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.683166    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53924c141a1783aa6f67bfbce1642f5a04163577e762e7de3dd4dd3d46e0e1ca"} err="failed to get container status \"53924c141a1783aa6f67bfbce1642f5a04163577e762e7de3dd4dd3d46e0e1ca\": rpc error: code = NotFound desc = could not find container \"53924c141a1783aa6f67bfbce1642f5a04163577e762e7de3dd4dd3d46e0e1ca\": container with ID starting with 53924c141a1783aa6f67bfbce1642f5a04163577e762e7de3dd4dd3d46e0e1ca not found: ID does not exist"
Sep 15 20:36:41.683220 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.683188    2577 scope.go:117] "RemoveContainer" containerID="15dde39be4fd6ef0b788d49d9afacb369a871c0aeb4c1123664ae1ec4de80353"
Sep 15 20:36:41.683478 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:36:41.683448    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15dde39be4fd6ef0b788d49d9afacb369a871c0aeb4c1123664ae1ec4de80353\": container with ID starting with 15dde39be4fd6ef0b788d49d9afacb369a871c0aeb4c1123664ae1ec4de80353 not found: ID does not exist" containerID="15dde39be4fd6ef0b788d49d9afacb369a871c0aeb4c1123664ae1ec4de80353"
Sep 15 20:36:41.683551 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.683484    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15dde39be4fd6ef0b788d49d9afacb369a871c0aeb4c1123664ae1ec4de80353"} err="failed to get container status \"15dde39be4fd6ef0b788d49d9afacb369a871c0aeb4c1123664ae1ec4de80353\": rpc error: code = NotFound desc = could not find container \"15dde39be4fd6ef0b788d49d9afacb369a871c0aeb4c1123664ae1ec4de80353\": container with ID starting with 15dde39be4fd6ef0b788d49d9afacb369a871c0aeb4c1123664ae1ec4de80353 not found: ID does not exist"
Sep 15 20:36:41.722613 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.722578    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa4b30cd-828e-4404-ab50-12cc8cf31f42-workload-certs\") pod \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\" (UID: \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\") "
Sep 15 20:36:41.722804 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.722649    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa4b30cd-828e-4404-ab50-12cc8cf31f42-credential-socket\") pod \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\" (UID: \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\") "
Sep 15 20:36:41.722804 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.722680    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa4b30cd-828e-4404-ab50-12cc8cf31f42-istio-token\") pod \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\" (UID: \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\") "
Sep 15 20:36:41.722804 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.722762    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa4b30cd-828e-4404-ab50-12cc8cf31f42-istio-podinfo\") pod \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\" (UID: \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\") "
Sep 15 20:36:41.722979 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.722806    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa4b30cd-828e-4404-ab50-12cc8cf31f42-workload-socket\") pod \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\" (UID: \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\") "
Sep 15 20:36:41.722979 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.722900    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa4b30cd-828e-4404-ab50-12cc8cf31f42-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fa4b30cd-828e-4404-ab50-12cc8cf31f42" (UID: "fa4b30cd-828e-4404-ab50-12cc8cf31f42"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:36:41.722979 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.722908    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa4b30cd-828e-4404-ab50-12cc8cf31f42-istiod-ca-cert\") pod \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\" (UID: \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\") "
Sep 15 20:36:41.722979 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.722963    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa4b30cd-828e-4404-ab50-12cc8cf31f42-istio-envoy\") pod \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\" (UID: \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\") "
Sep 15 20:36:41.722979 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.722968    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa4b30cd-828e-4404-ab50-12cc8cf31f42-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fa4b30cd-828e-4404-ab50-12cc8cf31f42" (UID: "fa4b30cd-828e-4404-ab50-12cc8cf31f42"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:36:41.723250 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.722992    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wd292\" (UniqueName: \"kubernetes.io/projected/fa4b30cd-828e-4404-ab50-12cc8cf31f42-kube-api-access-wd292\") pod \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\" (UID: \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\") "
Sep 15 20:36:41.723250 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.723017    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa4b30cd-828e-4404-ab50-12cc8cf31f42-istio-data\") pod \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\" (UID: \"fa4b30cd-828e-4404-ab50-12cc8cf31f42\") "
Sep 15 20:36:41.723250 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.723138    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa4b30cd-828e-4404-ab50-12cc8cf31f42-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fa4b30cd-828e-4404-ab50-12cc8cf31f42" (UID: "fa4b30cd-828e-4404-ab50-12cc8cf31f42"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:36:41.723399 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.723298    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa4b30cd-828e-4404-ab50-12cc8cf31f42-istio-data" (OuterVolumeSpecName: "istio-data") pod "fa4b30cd-828e-4404-ab50-12cc8cf31f42" (UID: "fa4b30cd-828e-4404-ab50-12cc8cf31f42"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:36:41.723399 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.723336    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa4b30cd-828e-4404-ab50-12cc8cf31f42-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fa4b30cd-828e-4404-ab50-12cc8cf31f42" (UID: "fa4b30cd-828e-4404-ab50-12cc8cf31f42"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:36:41.723535 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.723512    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa4b30cd-828e-4404-ab50-12cc8cf31f42-workload-socket\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:36:41.723590 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.723541    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa4b30cd-828e-4404-ab50-12cc8cf31f42-istiod-ca-cert\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:36:41.723590 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.723556    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa4b30cd-828e-4404-ab50-12cc8cf31f42-istio-data\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:36:41.723590 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.723569    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa4b30cd-828e-4404-ab50-12cc8cf31f42-workload-certs\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:36:41.723590 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.723581    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa4b30cd-828e-4404-ab50-12cc8cf31f42-credential-socket\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:36:41.725125 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.725095    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa4b30cd-828e-4404-ab50-12cc8cf31f42-istio-token" (OuterVolumeSpecName: "istio-token") pod "fa4b30cd-828e-4404-ab50-12cc8cf31f42" (UID: "fa4b30cd-828e-4404-ab50-12cc8cf31f42"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:36:41.725225 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.725145    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fa4b30cd-828e-4404-ab50-12cc8cf31f42-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fa4b30cd-828e-4404-ab50-12cc8cf31f42" (UID: "fa4b30cd-828e-4404-ab50-12cc8cf31f42"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 20:36:41.725225 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.725145    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa4b30cd-828e-4404-ab50-12cc8cf31f42-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fa4b30cd-828e-4404-ab50-12cc8cf31f42" (UID: "fa4b30cd-828e-4404-ab50-12cc8cf31f42"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:36:41.725225 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.725177    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa4b30cd-828e-4404-ab50-12cc8cf31f42-kube-api-access-wd292" (OuterVolumeSpecName: "kube-api-access-wd292") pod "fa4b30cd-828e-4404-ab50-12cc8cf31f42" (UID: "fa4b30cd-828e-4404-ab50-12cc8cf31f42"). InnerVolumeSpecName "kube-api-access-wd292". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:36:41.824369 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.824328    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa4b30cd-828e-4404-ab50-12cc8cf31f42-istio-envoy\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:36:41.824369 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.824366    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wd292\" (UniqueName: \"kubernetes.io/projected/fa4b30cd-828e-4404-ab50-12cc8cf31f42-kube-api-access-wd292\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:36:41.824369 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.824377    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa4b30cd-828e-4404-ab50-12cc8cf31f42-istio-token\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:36:41.824617 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.824386    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa4b30cd-828e-4404-ab50-12cc8cf31f42-istio-podinfo\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:36:41.986226 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.986158    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"]
Sep 15 20:36:41.989862 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:41.989834    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-725e67-openshift-default-85555c746d-rltdg"]
Sep 15 20:36:42.068323 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:36:42.068276    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa4b30cd-828e-4404-ab50-12cc8cf31f42" path="/var/lib/kubelet/pods/fa4b30cd-828e-4404-ab50-12cc8cf31f42/volumes"
Sep 15 20:37:28.885152 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:28.885114    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7966-d6e3bc"]
Sep 15 20:37:28.885613 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:28.885599    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="091f856b-9ed9-43e3-a60f-71b61af8a89e" containerName="maas-api"
Sep 15 20:37:28.885613 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:28.885614    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="091f856b-9ed9-43e3-a60f-71b61af8a89e" containerName="maas-api"
Sep 15 20:37:28.885690 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:28.885627    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa4b30cd-828e-4404-ab50-12cc8cf31f42" containerName="istio-proxy"
Sep 15 20:37:28.885690 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:28.885633    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa4b30cd-828e-4404-ab50-12cc8cf31f42" containerName="istio-proxy"
Sep 15 20:37:28.885690 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:28.885639    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51d9392d-564c-4997-838e-11167858b45d" containerName="test-curl-7966-270820"
Sep 15 20:37:28.885690 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:28.885645    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="51d9392d-564c-4997-838e-11167858b45d" containerName="test-curl-7966-270820"
Sep 15 20:37:28.885844 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:28.885752    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="51d9392d-564c-4997-838e-11167858b45d" containerName="test-curl-7966-270820"
Sep 15 20:37:28.885844 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:28.885760    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="091f856b-9ed9-43e3-a60f-71b61af8a89e" containerName="maas-api"
Sep 15 20:37:28.885844 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:28.885766    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3dae1622-aa93-4ccf-9c98-a09fbbc6a74d" containerName="istio-proxy"
Sep 15 20:37:28.885844 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:28.885772    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa4b30cd-828e-4404-ab50-12cc8cf31f42" containerName="istio-proxy"
Sep 15 20:37:28.885844 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:28.885780    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa4b30cd-828e-4404-ab50-12cc8cf31f42" containerName="istio-proxy"
Sep 15 20:37:28.888883 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:28.888865    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7966-d6e3bc"
Sep 15 20:37:28.891461 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:28.891437    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tppb5\""
Sep 15 20:37:28.894632 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:28.894608    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7966-d6e3bc"]
Sep 15 20:37:29.022637 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:29.022596    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhh72\" (UniqueName: \"kubernetes.io/projected/99bafb55-96c2-4740-a86f-e8cc3c4afda8-kube-api-access-nhh72\") pod \"test-curl-7966-d6e3bc\" (UID: \"99bafb55-96c2-4740-a86f-e8cc3c4afda8\") " pod="opendatahub/test-curl-7966-d6e3bc"
Sep 15 20:37:29.124097 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:29.124060    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhh72\" (UniqueName: \"kubernetes.io/projected/99bafb55-96c2-4740-a86f-e8cc3c4afda8-kube-api-access-nhh72\") pod \"test-curl-7966-d6e3bc\" (UID: \"99bafb55-96c2-4740-a86f-e8cc3c4afda8\") " pod="opendatahub/test-curl-7966-d6e3bc"
Sep 15 20:37:29.132413 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:29.132380    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhh72\" (UniqueName: \"kubernetes.io/projected/99bafb55-96c2-4740-a86f-e8cc3c4afda8-kube-api-access-nhh72\") pod \"test-curl-7966-d6e3bc\" (UID: \"99bafb55-96c2-4740-a86f-e8cc3c4afda8\") " pod="opendatahub/test-curl-7966-d6e3bc"
Sep 15 20:37:29.200076 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:29.199984    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7966-d6e3bc"
Sep 15 20:37:29.473690 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:29.473657    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7966-d6e3bc"]
Sep 15 20:37:29.475394 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:37:29.475363    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod99bafb55_96c2_4740_a86f_e8cc3c4afda8.slice/crio-2d3c54c6486a92f6a3f487b984875fdbc8f96f87b7d0b5163bceca84732bbe54 WatchSource:0}: Error finding container 2d3c54c6486a92f6a3f487b984875fdbc8f96f87b7d0b5163bceca84732bbe54: Status 404 returned error can't find the container with id 2d3c54c6486a92f6a3f487b984875fdbc8f96f87b7d0b5163bceca84732bbe54
Sep 15 20:37:29.980507 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:29.980466    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7966-d6e3bc" event={"ID":"99bafb55-96c2-4740-a86f-e8cc3c4afda8","Type":"ContainerStarted","Data":"8b59c6f82c5eb31e95af6ebb5a8af3b843d60a52e34e72fc4021b8ea758ca419"}
Sep 15 20:37:29.981036 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:29.980528    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7966-d6e3bc" event={"ID":"99bafb55-96c2-4740-a86f-e8cc3c4afda8","Type":"ContainerStarted","Data":"2d3c54c6486a92f6a3f487b984875fdbc8f96f87b7d0b5163bceca84732bbe54"}
Sep 15 20:37:29.996874 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:29.996822    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7966-d6e3bc" podStartSLOduration=1.99680291 podStartE2EDuration="1.99680291s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:37:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:37:29.993506887 +0000 UTC m=+1956.384459884" watchObservedRunningTime="2026-09-15 20:37:29.99680291 +0000 UTC m=+1956.387755907"
Sep 15 20:37:30.562370 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:30.562335    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7966-d6e3bc"]
Sep 15 20:37:30.566790 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:30.566754    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7966-d6e3bc"]
Sep 15 20:37:30.680464 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:30.680366    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7966-d5547d"]
Sep 15 20:37:30.680894 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:30.680879    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa4b30cd-828e-4404-ab50-12cc8cf31f42" containerName="istio-proxy"
Sep 15 20:37:30.680938 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:30.680895    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa4b30cd-828e-4404-ab50-12cc8cf31f42" containerName="istio-proxy"
Sep 15 20:37:30.680938 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:30.680921    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99bafb55-96c2-4740-a86f-e8cc3c4afda8" containerName="test-curl-7966-d6e3bc"
Sep 15 20:37:30.680938 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:30.680926    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="99bafb55-96c2-4740-a86f-e8cc3c4afda8" containerName="test-curl-7966-d6e3bc"
Sep 15 20:37:30.681034 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:30.681003    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="99bafb55-96c2-4740-a86f-e8cc3c4afda8" containerName="test-curl-7966-d6e3bc"
Sep 15 20:37:30.684029 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:30.684005    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7966-d5547d"
Sep 15 20:37:30.691638 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:30.691607    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7966-d5547d"]
Sep 15 20:37:30.843438 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:30.843392    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnbzf\" (UniqueName: \"kubernetes.io/projected/0fef2822-0fed-4982-af68-d77d25336287-kube-api-access-cnbzf\") pod \"test-curl-7966-d5547d\" (UID: \"0fef2822-0fed-4982-af68-d77d25336287\") " pod="opendatahub/test-curl-7966-d5547d"
Sep 15 20:37:30.945044 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:30.944945    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnbzf\" (UniqueName: \"kubernetes.io/projected/0fef2822-0fed-4982-af68-d77d25336287-kube-api-access-cnbzf\") pod \"test-curl-7966-d5547d\" (UID: \"0fef2822-0fed-4982-af68-d77d25336287\") " pod="opendatahub/test-curl-7966-d5547d"
Sep 15 20:37:30.953049 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:30.953023    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnbzf\" (UniqueName: \"kubernetes.io/projected/0fef2822-0fed-4982-af68-d77d25336287-kube-api-access-cnbzf\") pod \"test-curl-7966-d5547d\" (UID: \"0fef2822-0fed-4982-af68-d77d25336287\") " pod="opendatahub/test-curl-7966-d5547d"
Sep 15 20:37:30.988532 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:37:30.988499    2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7966-d6e3bc" container="test-curl-7966-d6e3bc"
Sep 15 20:37:30.995476 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:30.995455    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7966-d5547d"
Sep 15 20:37:31.268825 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:31.268791    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7966-d5547d"]
Sep 15 20:37:31.270069 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:37:31.270040    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0fef2822_0fed_4982_af68_d77d25336287.slice/crio-250e0f3b41c9ef486c54376e85e65ef5e7476e2e3f22a97e813fcb780a3e97eb WatchSource:0}: Error finding container 250e0f3b41c9ef486c54376e85e65ef5e7476e2e3f22a97e813fcb780a3e97eb: Status 404 returned error can't find the container with id 250e0f3b41c9ef486c54376e85e65ef5e7476e2e3f22a97e813fcb780a3e97eb
Sep 15 20:37:31.996904 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:31.996856    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7966-d5547d" event={"ID":"0fef2822-0fed-4982-af68-d77d25336287","Type":"ContainerStarted","Data":"ee784ecbf05c30f382d6b0b8a702c8c74c126e6332e4ac016a9bdd4c2929556f"}
Sep 15 20:37:31.996904 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:31.996908    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7966-d5547d" event={"ID":"0fef2822-0fed-4982-af68-d77d25336287","Type":"ContainerStarted","Data":"250e0f3b41c9ef486c54376e85e65ef5e7476e2e3f22a97e813fcb780a3e97eb"}
Sep 15 20:37:31.997405 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:31.997049    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7966-d6e3bc" podUID="99bafb55-96c2-4740-a86f-e8cc3c4afda8" containerName="test-curl-7966-d6e3bc" containerID="cri-o://8b59c6f82c5eb31e95af6ebb5a8af3b843d60a52e34e72fc4021b8ea758ca419" gracePeriod=2
Sep 15 20:37:32.009739 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:32.009688    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7966-d5547d" podStartSLOduration=2.009672535 podStartE2EDuration="2.009672535s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:37:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:37:32.009083246 +0000 UTC m=+1958.400036245" watchObservedRunningTime="2026-09-15 20:37:32.009672535 +0000 UTC m=+1958.400625552"
Sep 15 20:37:32.010991 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:37:32.010956    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7966-d6e3bc\" is forbidden: User \"system:node:ip-10-0-140-72.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-72.ec2.internal' and this object" podUID="99bafb55-96c2-4740-a86f-e8cc3c4afda8" pod="opendatahub/test-curl-7966-d6e3bc"
Sep 15 20:37:32.590034 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:32.589997    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7966-d5547d"]
Sep 15 20:37:32.605203 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:32.605166    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7966-d5547d"]
Sep 15 20:37:33.003455 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:37:33.003421    2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7966-d5547d" container="test-curl-7966-d5547d"
Sep 15 20:37:34.009931 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:34.009872    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7966-d5547d" podUID="0fef2822-0fed-4982-af68-d77d25336287" containerName="test-curl-7966-d5547d" containerID="cri-o://ee784ecbf05c30f382d6b0b8a702c8c74c126e6332e4ac016a9bdd4c2929556f" gracePeriod=2
Sep 15 20:37:34.012117 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:37:34.012072    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7966-d5547d\" is forbidden: User \"system:node:ip-10-0-140-72.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-72.ec2.internal' and this object" podUID="0fef2822-0fed-4982-af68-d77d25336287" pod="opendatahub/test-curl-7966-d5547d"
Sep 15 20:37:34.060353 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:37:34.060318    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7966-d5547d\" is forbidden: User \"system:node:ip-10-0-140-72.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-72.ec2.internal' and this object" podUID="0fef2822-0fed-4982-af68-d77d25336287" pod="opendatahub/test-curl-7966-d5547d"
Sep 15 20:37:34.192994 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:34.192968    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7966-d6e3bc"
Sep 15 20:37:34.195130 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:37:34.195104    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7966-d6e3bc\" is forbidden: User \"system:node:ip-10-0-140-72.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-72.ec2.internal' and this object" podUID="99bafb55-96c2-4740-a86f-e8cc3c4afda8" pod="opendatahub/test-curl-7966-d6e3bc"
Sep 15 20:37:34.379863 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:34.379774    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nhh72\" (UniqueName: \"kubernetes.io/projected/99bafb55-96c2-4740-a86f-e8cc3c4afda8-kube-api-access-nhh72\") pod \"99bafb55-96c2-4740-a86f-e8cc3c4afda8\" (UID: \"99bafb55-96c2-4740-a86f-e8cc3c4afda8\") "
Sep 15 20:37:34.381891 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:34.381855    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99bafb55-96c2-4740-a86f-e8cc3c4afda8-kube-api-access-nhh72" (OuterVolumeSpecName: "kube-api-access-nhh72") pod "99bafb55-96c2-4740-a86f-e8cc3c4afda8" (UID: "99bafb55-96c2-4740-a86f-e8cc3c4afda8"). InnerVolumeSpecName "kube-api-access-nhh72". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:37:34.481033 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:34.480995    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhh72\" (UniqueName: \"kubernetes.io/projected/99bafb55-96c2-4740-a86f-e8cc3c4afda8-kube-api-access-nhh72\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:37:35.017908 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:35.017869    2577 generic.go:356] "Generic (PLEG): container finished" podID="99bafb55-96c2-4740-a86f-e8cc3c4afda8" containerID="8b59c6f82c5eb31e95af6ebb5a8af3b843d60a52e34e72fc4021b8ea758ca419" exitCode=137
Sep 15 20:37:35.017908 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:35.017909    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7966-d6e3bc"
Sep 15 20:37:35.018435 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:35.017959    2577 scope.go:117] "RemoveContainer" containerID="8b59c6f82c5eb31e95af6ebb5a8af3b843d60a52e34e72fc4021b8ea758ca419"
Sep 15 20:37:35.020236 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:37:35.020213    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7966-d6e3bc\" is forbidden: User \"system:node:ip-10-0-140-72.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-72.ec2.internal' and this object" podUID="99bafb55-96c2-4740-a86f-e8cc3c4afda8" pod="opendatahub/test-curl-7966-d6e3bc"
Sep 15 20:37:35.028627 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:35.028613    2577 scope.go:117] "RemoveContainer" containerID="8b59c6f82c5eb31e95af6ebb5a8af3b843d60a52e34e72fc4021b8ea758ca419"
Sep 15 20:37:35.028925 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:37:35.028906    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b59c6f82c5eb31e95af6ebb5a8af3b843d60a52e34e72fc4021b8ea758ca419\": container with ID starting with 8b59c6f82c5eb31e95af6ebb5a8af3b843d60a52e34e72fc4021b8ea758ca419 not found: ID does not exist" containerID="8b59c6f82c5eb31e95af6ebb5a8af3b843d60a52e34e72fc4021b8ea758ca419"
Sep 15 20:37:35.028988 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:35.028934    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b59c6f82c5eb31e95af6ebb5a8af3b843d60a52e34e72fc4021b8ea758ca419"} err="failed to get container status \"8b59c6f82c5eb31e95af6ebb5a8af3b843d60a52e34e72fc4021b8ea758ca419\": rpc error: code = NotFound desc = could not find container \"8b59c6f82c5eb31e95af6ebb5a8af3b843d60a52e34e72fc4021b8ea758ca419\": container with ID starting with 8b59c6f82c5eb31e95af6ebb5a8af3b843d60a52e34e72fc4021b8ea758ca419 not found: ID does not exist"
Sep 15 20:37:35.029406 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:37:35.029388    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7966-d6e3bc\" is forbidden: User \"system:node:ip-10-0-140-72.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-72.ec2.internal' and this object" podUID="99bafb55-96c2-4740-a86f-e8cc3c4afda8" pod="opendatahub/test-curl-7966-d6e3bc"
Sep 15 20:37:36.062757 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:36.062710    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99bafb55-96c2-4740-a86f-e8cc3c4afda8" path="/var/lib/kubelet/pods/99bafb55-96c2-4740-a86f-e8cc3c4afda8/volumes"
Sep 15 20:37:36.193123 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:36.193097    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7966-d5547d"
Sep 15 20:37:36.301322 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:36.301215    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cnbzf\" (UniqueName: \"kubernetes.io/projected/0fef2822-0fed-4982-af68-d77d25336287-kube-api-access-cnbzf\") pod \"0fef2822-0fed-4982-af68-d77d25336287\" (UID: \"0fef2822-0fed-4982-af68-d77d25336287\") "
Sep 15 20:37:36.303489 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:36.303456    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fef2822-0fed-4982-af68-d77d25336287-kube-api-access-cnbzf" (OuterVolumeSpecName: "kube-api-access-cnbzf") pod "0fef2822-0fed-4982-af68-d77d25336287" (UID: "0fef2822-0fed-4982-af68-d77d25336287"). InnerVolumeSpecName "kube-api-access-cnbzf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:37:36.402206 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:36.402167    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cnbzf\" (UniqueName: \"kubernetes.io/projected/0fef2822-0fed-4982-af68-d77d25336287-kube-api-access-cnbzf\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:37:37.034394 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:37.034356    2577 generic.go:356] "Generic (PLEG): container finished" podID="0fef2822-0fed-4982-af68-d77d25336287" containerID="ee784ecbf05c30f382d6b0b8a702c8c74c126e6332e4ac016a9bdd4c2929556f" exitCode=137
Sep 15 20:37:37.034590 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:37.034400    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7966-d5547d"
Sep 15 20:37:37.034590 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:37.034430    2577 scope.go:117] "RemoveContainer" containerID="ee784ecbf05c30f382d6b0b8a702c8c74c126e6332e4ac016a9bdd4c2929556f"
Sep 15 20:37:37.044174 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:37.044153    2577 scope.go:117] "RemoveContainer" containerID="ee784ecbf05c30f382d6b0b8a702c8c74c126e6332e4ac016a9bdd4c2929556f"
Sep 15 20:37:37.044474 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:37:37.044451    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee784ecbf05c30f382d6b0b8a702c8c74c126e6332e4ac016a9bdd4c2929556f\": container with ID starting with ee784ecbf05c30f382d6b0b8a702c8c74c126e6332e4ac016a9bdd4c2929556f not found: ID does not exist" containerID="ee784ecbf05c30f382d6b0b8a702c8c74c126e6332e4ac016a9bdd4c2929556f"
Sep 15 20:37:37.044535 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:37.044486    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee784ecbf05c30f382d6b0b8a702c8c74c126e6332e4ac016a9bdd4c2929556f"} err="failed to get container status \"ee784ecbf05c30f382d6b0b8a702c8c74c126e6332e4ac016a9bdd4c2929556f\": rpc error: code = NotFound desc = could not find container \"ee784ecbf05c30f382d6b0b8a702c8c74c126e6332e4ac016a9bdd4c2929556f\": container with ID starting with ee784ecbf05c30f382d6b0b8a702c8c74c126e6332e4ac016a9bdd4c2929556f not found: ID does not exist"
Sep 15 20:37:37.045200 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:37:37.045167    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7966-d5547d\" is forbidden: User \"system:node:ip-10-0-140-72.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-72.ec2.internal' and this object" podUID="0fef2822-0fed-4982-af68-d77d25336287" pod="opendatahub/test-curl-7966-d5547d"
Sep 15 20:37:38.060822 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:37:38.060793    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0fef2822-0fed-4982-af68-d77d25336287" path="/var/lib/kubelet/pods/0fef2822-0fed-4982-af68-d77d25336287/volumes"
Sep 15 20:39:16.987707 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:16.987617    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562"]
Sep 15 20:39:16.988403 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:16.988104    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0fef2822-0fed-4982-af68-d77d25336287" containerName="test-curl-7966-d5547d"
Sep 15 20:39:16.988403 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:16.988118    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fef2822-0fed-4982-af68-d77d25336287" containerName="test-curl-7966-d5547d"
Sep 15 20:39:16.988403 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:16.988185    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0fef2822-0fed-4982-af68-d77d25336287" containerName="test-curl-7966-d5547d"
Sep 15 20:39:16.991274 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:16.991252    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562"
Sep 15 20:39:16.993810 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:16.993787    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-c005f8-serving-cert\""
Sep 15 20:39:17.001400 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:17.001374    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562"]
Sep 15 20:39:17.139028 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:17.138993    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/750e4921-2932-457a-af2b-2c90bfdf52da-metrics-certs\") pod \"maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562\" (UID: \"750e4921-2932-457a-af2b-2c90bfdf52da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562"
Sep 15 20:39:17.139215 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:17.139127    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5g566\" (UniqueName: \"kubernetes.io/projected/750e4921-2932-457a-af2b-2c90bfdf52da-kube-api-access-5g566\") pod \"maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562\" (UID: \"750e4921-2932-457a-af2b-2c90bfdf52da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562"
Sep 15 20:39:17.139279 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:17.139230    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/750e4921-2932-457a-af2b-2c90bfdf52da-maas-api-tls\") pod \"maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562\" (UID: \"750e4921-2932-457a-af2b-2c90bfdf52da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562"
Sep 15 20:39:17.139279 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:17.139258    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/750e4921-2932-457a-af2b-2c90bfdf52da-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562\" (UID: \"750e4921-2932-457a-af2b-2c90bfdf52da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562"
Sep 15 20:39:17.240502 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:17.240399    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g566\" (UniqueName: \"kubernetes.io/projected/750e4921-2932-457a-af2b-2c90bfdf52da-kube-api-access-5g566\") pod \"maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562\" (UID: \"750e4921-2932-457a-af2b-2c90bfdf52da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562"
Sep 15 20:39:17.240667 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:17.240504    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/750e4921-2932-457a-af2b-2c90bfdf52da-maas-api-tls\") pod \"maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562\" (UID: \"750e4921-2932-457a-af2b-2c90bfdf52da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562"
Sep 15 20:39:17.240667 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:17.240539    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/750e4921-2932-457a-af2b-2c90bfdf52da-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562\" (UID: \"750e4921-2932-457a-af2b-2c90bfdf52da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562"
Sep 15 20:39:17.240667 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:17.240614    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/750e4921-2932-457a-af2b-2c90bfdf52da-metrics-certs\") pod \"maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562\" (UID: \"750e4921-2932-457a-af2b-2c90bfdf52da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562"
Sep 15 20:39:17.241261 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:17.241234    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/750e4921-2932-457a-af2b-2c90bfdf52da-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562\" (UID: \"750e4921-2932-457a-af2b-2c90bfdf52da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562"
Sep 15 20:39:17.243164 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:17.243135    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/750e4921-2932-457a-af2b-2c90bfdf52da-maas-api-tls\") pod \"maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562\" (UID: \"750e4921-2932-457a-af2b-2c90bfdf52da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562"
Sep 15 20:39:17.243264 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:17.243135    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/750e4921-2932-457a-af2b-2c90bfdf52da-metrics-certs\") pod \"maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562\" (UID: \"750e4921-2932-457a-af2b-2c90bfdf52da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562"
Sep 15 20:39:17.248001 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:17.247978    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5g566\" (UniqueName: \"kubernetes.io/projected/750e4921-2932-457a-af2b-2c90bfdf52da-kube-api-access-5g566\") pod \"maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562\" (UID: \"750e4921-2932-457a-af2b-2c90bfdf52da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562"
Sep 15 20:39:17.306807 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:17.306769    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562"
Sep 15 20:39:17.593870 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:17.593838    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562"]
Sep 15 20:39:17.596414 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:39:17.596380    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod750e4921_2932_457a_af2b_2c90bfdf52da.slice/crio-02e8d3495711f4aa576792facf3d5f2d938daa8b1c95c90336153def50f28d0d WatchSource:0}: Error finding container 02e8d3495711f4aa576792facf3d5f2d938daa8b1c95c90336153def50f28d0d: Status 404 returned error can't find the container with id 02e8d3495711f4aa576792facf3d5f2d938daa8b1c95c90336153def50f28d0d
Sep 15 20:39:17.597892 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:17.597872    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 20:39:17.680273 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:17.680230    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562" event={"ID":"750e4921-2932-457a-af2b-2c90bfdf52da","Type":"ContainerStarted","Data":"02e8d3495711f4aa576792facf3d5f2d938daa8b1c95c90336153def50f28d0d"}
Sep 15 20:39:18.688636 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:18.688545    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562" event={"ID":"750e4921-2932-457a-af2b-2c90bfdf52da","Type":"ContainerStarted","Data":"7489cf2d798b67c71561f77bce7dc39c9c1ca88c35b4bb4bfa1359e16f0f5bb2"}
Sep 15 20:39:18.688636 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:18.688615    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562"
Sep 15 20:39:18.709518 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:18.709464    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562" podStartSLOduration=1.981663725 podStartE2EDuration="2.709447058s" totalImagesPullingTime="727.783333ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:39:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:39:18.705477458 +0000 UTC m=+2065.096430468" watchObservedRunningTime="2026-09-15 20:39:18.709447058 +0000 UTC m=+2065.100400054"
Sep 15 20:39:24.701793 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:24.701762    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562"
Sep 15 20:39:47.534741 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:47.534699    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w"]
Sep 15 20:39:47.537687 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:47.537667    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w"
Sep 15 20:39:47.539864 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:47.539846    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-8dc819-serving-cert\""
Sep 15 20:39:47.548745 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:47.548709    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w"]
Sep 15 20:39:47.649041 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:47.649001    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e06c5ee-e2f6-4b98-a77b-d6c29352edab-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w\" (UID: \"7e06c5ee-e2f6-4b98-a77b-d6c29352edab\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w"
Sep 15 20:39:47.649041 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:47.649041    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwnf6\" (UniqueName: \"kubernetes.io/projected/7e06c5ee-e2f6-4b98-a77b-d6c29352edab-kube-api-access-vwnf6\") pod \"maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w\" (UID: \"7e06c5ee-e2f6-4b98-a77b-d6c29352edab\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w"
Sep 15 20:39:47.649250 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:47.649176    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e06c5ee-e2f6-4b98-a77b-d6c29352edab-metrics-certs\") pod \"maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w\" (UID: \"7e06c5ee-e2f6-4b98-a77b-d6c29352edab\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w"
Sep 15 20:39:47.649250 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:47.649208    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e06c5ee-e2f6-4b98-a77b-d6c29352edab-maas-api-tls\") pod \"maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w\" (UID: \"7e06c5ee-e2f6-4b98-a77b-d6c29352edab\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w"
Sep 15 20:39:47.750122 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:47.750079    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e06c5ee-e2f6-4b98-a77b-d6c29352edab-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w\" (UID: \"7e06c5ee-e2f6-4b98-a77b-d6c29352edab\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w"
Sep 15 20:39:47.750122 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:47.750122    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwnf6\" (UniqueName: \"kubernetes.io/projected/7e06c5ee-e2f6-4b98-a77b-d6c29352edab-kube-api-access-vwnf6\") pod \"maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w\" (UID: \"7e06c5ee-e2f6-4b98-a77b-d6c29352edab\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w"
Sep 15 20:39:47.750356 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:47.750198    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e06c5ee-e2f6-4b98-a77b-d6c29352edab-metrics-certs\") pod \"maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w\" (UID: \"7e06c5ee-e2f6-4b98-a77b-d6c29352edab\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w"
Sep 15 20:39:47.750356 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:47.750215    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e06c5ee-e2f6-4b98-a77b-d6c29352edab-maas-api-tls\") pod \"maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w\" (UID: \"7e06c5ee-e2f6-4b98-a77b-d6c29352edab\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w"
Sep 15 20:39:47.750902 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:47.750873    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e06c5ee-e2f6-4b98-a77b-d6c29352edab-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w\" (UID: \"7e06c5ee-e2f6-4b98-a77b-d6c29352edab\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w"
Sep 15 20:39:47.752683 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:47.752659    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e06c5ee-e2f6-4b98-a77b-d6c29352edab-maas-api-tls\") pod \"maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w\" (UID: \"7e06c5ee-e2f6-4b98-a77b-d6c29352edab\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w"
Sep 15 20:39:47.752796 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:47.752778    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e06c5ee-e2f6-4b98-a77b-d6c29352edab-metrics-certs\") pod \"maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w\" (UID: \"7e06c5ee-e2f6-4b98-a77b-d6c29352edab\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w"
Sep 15 20:39:47.757909 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:47.757889    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwnf6\" (UniqueName: \"kubernetes.io/projected/7e06c5ee-e2f6-4b98-a77b-d6c29352edab-kube-api-access-vwnf6\") pod \"maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w\" (UID: \"7e06c5ee-e2f6-4b98-a77b-d6c29352edab\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w"
Sep 15 20:39:47.849675 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:47.849577    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w"
Sep 15 20:39:48.126019 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:48.125981    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w"]
Sep 15 20:39:48.127193 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:39:48.127159    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e06c5ee_e2f6_4b98_a77b_d6c29352edab.slice/crio-5976ed6addbc07279af6e51a101cc5fcb785d44ba6affb0184e6693ff8370321 WatchSource:0}: Error finding container 5976ed6addbc07279af6e51a101cc5fcb785d44ba6affb0184e6693ff8370321: Status 404 returned error can't find the container with id 5976ed6addbc07279af6e51a101cc5fcb785d44ba6affb0184e6693ff8370321
Sep 15 20:39:48.891614 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:48.891516    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w" event={"ID":"7e06c5ee-e2f6-4b98-a77b-d6c29352edab","Type":"ContainerStarted","Data":"18a5c4c04fe3379fdbf47394a4b3b1447aa4574441941712f6129cbdb1c1a896"}
Sep 15 20:39:48.891614 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:48.891583    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w" event={"ID":"7e06c5ee-e2f6-4b98-a77b-d6c29352edab","Type":"ContainerStarted","Data":"5976ed6addbc07279af6e51a101cc5fcb785d44ba6affb0184e6693ff8370321"}
Sep 15 20:39:48.892033 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:48.891616    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w"
Sep 15 20:39:48.911689 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:48.911616    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w" podStartSLOduration=1.572068282 podStartE2EDuration="1.911588749s" totalImagesPullingTime="339.520467ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:39:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:39:48.907053037 +0000 UTC m=+2095.298006058" watchObservedRunningTime="2026-09-15 20:39:48.911588749 +0000 UTC m=+2095.302541746"
Sep 15 20:39:54.318485 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:54.318450    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-r227b_ecab5774-79a4-42eb-9bb5-a1f09b18f3fb/console-operator/1.log"
Sep 15 20:39:54.329243 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:54.329210    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-r227b_ecab5774-79a4-42eb-9bb5-a1f09b18f3fb/console-operator/1.log"
Sep 15 20:39:54.903395 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:39:54.903363    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w"
Sep 15 20:40:20.036925 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:40:20.036891    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-69vj4_8b20137b-9766-427f-b140-5c75d6f77f23/payload-processing/0.log"
Sep 15 20:40:50.980861 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:40:50.980764    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-69vj4_8b20137b-9766-427f-b140-5c75d6f77f23/payload-processing/0.log"
Sep 15 20:40:57.361345 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:40:57.361309    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj"]
Sep 15 20:40:57.365402 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:40:57.365375    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj"
Sep 15 20:40:57.367940 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:40:57.367915    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-a6b11e-serving-cert\""
Sep 15 20:40:57.377714 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:40:57.377676    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj"]
Sep 15 20:40:57.482606 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:40:57.482562    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bh6c\" (UniqueName: \"kubernetes.io/projected/d9042937-f5db-4263-b2c8-9eebcf633f3a-kube-api-access-4bh6c\") pod \"maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj\" (UID: \"d9042937-f5db-4263-b2c8-9eebcf633f3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj"
Sep 15 20:40:57.482810 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:40:57.482657    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9042937-f5db-4263-b2c8-9eebcf633f3a-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj\" (UID: \"d9042937-f5db-4263-b2c8-9eebcf633f3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj"
Sep 15 20:40:57.482859 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:40:57.482824    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9042937-f5db-4263-b2c8-9eebcf633f3a-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj\" (UID: \"d9042937-f5db-4263-b2c8-9eebcf633f3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj"
Sep 15 20:40:57.482899 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:40:57.482862    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9042937-f5db-4263-b2c8-9eebcf633f3a-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj\" (UID: \"d9042937-f5db-4263-b2c8-9eebcf633f3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj"
Sep 15 20:40:57.583718 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:40:57.583672    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9042937-f5db-4263-b2c8-9eebcf633f3a-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj\" (UID: \"d9042937-f5db-4263-b2c8-9eebcf633f3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj"
Sep 15 20:40:57.583947 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:40:57.583791    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9042937-f5db-4263-b2c8-9eebcf633f3a-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj\" (UID: \"d9042937-f5db-4263-b2c8-9eebcf633f3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj"
Sep 15 20:40:57.583947 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:40:57.583811    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9042937-f5db-4263-b2c8-9eebcf633f3a-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj\" (UID: \"d9042937-f5db-4263-b2c8-9eebcf633f3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj"
Sep 15 20:40:57.583947 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:40:57.583856    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bh6c\" (UniqueName: \"kubernetes.io/projected/d9042937-f5db-4263-b2c8-9eebcf633f3a-kube-api-access-4bh6c\") pod \"maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj\" (UID: \"d9042937-f5db-4263-b2c8-9eebcf633f3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj"
Sep 15 20:40:57.584140 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:40:57.583949    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a6b11e-serving-cert: secret "maas-api-e2e-ipp-cleanup-a6b11e-serving-cert" not found
Sep 15 20:40:57.584140 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:40:57.584052    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9042937-f5db-4263-b2c8-9eebcf633f3a-maas-api-tls podName:d9042937-f5db-4263-b2c8-9eebcf633f3a nodeName:}" failed. No retries permitted until 2026-09-15 20:40:58.084024814 +0000 UTC m=+2164.474977796 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d9042937-f5db-4263-b2c8-9eebcf633f3a-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj" (UID: "d9042937-f5db-4263-b2c8-9eebcf633f3a") : secret "maas-api-e2e-ipp-cleanup-a6b11e-serving-cert" not found
Sep 15 20:40:57.584488 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:40:57.584458    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9042937-f5db-4263-b2c8-9eebcf633f3a-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj\" (UID: \"d9042937-f5db-4263-b2c8-9eebcf633f3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj"
Sep 15 20:40:57.586441 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:40:57.586419    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9042937-f5db-4263-b2c8-9eebcf633f3a-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj\" (UID: \"d9042937-f5db-4263-b2c8-9eebcf633f3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj"
Sep 15 20:40:57.592163 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:40:57.592135    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bh6c\" (UniqueName: \"kubernetes.io/projected/d9042937-f5db-4263-b2c8-9eebcf633f3a-kube-api-access-4bh6c\") pod \"maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj\" (UID: \"d9042937-f5db-4263-b2c8-9eebcf633f3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj"
Sep 15 20:40:58.089471 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:40:58.089420    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9042937-f5db-4263-b2c8-9eebcf633f3a-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj\" (UID: \"d9042937-f5db-4263-b2c8-9eebcf633f3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj"
Sep 15 20:40:58.092113 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:40:58.092086    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9042937-f5db-4263-b2c8-9eebcf633f3a-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj\" (UID: \"d9042937-f5db-4263-b2c8-9eebcf633f3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj"
Sep 15 20:40:58.280780 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:40:58.280740    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj"
Sep 15 20:40:58.561036 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:40:58.561011    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj"]
Sep 15 20:40:58.562470 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:40:58.562442    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9042937_f5db_4263_b2c8_9eebcf633f3a.slice/crio-4f55216339b2848c378e02a89cf1ff148af65c4b807c2406795fc1ad2d27f51a WatchSource:0}: Error finding container 4f55216339b2848c378e02a89cf1ff148af65c4b807c2406795fc1ad2d27f51a: Status 404 returned error can't find the container with id 4f55216339b2848c378e02a89cf1ff148af65c4b807c2406795fc1ad2d27f51a
Sep 15 20:40:59.364276 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:40:59.364232    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj" event={"ID":"d9042937-f5db-4263-b2c8-9eebcf633f3a","Type":"ContainerStarted","Data":"ad50e451f838c67cbf3b67d96e1bae1cdf85c7a7119163ebc56855a32a8dc979"}
Sep 15 20:40:59.364475 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:40:59.364291    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj" event={"ID":"d9042937-f5db-4263-b2c8-9eebcf633f3a","Type":"ContainerStarted","Data":"4f55216339b2848c378e02a89cf1ff148af65c4b807c2406795fc1ad2d27f51a"}
Sep 15 20:40:59.364475 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:40:59.364353    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj"
Sep 15 20:40:59.382228 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:40:59.382172    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj" podStartSLOduration=1.671554371 podStartE2EDuration="2.38215532s" totalImagesPullingTime="710.600949ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:40:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:40:59.379846518 +0000 UTC m=+2165.770799515" watchObservedRunningTime="2026-09-15 20:40:59.38215532 +0000 UTC m=+2165.773108340"
Sep 15 20:41:05.378680 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:05.378647    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj"
Sep 15 20:41:26.156288 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:26.156246    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj"]
Sep 15 20:41:26.156929 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:26.156531    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj" podUID="d9042937-f5db-4263-b2c8-9eebcf633f3a" containerName="maas-api" containerID="cri-o://ad50e451f838c67cbf3b67d96e1bae1cdf85c7a7119163ebc56855a32a8dc979" gracePeriod=30
Sep 15 20:41:26.432501 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:26.432476    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj"
Sep 15 20:41:26.494701 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:26.494663    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9042937-f5db-4263-b2c8-9eebcf633f3a-metrics-certs\") pod \"d9042937-f5db-4263-b2c8-9eebcf633f3a\" (UID: \"d9042937-f5db-4263-b2c8-9eebcf633f3a\") "
Sep 15 20:41:26.494899 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:26.494758    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9042937-f5db-4263-b2c8-9eebcf633f3a-openshift-service-ca\") pod \"d9042937-f5db-4263-b2c8-9eebcf633f3a\" (UID: \"d9042937-f5db-4263-b2c8-9eebcf633f3a\") "
Sep 15 20:41:26.494899 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:26.494819    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9042937-f5db-4263-b2c8-9eebcf633f3a-maas-api-tls\") pod \"d9042937-f5db-4263-b2c8-9eebcf633f3a\" (UID: \"d9042937-f5db-4263-b2c8-9eebcf633f3a\") "
Sep 15 20:41:26.494899 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:26.494885    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4bh6c\" (UniqueName: \"kubernetes.io/projected/d9042937-f5db-4263-b2c8-9eebcf633f3a-kube-api-access-4bh6c\") pod \"d9042937-f5db-4263-b2c8-9eebcf633f3a\" (UID: \"d9042937-f5db-4263-b2c8-9eebcf633f3a\") "
Sep 15 20:41:26.495396 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:26.495367    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9042937-f5db-4263-b2c8-9eebcf633f3a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d9042937-f5db-4263-b2c8-9eebcf633f3a" (UID: "d9042937-f5db-4263-b2c8-9eebcf633f3a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:41:26.498113 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:26.498075    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9042937-f5db-4263-b2c8-9eebcf633f3a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d9042937-f5db-4263-b2c8-9eebcf633f3a" (UID: "d9042937-f5db-4263-b2c8-9eebcf633f3a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:41:26.498687 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:26.498651    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9042937-f5db-4263-b2c8-9eebcf633f3a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d9042937-f5db-4263-b2c8-9eebcf633f3a" (UID: "d9042937-f5db-4263-b2c8-9eebcf633f3a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:41:26.499548 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:26.499502    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9042937-f5db-4263-b2c8-9eebcf633f3a-kube-api-access-4bh6c" (OuterVolumeSpecName: "kube-api-access-4bh6c") pod "d9042937-f5db-4263-b2c8-9eebcf633f3a" (UID: "d9042937-f5db-4263-b2c8-9eebcf633f3a"). InnerVolumeSpecName "kube-api-access-4bh6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:41:26.556815 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:26.556781    2577 generic.go:356] "Generic (PLEG): container finished" podID="d9042937-f5db-4263-b2c8-9eebcf633f3a" containerID="ad50e451f838c67cbf3b67d96e1bae1cdf85c7a7119163ebc56855a32a8dc979" exitCode=0
Sep 15 20:41:26.556957 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:26.556822    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj"
Sep 15 20:41:26.556957 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:26.556857    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj" event={"ID":"d9042937-f5db-4263-b2c8-9eebcf633f3a","Type":"ContainerDied","Data":"ad50e451f838c67cbf3b67d96e1bae1cdf85c7a7119163ebc56855a32a8dc979"}
Sep 15 20:41:26.556957 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:26.556909    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj" event={"ID":"d9042937-f5db-4263-b2c8-9eebcf633f3a","Type":"ContainerDied","Data":"4f55216339b2848c378e02a89cf1ff148af65c4b807c2406795fc1ad2d27f51a"}
Sep 15 20:41:26.556957 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:26.556932    2577 scope.go:117] "RemoveContainer" containerID="ad50e451f838c67cbf3b67d96e1bae1cdf85c7a7119163ebc56855a32a8dc979"
Sep 15 20:41:26.567677 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:26.567659    2577 scope.go:117] "RemoveContainer" containerID="ad50e451f838c67cbf3b67d96e1bae1cdf85c7a7119163ebc56855a32a8dc979"
Sep 15 20:41:26.567977 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:41:26.567957    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad50e451f838c67cbf3b67d96e1bae1cdf85c7a7119163ebc56855a32a8dc979\": container with ID starting with ad50e451f838c67cbf3b67d96e1bae1cdf85c7a7119163ebc56855a32a8dc979 not found: ID does not exist" containerID="ad50e451f838c67cbf3b67d96e1bae1cdf85c7a7119163ebc56855a32a8dc979"
Sep 15 20:41:26.568055 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:26.567985    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad50e451f838c67cbf3b67d96e1bae1cdf85c7a7119163ebc56855a32a8dc979"} err="failed to get container status \"ad50e451f838c67cbf3b67d96e1bae1cdf85c7a7119163ebc56855a32a8dc979\": rpc error: code = NotFound desc = could not find container \"ad50e451f838c67cbf3b67d96e1bae1cdf85c7a7119163ebc56855a32a8dc979\": container with ID starting with ad50e451f838c67cbf3b67d96e1bae1cdf85c7a7119163ebc56855a32a8dc979 not found: ID does not exist"
Sep 15 20:41:26.582077 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:26.582049    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj"]
Sep 15 20:41:26.585201 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:26.585177    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-a6b11e-86c4557bd9-4mtnj"]
Sep 15 20:41:26.596303 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:26.596277    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9042937-f5db-4263-b2c8-9eebcf633f3a-metrics-certs\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:41:26.596303 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:26.596302    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9042937-f5db-4263-b2c8-9eebcf633f3a-openshift-service-ca\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:41:26.596487 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:26.596313    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9042937-f5db-4263-b2c8-9eebcf633f3a-maas-api-tls\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:41:26.596487 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:26.596324    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bh6c\" (UniqueName: \"kubernetes.io/projected/d9042937-f5db-4263-b2c8-9eebcf633f3a-kube-api-access-4bh6c\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:41:28.062229 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:28.062193    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9042937-f5db-4263-b2c8-9eebcf633f3a" path="/var/lib/kubelet/pods/d9042937-f5db-4263-b2c8-9eebcf633f3a/volumes"
Sep 15 20:41:51.945334 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:51.945296    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562"]
Sep 15 20:41:51.945808 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:51.945574    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562" podUID="750e4921-2932-457a-af2b-2c90bfdf52da" containerName="maas-api" containerID="cri-o://7489cf2d798b67c71561f77bce7dc39c9c1ca88c35b4bb4bfa1359e16f0f5bb2" gracePeriod=30
Sep 15 20:41:52.200587 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:52.200515    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562"
Sep 15 20:41:52.264762 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:52.264707    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/750e4921-2932-457a-af2b-2c90bfdf52da-openshift-service-ca\") pod \"750e4921-2932-457a-af2b-2c90bfdf52da\" (UID: \"750e4921-2932-457a-af2b-2c90bfdf52da\") "
Sep 15 20:41:52.264933 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:52.264797    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/750e4921-2932-457a-af2b-2c90bfdf52da-metrics-certs\") pod \"750e4921-2932-457a-af2b-2c90bfdf52da\" (UID: \"750e4921-2932-457a-af2b-2c90bfdf52da\") "
Sep 15 20:41:52.264933 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:52.264839    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/750e4921-2932-457a-af2b-2c90bfdf52da-maas-api-tls\") pod \"750e4921-2932-457a-af2b-2c90bfdf52da\" (UID: \"750e4921-2932-457a-af2b-2c90bfdf52da\") "
Sep 15 20:41:52.264933 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:52.264925    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5g566\" (UniqueName: \"kubernetes.io/projected/750e4921-2932-457a-af2b-2c90bfdf52da-kube-api-access-5g566\") pod \"750e4921-2932-457a-af2b-2c90bfdf52da\" (UID: \"750e4921-2932-457a-af2b-2c90bfdf52da\") "
Sep 15 20:41:52.265148 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:52.265119    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/750e4921-2932-457a-af2b-2c90bfdf52da-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "750e4921-2932-457a-af2b-2c90bfdf52da" (UID: "750e4921-2932-457a-af2b-2c90bfdf52da"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:41:52.265296 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:52.265270    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/750e4921-2932-457a-af2b-2c90bfdf52da-openshift-service-ca\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:41:52.267270 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:52.267234    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/750e4921-2932-457a-af2b-2c90bfdf52da-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "750e4921-2932-457a-af2b-2c90bfdf52da" (UID: "750e4921-2932-457a-af2b-2c90bfdf52da"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:41:52.267382 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:52.267322    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/750e4921-2932-457a-af2b-2c90bfdf52da-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "750e4921-2932-457a-af2b-2c90bfdf52da" (UID: "750e4921-2932-457a-af2b-2c90bfdf52da"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:41:52.267382 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:52.267336    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/750e4921-2932-457a-af2b-2c90bfdf52da-kube-api-access-5g566" (OuterVolumeSpecName: "kube-api-access-5g566") pod "750e4921-2932-457a-af2b-2c90bfdf52da" (UID: "750e4921-2932-457a-af2b-2c90bfdf52da"). InnerVolumeSpecName "kube-api-access-5g566". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:41:52.366405 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:52.366363    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/750e4921-2932-457a-af2b-2c90bfdf52da-metrics-certs\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:41:52.366405 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:52.366397    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/750e4921-2932-457a-af2b-2c90bfdf52da-maas-api-tls\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:41:52.366405 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:52.366409    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5g566\" (UniqueName: \"kubernetes.io/projected/750e4921-2932-457a-af2b-2c90bfdf52da-kube-api-access-5g566\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:41:52.730062 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:52.730028    2577 generic.go:356] "Generic (PLEG): container finished" podID="750e4921-2932-457a-af2b-2c90bfdf52da" containerID="7489cf2d798b67c71561f77bce7dc39c9c1ca88c35b4bb4bfa1359e16f0f5bb2" exitCode=0
Sep 15 20:41:52.730220 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:52.730076    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562"
Sep 15 20:41:52.730220 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:52.730121    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562" event={"ID":"750e4921-2932-457a-af2b-2c90bfdf52da","Type":"ContainerDied","Data":"7489cf2d798b67c71561f77bce7dc39c9c1ca88c35b4bb4bfa1359e16f0f5bb2"}
Sep 15 20:41:52.730220 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:52.730174    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562" event={"ID":"750e4921-2932-457a-af2b-2c90bfdf52da","Type":"ContainerDied","Data":"02e8d3495711f4aa576792facf3d5f2d938daa8b1c95c90336153def50f28d0d"}
Sep 15 20:41:52.730220 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:52.730197    2577 scope.go:117] "RemoveContainer" containerID="7489cf2d798b67c71561f77bce7dc39c9c1ca88c35b4bb4bfa1359e16f0f5bb2"
Sep 15 20:41:52.740685 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:52.740668    2577 scope.go:117] "RemoveContainer" containerID="7489cf2d798b67c71561f77bce7dc39c9c1ca88c35b4bb4bfa1359e16f0f5bb2"
Sep 15 20:41:52.741026 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:41:52.740999    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7489cf2d798b67c71561f77bce7dc39c9c1ca88c35b4bb4bfa1359e16f0f5bb2\": container with ID starting with 7489cf2d798b67c71561f77bce7dc39c9c1ca88c35b4bb4bfa1359e16f0f5bb2 not found: ID does not exist" containerID="7489cf2d798b67c71561f77bce7dc39c9c1ca88c35b4bb4bfa1359e16f0f5bb2"
Sep 15 20:41:52.741099 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:52.741035    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7489cf2d798b67c71561f77bce7dc39c9c1ca88c35b4bb4bfa1359e16f0f5bb2"} err="failed to get container status \"7489cf2d798b67c71561f77bce7dc39c9c1ca88c35b4bb4bfa1359e16f0f5bb2\": rpc error: code = NotFound desc = could not find container \"7489cf2d798b67c71561f77bce7dc39c9c1ca88c35b4bb4bfa1359e16f0f5bb2\": container with ID starting with 7489cf2d798b67c71561f77bce7dc39c9c1ca88c35b4bb4bfa1359e16f0f5bb2 not found: ID does not exist"
Sep 15 20:41:52.753219 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:52.753194    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562"]
Sep 15 20:41:52.756883 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:52.756855    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-c005f8-64874f99f6-wt562"]
Sep 15 20:41:54.061921 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:41:54.061890    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="750e4921-2932-457a-af2b-2c90bfdf52da" path="/var/lib/kubelet/pods/750e4921-2932-457a-af2b-2c90bfdf52da/volumes"
Sep 15 20:42:56.961865 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:42:56.961822    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w"]
Sep 15 20:42:56.962452 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:42:56.962064    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w" podUID="7e06c5ee-e2f6-4b98-a77b-d6c29352edab" containerName="maas-api" containerID="cri-o://18a5c4c04fe3379fdbf47394a4b3b1447aa4574441941712f6129cbdb1c1a896" gracePeriod=30
Sep 15 20:42:57.148817 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:42:57.148780    2577 generic.go:356] "Generic (PLEG): container finished" podID="7e06c5ee-e2f6-4b98-a77b-d6c29352edab" containerID="18a5c4c04fe3379fdbf47394a4b3b1447aa4574441941712f6129cbdb1c1a896" exitCode=0
Sep 15 20:42:57.148989 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:42:57.148853    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w" event={"ID":"7e06c5ee-e2f6-4b98-a77b-d6c29352edab","Type":"ContainerDied","Data":"18a5c4c04fe3379fdbf47394a4b3b1447aa4574441941712f6129cbdb1c1a896"}
Sep 15 20:42:57.230318 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:42:57.230192    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w"
Sep 15 20:42:57.343545 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:42:57.343498    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e06c5ee-e2f6-4b98-a77b-d6c29352edab-openshift-service-ca\") pod \"7e06c5ee-e2f6-4b98-a77b-d6c29352edab\" (UID: \"7e06c5ee-e2f6-4b98-a77b-d6c29352edab\") "
Sep 15 20:42:57.343751 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:42:57.343583    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e06c5ee-e2f6-4b98-a77b-d6c29352edab-maas-api-tls\") pod \"7e06c5ee-e2f6-4b98-a77b-d6c29352edab\" (UID: \"7e06c5ee-e2f6-4b98-a77b-d6c29352edab\") "
Sep 15 20:42:57.343751 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:42:57.343714    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwnf6\" (UniqueName: \"kubernetes.io/projected/7e06c5ee-e2f6-4b98-a77b-d6c29352edab-kube-api-access-vwnf6\") pod \"7e06c5ee-e2f6-4b98-a77b-d6c29352edab\" (UID: \"7e06c5ee-e2f6-4b98-a77b-d6c29352edab\") "
Sep 15 20:42:57.343848 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:42:57.343781    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e06c5ee-e2f6-4b98-a77b-d6c29352edab-metrics-certs\") pod \"7e06c5ee-e2f6-4b98-a77b-d6c29352edab\" (UID: \"7e06c5ee-e2f6-4b98-a77b-d6c29352edab\") "
Sep 15 20:42:57.343997 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:42:57.343974    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e06c5ee-e2f6-4b98-a77b-d6c29352edab-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7e06c5ee-e2f6-4b98-a77b-d6c29352edab" (UID: "7e06c5ee-e2f6-4b98-a77b-d6c29352edab"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:42:57.344185 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:42:57.344165    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e06c5ee-e2f6-4b98-a77b-d6c29352edab-openshift-service-ca\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:42:57.345750 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:42:57.345707    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e06c5ee-e2f6-4b98-a77b-d6c29352edab-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7e06c5ee-e2f6-4b98-a77b-d6c29352edab" (UID: "7e06c5ee-e2f6-4b98-a77b-d6c29352edab"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:42:57.345914 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:42:57.345897    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e06c5ee-e2f6-4b98-a77b-d6c29352edab-kube-api-access-vwnf6" (OuterVolumeSpecName: "kube-api-access-vwnf6") pod "7e06c5ee-e2f6-4b98-a77b-d6c29352edab" (UID: "7e06c5ee-e2f6-4b98-a77b-d6c29352edab"). InnerVolumeSpecName "kube-api-access-vwnf6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:42:57.345914 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:42:57.345900    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e06c5ee-e2f6-4b98-a77b-d6c29352edab-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7e06c5ee-e2f6-4b98-a77b-d6c29352edab" (UID: "7e06c5ee-e2f6-4b98-a77b-d6c29352edab"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:42:57.445983 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:42:57.445941    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwnf6\" (UniqueName: \"kubernetes.io/projected/7e06c5ee-e2f6-4b98-a77b-d6c29352edab-kube-api-access-vwnf6\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:42:57.445983 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:42:57.445975    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e06c5ee-e2f6-4b98-a77b-d6c29352edab-metrics-certs\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:42:57.445983 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:42:57.445984    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e06c5ee-e2f6-4b98-a77b-d6c29352edab-maas-api-tls\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:42:58.157436 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:42:58.157401    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w"
Sep 15 20:42:58.157436 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:42:58.157428    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w" event={"ID":"7e06c5ee-e2f6-4b98-a77b-d6c29352edab","Type":"ContainerDied","Data":"5976ed6addbc07279af6e51a101cc5fcb785d44ba6affb0184e6693ff8370321"}
Sep 15 20:42:58.158017 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:42:58.157502    2577 scope.go:117] "RemoveContainer" containerID="18a5c4c04fe3379fdbf47394a4b3b1447aa4574441941712f6129cbdb1c1a896"
Sep 15 20:42:58.175184 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:42:58.175147    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w"]
Sep 15 20:42:58.178314 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:42:58.178285    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-8dc819-7d974c5bdd-fpr6w"]
Sep 15 20:43:00.060816 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:00.060784    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e06c5ee-e2f6-4b98-a77b-d6c29352edab" path="/var/lib/kubelet/pods/7e06c5ee-e2f6-4b98-a77b-d6c29352edab/volumes"
Sep 15 20:43:14.403127 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:14.403085    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf"]
Sep 15 20:43:14.403620 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:14.403360    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf" podUID="fdcedef1-0099-4a2d-99cc-b76ec9074ca1" containerName="maas-api" containerID="cri-o://b7150d533c89d3f2da8df4cf2e66d7635f3fe8b3f0a30d6444556b29401e9e22" gracePeriod=30
Sep 15 20:43:14.672280 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:14.672254    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf"
Sep 15 20:43:14.831238 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:14.831203    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdcedef1-0099-4a2d-99cc-b76ec9074ca1-openshift-service-ca\") pod \"fdcedef1-0099-4a2d-99cc-b76ec9074ca1\" (UID: \"fdcedef1-0099-4a2d-99cc-b76ec9074ca1\") "
Sep 15 20:43:14.831425 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:14.831255    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdcedef1-0099-4a2d-99cc-b76ec9074ca1-metrics-certs\") pod \"fdcedef1-0099-4a2d-99cc-b76ec9074ca1\" (UID: \"fdcedef1-0099-4a2d-99cc-b76ec9074ca1\") "
Sep 15 20:43:14.831425 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:14.831314    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jxlgn\" (UniqueName: \"kubernetes.io/projected/fdcedef1-0099-4a2d-99cc-b76ec9074ca1-kube-api-access-jxlgn\") pod \"fdcedef1-0099-4a2d-99cc-b76ec9074ca1\" (UID: \"fdcedef1-0099-4a2d-99cc-b76ec9074ca1\") "
Sep 15 20:43:14.831425 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:14.831353    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdcedef1-0099-4a2d-99cc-b76ec9074ca1-maas-api-tls\") pod \"fdcedef1-0099-4a2d-99cc-b76ec9074ca1\" (UID: \"fdcedef1-0099-4a2d-99cc-b76ec9074ca1\") "
Sep 15 20:43:14.831682 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:14.831659    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdcedef1-0099-4a2d-99cc-b76ec9074ca1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fdcedef1-0099-4a2d-99cc-b76ec9074ca1" (UID: "fdcedef1-0099-4a2d-99cc-b76ec9074ca1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:43:14.831793 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:14.831777    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdcedef1-0099-4a2d-99cc-b76ec9074ca1-openshift-service-ca\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:43:14.833610 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:14.833581    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdcedef1-0099-4a2d-99cc-b76ec9074ca1-kube-api-access-jxlgn" (OuterVolumeSpecName: "kube-api-access-jxlgn") pod "fdcedef1-0099-4a2d-99cc-b76ec9074ca1" (UID: "fdcedef1-0099-4a2d-99cc-b76ec9074ca1"). InnerVolumeSpecName "kube-api-access-jxlgn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:43:14.833610 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:14.833592    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fdcedef1-0099-4a2d-99cc-b76ec9074ca1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fdcedef1-0099-4a2d-99cc-b76ec9074ca1" (UID: "fdcedef1-0099-4a2d-99cc-b76ec9074ca1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:43:14.833786 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:14.833617    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fdcedef1-0099-4a2d-99cc-b76ec9074ca1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "fdcedef1-0099-4a2d-99cc-b76ec9074ca1" (UID: "fdcedef1-0099-4a2d-99cc-b76ec9074ca1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:43:14.932399 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:14.932310    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdcedef1-0099-4a2d-99cc-b76ec9074ca1-maas-api-tls\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:43:14.932399 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:14.932338    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdcedef1-0099-4a2d-99cc-b76ec9074ca1-metrics-certs\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:43:14.932399 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:14.932352    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxlgn\" (UniqueName: \"kubernetes.io/projected/fdcedef1-0099-4a2d-99cc-b76ec9074ca1-kube-api-access-jxlgn\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:43:15.269268 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:15.269230    2577 generic.go:356] "Generic (PLEG): container finished" podID="fdcedef1-0099-4a2d-99cc-b76ec9074ca1" containerID="b7150d533c89d3f2da8df4cf2e66d7635f3fe8b3f0a30d6444556b29401e9e22" exitCode=0
Sep 15 20:43:15.269439 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:15.269275    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf"
Sep 15 20:43:15.269439 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:15.269304    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf" event={"ID":"fdcedef1-0099-4a2d-99cc-b76ec9074ca1","Type":"ContainerDied","Data":"b7150d533c89d3f2da8df4cf2e66d7635f3fe8b3f0a30d6444556b29401e9e22"}
Sep 15 20:43:15.269439 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:15.269349    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf" event={"ID":"fdcedef1-0099-4a2d-99cc-b76ec9074ca1","Type":"ContainerDied","Data":"4edeeac597fa3813e8a1de38cf47cd3eafbac6447ccb4318b5663e6ca65f163f"}
Sep 15 20:43:15.269439 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:15.269374    2577 scope.go:117] "RemoveContainer" containerID="b7150d533c89d3f2da8df4cf2e66d7635f3fe8b3f0a30d6444556b29401e9e22"
Sep 15 20:43:15.284496 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:15.284451    2577 scope.go:117] "RemoveContainer" containerID="b7150d533c89d3f2da8df4cf2e66d7635f3fe8b3f0a30d6444556b29401e9e22"
Sep 15 20:43:15.285062 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:43:15.285032    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7150d533c89d3f2da8df4cf2e66d7635f3fe8b3f0a30d6444556b29401e9e22\": container with ID starting with b7150d533c89d3f2da8df4cf2e66d7635f3fe8b3f0a30d6444556b29401e9e22 not found: ID does not exist" containerID="b7150d533c89d3f2da8df4cf2e66d7635f3fe8b3f0a30d6444556b29401e9e22"
Sep 15 20:43:15.285166 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:15.285084    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7150d533c89d3f2da8df4cf2e66d7635f3fe8b3f0a30d6444556b29401e9e22"} err="failed to get container status \"b7150d533c89d3f2da8df4cf2e66d7635f3fe8b3f0a30d6444556b29401e9e22\": rpc error: code = NotFound desc = could not find container \"b7150d533c89d3f2da8df4cf2e66d7635f3fe8b3f0a30d6444556b29401e9e22\": container with ID starting with b7150d533c89d3f2da8df4cf2e66d7635f3fe8b3f0a30d6444556b29401e9e22 not found: ID does not exist"
Sep 15 20:43:15.295106 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:15.295080    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf"]
Sep 15 20:43:15.299406 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:15.299378    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-92702a-7df457957b-6xcjf"]
Sep 15 20:43:16.061629 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:16.061594    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fdcedef1-0099-4a2d-99cc-b76ec9074ca1" path="/var/lib/kubelet/pods/fdcedef1-0099-4a2d-99cc-b76ec9074ca1/volumes"
Sep 15 20:43:21.030874 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:21.030834    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"]
Sep 15 20:43:21.031362 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:21.031100    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk" podUID="d311d46a-a8a2-4f83-9559-39fd8630e3f3" containerName="istio-proxy" containerID="cri-o://afd25d786d2c8f7ed36039050404f061f828bab2b638aaffa5c7ed789a5b3ca7" gracePeriod=30
Sep 15 20:43:26.286700 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.286662    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:43:26.339933 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.339833    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d311d46a-a8a2-4f83-9559-39fd8630e3f3-workload-certs\") pod \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\" (UID: \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\") "
Sep 15 20:43:26.339933 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.339885    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d311d46a-a8a2-4f83-9559-39fd8630e3f3-istio-podinfo\") pod \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\" (UID: \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\") "
Sep 15 20:43:26.340173 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.339947    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d311d46a-a8a2-4f83-9559-39fd8630e3f3-workload-socket\") pod \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\" (UID: \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\") "
Sep 15 20:43:26.340173 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.339965    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d311d46a-a8a2-4f83-9559-39fd8630e3f3-istio-token\") pod \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\" (UID: \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\") "
Sep 15 20:43:26.340173 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.339996    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wvppj\" (UniqueName: \"kubernetes.io/projected/d311d46a-a8a2-4f83-9559-39fd8630e3f3-kube-api-access-wvppj\") pod \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\" (UID: \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\") "
Sep 15 20:43:26.340173 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.340024    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d311d46a-a8a2-4f83-9559-39fd8630e3f3-istiod-ca-cert\") pod \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\" (UID: \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\") "
Sep 15 20:43:26.340173 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.340073    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d311d46a-a8a2-4f83-9559-39fd8630e3f3-istio-envoy\") pod \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\" (UID: \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\") "
Sep 15 20:43:26.340173 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.340100    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d311d46a-a8a2-4f83-9559-39fd8630e3f3-istio-data\") pod \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\" (UID: \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\") "
Sep 15 20:43:26.340173 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.340130    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d311d46a-a8a2-4f83-9559-39fd8630e3f3-credential-socket\") pod \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\" (UID: \"d311d46a-a8a2-4f83-9559-39fd8630e3f3\") "
Sep 15 20:43:26.340173 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.340136    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d311d46a-a8a2-4f83-9559-39fd8630e3f3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d311d46a-a8a2-4f83-9559-39fd8630e3f3" (UID: "d311d46a-a8a2-4f83-9559-39fd8630e3f3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:43:26.340599 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.340213    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d311d46a-a8a2-4f83-9559-39fd8630e3f3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d311d46a-a8a2-4f83-9559-39fd8630e3f3" (UID: "d311d46a-a8a2-4f83-9559-39fd8630e3f3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:43:26.340599 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.340444    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d311d46a-a8a2-4f83-9559-39fd8630e3f3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d311d46a-a8a2-4f83-9559-39fd8630e3f3" (UID: "d311d46a-a8a2-4f83-9559-39fd8630e3f3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:43:26.340599 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.340520    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d311d46a-a8a2-4f83-9559-39fd8630e3f3-workload-certs\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:43:26.340599 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.340542    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d311d46a-a8a2-4f83-9559-39fd8630e3f3-workload-socket\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:43:26.340599 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.340555    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d311d46a-a8a2-4f83-9559-39fd8630e3f3-credential-socket\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:43:26.340905 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.340618    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d311d46a-a8a2-4f83-9559-39fd8630e3f3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d311d46a-a8a2-4f83-9559-39fd8630e3f3" (UID: "d311d46a-a8a2-4f83-9559-39fd8630e3f3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:43:26.340905 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.340702    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d311d46a-a8a2-4f83-9559-39fd8630e3f3-istio-data" (OuterVolumeSpecName: "istio-data") pod "d311d46a-a8a2-4f83-9559-39fd8630e3f3" (UID: "d311d46a-a8a2-4f83-9559-39fd8630e3f3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:43:26.342942 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.342901    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d311d46a-a8a2-4f83-9559-39fd8630e3f3-kube-api-access-wvppj" (OuterVolumeSpecName: "kube-api-access-wvppj") pod "d311d46a-a8a2-4f83-9559-39fd8630e3f3" (UID: "d311d46a-a8a2-4f83-9559-39fd8630e3f3"). InnerVolumeSpecName "kube-api-access-wvppj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:43:26.342942 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.342914    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d311d46a-a8a2-4f83-9559-39fd8630e3f3-istio-token" (OuterVolumeSpecName: "istio-token") pod "d311d46a-a8a2-4f83-9559-39fd8630e3f3" (UID: "d311d46a-a8a2-4f83-9559-39fd8630e3f3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:43:26.343123 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.342979    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d311d46a-a8a2-4f83-9559-39fd8630e3f3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d311d46a-a8a2-4f83-9559-39fd8630e3f3" (UID: "d311d46a-a8a2-4f83-9559-39fd8630e3f3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:43:26.343372 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.343341    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d311d46a-a8a2-4f83-9559-39fd8630e3f3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d311d46a-a8a2-4f83-9559-39fd8630e3f3" (UID: "d311d46a-a8a2-4f83-9559-39fd8630e3f3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 20:43:26.344632 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.344605    2577 generic.go:356] "Generic (PLEG): container finished" podID="d311d46a-a8a2-4f83-9559-39fd8630e3f3" containerID="afd25d786d2c8f7ed36039050404f061f828bab2b638aaffa5c7ed789a5b3ca7" exitCode=0
Sep 15 20:43:26.344777 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.344654    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"
Sep 15 20:43:26.344777 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.344682    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk" event={"ID":"d311d46a-a8a2-4f83-9559-39fd8630e3f3","Type":"ContainerDied","Data":"afd25d786d2c8f7ed36039050404f061f828bab2b638aaffa5c7ed789a5b3ca7"}
Sep 15 20:43:26.344777 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.344764    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk" event={"ID":"d311d46a-a8a2-4f83-9559-39fd8630e3f3","Type":"ContainerDied","Data":"0a000a196bc66d6bcd5c69a6f151c0e0d758da1860be15515a5e8d596f3a9546"}
Sep 15 20:43:26.344953 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.344789    2577 scope.go:117] "RemoveContainer" containerID="afd25d786d2c8f7ed36039050404f061f828bab2b638aaffa5c7ed789a5b3ca7"
Sep 15 20:43:26.365801 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.365777    2577 scope.go:117] "RemoveContainer" containerID="bcc8b5607ad60d7265250701e59d15acb1ace8814b4e754dcf8eca193b6ee040"
Sep 15 20:43:26.375137 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.375118    2577 scope.go:117] "RemoveContainer" containerID="afd25d786d2c8f7ed36039050404f061f828bab2b638aaffa5c7ed789a5b3ca7"
Sep 15 20:43:26.375485 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:43:26.375456    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afd25d786d2c8f7ed36039050404f061f828bab2b638aaffa5c7ed789a5b3ca7\": container with ID starting with afd25d786d2c8f7ed36039050404f061f828bab2b638aaffa5c7ed789a5b3ca7 not found: ID does not exist" containerID="afd25d786d2c8f7ed36039050404f061f828bab2b638aaffa5c7ed789a5b3ca7"
Sep 15 20:43:26.375620 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.375494    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afd25d786d2c8f7ed36039050404f061f828bab2b638aaffa5c7ed789a5b3ca7"} err="failed to get container status \"afd25d786d2c8f7ed36039050404f061f828bab2b638aaffa5c7ed789a5b3ca7\": rpc error: code = NotFound desc = could not find container \"afd25d786d2c8f7ed36039050404f061f828bab2b638aaffa5c7ed789a5b3ca7\": container with ID starting with afd25d786d2c8f7ed36039050404f061f828bab2b638aaffa5c7ed789a5b3ca7 not found: ID does not exist"
Sep 15 20:43:26.375620 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.375524    2577 scope.go:117] "RemoveContainer" containerID="bcc8b5607ad60d7265250701e59d15acb1ace8814b4e754dcf8eca193b6ee040"
Sep 15 20:43:26.375848 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:43:26.375827    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcc8b5607ad60d7265250701e59d15acb1ace8814b4e754dcf8eca193b6ee040\": container with ID starting with bcc8b5607ad60d7265250701e59d15acb1ace8814b4e754dcf8eca193b6ee040 not found: ID does not exist" containerID="bcc8b5607ad60d7265250701e59d15acb1ace8814b4e754dcf8eca193b6ee040"
Sep 15 20:43:26.375917 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.375853    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcc8b5607ad60d7265250701e59d15acb1ace8814b4e754dcf8eca193b6ee040"} err="failed to get container status \"bcc8b5607ad60d7265250701e59d15acb1ace8814b4e754dcf8eca193b6ee040\": rpc error: code = NotFound desc = could not find container \"bcc8b5607ad60d7265250701e59d15acb1ace8814b4e754dcf8eca193b6ee040\": container with ID starting with bcc8b5607ad60d7265250701e59d15acb1ace8814b4e754dcf8eca193b6ee040 not found: ID does not exist"
Sep 15 20:43:26.376607 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.376582    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"]
Sep 15 20:43:26.379861 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.379840    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-92702a-openshift-default-5cd489db8b-dzcxk"]
Sep 15 20:43:26.441100 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.441062    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d311d46a-a8a2-4f83-9559-39fd8630e3f3-istio-token\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:43:26.441100 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.441096    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvppj\" (UniqueName: \"kubernetes.io/projected/d311d46a-a8a2-4f83-9559-39fd8630e3f3-kube-api-access-wvppj\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:43:26.441100 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.441107    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d311d46a-a8a2-4f83-9559-39fd8630e3f3-istiod-ca-cert\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:43:26.441351 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.441118    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d311d46a-a8a2-4f83-9559-39fd8630e3f3-istio-envoy\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:43:26.441351 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.441126    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d311d46a-a8a2-4f83-9559-39fd8630e3f3-istio-data\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:43:26.441351 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:26.441133    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d311d46a-a8a2-4f83-9559-39fd8630e3f3-istio-podinfo\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:43:28.061117 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:28.061084    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d311d46a-a8a2-4f83-9559-39fd8630e3f3" path="/var/lib/kubelet/pods/d311d46a-a8a2-4f83-9559-39fd8630e3f3/volumes"
Sep 15 20:43:31.862853 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:31.862811    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk"]
Sep 15 20:43:31.863307 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:31.863074    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk" podUID="730a8f9a-0c38-453f-8cd0-56321c4592e0" containerName="maas-api" containerID="cri-o://1f63f3dd69168d106f800b838ffe9e9c3ad26f75469b2c18ca2a0e2a3b545f5a" gracePeriod=30
Sep 15 20:43:32.123941 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:32.123874    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk"
Sep 15 20:43:32.195584 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:32.195545    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x5ggg\" (UniqueName: \"kubernetes.io/projected/730a8f9a-0c38-453f-8cd0-56321c4592e0-kube-api-access-x5ggg\") pod \"730a8f9a-0c38-453f-8cd0-56321c4592e0\" (UID: \"730a8f9a-0c38-453f-8cd0-56321c4592e0\") "
Sep 15 20:43:32.195794 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:32.195717    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/730a8f9a-0c38-453f-8cd0-56321c4592e0-openshift-service-ca\") pod \"730a8f9a-0c38-453f-8cd0-56321c4592e0\" (UID: \"730a8f9a-0c38-453f-8cd0-56321c4592e0\") "
Sep 15 20:43:32.195876 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:32.195793    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/730a8f9a-0c38-453f-8cd0-56321c4592e0-metrics-certs\") pod \"730a8f9a-0c38-453f-8cd0-56321c4592e0\" (UID: \"730a8f9a-0c38-453f-8cd0-56321c4592e0\") "
Sep 15 20:43:32.195876 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:32.195824    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/730a8f9a-0c38-453f-8cd0-56321c4592e0-maas-api-tls\") pod \"730a8f9a-0c38-453f-8cd0-56321c4592e0\" (UID: \"730a8f9a-0c38-453f-8cd0-56321c4592e0\") "
Sep 15 20:43:32.196133 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:32.196102    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/730a8f9a-0c38-453f-8cd0-56321c4592e0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "730a8f9a-0c38-453f-8cd0-56321c4592e0" (UID: "730a8f9a-0c38-453f-8cd0-56321c4592e0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:43:32.197970 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:32.197936    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/730a8f9a-0c38-453f-8cd0-56321c4592e0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "730a8f9a-0c38-453f-8cd0-56321c4592e0" (UID: "730a8f9a-0c38-453f-8cd0-56321c4592e0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:43:32.197970 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:32.197956    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/730a8f9a-0c38-453f-8cd0-56321c4592e0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "730a8f9a-0c38-453f-8cd0-56321c4592e0" (UID: "730a8f9a-0c38-453f-8cd0-56321c4592e0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:43:32.198132 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:32.197995    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/730a8f9a-0c38-453f-8cd0-56321c4592e0-kube-api-access-x5ggg" (OuterVolumeSpecName: "kube-api-access-x5ggg") pod "730a8f9a-0c38-453f-8cd0-56321c4592e0" (UID: "730a8f9a-0c38-453f-8cd0-56321c4592e0"). InnerVolumeSpecName "kube-api-access-x5ggg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:43:32.297539 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:32.297495    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/730a8f9a-0c38-453f-8cd0-56321c4592e0-openshift-service-ca\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:43:32.297539 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:32.297532    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/730a8f9a-0c38-453f-8cd0-56321c4592e0-metrics-certs\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:43:32.297539 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:32.297545    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/730a8f9a-0c38-453f-8cd0-56321c4592e0-maas-api-tls\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:43:32.297828 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:32.297556    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x5ggg\" (UniqueName: \"kubernetes.io/projected/730a8f9a-0c38-453f-8cd0-56321c4592e0-kube-api-access-x5ggg\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:43:32.387359 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:32.387260    2577 generic.go:356] "Generic (PLEG): container finished" podID="730a8f9a-0c38-453f-8cd0-56321c4592e0" containerID="1f63f3dd69168d106f800b838ffe9e9c3ad26f75469b2c18ca2a0e2a3b545f5a" exitCode=0
Sep 15 20:43:32.387359 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:32.387300    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk"
Sep 15 20:43:32.387593 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:32.387351    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk" event={"ID":"730a8f9a-0c38-453f-8cd0-56321c4592e0","Type":"ContainerDied","Data":"1f63f3dd69168d106f800b838ffe9e9c3ad26f75469b2c18ca2a0e2a3b545f5a"}
Sep 15 20:43:32.387593 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:32.387429    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk" event={"ID":"730a8f9a-0c38-453f-8cd0-56321c4592e0","Type":"ContainerDied","Data":"03a1205d83e03b61364999c2c9c0f1c820654dcd904b4046c6489d054c75685c"}
Sep 15 20:43:32.387593 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:32.387460    2577 scope.go:117] "RemoveContainer" containerID="1f63f3dd69168d106f800b838ffe9e9c3ad26f75469b2c18ca2a0e2a3b545f5a"
Sep 15 20:43:32.399458 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:32.399439    2577 scope.go:117] "RemoveContainer" containerID="1f63f3dd69168d106f800b838ffe9e9c3ad26f75469b2c18ca2a0e2a3b545f5a"
Sep 15 20:43:32.399788 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:43:32.399768    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f63f3dd69168d106f800b838ffe9e9c3ad26f75469b2c18ca2a0e2a3b545f5a\": container with ID starting with 1f63f3dd69168d106f800b838ffe9e9c3ad26f75469b2c18ca2a0e2a3b545f5a not found: ID does not exist" containerID="1f63f3dd69168d106f800b838ffe9e9c3ad26f75469b2c18ca2a0e2a3b545f5a"
Sep 15 20:43:32.399872 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:32.399799    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f63f3dd69168d106f800b838ffe9e9c3ad26f75469b2c18ca2a0e2a3b545f5a"} err="failed to get container status \"1f63f3dd69168d106f800b838ffe9e9c3ad26f75469b2c18ca2a0e2a3b545f5a\": rpc error: code = NotFound desc = could not find container \"1f63f3dd69168d106f800b838ffe9e9c3ad26f75469b2c18ca2a0e2a3b545f5a\": container with ID starting with 1f63f3dd69168d106f800b838ffe9e9c3ad26f75469b2c18ca2a0e2a3b545f5a not found: ID does not exist"
Sep 15 20:43:32.414569 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:32.414540    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk"]
Sep 15 20:43:32.418142 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:32.418117    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-e80b38-697dddf548-rmvlk"]
Sep 15 20:43:34.061159 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:34.061128    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="730a8f9a-0c38-453f-8cd0-56321c4592e0" path="/var/lib/kubelet/pods/730a8f9a-0c38-453f-8cd0-56321c4592e0/volumes"
Sep 15 20:43:38.863507 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:38.863470    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"]
Sep 15 20:43:38.863999 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:38.863761    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf" podUID="e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb" containerName="istio-proxy" containerID="cri-o://3d9b9dd0cbb281481cdec6de5289cf87ba55eab04b3eee93b935865a41a6df11" gracePeriod=30
Sep 15 20:43:41.585286 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:41.585239    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5b579c7cb4-gg2pv"]
Sep 15 20:43:41.585795 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:41.585510    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5b579c7cb4-gg2pv" podUID="3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774" containerName="manager" containerID="cri-o://53ced0205dd9e2362e41e709962695b259120bee5654617822c011ef43a7d905" gracePeriod=30
Sep 15 20:43:41.839228 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:41.839162    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b579c7cb4-gg2pv"
Sep 15 20:43:41.892251 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:41.892211    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774-webhook-cert\") pod \"3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774\" (UID: \"3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774\") "
Sep 15 20:43:41.892414 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:41.892357    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774-metrics-certs\") pod \"3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774\" (UID: \"3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774\") "
Sep 15 20:43:41.892464 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:41.892433    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2k2k\" (UniqueName: \"kubernetes.io/projected/3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774-kube-api-access-w2k2k\") pod \"3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774\" (UID: \"3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774\") "
Sep 15 20:43:41.894526 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:41.894491    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774" (UID: "3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:43:41.894665 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:41.894610    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774-kube-api-access-w2k2k" (OuterVolumeSpecName: "kube-api-access-w2k2k") pod "3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774" (UID: "3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774"). InnerVolumeSpecName "kube-api-access-w2k2k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:43:41.894920 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:41.894893    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774" (UID: "3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:43:41.993703 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:41.993667    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774-metrics-certs\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:43:41.993703 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:41.993699    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2k2k\" (UniqueName: \"kubernetes.io/projected/3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774-kube-api-access-w2k2k\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:43:41.993703 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:41.993709    2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774-webhook-cert\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:43:42.454373 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:42.454341    2577 generic.go:356] "Generic (PLEG): container finished" podID="3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774" containerID="53ced0205dd9e2362e41e709962695b259120bee5654617822c011ef43a7d905" exitCode=0
Sep 15 20:43:42.454595 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:42.454400    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b579c7cb4-gg2pv"
Sep 15 20:43:42.454595 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:42.454429    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b579c7cb4-gg2pv" event={"ID":"3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774","Type":"ContainerDied","Data":"53ced0205dd9e2362e41e709962695b259120bee5654617822c011ef43a7d905"}
Sep 15 20:43:42.454595 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:42.454524    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b579c7cb4-gg2pv" event={"ID":"3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774","Type":"ContainerDied","Data":"1835036539be31494f67e56316771dc9813468f9a4aa9b0ed1e3762f00e7d0c4"}
Sep 15 20:43:42.454595 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:42.454555    2577 scope.go:117] "RemoveContainer" containerID="53ced0205dd9e2362e41e709962695b259120bee5654617822c011ef43a7d905"
Sep 15 20:43:42.464487 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:42.464470    2577 scope.go:117] "RemoveContainer" containerID="53ced0205dd9e2362e41e709962695b259120bee5654617822c011ef43a7d905"
Sep 15 20:43:42.464784 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:43:42.464760    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53ced0205dd9e2362e41e709962695b259120bee5654617822c011ef43a7d905\": container with ID starting with 53ced0205dd9e2362e41e709962695b259120bee5654617822c011ef43a7d905 not found: ID does not exist" containerID="53ced0205dd9e2362e41e709962695b259120bee5654617822c011ef43a7d905"
Sep 15 20:43:42.464873 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:42.464790    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53ced0205dd9e2362e41e709962695b259120bee5654617822c011ef43a7d905"} err="failed to get container status \"53ced0205dd9e2362e41e709962695b259120bee5654617822c011ef43a7d905\": rpc error: code = NotFound desc = could not find container \"53ced0205dd9e2362e41e709962695b259120bee5654617822c011ef43a7d905\": container with ID starting with 53ced0205dd9e2362e41e709962695b259120bee5654617822c011ef43a7d905 not found: ID does not exist"
Sep 15 20:43:42.472651 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:42.472629    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5b579c7cb4-gg2pv"]
Sep 15 20:43:42.475452 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:42.475430    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5b579c7cb4-gg2pv"]
Sep 15 20:43:43.436504 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:43.436465    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 20:43:43.436904 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:43.436537    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf" podUID="e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 20:43:44.064513 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.064483    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774" path="/var/lib/kubelet/pods/3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774/volumes"
Sep 15 20:43:44.128125 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.128097    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:43:44.215291 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.215258    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ggbsm\" (UniqueName: \"kubernetes.io/projected/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-kube-api-access-ggbsm\") pod \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\" (UID: \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\") "
Sep 15 20:43:44.215487 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.215305    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-istio-envoy\") pod \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\" (UID: \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\") "
Sep 15 20:43:44.215487 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.215350    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-workload-socket\") pod \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\" (UID: \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\") "
Sep 15 20:43:44.215487 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.215388    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-workload-certs\") pod \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\" (UID: \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\") "
Sep 15 20:43:44.215487 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.215419    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-istiod-ca-cert\") pod \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\" (UID: \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\") "
Sep 15 20:43:44.215487 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.215466    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-istio-data\") pod \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\" (UID: \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\") "
Sep 15 20:43:44.215803 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.215494    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-credential-socket\") pod \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\" (UID: \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\") "
Sep 15 20:43:44.215803 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.215542    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-istio-token\") pod \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\" (UID: \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\") "
Sep 15 20:43:44.215803 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.215572    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-istio-podinfo\") pod \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\" (UID: \"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb\") "
Sep 15 20:43:44.215803 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.215746    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb" (UID: "e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:43:44.215803 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.215767    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb" (UID: "e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:43:44.216085 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.215881    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb" (UID: "e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:43:44.216085 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.215981    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-workload-socket\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:43:44.216085 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.215995    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb" (UID: "e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:43:44.216085 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.216000    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-workload-certs\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:43:44.216085 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.216034    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-istiod-ca-cert\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:43:44.216519 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.216490    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-istio-data" (OuterVolumeSpecName: "istio-data") pod "e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb" (UID: "e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:43:44.217806 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.217779    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-kube-api-access-ggbsm" (OuterVolumeSpecName: "kube-api-access-ggbsm") pod "e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb" (UID: "e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb"). InnerVolumeSpecName "kube-api-access-ggbsm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:43:44.217916 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.217861    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb" (UID: "e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:43:44.217993 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.217959    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb" (UID: "e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 20:43:44.218190 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.218167    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-istio-token" (OuterVolumeSpecName: "istio-token") pod "e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb" (UID: "e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:43:44.317053 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.317015    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ggbsm\" (UniqueName: \"kubernetes.io/projected/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-kube-api-access-ggbsm\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:43:44.317053 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.317048    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-istio-envoy\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:43:44.317053 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.317057    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-istio-data\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:43:44.317053 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.317065    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-credential-socket\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:43:44.317330 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.317073    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-istio-token\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:43:44.317330 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.317082    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb-istio-podinfo\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:43:44.471411 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.471319    2577 generic.go:356] "Generic (PLEG): container finished" podID="e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb" containerID="3d9b9dd0cbb281481cdec6de5289cf87ba55eab04b3eee93b935865a41a6df11" exitCode=0
Sep 15 20:43:44.471411 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.471368    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"
Sep 15 20:43:44.471943 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.471403    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf" event={"ID":"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb","Type":"ContainerDied","Data":"3d9b9dd0cbb281481cdec6de5289cf87ba55eab04b3eee93b935865a41a6df11"}
Sep 15 20:43:44.471943 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.471462    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf" event={"ID":"e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb","Type":"ContainerDied","Data":"2c517683f2209ff1d554e133a4ded480c6b1237d4bffad258e98b1525b42f1e6"}
Sep 15 20:43:44.471943 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.471488    2577 scope.go:117] "RemoveContainer" containerID="3d9b9dd0cbb281481cdec6de5289cf87ba55eab04b3eee93b935865a41a6df11"
Sep 15 20:43:44.482716 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.482691    2577 scope.go:117] "RemoveContainer" containerID="d6d50268b388edf5d86f5e12cdce13731a938248b16c80597195253d7e000d5a"
Sep 15 20:43:44.492550 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.492528    2577 scope.go:117] "RemoveContainer" containerID="3d9b9dd0cbb281481cdec6de5289cf87ba55eab04b3eee93b935865a41a6df11"
Sep 15 20:43:44.492906 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:43:44.492885    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d9b9dd0cbb281481cdec6de5289cf87ba55eab04b3eee93b935865a41a6df11\": container with ID starting with 3d9b9dd0cbb281481cdec6de5289cf87ba55eab04b3eee93b935865a41a6df11 not found: ID does not exist" containerID="3d9b9dd0cbb281481cdec6de5289cf87ba55eab04b3eee93b935865a41a6df11"
Sep 15 20:43:44.492995 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.492920    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d9b9dd0cbb281481cdec6de5289cf87ba55eab04b3eee93b935865a41a6df11"} err="failed to get container status \"3d9b9dd0cbb281481cdec6de5289cf87ba55eab04b3eee93b935865a41a6df11\": rpc error: code = NotFound desc = could not find container \"3d9b9dd0cbb281481cdec6de5289cf87ba55eab04b3eee93b935865a41a6df11\": container with ID starting with 3d9b9dd0cbb281481cdec6de5289cf87ba55eab04b3eee93b935865a41a6df11 not found: ID does not exist"
Sep 15 20:43:44.492995 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.492944    2577 scope.go:117] "RemoveContainer" containerID="d6d50268b388edf5d86f5e12cdce13731a938248b16c80597195253d7e000d5a"
Sep 15 20:43:44.493247 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:43:44.493229    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6d50268b388edf5d86f5e12cdce13731a938248b16c80597195253d7e000d5a\": container with ID starting with d6d50268b388edf5d86f5e12cdce13731a938248b16c80597195253d7e000d5a not found: ID does not exist" containerID="d6d50268b388edf5d86f5e12cdce13731a938248b16c80597195253d7e000d5a"
Sep 15 20:43:44.493306 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.493255    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6d50268b388edf5d86f5e12cdce13731a938248b16c80597195253d7e000d5a"} err="failed to get container status \"d6d50268b388edf5d86f5e12cdce13731a938248b16c80597195253d7e000d5a\": rpc error: code = NotFound desc = could not find container \"d6d50268b388edf5d86f5e12cdce13731a938248b16c80597195253d7e000d5a\": container with ID starting with d6d50268b388edf5d86f5e12cdce13731a938248b16c80597195253d7e000d5a not found: ID does not exist"
Sep 15 20:43:44.495767 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.495744    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"]
Sep 15 20:43:44.498258 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:44.498236    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-e80b38-openshift-default-7f95788494-dhflf"]
Sep 15 20:43:46.061249 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:43:46.061215    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb" path="/var/lib/kubelet/pods/e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb/volumes"
Sep 15 20:44:43.242420 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.242374    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5b579c7cb4-mp9vp"]
Sep 15 20:44:43.244744 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.242879    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="750e4921-2932-457a-af2b-2c90bfdf52da" containerName="maas-api"
Sep 15 20:44:43.244744 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.242893    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="750e4921-2932-457a-af2b-2c90bfdf52da" containerName="maas-api"
Sep 15 20:44:43.244744 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.242907    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d311d46a-a8a2-4f83-9559-39fd8630e3f3" containerName="istio-proxy"
Sep 15 20:44:43.244744 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.242913    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d311d46a-a8a2-4f83-9559-39fd8630e3f3" containerName="istio-proxy"
Sep 15 20:44:43.244744 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.242936    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e06c5ee-e2f6-4b98-a77b-d6c29352edab" containerName="maas-api"
Sep 15 20:44:43.244744 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.242943    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e06c5ee-e2f6-4b98-a77b-d6c29352edab" containerName="maas-api"
Sep 15 20:44:43.244744 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.242954    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774" containerName="manager"
Sep 15 20:44:43.244744 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.242961    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774" containerName="manager"
Sep 15 20:44:43.244744 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.242968    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdcedef1-0099-4a2d-99cc-b76ec9074ca1" containerName="maas-api"
Sep 15 20:44:43.244744 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.242973    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdcedef1-0099-4a2d-99cc-b76ec9074ca1" containerName="maas-api"
Sep 15 20:44:43.244744 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.242981    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb" containerName="istio-proxy"
Sep 15 20:44:43.244744 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.242986    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb" containerName="istio-proxy"
Sep 15 20:44:43.244744 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.242992    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="730a8f9a-0c38-453f-8cd0-56321c4592e0" containerName="maas-api"
Sep 15 20:44:43.244744 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.242997    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="730a8f9a-0c38-453f-8cd0-56321c4592e0" containerName="maas-api"
Sep 15 20:44:43.244744 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.243003    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb" containerName="istio-proxy"
Sep 15 20:44:43.244744 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.243012    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb" containerName="istio-proxy"
Sep 15 20:44:43.244744 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.243021    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9042937-f5db-4263-b2c8-9eebcf633f3a" containerName="maas-api"
Sep 15 20:44:43.244744 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.243026    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9042937-f5db-4263-b2c8-9eebcf633f3a" containerName="maas-api"
Sep 15 20:44:43.244744 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.243098    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f36b4cb-ff71-4d7d-a4e2-dd3b2d138774" containerName="manager"
Sep 15 20:44:43.244744 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.243107    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdcedef1-0099-4a2d-99cc-b76ec9074ca1" containerName="maas-api"
Sep 15 20:44:43.244744 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.243113    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="730a8f9a-0c38-453f-8cd0-56321c4592e0" containerName="maas-api"
Sep 15 20:44:43.244744 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.243119    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e06c5ee-e2f6-4b98-a77b-d6c29352edab" containerName="maas-api"
Sep 15 20:44:43.244744 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.243124    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9042937-f5db-4263-b2c8-9eebcf633f3a" containerName="maas-api"
Sep 15 20:44:43.244744 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.243130    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d311d46a-a8a2-4f83-9559-39fd8630e3f3" containerName="istio-proxy"
Sep 15 20:44:43.244744 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.243138    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb" containerName="istio-proxy"
Sep 15 20:44:43.244744 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.243145    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1c8d5ae-098d-47d7-9476-afc4bb8ec4cb" containerName="istio-proxy"
Sep 15 20:44:43.244744 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.243151    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="750e4921-2932-457a-af2b-2c90bfdf52da" containerName="maas-api"
Sep 15 20:44:43.245934 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.245918    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b579c7cb4-mp9vp"
Sep 15 20:44:43.248504 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.248480    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 15 20:44:43.249817 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.249795    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 15 20:44:43.249936 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.249802    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 15 20:44:43.250009 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.249980    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8wcdj\""
Sep 15 20:44:43.256620 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.256595    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b579c7cb4-mp9vp"]
Sep 15 20:44:43.294250 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.294208    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/52b5d095-986b-4aba-bc79-ee465bbedc36-webhook-cert\") pod \"maas-controller-5b579c7cb4-mp9vp\" (UID: \"52b5d095-986b-4aba-bc79-ee465bbedc36\") " pod="opendatahub/maas-controller-5b579c7cb4-mp9vp"
Sep 15 20:44:43.294444 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.294293    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlz7q\" (UniqueName: \"kubernetes.io/projected/52b5d095-986b-4aba-bc79-ee465bbedc36-kube-api-access-rlz7q\") pod \"maas-controller-5b579c7cb4-mp9vp\" (UID: \"52b5d095-986b-4aba-bc79-ee465bbedc36\") " pod="opendatahub/maas-controller-5b579c7cb4-mp9vp"
Sep 15 20:44:43.294444 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.294366    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52b5d095-986b-4aba-bc79-ee465bbedc36-metrics-certs\") pod \"maas-controller-5b579c7cb4-mp9vp\" (UID: \"52b5d095-986b-4aba-bc79-ee465bbedc36\") " pod="opendatahub/maas-controller-5b579c7cb4-mp9vp"
Sep 15 20:44:43.395211 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.395167    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlz7q\" (UniqueName: \"kubernetes.io/projected/52b5d095-986b-4aba-bc79-ee465bbedc36-kube-api-access-rlz7q\") pod \"maas-controller-5b579c7cb4-mp9vp\" (UID: \"52b5d095-986b-4aba-bc79-ee465bbedc36\") " pod="opendatahub/maas-controller-5b579c7cb4-mp9vp"
Sep 15 20:44:43.395400 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.395219    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52b5d095-986b-4aba-bc79-ee465bbedc36-metrics-certs\") pod \"maas-controller-5b579c7cb4-mp9vp\" (UID: \"52b5d095-986b-4aba-bc79-ee465bbedc36\") " pod="opendatahub/maas-controller-5b579c7cb4-mp9vp"
Sep 15 20:44:43.395400 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.395271    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/52b5d095-986b-4aba-bc79-ee465bbedc36-webhook-cert\") pod \"maas-controller-5b579c7cb4-mp9vp\" (UID: \"52b5d095-986b-4aba-bc79-ee465bbedc36\") " pod="opendatahub/maas-controller-5b579c7cb4-mp9vp"
Sep 15 20:44:43.398048 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.398023    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52b5d095-986b-4aba-bc79-ee465bbedc36-metrics-certs\") pod \"maas-controller-5b579c7cb4-mp9vp\" (UID: \"52b5d095-986b-4aba-bc79-ee465bbedc36\") " pod="opendatahub/maas-controller-5b579c7cb4-mp9vp"
Sep 15 20:44:43.398129 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.398023    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/52b5d095-986b-4aba-bc79-ee465bbedc36-webhook-cert\") pod \"maas-controller-5b579c7cb4-mp9vp\" (UID: \"52b5d095-986b-4aba-bc79-ee465bbedc36\") " pod="opendatahub/maas-controller-5b579c7cb4-mp9vp"
Sep 15 20:44:43.402777 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.402754    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlz7q\" (UniqueName: \"kubernetes.io/projected/52b5d095-986b-4aba-bc79-ee465bbedc36-kube-api-access-rlz7q\") pod \"maas-controller-5b579c7cb4-mp9vp\" (UID: \"52b5d095-986b-4aba-bc79-ee465bbedc36\") " pod="opendatahub/maas-controller-5b579c7cb4-mp9vp"
Sep 15 20:44:43.557661 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.557566    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b579c7cb4-mp9vp"
Sep 15 20:44:43.841795 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.836280    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b579c7cb4-mp9vp"]
Sep 15 20:44:43.842895 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:43.842866    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 20:44:44.842856 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:44.842815    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b579c7cb4-mp9vp" event={"ID":"52b5d095-986b-4aba-bc79-ee465bbedc36","Type":"ContainerStarted","Data":"4c80710c7638007dfde1774359244897dd9da192fda04d07a7447829f4cd3345"}
Sep 15 20:44:44.843227 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:44.842887    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5b579c7cb4-mp9vp"
Sep 15 20:44:44.843227 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:44.842906    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b579c7cb4-mp9vp" event={"ID":"52b5d095-986b-4aba-bc79-ee465bbedc36","Type":"ContainerStarted","Data":"39f996eefa0eef59154edd159d43420c154a4db912c464e0187da1d487ef9d2a"}
Sep 15 20:44:44.861226 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:44.861174    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5b579c7cb4-mp9vp" podStartSLOduration=1.083516423 podStartE2EDuration="1.861158415s" totalImagesPullingTime="777.641992ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:44:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:44:44.86029832 +0000 UTC m=+2391.251251317" watchObservedRunningTime="2026-09-15 20:44:44.861158415 +0000 UTC m=+2391.252111411"
Sep 15 20:44:54.361706 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:54.361678    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-r227b_ecab5774-79a4-42eb-9bb5-a1f09b18f3fb/console-operator/1.log"
Sep 15 20:44:54.373446 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:54.373424    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-r227b_ecab5774-79a4-42eb-9bb5-a1f09b18f3fb/console-operator/1.log"
Sep 15 20:44:55.850361 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:44:55.850327    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5b579c7cb4-mp9vp"
Sep 15 20:45:05.181787 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:05.181744    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5b579c7cb4-mp9vp"]
Sep 15 20:45:05.182277 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:05.182061    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5b579c7cb4-mp9vp" podUID="52b5d095-986b-4aba-bc79-ee465bbedc36" containerName="manager" containerID="cri-o://4c80710c7638007dfde1774359244897dd9da192fda04d07a7447829f4cd3345" gracePeriod=30
Sep 15 20:45:05.438819 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:05.438747    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b579c7cb4-mp9vp"
Sep 15 20:45:05.622396 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:05.622357    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rlz7q\" (UniqueName: \"kubernetes.io/projected/52b5d095-986b-4aba-bc79-ee465bbedc36-kube-api-access-rlz7q\") pod \"52b5d095-986b-4aba-bc79-ee465bbedc36\" (UID: \"52b5d095-986b-4aba-bc79-ee465bbedc36\") "
Sep 15 20:45:05.622568 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:05.622456    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52b5d095-986b-4aba-bc79-ee465bbedc36-metrics-certs\") pod \"52b5d095-986b-4aba-bc79-ee465bbedc36\" (UID: \"52b5d095-986b-4aba-bc79-ee465bbedc36\") "
Sep 15 20:45:05.622568 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:05.622526    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/52b5d095-986b-4aba-bc79-ee465bbedc36-webhook-cert\") pod \"52b5d095-986b-4aba-bc79-ee465bbedc36\" (UID: \"52b5d095-986b-4aba-bc79-ee465bbedc36\") "
Sep 15 20:45:05.624646 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:05.624611    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52b5d095-986b-4aba-bc79-ee465bbedc36-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "52b5d095-986b-4aba-bc79-ee465bbedc36" (UID: "52b5d095-986b-4aba-bc79-ee465bbedc36"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:45:05.624803 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:05.624661    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52b5d095-986b-4aba-bc79-ee465bbedc36-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "52b5d095-986b-4aba-bc79-ee465bbedc36" (UID: "52b5d095-986b-4aba-bc79-ee465bbedc36"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:45:05.624803 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:05.624692    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52b5d095-986b-4aba-bc79-ee465bbedc36-kube-api-access-rlz7q" (OuterVolumeSpecName: "kube-api-access-rlz7q") pod "52b5d095-986b-4aba-bc79-ee465bbedc36" (UID: "52b5d095-986b-4aba-bc79-ee465bbedc36"). InnerVolumeSpecName "kube-api-access-rlz7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:45:05.723635 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:05.723596    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/52b5d095-986b-4aba-bc79-ee465bbedc36-metrics-certs\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:45:05.723635 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:05.723629    2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/52b5d095-986b-4aba-bc79-ee465bbedc36-webhook-cert\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:45:05.723884 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:05.723643    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rlz7q\" (UniqueName: \"kubernetes.io/projected/52b5d095-986b-4aba-bc79-ee465bbedc36-kube-api-access-rlz7q\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:45:05.986583 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:05.986485    2577 generic.go:356] "Generic (PLEG): container finished" podID="52b5d095-986b-4aba-bc79-ee465bbedc36" containerID="4c80710c7638007dfde1774359244897dd9da192fda04d07a7447829f4cd3345" exitCode=0
Sep 15 20:45:05.986583 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:05.986536    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b579c7cb4-mp9vp"
Sep 15 20:45:05.986583 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:05.986572    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b579c7cb4-mp9vp" event={"ID":"52b5d095-986b-4aba-bc79-ee465bbedc36","Type":"ContainerDied","Data":"4c80710c7638007dfde1774359244897dd9da192fda04d07a7447829f4cd3345"}
Sep 15 20:45:05.986892 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:05.986632    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b579c7cb4-mp9vp" event={"ID":"52b5d095-986b-4aba-bc79-ee465bbedc36","Type":"ContainerDied","Data":"39f996eefa0eef59154edd159d43420c154a4db912c464e0187da1d487ef9d2a"}
Sep 15 20:45:05.986892 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:05.986658    2577 scope.go:117] "RemoveContainer" containerID="4c80710c7638007dfde1774359244897dd9da192fda04d07a7447829f4cd3345"
Sep 15 20:45:05.996857 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:05.996839    2577 scope.go:117] "RemoveContainer" containerID="4c80710c7638007dfde1774359244897dd9da192fda04d07a7447829f4cd3345"
Sep 15 20:45:05.997149 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:45:05.997129    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c80710c7638007dfde1774359244897dd9da192fda04d07a7447829f4cd3345\": container with ID starting with 4c80710c7638007dfde1774359244897dd9da192fda04d07a7447829f4cd3345 not found: ID does not exist" containerID="4c80710c7638007dfde1774359244897dd9da192fda04d07a7447829f4cd3345"
Sep 15 20:45:05.997210 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:05.997158    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c80710c7638007dfde1774359244897dd9da192fda04d07a7447829f4cd3345"} err="failed to get container status \"4c80710c7638007dfde1774359244897dd9da192fda04d07a7447829f4cd3345\": rpc error: code = NotFound desc = could not find container \"4c80710c7638007dfde1774359244897dd9da192fda04d07a7447829f4cd3345\": container with ID starting with 4c80710c7638007dfde1774359244897dd9da192fda04d07a7447829f4cd3345 not found: ID does not exist"
Sep 15 20:45:06.008624 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:06.008598    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5b579c7cb4-mp9vp"]
Sep 15 20:45:06.011930 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:06.011903    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5b579c7cb4-mp9vp"]
Sep 15 20:45:06.061405 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:06.061375    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52b5d095-986b-4aba-bc79-ee465bbedc36" path="/var/lib/kubelet/pods/52b5d095-986b-4aba-bc79-ee465bbedc36/volumes"
Sep 15 20:45:06.757684 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:06.757641    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5b579c7cb4-x99jn"]
Sep 15 20:45:06.758146 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:06.758131    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d311d46a-a8a2-4f83-9559-39fd8630e3f3" containerName="istio-proxy"
Sep 15 20:45:06.758195 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:06.758148    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d311d46a-a8a2-4f83-9559-39fd8630e3f3" containerName="istio-proxy"
Sep 15 20:45:06.758195 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:06.758165    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52b5d095-986b-4aba-bc79-ee465bbedc36" containerName="manager"
Sep 15 20:45:06.758195 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:06.758171    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="52b5d095-986b-4aba-bc79-ee465bbedc36" containerName="manager"
Sep 15 20:45:06.758298 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:06.758243    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="52b5d095-986b-4aba-bc79-ee465bbedc36" containerName="manager"
Sep 15 20:45:06.758298 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:06.758253    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d311d46a-a8a2-4f83-9559-39fd8630e3f3" containerName="istio-proxy"
Sep 15 20:45:06.762815 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:06.762798    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b579c7cb4-x99jn"
Sep 15 20:45:06.767326 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:06.767297    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8wcdj\""
Sep 15 20:45:06.767457 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:06.767332    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 15 20:45:06.767457 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:06.767395    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 15 20:45:06.767457 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:06.767397    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 15 20:45:06.772750 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:06.772706    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b579c7cb4-x99jn"]
Sep 15 20:45:06.936973 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:06.936938    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-549nn\" (UniqueName: \"kubernetes.io/projected/3d841d3a-d711-4ffd-a12b-71c614144ed7-kube-api-access-549nn\") pod \"maas-controller-5b579c7cb4-x99jn\" (UID: \"3d841d3a-d711-4ffd-a12b-71c614144ed7\") " pod="opendatahub/maas-controller-5b579c7cb4-x99jn"
Sep 15 20:45:06.937160 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:06.936986    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d841d3a-d711-4ffd-a12b-71c614144ed7-metrics-certs\") pod \"maas-controller-5b579c7cb4-x99jn\" (UID: \"3d841d3a-d711-4ffd-a12b-71c614144ed7\") " pod="opendatahub/maas-controller-5b579c7cb4-x99jn"
Sep 15 20:45:06.937160 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:06.937012    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3d841d3a-d711-4ffd-a12b-71c614144ed7-webhook-cert\") pod \"maas-controller-5b579c7cb4-x99jn\" (UID: \"3d841d3a-d711-4ffd-a12b-71c614144ed7\") " pod="opendatahub/maas-controller-5b579c7cb4-x99jn"
Sep 15 20:45:07.038625 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:07.038531    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-549nn\" (UniqueName: \"kubernetes.io/projected/3d841d3a-d711-4ffd-a12b-71c614144ed7-kube-api-access-549nn\") pod \"maas-controller-5b579c7cb4-x99jn\" (UID: \"3d841d3a-d711-4ffd-a12b-71c614144ed7\") " pod="opendatahub/maas-controller-5b579c7cb4-x99jn"
Sep 15 20:45:07.038625 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:07.038581    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d841d3a-d711-4ffd-a12b-71c614144ed7-metrics-certs\") pod \"maas-controller-5b579c7cb4-x99jn\" (UID: \"3d841d3a-d711-4ffd-a12b-71c614144ed7\") " pod="opendatahub/maas-controller-5b579c7cb4-x99jn"
Sep 15 20:45:07.038845 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:07.038628    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3d841d3a-d711-4ffd-a12b-71c614144ed7-webhook-cert\") pod \"maas-controller-5b579c7cb4-x99jn\" (UID: \"3d841d3a-d711-4ffd-a12b-71c614144ed7\") " pod="opendatahub/maas-controller-5b579c7cb4-x99jn"
Sep 15 20:45:07.041270 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:07.041243    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d841d3a-d711-4ffd-a12b-71c614144ed7-metrics-certs\") pod \"maas-controller-5b579c7cb4-x99jn\" (UID: \"3d841d3a-d711-4ffd-a12b-71c614144ed7\") " pod="opendatahub/maas-controller-5b579c7cb4-x99jn"
Sep 15 20:45:07.041270 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:07.041266    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3d841d3a-d711-4ffd-a12b-71c614144ed7-webhook-cert\") pod \"maas-controller-5b579c7cb4-x99jn\" (UID: \"3d841d3a-d711-4ffd-a12b-71c614144ed7\") " pod="opendatahub/maas-controller-5b579c7cb4-x99jn"
Sep 15 20:45:07.046493 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:07.046468    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-549nn\" (UniqueName: \"kubernetes.io/projected/3d841d3a-d711-4ffd-a12b-71c614144ed7-kube-api-access-549nn\") pod \"maas-controller-5b579c7cb4-x99jn\" (UID: \"3d841d3a-d711-4ffd-a12b-71c614144ed7\") " pod="opendatahub/maas-controller-5b579c7cb4-x99jn"
Sep 15 20:45:07.075266 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:07.075234    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b579c7cb4-x99jn"
Sep 15 20:45:07.348175 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:07.348144    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b579c7cb4-x99jn"]
Sep 15 20:45:07.348899 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:45:07.348870    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d841d3a_d711_4ffd_a12b_71c614144ed7.slice/crio-960a61cf68733150315033a20cf4fc32776768965565aaede17fd43fc4301f1c WatchSource:0}: Error finding container 960a61cf68733150315033a20cf4fc32776768965565aaede17fd43fc4301f1c: Status 404 returned error can't find the container with id 960a61cf68733150315033a20cf4fc32776768965565aaede17fd43fc4301f1c
Sep 15 20:45:08.002875 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:08.002838    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b579c7cb4-x99jn" event={"ID":"3d841d3a-d711-4ffd-a12b-71c614144ed7","Type":"ContainerStarted","Data":"1ae95e49e367fc04dbb08825df9157729baea162fc46006d11f466a006bb1854"}
Sep 15 20:45:08.003247 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:08.002898    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b579c7cb4-x99jn" event={"ID":"3d841d3a-d711-4ffd-a12b-71c614144ed7","Type":"ContainerStarted","Data":"960a61cf68733150315033a20cf4fc32776768965565aaede17fd43fc4301f1c"}
Sep 15 20:45:08.003247 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:08.003067    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5b579c7cb4-x99jn"
Sep 15 20:45:08.020666 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:08.020591    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5b579c7cb4-x99jn" podStartSLOduration=1.595786981 podStartE2EDuration="2.020574555s" totalImagesPullingTime="424.787574ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:45:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:45:08.019844938 +0000 UTC m=+2414.410797930" watchObservedRunningTime="2026-09-15 20:45:08.020574555 +0000 UTC m=+2414.411527549"
Sep 15 20:45:19.011455 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:45:19.011374    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5b579c7cb4-x99jn"
Sep 15 20:46:17.673512 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:46:17.673470    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qrq84"]
Sep 15 20:46:17.674083 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:46:17.673684    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qrq84" podUID="c3c6f523-42a6-471f-ab50-f0fd4b821101" containerName="manager" containerID="cri-o://f002a1493ea8b0c470d1341d6248f314642a40b40768c1eab56c8c048acf05ea" gracePeriod=10
Sep 15 20:46:17.940641 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:46:17.940570    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qrq84"
Sep 15 20:46:18.025212 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:46:18.025173    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m6dlg\" (UniqueName: \"kubernetes.io/projected/c3c6f523-42a6-471f-ab50-f0fd4b821101-kube-api-access-m6dlg\") pod \"c3c6f523-42a6-471f-ab50-f0fd4b821101\" (UID: \"c3c6f523-42a6-471f-ab50-f0fd4b821101\") "
Sep 15 20:46:18.025364 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:46:18.025322    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c3c6f523-42a6-471f-ab50-f0fd4b821101-extensions-socket-volume\") pod \"c3c6f523-42a6-471f-ab50-f0fd4b821101\" (UID: \"c3c6f523-42a6-471f-ab50-f0fd4b821101\") "
Sep 15 20:46:18.025703 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:46:18.025677    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3c6f523-42a6-471f-ab50-f0fd4b821101-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c3c6f523-42a6-471f-ab50-f0fd4b821101" (UID: "c3c6f523-42a6-471f-ab50-f0fd4b821101"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:46:18.027459 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:46:18.027433    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3c6f523-42a6-471f-ab50-f0fd4b821101-kube-api-access-m6dlg" (OuterVolumeSpecName: "kube-api-access-m6dlg") pod "c3c6f523-42a6-471f-ab50-f0fd4b821101" (UID: "c3c6f523-42a6-471f-ab50-f0fd4b821101"). InnerVolumeSpecName "kube-api-access-m6dlg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:46:18.126972 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:46:18.126923    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c3c6f523-42a6-471f-ab50-f0fd4b821101-extensions-socket-volume\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:46:18.126972 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:46:18.126965    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6dlg\" (UniqueName: \"kubernetes.io/projected/c3c6f523-42a6-471f-ab50-f0fd4b821101-kube-api-access-m6dlg\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:46:18.441135 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:46:18.441051    2577 generic.go:356] "Generic (PLEG): container finished" podID="c3c6f523-42a6-471f-ab50-f0fd4b821101" containerID="f002a1493ea8b0c470d1341d6248f314642a40b40768c1eab56c8c048acf05ea" exitCode=0
Sep 15 20:46:18.441590 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:46:18.441544    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qrq84" event={"ID":"c3c6f523-42a6-471f-ab50-f0fd4b821101","Type":"ContainerDied","Data":"f002a1493ea8b0c470d1341d6248f314642a40b40768c1eab56c8c048acf05ea"}
Sep 15 20:46:18.441866 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:46:18.441789    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qrq84" event={"ID":"c3c6f523-42a6-471f-ab50-f0fd4b821101","Type":"ContainerDied","Data":"fb31f1d9e16b11021715c20f3de69cbeb8f263327c5541a34bd829d4afd60d35"}
Sep 15 20:46:18.441866 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:46:18.441830    2577 scope.go:117] "RemoveContainer" containerID="f002a1493ea8b0c470d1341d6248f314642a40b40768c1eab56c8c048acf05ea"
Sep 15 20:46:18.442239 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:46:18.442221    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qrq84"
Sep 15 20:46:18.453640 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:46:18.453618    2577 scope.go:117] "RemoveContainer" containerID="20e757d2a4f7eabdac679565ba6a7a92ce457e8d86d132851ccaba02d3703ec5"
Sep 15 20:46:18.463195 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:46:18.463097    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qrq84"]
Sep 15 20:46:18.463195 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:46:18.463181    2577 scope.go:117] "RemoveContainer" containerID="f002a1493ea8b0c470d1341d6248f314642a40b40768c1eab56c8c048acf05ea"
Sep 15 20:46:18.463515 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:46:18.463492    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f002a1493ea8b0c470d1341d6248f314642a40b40768c1eab56c8c048acf05ea\": container with ID starting with f002a1493ea8b0c470d1341d6248f314642a40b40768c1eab56c8c048acf05ea not found: ID does not exist" containerID="f002a1493ea8b0c470d1341d6248f314642a40b40768c1eab56c8c048acf05ea"
Sep 15 20:46:18.463588 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:46:18.463522    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f002a1493ea8b0c470d1341d6248f314642a40b40768c1eab56c8c048acf05ea"} err="failed to get container status \"f002a1493ea8b0c470d1341d6248f314642a40b40768c1eab56c8c048acf05ea\": rpc error: code = NotFound desc = could not find container \"f002a1493ea8b0c470d1341d6248f314642a40b40768c1eab56c8c048acf05ea\": container with ID starting with f002a1493ea8b0c470d1341d6248f314642a40b40768c1eab56c8c048acf05ea not found: ID does not exist"
Sep 15 20:46:18.463588 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:46:18.463541    2577 scope.go:117] "RemoveContainer" containerID="20e757d2a4f7eabdac679565ba6a7a92ce457e8d86d132851ccaba02d3703ec5"
Sep 15 20:46:18.463831 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:46:18.463813    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20e757d2a4f7eabdac679565ba6a7a92ce457e8d86d132851ccaba02d3703ec5\": container with ID starting with 20e757d2a4f7eabdac679565ba6a7a92ce457e8d86d132851ccaba02d3703ec5 not found: ID does not exist" containerID="20e757d2a4f7eabdac679565ba6a7a92ce457e8d86d132851ccaba02d3703ec5"
Sep 15 20:46:18.463881 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:46:18.463835    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20e757d2a4f7eabdac679565ba6a7a92ce457e8d86d132851ccaba02d3703ec5"} err="failed to get container status \"20e757d2a4f7eabdac679565ba6a7a92ce457e8d86d132851ccaba02d3703ec5\": rpc error: code = NotFound desc = could not find container \"20e757d2a4f7eabdac679565ba6a7a92ce457e8d86d132851ccaba02d3703ec5\": container with ID starting with 20e757d2a4f7eabdac679565ba6a7a92ce457e8d86d132851ccaba02d3703ec5 not found: ID does not exist"
Sep 15 20:46:18.467566 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:46:18.467531    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-qrq84"]
Sep 15 20:46:20.060572 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:46:20.060535    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3c6f523-42a6-471f-ab50-f0fd4b821101" path="/var/lib/kubelet/pods/c3c6f523-42a6-471f-ab50-f0fd4b821101/volumes"
Sep 15 20:47:18.399987 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:47:18.399945    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-jk4zv"]
Sep 15 20:47:18.400578 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:47:18.400556    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3c6f523-42a6-471f-ab50-f0fd4b821101" containerName="manager"
Sep 15 20:47:18.400669 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:47:18.400580    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3c6f523-42a6-471f-ab50-f0fd4b821101" containerName="manager"
Sep 15 20:47:18.400669 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:47:18.400594    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3c6f523-42a6-471f-ab50-f0fd4b821101" containerName="manager"
Sep 15 20:47:18.400669 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:47:18.400602    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3c6f523-42a6-471f-ab50-f0fd4b821101" containerName="manager"
Sep 15 20:47:18.400865 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:47:18.400742    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3c6f523-42a6-471f-ab50-f0fd4b821101" containerName="manager"
Sep 15 20:47:18.400865 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:47:18.400757    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3c6f523-42a6-471f-ab50-f0fd4b821101" containerName="manager"
Sep 15 20:47:18.404000 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:47:18.403978    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-jk4zv"
Sep 15 20:47:18.406960 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:47:18.406939    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-dbq9q\""
Sep 15 20:47:18.416809 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:47:18.416778    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-jk4zv"]
Sep 15 20:47:18.529440 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:47:18.529395    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/142a617d-92bf-4b8c-95e0-00dfa6e4d894-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-jk4zv\" (UID: \"142a617d-92bf-4b8c-95e0-00dfa6e4d894\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-jk4zv"
Sep 15 20:47:18.529632 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:47:18.529535    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78mwj\" (UniqueName: \"kubernetes.io/projected/142a617d-92bf-4b8c-95e0-00dfa6e4d894-kube-api-access-78mwj\") pod \"kuadrant-operator-controller-manager-5769f99d9f-jk4zv\" (UID: \"142a617d-92bf-4b8c-95e0-00dfa6e4d894\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-jk4zv"
Sep 15 20:47:18.630799 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:47:18.630754    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78mwj\" (UniqueName: \"kubernetes.io/projected/142a617d-92bf-4b8c-95e0-00dfa6e4d894-kube-api-access-78mwj\") pod \"kuadrant-operator-controller-manager-5769f99d9f-jk4zv\" (UID: \"142a617d-92bf-4b8c-95e0-00dfa6e4d894\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-jk4zv"
Sep 15 20:47:18.630995 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:47:18.630856    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/142a617d-92bf-4b8c-95e0-00dfa6e4d894-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-jk4zv\" (UID: \"142a617d-92bf-4b8c-95e0-00dfa6e4d894\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-jk4zv"
Sep 15 20:47:18.631278 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:47:18.631253    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/142a617d-92bf-4b8c-95e0-00dfa6e4d894-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-jk4zv\" (UID: \"142a617d-92bf-4b8c-95e0-00dfa6e4d894\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-jk4zv"
Sep 15 20:47:18.639112 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:47:18.639080    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78mwj\" (UniqueName: \"kubernetes.io/projected/142a617d-92bf-4b8c-95e0-00dfa6e4d894-kube-api-access-78mwj\") pod \"kuadrant-operator-controller-manager-5769f99d9f-jk4zv\" (UID: \"142a617d-92bf-4b8c-95e0-00dfa6e4d894\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-jk4zv"
Sep 15 20:47:18.715130 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:47:18.715098    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-jk4zv"
Sep 15 20:47:18.998881 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:47:18.998854    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-jk4zv"]
Sep 15 20:47:19.000429 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:47:19.000403    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod142a617d_92bf_4b8c_95e0_00dfa6e4d894.slice/crio-186a1df4906dc3b8c025123b8fb571a8792f13ce5239c0d172d81dcb804d1642 WatchSource:0}: Error finding container 186a1df4906dc3b8c025123b8fb571a8792f13ce5239c0d172d81dcb804d1642: Status 404 returned error can't find the container with id 186a1df4906dc3b8c025123b8fb571a8792f13ce5239c0d172d81dcb804d1642
Sep 15 20:47:19.830640 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:47:19.830602    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-jk4zv" event={"ID":"142a617d-92bf-4b8c-95e0-00dfa6e4d894","Type":"ContainerStarted","Data":"2a391c02883d2832e39df4ca014d3c1789399e61fc6d66c09e7a67c2c9556925"}
Sep 15 20:47:19.831041 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:47:19.830664    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-jk4zv" event={"ID":"142a617d-92bf-4b8c-95e0-00dfa6e4d894","Type":"ContainerStarted","Data":"186a1df4906dc3b8c025123b8fb571a8792f13ce5239c0d172d81dcb804d1642"}
Sep 15 20:47:19.831041 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:47:19.830689    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-jk4zv"
Sep 15 20:47:19.854599 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:47:19.854547    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-jk4zv" podStartSLOduration=1.8545287689999999 podStartE2EDuration="1.854528769s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:47:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:47:19.852704037 +0000 UTC m=+2546.243657033" watchObservedRunningTime="2026-09-15 20:47:19.854528769 +0000 UTC m=+2546.245481766"
Sep 15 20:47:30.839200 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:47:30.839164    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-jk4zv"
Sep 15 20:49:16.953211 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:16.953171    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6b4c4886c6-xwl8p"]
Sep 15 20:49:16.958031 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:16.958008    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6b4c4886c6-xwl8p"
Sep 15 20:49:16.968714 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:16.968594    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b4c4886c6-xwl8p"]
Sep 15 20:49:17.071817 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:17.071779    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/081034c0-395f-47f6-b655-01f39f08504e-metrics-certs\") pod \"maas-api-6b4c4886c6-xwl8p\" (UID: \"081034c0-395f-47f6-b655-01f39f08504e\") " pod="odh-ai-gateway-infra/maas-api-6b4c4886c6-xwl8p"
Sep 15 20:49:17.072009 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:17.071859    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/081034c0-395f-47f6-b655-01f39f08504e-openshift-service-ca\") pod \"maas-api-6b4c4886c6-xwl8p\" (UID: \"081034c0-395f-47f6-b655-01f39f08504e\") " pod="odh-ai-gateway-infra/maas-api-6b4c4886c6-xwl8p"
Sep 15 20:49:17.072009 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:17.071892    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/081034c0-395f-47f6-b655-01f39f08504e-maas-api-tls\") pod \"maas-api-6b4c4886c6-xwl8p\" (UID: \"081034c0-395f-47f6-b655-01f39f08504e\") " pod="odh-ai-gateway-infra/maas-api-6b4c4886c6-xwl8p"
Sep 15 20:49:17.072009 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:17.071913    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwtxx\" (UniqueName: \"kubernetes.io/projected/081034c0-395f-47f6-b655-01f39f08504e-kube-api-access-dwtxx\") pod \"maas-api-6b4c4886c6-xwl8p\" (UID: \"081034c0-395f-47f6-b655-01f39f08504e\") " pod="odh-ai-gateway-infra/maas-api-6b4c4886c6-xwl8p"
Sep 15 20:49:17.173095 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:17.173050    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/081034c0-395f-47f6-b655-01f39f08504e-metrics-certs\") pod \"maas-api-6b4c4886c6-xwl8p\" (UID: \"081034c0-395f-47f6-b655-01f39f08504e\") " pod="odh-ai-gateway-infra/maas-api-6b4c4886c6-xwl8p"
Sep 15 20:49:17.173298 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:17.173177    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/081034c0-395f-47f6-b655-01f39f08504e-openshift-service-ca\") pod \"maas-api-6b4c4886c6-xwl8p\" (UID: \"081034c0-395f-47f6-b655-01f39f08504e\") " pod="odh-ai-gateway-infra/maas-api-6b4c4886c6-xwl8p"
Sep 15 20:49:17.173298 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:17.173238    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/081034c0-395f-47f6-b655-01f39f08504e-maas-api-tls\") pod \"maas-api-6b4c4886c6-xwl8p\" (UID: \"081034c0-395f-47f6-b655-01f39f08504e\") " pod="odh-ai-gateway-infra/maas-api-6b4c4886c6-xwl8p"
Sep 15 20:49:17.173298 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:17.173270    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwtxx\" (UniqueName: \"kubernetes.io/projected/081034c0-395f-47f6-b655-01f39f08504e-kube-api-access-dwtxx\") pod \"maas-api-6b4c4886c6-xwl8p\" (UID: \"081034c0-395f-47f6-b655-01f39f08504e\") " pod="odh-ai-gateway-infra/maas-api-6b4c4886c6-xwl8p"
Sep 15 20:49:17.174209 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:17.174179    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/081034c0-395f-47f6-b655-01f39f08504e-openshift-service-ca\") pod \"maas-api-6b4c4886c6-xwl8p\" (UID: \"081034c0-395f-47f6-b655-01f39f08504e\") " pod="odh-ai-gateway-infra/maas-api-6b4c4886c6-xwl8p"
Sep 15 20:49:17.175953 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:17.175924    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/081034c0-395f-47f6-b655-01f39f08504e-maas-api-tls\") pod \"maas-api-6b4c4886c6-xwl8p\" (UID: \"081034c0-395f-47f6-b655-01f39f08504e\") " pod="odh-ai-gateway-infra/maas-api-6b4c4886c6-xwl8p"
Sep 15 20:49:17.176194 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:17.176175    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/081034c0-395f-47f6-b655-01f39f08504e-metrics-certs\") pod \"maas-api-6b4c4886c6-xwl8p\" (UID: \"081034c0-395f-47f6-b655-01f39f08504e\") " pod="odh-ai-gateway-infra/maas-api-6b4c4886c6-xwl8p"
Sep 15 20:49:17.181166 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:17.181134    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwtxx\" (UniqueName: \"kubernetes.io/projected/081034c0-395f-47f6-b655-01f39f08504e-kube-api-access-dwtxx\") pod \"maas-api-6b4c4886c6-xwl8p\" (UID: \"081034c0-395f-47f6-b655-01f39f08504e\") " pod="odh-ai-gateway-infra/maas-api-6b4c4886c6-xwl8p"
Sep 15 20:49:17.271967 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:17.271926    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6b4c4886c6-xwl8p"
Sep 15 20:49:17.550135 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:17.550108    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b4c4886c6-xwl8p"]
Sep 15 20:49:17.552163 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:49:17.552136    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod081034c0_395f_47f6_b655_01f39f08504e.slice/crio-31c464a570fb17e9a82847c1c204b3e48f839b62506600f726c4c62a3c672f56 WatchSource:0}: Error finding container 31c464a570fb17e9a82847c1c204b3e48f839b62506600f726c4c62a3c672f56: Status 404 returned error can't find the container with id 31c464a570fb17e9a82847c1c204b3e48f839b62506600f726c4c62a3c672f56
Sep 15 20:49:18.554935 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:18.554892    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6b4c4886c6-xwl8p" event={"ID":"081034c0-395f-47f6-b655-01f39f08504e","Type":"ContainerStarted","Data":"3584f6c78d36924d189c6ba0a995de73c8a649705d6d66fe3244aa3c09f2d15c"}
Sep 15 20:49:18.555327 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:18.554956    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6b4c4886c6-xwl8p" event={"ID":"081034c0-395f-47f6-b655-01f39f08504e","Type":"ContainerStarted","Data":"31c464a570fb17e9a82847c1c204b3e48f839b62506600f726c4c62a3c672f56"}
Sep 15 20:49:18.555327 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:18.554986    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6b4c4886c6-xwl8p"
Sep 15 20:49:18.573118 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:18.573064    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6b4c4886c6-xwl8p" podStartSLOduration=2.12923216 podStartE2EDuration="2.573046282s" totalImagesPullingTime="443.814122ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:49:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:49:18.570522519 +0000 UTC m=+2664.961475516" watchObservedRunningTime="2026-09-15 20:49:18.573046282 +0000 UTC m=+2664.963999279"
Sep 15 20:49:24.566400 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:24.566366    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6b4c4886c6-xwl8p"
Sep 15 20:49:34.240174 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:34.240135    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7d5d7c6468-hhdcz"]
Sep 15 20:49:34.240711 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:34.240364    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-7d5d7c6468-hhdcz" podUID="be7ccdf9-7d15-4f0a-9c68-32a90c5031f1" containerName="maas-api" containerID="cri-o://b5e07feefe1a585a3f490ac73fe77aab35d28beac880c4501969318d08fa24eb" gracePeriod=30
Sep 15 20:49:35.493624 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:35.493596    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7d5d7c6468-hhdcz"
Sep 15 20:49:35.534224 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:35.534182    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be7ccdf9-7d15-4f0a-9c68-32a90c5031f1-maas-api-tls\") pod \"be7ccdf9-7d15-4f0a-9c68-32a90c5031f1\" (UID: \"be7ccdf9-7d15-4f0a-9c68-32a90c5031f1\") "
Sep 15 20:49:35.534417 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:35.534304    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be7ccdf9-7d15-4f0a-9c68-32a90c5031f1-openshift-service-ca\") pod \"be7ccdf9-7d15-4f0a-9c68-32a90c5031f1\" (UID: \"be7ccdf9-7d15-4f0a-9c68-32a90c5031f1\") "
Sep 15 20:49:35.534417 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:35.534345    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be7ccdf9-7d15-4f0a-9c68-32a90c5031f1-metrics-certs\") pod \"be7ccdf9-7d15-4f0a-9c68-32a90c5031f1\" (UID: \"be7ccdf9-7d15-4f0a-9c68-32a90c5031f1\") "
Sep 15 20:49:35.534555 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:35.534415    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftqc2\" (UniqueName: \"kubernetes.io/projected/be7ccdf9-7d15-4f0a-9c68-32a90c5031f1-kube-api-access-ftqc2\") pod \"be7ccdf9-7d15-4f0a-9c68-32a90c5031f1\" (UID: \"be7ccdf9-7d15-4f0a-9c68-32a90c5031f1\") "
Sep 15 20:49:35.534764 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:35.534716    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be7ccdf9-7d15-4f0a-9c68-32a90c5031f1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "be7ccdf9-7d15-4f0a-9c68-32a90c5031f1" (UID: "be7ccdf9-7d15-4f0a-9c68-32a90c5031f1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:49:35.536428 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:35.536402    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be7ccdf9-7d15-4f0a-9c68-32a90c5031f1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "be7ccdf9-7d15-4f0a-9c68-32a90c5031f1" (UID: "be7ccdf9-7d15-4f0a-9c68-32a90c5031f1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:49:35.536611 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:35.536589    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be7ccdf9-7d15-4f0a-9c68-32a90c5031f1-kube-api-access-ftqc2" (OuterVolumeSpecName: "kube-api-access-ftqc2") pod "be7ccdf9-7d15-4f0a-9c68-32a90c5031f1" (UID: "be7ccdf9-7d15-4f0a-9c68-32a90c5031f1"). InnerVolumeSpecName "kube-api-access-ftqc2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:49:35.536663 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:35.536592    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be7ccdf9-7d15-4f0a-9c68-32a90c5031f1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "be7ccdf9-7d15-4f0a-9c68-32a90c5031f1" (UID: "be7ccdf9-7d15-4f0a-9c68-32a90c5031f1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:49:35.635438 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:35.635343    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be7ccdf9-7d15-4f0a-9c68-32a90c5031f1-maas-api-tls\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:49:35.635438 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:35.635382    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be7ccdf9-7d15-4f0a-9c68-32a90c5031f1-openshift-service-ca\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:49:35.635438 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:35.635392    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be7ccdf9-7d15-4f0a-9c68-32a90c5031f1-metrics-certs\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:49:35.635438 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:35.635401    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftqc2\" (UniqueName: \"kubernetes.io/projected/be7ccdf9-7d15-4f0a-9c68-32a90c5031f1-kube-api-access-ftqc2\") on node \"ip-10-0-140-72.ec2.internal\" DevicePath \"\""
Sep 15 20:49:35.661979 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:35.661947    2577 generic.go:356] "Generic (PLEG): container finished" podID="be7ccdf9-7d15-4f0a-9c68-32a90c5031f1" containerID="b5e07feefe1a585a3f490ac73fe77aab35d28beac880c4501969318d08fa24eb" exitCode=0
Sep 15 20:49:35.662169 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:35.661998    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7d5d7c6468-hhdcz"
Sep 15 20:49:35.662169 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:35.662031    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7d5d7c6468-hhdcz" event={"ID":"be7ccdf9-7d15-4f0a-9c68-32a90c5031f1","Type":"ContainerDied","Data":"b5e07feefe1a585a3f490ac73fe77aab35d28beac880c4501969318d08fa24eb"}
Sep 15 20:49:35.662169 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:35.662086    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7d5d7c6468-hhdcz" event={"ID":"be7ccdf9-7d15-4f0a-9c68-32a90c5031f1","Type":"ContainerDied","Data":"60ed349f8a4e1404b6ce70aab5fb145c96136a2a3b522be0c25baec35062c17e"}
Sep 15 20:49:35.662169 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:35.662108    2577 scope.go:117] "RemoveContainer" containerID="b5e07feefe1a585a3f490ac73fe77aab35d28beac880c4501969318d08fa24eb"
Sep 15 20:49:35.673473 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:35.673455    2577 scope.go:117] "RemoveContainer" containerID="b5e07feefe1a585a3f490ac73fe77aab35d28beac880c4501969318d08fa24eb"
Sep 15 20:49:35.673829 ip-10-0-140-72 kubenswrapper[2577]: E0915 20:49:35.673805    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5e07feefe1a585a3f490ac73fe77aab35d28beac880c4501969318d08fa24eb\": container with ID starting with b5e07feefe1a585a3f490ac73fe77aab35d28beac880c4501969318d08fa24eb not found: ID does not exist" containerID="b5e07feefe1a585a3f490ac73fe77aab35d28beac880c4501969318d08fa24eb"
Sep 15 20:49:35.673889 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:35.673842    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5e07feefe1a585a3f490ac73fe77aab35d28beac880c4501969318d08fa24eb"} err="failed to get container status \"b5e07feefe1a585a3f490ac73fe77aab35d28beac880c4501969318d08fa24eb\": rpc error: code = NotFound desc = could not find container \"b5e07feefe1a585a3f490ac73fe77aab35d28beac880c4501969318d08fa24eb\": container with ID starting with b5e07feefe1a585a3f490ac73fe77aab35d28beac880c4501969318d08fa24eb not found: ID does not exist"
Sep 15 20:49:35.685890 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:35.685859    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7d5d7c6468-hhdcz"]
Sep 15 20:49:35.688610 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:35.688588    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-7d5d7c6468-hhdcz"]
Sep 15 20:49:36.061404 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:36.061370    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be7ccdf9-7d15-4f0a-9c68-32a90c5031f1" path="/var/lib/kubelet/pods/be7ccdf9-7d15-4f0a-9c68-32a90c5031f1/volumes"
Sep 15 20:49:51.915320 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:51.915240    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-bmnwk_8b2eb956-9488-4880-a693-464ef27c2e72/manager/0.log"
Sep 15 20:49:52.030011 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:52.029982    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-8gvgr_c53c751d-5dda-4a25-9112-8c3d9f89c665/manager/0.log"
Sep 15 20:49:52.144747 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:52.144705    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5b579c7cb4-x99jn_3d841d3a-d711-4ffd-a12b-71c614144ed7/manager/0.log"
Sep 15 20:49:52.247229 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:52.247202    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-p4gsl_f7d0a0cc-441e-4de1-93f7-99bf335416d2/server/0.log"
Sep 15 20:49:52.362264 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:52.362229    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-b4sg4_8e8dc63c-00bd-4917-8463-1795690501b1/manager/0.log"
Sep 15 20:49:52.618604 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:52.618518    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-75fcf566-jwslv_2db0feff-46a2-4a97-aac2-ccb349c31ea0/copy-manifests/0.log"
Sep 15 20:49:52.624834 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:52.624806    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-75fcf566-jwslv_2db0feff-46a2-4a97-aac2-ccb349c31ea0/manager/0.log"
Sep 15 20:49:53.169679 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:53.169645    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6b4c4886c6-xwl8p_081034c0-395f-47f6-b655-01f39f08504e/maas-api/0.log"
Sep 15 20:49:53.487319 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:53.487288    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-6w4pt_b60a69e7-cf8d-4848-8815-72eb0f3f89ae/postgres/0.log"
Sep 15 20:49:54.372509 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:54.372473    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-d67zs_f49caa1a-07d8-4a70-8260-bb16b54a7b03/manager/0.log"
Sep 15 20:49:54.404177 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:54.404140    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-r227b_ecab5774-79a4-42eb-9bb5-a1f09b18f3fb/console-operator/1.log"
Sep 15 20:49:54.424051 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:54.424023    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-r227b_ecab5774-79a4-42eb-9bb5-a1f09b18f3fb/console-operator/1.log"
Sep 15 20:49:54.482247 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:54.482216    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-vrwfm_947328b0-ab34-4b57-b9ef-275efbc95ed4/manager/0.log"
Sep 15 20:49:54.589593 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:54.589568    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-6nkd2_4f6ccaaa-6260-439a-8d7d-36dc24fe7ecc/kuadrant-console-plugin/0.log"
Sep 15 20:49:54.715246 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:54.715213    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-jk4zv_142a617d-92bf-4b8c-95e0-00dfa6e4d894/manager/0.log"
Sep 15 20:49:55.295851 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:55.295819    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799csl74f_6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf/istio-proxy/0.log"
Sep 15 20:49:55.638818 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:55.638714    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-565c49d7c-smwm7_d92f8a14-740a-4a09-87e9-3e39da9d2788/kube-auth-proxy/0.log"
Sep 15 20:49:55.989211 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:55.989177    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-69vj4_8b20137b-9766-427f-b140-5c75d6f77f23/payload-processing/0.log"
Sep 15 20:49:56.541022 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:49:56.540989    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-45wkt_2ea3f838-a397-4de2-b56a-200b482a72c3/main/0.log"
Sep 15 20:50:02.818681 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:02.818639    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8bqd6/must-gather-xgr8f"]
Sep 15 20:50:02.819402 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:02.819383    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be7ccdf9-7d15-4f0a-9c68-32a90c5031f1" containerName="maas-api"
Sep 15 20:50:02.819452 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:02.819409    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="be7ccdf9-7d15-4f0a-9c68-32a90c5031f1" containerName="maas-api"
Sep 15 20:50:02.819522 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:02.819512    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="be7ccdf9-7d15-4f0a-9c68-32a90c5031f1" containerName="maas-api"
Sep 15 20:50:02.822989 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:02.822968    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8bqd6/must-gather-xgr8f"
Sep 15 20:50:02.825770 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:02.825714    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8bqd6\"/\"openshift-service-ca.crt\""
Sep 15 20:50:02.826932 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:02.826893    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8bqd6\"/\"default-dockercfg-fgwsj\""
Sep 15 20:50:02.827272 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:02.827253    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8bqd6\"/\"kube-root-ca.crt\""
Sep 15 20:50:02.832004 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:02.831575    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8bqd6/must-gather-xgr8f"]
Sep 15 20:50:02.999282 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:02.999223    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd5zq\" (UniqueName: \"kubernetes.io/projected/f6498b04-0e45-4451-ae25-4562c9644a3f-kube-api-access-dd5zq\") pod \"must-gather-xgr8f\" (UID: \"f6498b04-0e45-4451-ae25-4562c9644a3f\") " pod="openshift-must-gather-8bqd6/must-gather-xgr8f"
Sep 15 20:50:02.999282 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:02.999280    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f6498b04-0e45-4451-ae25-4562c9644a3f-must-gather-output\") pod \"must-gather-xgr8f\" (UID: \"f6498b04-0e45-4451-ae25-4562c9644a3f\") " pod="openshift-must-gather-8bqd6/must-gather-xgr8f"
Sep 15 20:50:03.100155 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:03.100058    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd5zq\" (UniqueName: \"kubernetes.io/projected/f6498b04-0e45-4451-ae25-4562c9644a3f-kube-api-access-dd5zq\") pod \"must-gather-xgr8f\" (UID: \"f6498b04-0e45-4451-ae25-4562c9644a3f\") " pod="openshift-must-gather-8bqd6/must-gather-xgr8f"
Sep 15 20:50:03.100155 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:03.100100    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f6498b04-0e45-4451-ae25-4562c9644a3f-must-gather-output\") pod \"must-gather-xgr8f\" (UID: \"f6498b04-0e45-4451-ae25-4562c9644a3f\") " pod="openshift-must-gather-8bqd6/must-gather-xgr8f"
Sep 15 20:50:03.100421 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:03.100404    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f6498b04-0e45-4451-ae25-4562c9644a3f-must-gather-output\") pod \"must-gather-xgr8f\" (UID: \"f6498b04-0e45-4451-ae25-4562c9644a3f\") " pod="openshift-must-gather-8bqd6/must-gather-xgr8f"
Sep 15 20:50:03.107803 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:03.107778    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd5zq\" (UniqueName: \"kubernetes.io/projected/f6498b04-0e45-4451-ae25-4562c9644a3f-kube-api-access-dd5zq\") pod \"must-gather-xgr8f\" (UID: \"f6498b04-0e45-4451-ae25-4562c9644a3f\") " pod="openshift-must-gather-8bqd6/must-gather-xgr8f"
Sep 15 20:50:03.134867 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:03.134832    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8bqd6/must-gather-xgr8f"
Sep 15 20:50:03.411942 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:03.411916    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8bqd6/must-gather-xgr8f"]
Sep 15 20:50:03.414235 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:50:03.414204    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6498b04_0e45_4451_ae25_4562c9644a3f.slice/crio-13753246dbe94a7b91c722517e6ece81c0cbb4ca8d12521121e3ca124749340e WatchSource:0}: Error finding container 13753246dbe94a7b91c722517e6ece81c0cbb4ca8d12521121e3ca124749340e: Status 404 returned error can't find the container with id 13753246dbe94a7b91c722517e6ece81c0cbb4ca8d12521121e3ca124749340e
Sep 15 20:50:03.416202 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:03.416185    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 20:50:03.840103 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:03.840064    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8bqd6/must-gather-xgr8f" event={"ID":"f6498b04-0e45-4451-ae25-4562c9644a3f","Type":"ContainerStarted","Data":"13753246dbe94a7b91c722517e6ece81c0cbb4ca8d12521121e3ca124749340e"}
Sep 15 20:50:04.852044 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:04.851958    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8bqd6/must-gather-xgr8f" event={"ID":"f6498b04-0e45-4451-ae25-4562c9644a3f","Type":"ContainerStarted","Data":"de007902c4437d113d222f7af656603d85fa412e537cff29d2896da92566411f"}
Sep 15 20:50:04.852044 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:04.852013    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8bqd6/must-gather-xgr8f" event={"ID":"f6498b04-0e45-4451-ae25-4562c9644a3f","Type":"ContainerStarted","Data":"6e5ee63c00daae9ae60681827b85226bbbb67aaa4925f0945639e54fc53fb37e"}
Sep 15 20:50:04.869323 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:04.869255    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8bqd6/must-gather-xgr8f" podStartSLOduration=2.199090468 podStartE2EDuration="2.869238282s" totalImagesPullingTime="670.147814ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:50:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:50:04.867437682 +0000 UTC m=+2711.258390677" watchObservedRunningTime="2026-09-15 20:50:04.869238282 +0000 UTC m=+2711.260191279"
Sep 15 20:50:05.670808 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:05.670770    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9dwjh_7e124866-fa5b-4ecb-bca9-075fa8bdb5a5/global-pull-secret-syncer/0.log"
Sep 15 20:50:05.821746 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:05.821689    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qvw2x_1547d350-30fe-4c0b-9448-1cbd9c4ae5b9/konnectivity-agent/0.log"
Sep 15 20:50:05.887504 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:05.887470    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-72.ec2.internal_6967c7c3d2c13b8fd404f2bb5001259f/haproxy/0.log"
Sep 15 20:50:10.333643 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:10.333606    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-d67zs_f49caa1a-07d8-4a70-8260-bb16b54a7b03/manager/0.log"
Sep 15 20:50:10.382638 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:10.382582    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-vrwfm_947328b0-ab34-4b57-b9ef-275efbc95ed4/manager/0.log"
Sep 15 20:50:10.406275 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:10.406216    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-6nkd2_4f6ccaaa-6260-439a-8d7d-36dc24fe7ecc/kuadrant-console-plugin/0.log"
Sep 15 20:50:10.448118 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:10.448079    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-jk4zv_142a617d-92bf-4b8c-95e0-00dfa6e4d894/manager/0.log"
Sep 15 20:50:12.104343 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:12.104301    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_93569234-ca8e-4e34-acc8-1d413f92b96e/alertmanager/0.log"
Sep 15 20:50:12.128269 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:12.128237    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_93569234-ca8e-4e34-acc8-1d413f92b96e/config-reloader/0.log"
Sep 15 20:50:12.153454 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:12.153417    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_93569234-ca8e-4e34-acc8-1d413f92b96e/kube-rbac-proxy-web/0.log"
Sep 15 20:50:12.179714 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:12.179678    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_93569234-ca8e-4e34-acc8-1d413f92b96e/kube-rbac-proxy/0.log"
Sep 15 20:50:12.223778 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:12.223748    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_93569234-ca8e-4e34-acc8-1d413f92b96e/kube-rbac-proxy-metric/0.log"
Sep 15 20:50:12.278160 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:12.278121    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_93569234-ca8e-4e34-acc8-1d413f92b96e/prom-label-proxy/0.log"
Sep 15 20:50:12.309443 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:12.309412    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_93569234-ca8e-4e34-acc8-1d413f92b96e/init-config-reloader/0.log"
Sep 15 20:50:12.444307 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:12.444183    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-599ffb4745-n48qx_fb2a17c4-0ab8-47ec-ad13-694ca3e2c537/metrics-server/0.log"
Sep 15 20:50:12.501715 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:12.501686    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cnlkd_605e5608-f132-43d8-a691-c56f2f342c9c/node-exporter/0.log"
Sep 15 20:50:12.520440 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:12.520409    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cnlkd_605e5608-f132-43d8-a691-c56f2f342c9c/kube-rbac-proxy/0.log"
Sep 15 20:50:12.540716 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:12.540689    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cnlkd_605e5608-f132-43d8-a691-c56f2f342c9c/init-textfile/0.log"
Sep 15 20:50:13.111808 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:13.111773    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7686f988d4-slqm5_ec8c4c90-e728-48af-9caf-0e11b18e85db/thanos-query/0.log"
Sep 15 20:50:13.137919 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:13.137893    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7686f988d4-slqm5_ec8c4c90-e728-48af-9caf-0e11b18e85db/kube-rbac-proxy-web/0.log"
Sep 15 20:50:13.166608 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:13.165716    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7686f988d4-slqm5_ec8c4c90-e728-48af-9caf-0e11b18e85db/kube-rbac-proxy/0.log"
Sep 15 20:50:13.206307 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:13.206268    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7686f988d4-slqm5_ec8c4c90-e728-48af-9caf-0e11b18e85db/prom-label-proxy/0.log"
Sep 15 20:50:13.242045 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:13.242009    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7686f988d4-slqm5_ec8c4c90-e728-48af-9caf-0e11b18e85db/kube-rbac-proxy-rules/0.log"
Sep 15 20:50:13.266415 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:13.266376    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7686f988d4-slqm5_ec8c4c90-e728-48af-9caf-0e11b18e85db/kube-rbac-proxy-metrics/0.log"
Sep 15 20:50:15.505530 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:15.505487    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-r227b_ecab5774-79a4-42eb-9bb5-a1f09b18f3fb/console-operator/1.log"
Sep 15 20:50:15.512386 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:15.512358    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-r227b_ecab5774-79a4-42eb-9bb5-a1f09b18f3fb/console-operator/2.log"
Sep 15 20:50:16.052160 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:16.052092    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7f89cb66dd-rhb7p_4193569c-6ce6-4671-952b-e1601ee2f70b/console/0.log"
Sep 15 20:50:16.216879 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:16.216834    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8bqd6/perf-node-gather-daemonset-p78z2"]
Sep 15 20:50:16.227394 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:16.227355    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-p78z2"
Sep 15 20:50:16.236161 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:16.232930    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8bqd6/perf-node-gather-daemonset-p78z2"]
Sep 15 20:50:16.361321 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:16.361235    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/59e75a3d-40a4-4e75-a6a0-a5ed5a3ce115-sys\") pod \"perf-node-gather-daemonset-p78z2\" (UID: \"59e75a3d-40a4-4e75-a6a0-a5ed5a3ce115\") " pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-p78z2"
Sep 15 20:50:16.361321 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:16.361303    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/59e75a3d-40a4-4e75-a6a0-a5ed5a3ce115-lib-modules\") pod \"perf-node-gather-daemonset-p78z2\" (UID: \"59e75a3d-40a4-4e75-a6a0-a5ed5a3ce115\") " pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-p78z2"
Sep 15 20:50:16.361563 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:16.361410    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbwpc\" (UniqueName: \"kubernetes.io/projected/59e75a3d-40a4-4e75-a6a0-a5ed5a3ce115-kube-api-access-vbwpc\") pod \"perf-node-gather-daemonset-p78z2\" (UID: \"59e75a3d-40a4-4e75-a6a0-a5ed5a3ce115\") " pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-p78z2"
Sep 15 20:50:16.361563 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:16.361459    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/59e75a3d-40a4-4e75-a6a0-a5ed5a3ce115-proc\") pod \"perf-node-gather-daemonset-p78z2\" (UID: \"59e75a3d-40a4-4e75-a6a0-a5ed5a3ce115\") " pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-p78z2"
Sep 15 20:50:16.361563 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:16.361483    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/59e75a3d-40a4-4e75-a6a0-a5ed5a3ce115-podres\") pod \"perf-node-gather-daemonset-p78z2\" (UID: \"59e75a3d-40a4-4e75-a6a0-a5ed5a3ce115\") " pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-p78z2"
Sep 15 20:50:16.462549 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:16.462506    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/59e75a3d-40a4-4e75-a6a0-a5ed5a3ce115-sys\") pod \"perf-node-gather-daemonset-p78z2\" (UID: \"59e75a3d-40a4-4e75-a6a0-a5ed5a3ce115\") " pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-p78z2"
Sep 15 20:50:16.463004 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:16.462982    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/59e75a3d-40a4-4e75-a6a0-a5ed5a3ce115-lib-modules\") pod \"perf-node-gather-daemonset-p78z2\" (UID: \"59e75a3d-40a4-4e75-a6a0-a5ed5a3ce115\") " pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-p78z2"
Sep 15 20:50:16.463261 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:16.463236    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/59e75a3d-40a4-4e75-a6a0-a5ed5a3ce115-lib-modules\") pod \"perf-node-gather-daemonset-p78z2\" (UID: \"59e75a3d-40a4-4e75-a6a0-a5ed5a3ce115\") " pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-p78z2"
Sep 15 20:50:16.463261 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:16.462874    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/59e75a3d-40a4-4e75-a6a0-a5ed5a3ce115-sys\") pod \"perf-node-gather-daemonset-p78z2\" (UID: \"59e75a3d-40a4-4e75-a6a0-a5ed5a3ce115\") " pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-p78z2"
Sep 15 20:50:16.463444 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:16.463313    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbwpc\" (UniqueName: \"kubernetes.io/projected/59e75a3d-40a4-4e75-a6a0-a5ed5a3ce115-kube-api-access-vbwpc\") pod \"perf-node-gather-daemonset-p78z2\" (UID: \"59e75a3d-40a4-4e75-a6a0-a5ed5a3ce115\") " pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-p78z2"
Sep 15 20:50:16.463444 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:16.463348    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/59e75a3d-40a4-4e75-a6a0-a5ed5a3ce115-proc\") pod \"perf-node-gather-daemonset-p78z2\" (UID: \"59e75a3d-40a4-4e75-a6a0-a5ed5a3ce115\") " pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-p78z2"
Sep 15 20:50:16.463444 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:16.463363    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/59e75a3d-40a4-4e75-a6a0-a5ed5a3ce115-podres\") pod \"perf-node-gather-daemonset-p78z2\" (UID: \"59e75a3d-40a4-4e75-a6a0-a5ed5a3ce115\") " pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-p78z2"
Sep 15 20:50:16.463619 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:16.463505    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/59e75a3d-40a4-4e75-a6a0-a5ed5a3ce115-podres\") pod \"perf-node-gather-daemonset-p78z2\" (UID: \"59e75a3d-40a4-4e75-a6a0-a5ed5a3ce115\") " pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-p78z2"
Sep 15 20:50:16.463619 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:16.463511    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/59e75a3d-40a4-4e75-a6a0-a5ed5a3ce115-proc\") pod \"perf-node-gather-daemonset-p78z2\" (UID: \"59e75a3d-40a4-4e75-a6a0-a5ed5a3ce115\") " pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-p78z2"
Sep 15 20:50:16.472237 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:16.472198    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbwpc\" (UniqueName: \"kubernetes.io/projected/59e75a3d-40a4-4e75-a6a0-a5ed5a3ce115-kube-api-access-vbwpc\") pod \"perf-node-gather-daemonset-p78z2\" (UID: \"59e75a3d-40a4-4e75-a6a0-a5ed5a3ce115\") " pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-p78z2"
Sep 15 20:50:16.543343 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:16.543303    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-p78z2"
Sep 15 20:50:16.831004 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:16.830942    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-zjtlc_920cddf0-219e-4b86-84cd-b4111619a92b/volume-data-source-validator/0.log"
Sep 15 20:50:16.854312 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:16.854087    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8bqd6/perf-node-gather-daemonset-p78z2"]
Sep 15 20:50:16.861422 ip-10-0-140-72 kubenswrapper[2577]: W0915 20:50:16.861375    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod59e75a3d_40a4_4e75_a6a0_a5ed5a3ce115.slice/crio-95b30663edaa90572c9bfa8f249188a4004dcbf44f4feea2a1354831335bc72a WatchSource:0}: Error finding container 95b30663edaa90572c9bfa8f249188a4004dcbf44f4feea2a1354831335bc72a: Status 404 returned error can't find the container with id 95b30663edaa90572c9bfa8f249188a4004dcbf44f4feea2a1354831335bc72a
Sep 15 20:50:16.972535 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:16.972490    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-p78z2" event={"ID":"59e75a3d-40a4-4e75-a6a0-a5ed5a3ce115","Type":"ContainerStarted","Data":"95b30663edaa90572c9bfa8f249188a4004dcbf44f4feea2a1354831335bc72a"}
Sep 15 20:50:17.983526 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:17.983485    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-p78z2" event={"ID":"59e75a3d-40a4-4e75-a6a0-a5ed5a3ce115","Type":"ContainerStarted","Data":"2606c870dabc4219756941885d5f92413749f8149a1861fe05cb3d5e72f26fd3"}
Sep 15 20:50:17.984030 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:17.983702    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-p78z2"
Sep 15 20:50:18.002564 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:18.002506    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-p78z2" podStartSLOduration=2.002489457 podStartE2EDuration="2.002489457s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:50:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:50:17.998561725 +0000 UTC m=+2724.389514735" watchObservedRunningTime="2026-09-15 20:50:18.002489457 +0000 UTC m=+2724.393442479"
Sep 15 20:50:18.035460 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:18.035426    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-h67tx_5360f750-45a7-42d6-8b65-be039f6ee235/dns/0.log"
Sep 15 20:50:18.056153 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:18.056123    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-h67tx_5360f750-45a7-42d6-8b65-be039f6ee235/kube-rbac-proxy/0.log"
Sep 15 20:50:18.168420 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:18.168392    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-z5wjn_b6f901c3-0f40-46dc-8406-eaee847b1c10/dns-node-resolver/0.log"
Sep 15 20:50:18.790257 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:18.790223    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6d4544bc7f-64cxr_5d9260ed-5dcc-488e-9d13-a49675580c16/registry/0.log"
Sep 15 20:50:18.852869 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:18.852835    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vm6lc_81e9d392-e3c5-48f6-b98a-4bcaddda3336/node-ca/0.log"
Sep 15 20:50:19.787749 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:19.787689    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799csl74f_6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf/istio-proxy/0.log"
Sep 15 20:50:20.296150 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:20.296122    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-565c49d7c-smwm7_d92f8a14-740a-4a09-87e9-3e39da9d2788/kube-auth-proxy/0.log"
Sep 15 20:50:20.395291 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:20.395252    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-69vj4_8b20137b-9766-427f-b140-5c75d6f77f23/payload-processing/0.log"
Sep 15 20:50:21.370335 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:21.370302    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5l278_64743302-a5a8-4b6f-b7c7-d574edd19211/serve-healthcheck-canary/0.log"
Sep 15 20:50:21.942523 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:21.942495    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qflpq_866ab681-b97a-4958-a622-85f76910920a/kube-rbac-proxy/0.log"
Sep 15 20:50:21.962381 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:21.962348    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qflpq_866ab681-b97a-4958-a622-85f76910920a/exporter/0.log"
Sep 15 20:50:21.984743 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:21.984699    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qflpq_866ab681-b97a-4958-a622-85f76910920a/extractor/0.log"
Sep 15 20:50:24.001870 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:24.001842    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8bqd6/perf-node-gather-daemonset-p78z2"
Sep 15 20:50:24.109374 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:24.109311    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-bmnwk_8b2eb956-9488-4880-a693-464ef27c2e72/manager/0.log"
Sep 15 20:50:24.457865 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:24.457818    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-8gvgr_c53c751d-5dda-4a25-9112-8c3d9f89c665/manager/0.log"
Sep 15 20:50:24.512945 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:24.512647    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5b579c7cb4-x99jn_3d841d3a-d711-4ffd-a12b-71c614144ed7/manager/0.log"
Sep 15 20:50:24.530192 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:24.530167    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-p4gsl_f7d0a0cc-441e-4de1-93f7-99bf335416d2/server/0.log"
Sep 15 20:50:24.641208 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:24.641169    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-b4sg4_8e8dc63c-00bd-4917-8463-1795690501b1/manager/0.log"
Sep 15 20:50:24.765927 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:24.765844    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-75fcf566-jwslv_2db0feff-46a2-4a97-aac2-ccb349c31ea0/manager/0.log"
Sep 15 20:50:24.787820 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:24.787784    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-75fcf566-jwslv_2db0feff-46a2-4a97-aac2-ccb349c31ea0/copy-manifests/0.log"
Sep 15 20:50:26.352423 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:26.352385    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-bt4f9_dfba85cc-ef2b-4d44-b886-8821a367c70e/openshift-lws-operator/0.log"
Sep 15 20:50:27.343261 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:27.343220    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799csl74f_6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf/istio-proxy/0.log"
Sep 15 20:50:27.823944 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:27.823915    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-565c49d7c-smwm7_d92f8a14-740a-4a09-87e9-3e39da9d2788/kube-auth-proxy/0.log"
Sep 15 20:50:27.889394 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:27.889352    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-69vj4_8b20137b-9766-427f-b140-5c75d6f77f23/payload-processing/0.log"
Sep 15 20:50:29.447841 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:29.447808    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799csl74f_6e72cd0b-7ee7-418d-96fd-fb2c31b99ecf/istio-proxy/0.log"
Sep 15 20:50:29.950831 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:29.950802    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-565c49d7c-smwm7_d92f8a14-740a-4a09-87e9-3e39da9d2788/kube-auth-proxy/0.log"
Sep 15 20:50:30.036094 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:30.036065    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-69vj4_8b20137b-9766-427f-b140-5c75d6f77f23/payload-processing/0.log"
Sep 15 20:50:31.565238 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:31.565206    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-56d9f_c2db4be2-3083-4ba0-aecc-e3ce368114ce/migrator/0.log"
Sep 15 20:50:31.584648 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:31.584608    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-56d9f_c2db4be2-3083-4ba0-aecc-e3ce368114ce/graceful-termination/0.log"
Sep 15 20:50:31.946761 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:31.946651    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-xsqd9_c567a4c3-d608-46e7-b93a-88ad3486e709/kube-storage-version-migrator-operator/1.log"
Sep 15 20:50:31.948204 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:31.948165    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-xsqd9_c567a4c3-d608-46e7-b93a-88ad3486e709/kube-storage-version-migrator-operator/0.log"
Sep 15 20:50:33.162395 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:33.162366    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kjp4l_fd4ce615-2127-43bd-b843-f1b5a4bfcdc7/kube-multus-additional-cni-plugins/0.log"
Sep 15 20:50:33.181802 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:33.181774    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kjp4l_fd4ce615-2127-43bd-b843-f1b5a4bfcdc7/egress-router-binary-copy/0.log"
Sep 15 20:50:33.206071 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:33.206040    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kjp4l_fd4ce615-2127-43bd-b843-f1b5a4bfcdc7/cni-plugins/0.log"
Sep 15 20:50:33.225293 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:33.225261    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kjp4l_fd4ce615-2127-43bd-b843-f1b5a4bfcdc7/bond-cni-plugin/0.log"
Sep 15 20:50:33.249644 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:33.249584    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kjp4l_fd4ce615-2127-43bd-b843-f1b5a4bfcdc7/routeoverride-cni/0.log"
Sep 15 20:50:33.269185 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:33.269152    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kjp4l_fd4ce615-2127-43bd-b843-f1b5a4bfcdc7/whereabouts-cni-bincopy/0.log"
Sep 15 20:50:33.288335 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:33.288307    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kjp4l_fd4ce615-2127-43bd-b843-f1b5a4bfcdc7/whereabouts-cni/0.log"
Sep 15 20:50:33.547650 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:33.547617    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-brdnq_a8a416c3-51a5-4b55-9c73-9aaca4f93242/kube-multus/0.log"
Sep 15 20:50:33.607705 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:33.607671    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-2v4rt_5482d254-b248-4eda-a7d8-37bcb3289d69/network-metrics-daemon/0.log"
Sep 15 20:50:33.627596 ip-10-0-140-72 kubenswrapper[2577]: I0915 20:50:33.627562    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-2v4rt_5482d254-b248-4eda-a7d8-37bcb3289d69/kube-rbac-proxy/0.log"