Sep 10 19:13:36.175308 ip-10-0-142-20 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 10 19:13:36.175321 ip-10-0-142-20 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 10 19:13:36.175330 ip-10-0-142-20 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 10 19:13:36.175535 ip-10-0-142-20 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 10 19:13:46.424374 ip-10-0-142-20 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 10 19:13:46.424392 ip-10-0-142-20 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot d30d717a96d74228b3f7d69c1592c620 --
Sep 10 19:15:44.679132 ip-10-0-142-20 systemd[1]: Starting Kubernetes Kubelet...
Sep 10 19:15:45.009017 ip-10-0-142-20 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 19:15:45.009017 ip-10-0-142-20 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 10 19:15:45.009017 ip-10-0-142-20 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 19:15:45.009017 ip-10-0-142-20 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 10 19:15:45.009017 ip-10-0-142-20 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 19:15:45.009826 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.009723    2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 10 19:15:45.014284 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014257    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 19:15:45.014284 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014278    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 19:15:45.014284 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014282    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 19:15:45.014284 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014285    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 19:15:45.014284 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014288    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 19:15:45.014284 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014292    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 19:15:45.014284 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014295    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 19:15:45.014531 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014297    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 19:15:45.014531 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014300    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 19:15:45.014531 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014303    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 19:15:45.014531 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014305    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 19:15:45.014531 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014309    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 19:15:45.014531 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014311    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 19:15:45.014531 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014314    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 19:15:45.014531 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014316    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 19:15:45.014531 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014319    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 19:15:45.014531 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014322    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 19:15:45.014531 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014324    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 19:15:45.014531 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014327    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 19:15:45.014531 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014329    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 19:15:45.014531 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014332    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 19:15:45.014531 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014334    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 19:15:45.014531 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014337    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 19:15:45.014531 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014339    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 19:15:45.014531 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014342    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 19:15:45.014531 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014344    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 19:15:45.014531 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014347    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 19:15:45.015026 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014350    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 19:15:45.015026 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014352    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 19:15:45.015026 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014355    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 19:15:45.015026 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014357    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 19:15:45.015026 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014360    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 19:15:45.015026 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014363    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 19:15:45.015026 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014366    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 19:15:45.015026 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014370    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 19:15:45.015026 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014372    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 19:15:45.015026 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014375    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 19:15:45.015026 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014377    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 19:15:45.015026 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014380    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 19:15:45.015026 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014383    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 19:15:45.015026 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014386    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 19:15:45.015026 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014389    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 19:15:45.015026 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014391    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 19:15:45.015026 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014394    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 19:15:45.015026 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014397    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 19:15:45.015026 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014401    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 19:15:45.015026 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014404    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 19:15:45.015528 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014407    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 19:15:45.015528 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014411    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 19:15:45.015528 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014414    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 19:15:45.015528 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014417    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 19:15:45.015528 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014419    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 19:15:45.015528 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014422    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 19:15:45.015528 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014425    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 19:15:45.015528 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014428    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 19:15:45.015528 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014430    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 19:15:45.015528 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014433    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 19:15:45.015528 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014436    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 19:15:45.015528 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014439    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 19:15:45.015528 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014441    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 19:15:45.015528 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014444    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 19:15:45.015528 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014447    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 19:15:45.015528 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014450    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 19:15:45.015528 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014452    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 19:15:45.015528 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014455    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 19:15:45.015528 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014457    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 19:15:45.015528 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014460    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 19:15:45.016037 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014462    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 19:15:45.016037 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014465    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 19:15:45.016037 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014467    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 19:15:45.016037 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014472    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 19:15:45.016037 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014476    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 19:15:45.016037 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014480    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 19:15:45.016037 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014483    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 19:15:45.016037 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014486    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 19:15:45.016037 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014489    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 19:15:45.016037 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014492    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 19:15:45.016037 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014495    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 19:15:45.016037 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014498    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 19:15:45.016037 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014500    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 19:15:45.016037 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014502    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 19:15:45.016037 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014505    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 19:15:45.016037 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014507    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 19:15:45.016037 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014511    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 19:15:45.016037 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014513    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 19:15:45.016037 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014516    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 19:15:45.016515 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014518    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 19:15:45.016515 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014521    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 19:15:45.016515 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014523    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 19:15:45.016515 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014526    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 19:15:45.016515 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014528    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 19:15:45.016515 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014531    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 19:15:45.016515 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014533    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 19:15:45.016515 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014536    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 19:15:45.016515 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014539    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 19:15:45.016515 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014541    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 19:15:45.016515 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014544    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 19:15:45.016515 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014547    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 19:15:45.016515 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.014550    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 19:15:45.016515 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015280    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 19:15:45.016515 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015288    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 19:15:45.016515 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015293    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 19:15:45.016515 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015296    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 19:15:45.016515 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015300    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 19:15:45.016515 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015303    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 19:15:45.016988 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015305    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 19:15:45.016988 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015308    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 19:15:45.016988 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015310    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 19:15:45.016988 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015313    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 19:15:45.016988 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015315    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 19:15:45.016988 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015318    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 19:15:45.016988 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015321    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 19:15:45.016988 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015323    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 19:15:45.016988 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015326    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 19:15:45.016988 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015328    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 19:15:45.016988 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015331    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 19:15:45.016988 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015334    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 19:15:45.016988 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015337    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 19:15:45.016988 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015340    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 19:15:45.016988 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015342    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 19:15:45.016988 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015344    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 19:15:45.016988 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015347    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 19:15:45.016988 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015349    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 19:15:45.016988 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015352    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 19:15:45.016988 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015354    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 19:15:45.017473 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015357    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 19:15:45.017473 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015359    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 19:15:45.017473 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015362    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 19:15:45.017473 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015364    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 19:15:45.017473 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015367    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 19:15:45.017473 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015369    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 19:15:45.017473 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015372    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 19:15:45.017473 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015374    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 19:15:45.017473 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015377    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 19:15:45.017473 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015379    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 19:15:45.017473 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015382    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 19:15:45.017473 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015385    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 19:15:45.017473 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015387    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 19:15:45.017473 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015390    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 19:15:45.017473 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015392    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 19:15:45.017473 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015395    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 19:15:45.017473 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015398    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 19:15:45.017473 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015401    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 19:15:45.017473 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015403    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 19:15:45.017473 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015406    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 19:15:45.017978 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015408    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 19:15:45.017978 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015411    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 19:15:45.017978 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015413    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 19:15:45.017978 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015416    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 19:15:45.017978 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015419    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 19:15:45.017978 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015422    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 19:15:45.017978 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015424    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 19:15:45.017978 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015427    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 19:15:45.017978 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015429    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 19:15:45.017978 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015431    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 19:15:45.017978 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015434    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 19:15:45.017978 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015436    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 19:15:45.017978 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015439    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 19:15:45.017978 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015441    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 19:15:45.017978 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015443    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 19:15:45.017978 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015447    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 19:15:45.017978 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015451    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 19:15:45.017978 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015454    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 19:15:45.017978 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015456    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 19:15:45.017978 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015459    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 19:15:45.018463 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015463    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 19:15:45.018463 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015466    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 19:15:45.018463 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015469    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 19:15:45.018463 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015472    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 19:15:45.018463 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015475    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 19:15:45.018463 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015478    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 19:15:45.018463 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015481    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 19:15:45.018463 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015493    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 19:15:45.018463 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015496    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 19:15:45.018463 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015498    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 19:15:45.018463 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015501    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 19:15:45.018463 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015503    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 19:15:45.018463 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015506    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 19:15:45.018463 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015508    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 19:15:45.018463 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015511    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 19:15:45.018463 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015515    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 19:15:45.018463 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015517    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 19:15:45.018463 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015520    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 19:15:45.018463 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015523    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 19:15:45.018463 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015526    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 19:15:45.018973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015528    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 19:15:45.018973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015531    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 19:15:45.018973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015534    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 19:15:45.018973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015536    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 19:15:45.018973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015539    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 19:15:45.018973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015542    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 19:15:45.018973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015545    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 19:15:45.018973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015549    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 19:15:45.018973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015551    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 19:15:45.018973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015554    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 19:15:45.018973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015556    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 19:15:45.018973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015559    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 19:15:45.018973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015561    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 19:15:45.018973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015946    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 19:15:45.018973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015952    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 19:15:45.018973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015955    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 19:15:45.018973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015957    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 19:15:45.018973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015960    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 19:15:45.018973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015963    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 19:15:45.019448 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015966    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 19:15:45.019448 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015968    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 19:15:45.019448 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015971    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 19:15:45.019448 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015973    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 19:15:45.019448 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015976    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 19:15:45.019448 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015979    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 19:15:45.019448 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015981    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 19:15:45.019448 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015984    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 19:15:45.019448 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015986    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 19:15:45.019448 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015988    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 19:15:45.019448 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015992    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 19:15:45.019448 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015995    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 19:15:45.019448 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.015997    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 19:15:45.019448 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016000    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 19:15:45.019448 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016003    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 19:15:45.019448 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016005    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 19:15:45.019448 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016007    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 19:15:45.019448 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016010    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 19:15:45.019448 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016012    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 19:15:45.019448 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016015    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 19:15:45.019973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016018    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 19:15:45.019973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016020    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 19:15:45.019973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016023    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 19:15:45.019973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016025    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 19:15:45.019973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016027    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 19:15:45.019973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016030    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 19:15:45.019973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016032    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 19:15:45.019973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016035    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 19:15:45.019973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016037    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 19:15:45.019973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016040    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 19:15:45.019973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016043    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 19:15:45.019973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016045    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 19:15:45.019973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016048    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 19:15:45.019973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016051    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 19:15:45.019973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016053    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 19:15:45.019973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016056    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 19:15:45.019973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016059    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 19:15:45.019973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016064    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 19:15:45.019973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016067    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 19:15:45.020439 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016070    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 19:15:45.020439 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016073    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 19:15:45.020439 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016076    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 19:15:45.020439 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016078    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 19:15:45.020439 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016081    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 19:15:45.020439 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016084    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 19:15:45.020439 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016087    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 19:15:45.020439 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016089    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 19:15:45.020439 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016092    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 19:15:45.020439 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016095    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 19:15:45.020439 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016097    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 19:15:45.020439 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016100    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 19:15:45.020439 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016102    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 19:15:45.020439 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016105    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 19:15:45.020439 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016108    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 19:15:45.020439 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016110    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 19:15:45.020439 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016113    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 19:15:45.020439 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016115    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 19:15:45.020439 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016118    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 19:15:45.020439 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016122    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 19:15:45.020968 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016125    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 19:15:45.020968 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016127    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 19:15:45.020968 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016130    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 19:15:45.020968 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016132    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 19:15:45.020968 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016135    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 19:15:45.020968 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016137    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 19:15:45.020968 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016140    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 19:15:45.020968 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016142    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 19:15:45.020968 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016145    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 19:15:45.020968 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016147    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 19:15:45.020968 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016149    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 19:15:45.020968 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016153    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 19:15:45.020968 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016157    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 19:15:45.020968 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016160    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 19:15:45.020968 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016163    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 19:15:45.020968 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016165    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 19:15:45.020968 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016168    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 19:15:45.020968 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016170    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 19:15:45.020968 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016173    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 19:15:45.021466 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016175    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 19:15:45.021466 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016178    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 19:15:45.021466 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016180    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 19:15:45.021466 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016183    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 19:15:45.021466 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016185    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 19:15:45.021466 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016187    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 19:15:45.021466 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016190    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 19:15:45.021466 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016193    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 19:15:45.021466 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016195    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 19:15:45.021466 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016198    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 19:15:45.021466 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016200    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 19:15:45.021466 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016202    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 19:15:45.021466 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016205    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 19:15:45.021466 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016208    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 19:15:45.021466 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016210    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 19:15:45.021466 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016286    2579 flags.go:64] FLAG: --address="0.0.0.0"
Sep 10 19:15:45.021466 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016322    2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 10 19:15:45.021466 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016329    2579 flags.go:64] FLAG: --anonymous-auth="true"
Sep 10 19:15:45.021466 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016334    2579 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 10 19:15:45.021466 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016338    2579 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 10 19:15:45.021466 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016341    2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 10 19:15:45.022097 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016346    2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 10 19:15:45.022097 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016354    2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 10 19:15:45.022097 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016357    2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 10 19:15:45.022097 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016360    2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 10 19:15:45.022097 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016364    2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 10 19:15:45.022097 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016367    2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 10 19:15:45.022097 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016370    2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 10 19:15:45.022097 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016373    2579 flags.go:64] FLAG: --cgroup-root=""
Sep 10 19:15:45.022097 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016375    2579 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 10 19:15:45.022097 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016378    2579 flags.go:64] FLAG: --client-ca-file=""
Sep 10 19:15:45.022097 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016381    2579 flags.go:64] FLAG: --cloud-provider="external"
Sep 10 19:15:45.022097 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016384    2579 flags.go:64] FLAG: --cluster-dns="[]"
Sep 10 19:15:45.022097 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016388    2579 flags.go:64] FLAG: --cluster-domain=""
Sep 10 19:15:45.022097 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016391    2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 10 19:15:45.022097 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016394    2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 10 19:15:45.022097 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016397    2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 10 19:15:45.022097 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016400    2579 flags.go:64] FLAG: --container-log-max-files="5"
Sep 10 19:15:45.022097 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016406    2579 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 10 19:15:45.022097 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016409    2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 10 19:15:45.022097 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016412    2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 10 19:15:45.022097 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016415    2579 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 10 19:15:45.022097 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016418    2579 flags.go:64] FLAG: --contention-profiling="false"
Sep 10 19:15:45.022097 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016421    2579 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 10 19:15:45.022097 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016424    2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 10 19:15:45.022683 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016427    2579 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 10 19:15:45.022683 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016430    2579 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 10 19:15:45.022683 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016434    2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 10 19:15:45.022683 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016436    2579 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 10 19:15:45.022683 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016439    2579 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 10 19:15:45.022683 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016442    2579 flags.go:64] FLAG: --enable-load-reader="false"
Sep 10 19:15:45.022683 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016445    2579 flags.go:64] FLAG: --enable-server="true"
Sep 10 19:15:45.022683 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016447    2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 10 19:15:45.022683 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016452    2579 flags.go:64] FLAG: --event-burst="100"
Sep 10 19:15:45.022683 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016455    2579 flags.go:64] FLAG: --event-qps="50"
Sep 10 19:15:45.022683 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016458    2579 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 10 19:15:45.022683 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016461    2579 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 10 19:15:45.022683 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016464    2579 flags.go:64] FLAG: --eviction-hard=""
Sep 10 19:15:45.022683 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016467    2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 10 19:15:45.022683 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016470    2579 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 10 19:15:45.022683 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016473    2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 10 19:15:45.022683 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016476    2579 flags.go:64] FLAG: --eviction-soft=""
Sep 10 19:15:45.022683 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016478    2579 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 10 19:15:45.022683 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016481    2579 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 10 19:15:45.022683 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016485    2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 10 19:15:45.022683 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016488    2579 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 10 19:15:45.022683 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016491    2579 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 10 19:15:45.022683 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016493    2579 flags.go:64] FLAG: --fail-swap-on="true"
Sep 10 19:15:45.022683 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016496    2579 flags.go:64] FLAG: --feature-gates=""
Sep 10 19:15:45.022683 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016499    2579 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 10 19:15:45.023317 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016503    2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 10 19:15:45.023317 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016506    2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 10 19:15:45.023317 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016509    2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 10 19:15:45.023317 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016512    2579 flags.go:64] FLAG: --healthz-port="10248"
Sep 10 19:15:45.023317 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016515    2579 flags.go:64] FLAG: --help="false"
Sep 10 19:15:45.023317 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016518    2579 flags.go:64] FLAG: --hostname-override="ip-10-0-142-20.ec2.internal"
Sep 10 19:15:45.023317 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016521    2579 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 10 19:15:45.023317 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016524    2579 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 10 19:15:45.023317 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016527    2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 10 19:15:45.023317 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016530    2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 10 19:15:45.023317 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016533    2579 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 10 19:15:45.023317 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016536    2579 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 10 19:15:45.023317 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016538    2579 flags.go:64] FLAG: --image-service-endpoint=""
Sep 10 19:15:45.023317 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016541    2579 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 10 19:15:45.023317 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016544    2579 flags.go:64] FLAG: --kube-api-burst="100"
Sep 10 19:15:45.023317 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016546    2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 10 19:15:45.023317 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016549    2579 flags.go:64] FLAG: --kube-api-qps="50"
Sep 10 19:15:45.023317 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016552    2579 flags.go:64] FLAG: --kube-reserved=""
Sep 10 19:15:45.023317 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016555    2579 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 10 19:15:45.023317 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016557    2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 10 19:15:45.023317 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016560    2579 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 10 19:15:45.023317 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016563    2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 10 19:15:45.023317 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016566    2579 flags.go:64] FLAG: --lock-file=""
Sep 10 19:15:45.023317 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016568    2579 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 10 19:15:45.024338 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016571    2579 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 10 19:15:45.024338 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016573    2579 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 10 19:15:45.024338 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016579    2579 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 10 19:15:45.024338 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016582    2579 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 10 19:15:45.024338 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016584    2579 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 10 19:15:45.024338 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016587    2579 flags.go:64] FLAG: --logging-format="text"
Sep 10 19:15:45.024338 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016590    2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 10 19:15:45.024338 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016593    2579 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 10 19:15:45.024338 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016596    2579 flags.go:64] FLAG: --manifest-url=""
Sep 10 19:15:45.024338 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016599    2579 flags.go:64] FLAG: --manifest-url-header=""
Sep 10 19:15:45.024338 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016603    2579 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 10 19:15:45.024338 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016606    2579 flags.go:64] FLAG: --max-open-files="1000000"
Sep 10 19:15:45.024338 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016609    2579 flags.go:64] FLAG: --max-pods="110"
Sep 10 19:15:45.024338 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016612    2579 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 10 19:15:45.024338 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016617    2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 10 19:15:45.024338 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016620    2579 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 10 19:15:45.024338 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016623    2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 10 19:15:45.024338 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016626    2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 10 19:15:45.024338 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016629    2579 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 10 19:15:45.024338 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016632    2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 10 19:15:45.024338 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016639    2579 flags.go:64] FLAG: --node-status-max-images="50"
Sep 10 19:15:45.024338 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016642    2579 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 10 19:15:45.024338 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016644    2579 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 10 19:15:45.024338 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016648    2579 flags.go:64] FLAG: --pod-cidr=""
Sep 10 19:15:45.025415 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016650    2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 10 19:15:45.025415 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016656    2579 flags.go:64] FLAG: --pod-manifest-path=""
Sep 10 19:15:45.025415 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016659    2579 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 10 19:15:45.025415 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016662    2579 flags.go:64] FLAG: --pods-per-core="0"
Sep 10 19:15:45.025415 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016665    2579 flags.go:64] FLAG: --port="10250"
Sep 10 19:15:45.025415 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016668    2579 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 10 19:15:45.025415 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016671    2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08b59f5e20036407b"
Sep 10 19:15:45.025415 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016674    2579 flags.go:64] FLAG: --qos-reserved=""
Sep 10 19:15:45.025415 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016677    2579 flags.go:64] FLAG: --read-only-port="10255"
Sep 10 19:15:45.025415 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016680    2579 flags.go:64] FLAG: --register-node="true"
Sep 10 19:15:45.025415 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016683    2579 flags.go:64] FLAG: --register-with-taints=""
Sep 10 19:15:45.025415 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016686    2579 flags.go:64] FLAG: --registry-burst="10"
Sep 10 19:15:45.025415 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016690    2579 flags.go:64] FLAG: --registry-qps="5"
Sep 10 19:15:45.025415 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016692    2579 flags.go:64] FLAG: --reserved-cpus=""
Sep 10 19:15:45.025415 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016695    2579 flags.go:64] FLAG: --reserved-memory=""
Sep 10 19:15:45.025415 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016698    2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 10 19:15:45.025415 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016705    2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 10 19:15:45.025415 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016708    2579 flags.go:64] FLAG: --rotate-certificates="false"
Sep 10 19:15:45.025415 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016711    2579 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 10 19:15:45.025415 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016713    2579 flags.go:64] FLAG: --runonce="false"
Sep 10 19:15:45.025415 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016716    2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 10 19:15:45.025415 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016719    2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 10 19:15:45.025415 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016722    2579 flags.go:64] FLAG: --seccomp-default="false"
Sep 10 19:15:45.025415 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016726    2579 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 10 19:15:45.025415 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016729    2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 10 19:15:45.026510 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016732    2579 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 10 19:15:45.026510 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016735    2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 10 19:15:45.026510 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016738    2579 flags.go:64] FLAG: --storage-driver-password="root"
Sep 10 19:15:45.026510 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016741    2579 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 10 19:15:45.026510 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016743    2579 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 10 19:15:45.026510 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016762    2579 flags.go:64] FLAG: --storage-driver-user="root"
Sep 10 19:15:45.026510 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016765    2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 10 19:15:45.026510 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016768    2579 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 10 19:15:45.026510 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016771    2579 flags.go:64] FLAG: --system-cgroups=""
Sep 10 19:15:45.026510 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016774    2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 10 19:15:45.026510 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016779    2579 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 10 19:15:45.026510 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016781    2579 flags.go:64] FLAG: --tls-cert-file=""
Sep 10 19:15:45.026510 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016784    2579 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 10 19:15:45.026510 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016788    2579 flags.go:64] FLAG: --tls-min-version=""
Sep 10 19:15:45.026510 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016790    2579 flags.go:64] FLAG: --tls-private-key-file=""
Sep 10 19:15:45.026510 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016793    2579 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 10 19:15:45.026510 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016796    2579 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 10 19:15:45.026510 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016799    2579 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 10 19:15:45.026510 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016802    2579 flags.go:64] FLAG: --v="2"
Sep 10 19:15:45.026510 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016805    2579 flags.go:64] FLAG: --version="false"
Sep 10 19:15:45.026510 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016810    2579 flags.go:64] FLAG: --vmodule=""
Sep 10 19:15:45.026510 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016813    2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 10 19:15:45.026510 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.016816    2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 10 19:15:45.026510 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016926    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 19:15:45.026510 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016930    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 19:15:45.027471 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016933    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 19:15:45.027471 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016935    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 19:15:45.027471 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016939    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 19:15:45.027471 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016942    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 19:15:45.027471 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016945    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 19:15:45.027471 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016948    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 19:15:45.027471 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016951    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 19:15:45.027471 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016953    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 19:15:45.027471 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016956    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 19:15:45.027471 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016958    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 19:15:45.027471 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016961    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 19:15:45.027471 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016963    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 19:15:45.027471 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016966    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 19:15:45.027471 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016969    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 19:15:45.027471 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016971    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 19:15:45.027471 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016974    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 19:15:45.027471 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016977    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 19:15:45.027471 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016979    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 19:15:45.027471 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016981    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 19:15:45.028142 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016985    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 19:15:45.028142 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016989    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 19:15:45.028142 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016992    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 19:15:45.028142 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016995    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 19:15:45.028142 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.016998    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 19:15:45.028142 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017001    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 19:15:45.028142 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017004    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 19:15:45.028142 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017006    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 19:15:45.028142 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017009    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 19:15:45.028142 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017012    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 19:15:45.028142 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017014    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 19:15:45.028142 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017019    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 19:15:45.028142 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017022    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 19:15:45.028142 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017025    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 19:15:45.028142 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017029    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 19:15:45.028142 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017033    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 19:15:45.028142 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017037    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 19:15:45.028142 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017041    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 19:15:45.028142 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017047    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 19:15:45.029053 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017050    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 19:15:45.029053 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017053    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 19:15:45.029053 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017055    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 19:15:45.029053 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017057    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 19:15:45.029053 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017060    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 19:15:45.029053 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017062    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 19:15:45.029053 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017065    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 19:15:45.029053 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017067    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 19:15:45.029053 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017070    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 19:15:45.029053 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017072    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 19:15:45.029053 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017075    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 19:15:45.029053 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017078    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 19:15:45.029053 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017080    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 19:15:45.029053 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017082    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 19:15:45.029053 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017085    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 19:15:45.029053 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017087    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 19:15:45.029053 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017089    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 19:15:45.029053 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017092    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 19:15:45.029053 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017094    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 19:15:45.029053 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017097    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 19:15:45.029644 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017099    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 19:15:45.029644 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017102    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 19:15:45.029644 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017105    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 19:15:45.029644 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017108    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 19:15:45.029644 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017112    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 19:15:45.029644 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017115    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 19:15:45.029644 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017118    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 19:15:45.029644 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017121    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 19:15:45.029644 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017124    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 19:15:45.029644 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017127    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 19:15:45.029644 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017129    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 19:15:45.029644 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017134    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 19:15:45.029644 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017137    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 19:15:45.029644 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017139    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 19:15:45.029644 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017142    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 19:15:45.029644 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017144    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 19:15:45.029644 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017147    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 19:15:45.029644 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017149    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 19:15:45.029644 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017152    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 19:15:45.030298 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017154    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 19:15:45.030298 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017157    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 19:15:45.030298 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017159    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 19:15:45.030298 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017162    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 19:15:45.030298 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017164    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 19:15:45.030298 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017167    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 19:15:45.030298 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017169    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 19:15:45.030298 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017171    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 19:15:45.030298 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017174    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 19:15:45.030298 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017176    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 19:15:45.030298 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017178    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 19:15:45.030298 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017181    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 19:15:45.030298 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017184    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 19:15:45.030298 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017186    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 19:15:45.030298 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017189    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 19:15:45.030298 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017191    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 19:15:45.030298 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017193    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 19:15:45.030298 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017197    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 19:15:45.030298 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017200    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 19:15:45.030859 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.017203    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 19:15:45.030859 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.017824    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 19:15:45.030859 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.025633    2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 10 19:15:45.030859 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.025836    2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 10 19:15:45.030859 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.025857    2579 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 10 19:15:45.030859 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.025866    2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 10 19:15:45.030859 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.025948    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 19:15:45.030859 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.025957    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 19:15:45.030859 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.025962    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 19:15:45.030859 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.025967    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 19:15:45.030859 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.025972    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 19:15:45.030859 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.025977    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 19:15:45.030859 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.025982    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 19:15:45.030859 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.025986    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 19:15:45.030859 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.025991    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 19:15:45.031257 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.025996    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 19:15:45.031257 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026000    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 19:15:45.031257 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026004    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 19:15:45.031257 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026008    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 19:15:45.031257 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026012    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 19:15:45.031257 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026017    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 19:15:45.031257 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026021    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 19:15:45.031257 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026025    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 19:15:45.031257 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026029    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 19:15:45.031257 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026033    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 19:15:45.031257 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026037    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 19:15:45.031257 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026041    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 19:15:45.031257 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026045    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 19:15:45.031257 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026049    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 19:15:45.031257 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026053    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 19:15:45.031257 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026059    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 19:15:45.031257 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026063    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 19:15:45.031257 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026067    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 19:15:45.031257 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026071    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 19:15:45.031257 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026075    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 19:15:45.031862 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026079    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 19:15:45.031862 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026083    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 19:15:45.031862 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026088    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 19:15:45.031862 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026092    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 19:15:45.031862 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026096    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 19:15:45.031862 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026101    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 19:15:45.031862 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026104    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 19:15:45.031862 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026109    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 19:15:45.031862 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026112    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 19:15:45.031862 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026116    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 19:15:45.031862 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026120    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 19:15:45.031862 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026124    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 19:15:45.031862 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026128    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 19:15:45.031862 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026133    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 19:15:45.031862 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026136    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 19:15:45.031862 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026141    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 19:15:45.031862 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026145    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 19:15:45.031862 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026149    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 19:15:45.031862 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026153    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 19:15:45.032352 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026157    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 19:15:45.032352 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026161    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 19:15:45.032352 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026165    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 19:15:45.032352 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026169    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 19:15:45.032352 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026173    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 19:15:45.032352 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026177    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 19:15:45.032352 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026181    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 19:15:45.032352 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026186    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 19:15:45.032352 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026190    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 19:15:45.032352 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026194    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 19:15:45.032352 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026199    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 19:15:45.032352 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026203    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 19:15:45.032352 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026207    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 19:15:45.032352 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026213    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 19:15:45.032352 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026220    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 19:15:45.032352 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026224    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 19:15:45.032352 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026229    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 19:15:45.032352 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026232    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 19:15:45.032352 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026237    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 19:15:45.032352 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026240    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 19:15:45.032987 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026244    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 19:15:45.032987 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026248    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 19:15:45.032987 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026252    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 19:15:45.032987 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026256    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 19:15:45.032987 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026260    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 19:15:45.032987 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026264    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 19:15:45.032987 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026269    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 19:15:45.032987 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026272    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 19:15:45.032987 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026277    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 19:15:45.032987 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026281    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 19:15:45.032987 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026285    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 19:15:45.032987 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026289    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 19:15:45.032987 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026292    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 19:15:45.032987 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026296    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 19:15:45.032987 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026300    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 19:15:45.032987 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026304    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 19:15:45.032987 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026308    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 19:15:45.032987 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026313    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 19:15:45.032987 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026317    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 19:15:45.032987 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026321    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 19:15:45.033847 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026325    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 19:15:45.033847 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026329    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 19:15:45.033847 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026334    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 19:15:45.033847 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026338    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 19:15:45.033847 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026342    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 19:15:45.033847 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026348    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 19:15:45.033847 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026354    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 19:15:45.033847 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026361    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 19:15:45.033847 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026366    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 19:15:45.033847 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026371    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 19:15:45.033847 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026376    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 19:15:45.033847 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.026383    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 19:15:45.033847 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026559    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 19:15:45.033847 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026567    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 19:15:45.034452 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026571    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 19:15:45.034452 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026576    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 19:15:45.034452 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026580    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 19:15:45.034452 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026584    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 19:15:45.034452 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026588    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 19:15:45.034452 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026592    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 19:15:45.034452 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026596    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 19:15:45.034452 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026600    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 19:15:45.034452 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026604    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 19:15:45.034452 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026608    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 19:15:45.034452 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026612    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 19:15:45.034452 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026616    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 19:15:45.034452 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026620    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 19:15:45.034452 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026624    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 19:15:45.034452 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026628    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 19:15:45.034452 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026632    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 19:15:45.034452 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026636    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 19:15:45.034452 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026640    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 19:15:45.034452 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026644    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 19:15:45.035286 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026648    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 19:15:45.035286 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026652    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 19:15:45.035286 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026656    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 19:15:45.035286 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026660    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 19:15:45.035286 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026664    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 19:15:45.035286 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026669    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 19:15:45.035286 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026673    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 19:15:45.035286 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026677    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 19:15:45.035286 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026681    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 19:15:45.035286 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026685    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 19:15:45.035286 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026689    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 19:15:45.035286 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026693    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 19:15:45.035286 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026697    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 19:15:45.035286 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026701    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 19:15:45.035286 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026705    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 19:15:45.035286 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026709    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 19:15:45.035286 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026713    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 19:15:45.035286 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026717    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 19:15:45.035286 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026721    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 19:15:45.035286 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026725    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 19:15:45.035863 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026729    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 19:15:45.035863 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026733    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 19:15:45.035863 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026738    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 19:15:45.035863 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026742    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 19:15:45.035863 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026769    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 19:15:45.035863 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026774    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 19:15:45.035863 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026779    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 19:15:45.035863 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026783    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 19:15:45.035863 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026787    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 19:15:45.035863 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026791    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 19:15:45.035863 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026795    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 19:15:45.035863 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026799    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 19:15:45.035863 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026804    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 19:15:45.035863 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026807    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 19:15:45.035863 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026811    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 19:15:45.035863 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026818    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 19:15:45.035863 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026824    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 19:15:45.035863 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026828    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 19:15:45.035863 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026832    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 19:15:45.035863 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026836    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 19:15:45.036571 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026840    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 19:15:45.036571 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026844    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 19:15:45.036571 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026848    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 19:15:45.036571 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026852    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 19:15:45.036571 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026856    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 19:15:45.036571 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026859    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 19:15:45.036571 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026863    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 19:15:45.036571 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026867    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 19:15:45.036571 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026871    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 19:15:45.036571 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026875    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 19:15:45.036571 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026879    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 19:15:45.036571 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026884    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 19:15:45.036571 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026888    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 19:15:45.036571 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026892    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 19:15:45.036571 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026896    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 19:15:45.036571 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026900    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 19:15:45.036571 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026903    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 19:15:45.036571 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026908    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 19:15:45.036571 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026911    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 19:15:45.036571 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026915    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 19:15:45.037095 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026919    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 19:15:45.037095 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026932    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 19:15:45.037095 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026937    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 19:15:45.037095 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026941    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 19:15:45.037095 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026945    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 19:15:45.037095 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026949    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 19:15:45.037095 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026953    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 19:15:45.037095 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026957    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 19:15:45.037095 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026963    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 19:15:45.037095 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026970    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 19:15:45.037095 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026976    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 19:15:45.037095 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026981    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 19:15:45.037095 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026985    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 19:15:45.037095 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026991    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 19:15:45.037095 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.026997    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 19:15:45.037095 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.027002    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 19:15:45.037095 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.027007    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 19:15:45.037095 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:45.027011    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 19:15:45.037623 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.027020    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 19:15:45.037623 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.027175    2579 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 10 19:15:45.037623 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.029905    2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 10 19:15:45.037623 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.030794    2579 server.go:1019] "Starting client certificate rotation"
Sep 10 19:15:45.037623 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.030902    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 19:15:45.037623 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.030939    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 19:15:45.037623 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.033541    2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 19:15:45.037623 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.035315    2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 19:15:45.043734 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.043713    2579 log.go:25] "Validated CRI v1 runtime API"
Sep 10 19:15:45.046733 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.046714    2579 log.go:25] "Validated CRI v1 image API"
Sep 10 19:15:45.047562 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.047543    2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 10 19:15:45.052481 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.052462    2579 fs.go:135] Filesystem UUIDs: map[1aab1052-44dc-4394-a784-c72fa55b9870:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 b00f386c-b5d4-44da-ba73-6000357eb6ee:/dev/nvme0n1p4]
Sep 10 19:15:45.052566 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.052479    2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 10 19:15:45.058368 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.058258    2579 manager.go:216] Machine: {Timestamp:2026-09-10 19:15:45.056930822 +0000 UTC m=+0.276779302 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101351 MemoryCapacity:32812158976 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec218a730a9dd7471d6444f045c7b29c SystemUUID:ec218a73-0a9d-d747-1d64-44f045c7b29c BootID:d30d717a-96d7-4228-b3f7-d69c1592c620 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f8:59:3a:f0:71 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f8:59:3a:f0:71 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9a:9d:be:11:e1:8d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812158976 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 10 19:15:45.058368 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.058366    2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 10 19:15:45.058487 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.058478    2579 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 10 19:15:45.059975 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.059958    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 19:15:45.060999 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.060977    2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 10 19:15:45.061134 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.061004    2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-20.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 10 19:15:45.062197 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.062186    2579 topology_manager.go:138] "Creating topology manager with none policy"
Sep 10 19:15:45.062232 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.062200    2579 container_manager_linux.go:309] "Creating device plugin manager"
Sep 10 19:15:45.062232 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.062208    2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 19:15:45.062772 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.062761    2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 19:15:45.063618 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.063608    2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 10 19:15:45.067701 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.067689    2579 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 19:15:45.068005 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.067994    2579 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 10 19:15:45.068041 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.068026    2579 kubelet.go:486] "Attempting to sync node with API server"
Sep 10 19:15:45.068041 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.068036    2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 10 19:15:45.068097 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.068056    2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 10 19:15:45.068097 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.068065    2579 kubelet.go:398] "Adding apiserver pod source"
Sep 10 19:15:45.068097 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.068074    2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 10 19:15:45.068364 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.068353    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 19:15:45.068419 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.068370    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 19:15:45.070595 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.070577    2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 10 19:15:45.072833 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.072820    2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 10 19:15:45.072891 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.072849    2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 10 19:15:45.073869 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.073851    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 10 19:15:45.073944 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.073876    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 10 19:15:45.073944 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.073886    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 10 19:15:45.073944 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.073906    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 10 19:15:45.073944 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.073922    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 10 19:15:45.073944 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.073936    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 10 19:15:45.074080 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.073954    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 10 19:15:45.074080 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.073960    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 10 19:15:45.074080 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.073966    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 10 19:15:45.074080 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.073972    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 10 19:15:45.074080 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.073985    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 10 19:15:45.074080 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.073990    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 10 19:15:45.074674 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.074661    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 10 19:15:45.074674 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.074669    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 10 19:15:45.078415 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.078393    2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-20.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 19:15:45.078570 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:45.078538    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 19:15:45.078911 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:45.078878    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-20.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 19:15:45.079363 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.079332    2579 server.go:1268] "Started kubelet"
Sep 10 19:15:45.080494 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.080011    2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 10 19:15:45.080494 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.080170    2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 10 19:15:45.080494 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.080346    2579 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 10 19:15:45.080365 ip-10-0-142-20 systemd[1]: Started Kubernetes Kubelet.
Sep 10 19:15:45.081972 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.081951    2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 10 19:15:45.083790 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.083773    2579 server.go:310] "Adding debug handlers to kubelet server"
Sep 10 19:15:45.084896 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:45.084878    2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 10 19:15:45.087126 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:45.086320    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-20.ec2.internal.18d40ba2a088dde6  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-20.ec2.internal,UID:ip-10-0-142-20.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-20.ec2.internal,},FirstTimestamp:2026-09-10 19:15:45.078861286 +0000 UTC m=+0.298709767,LastTimestamp:2026-09-10 19:15:45.078861286 +0000 UTC m=+0.298709767,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-20.ec2.internal,}"
Sep 10 19:15:45.087394 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.087373    2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 10 19:15:45.087499 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.087398    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 10 19:15:45.087561 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:45.087544    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-20.ec2.internal\" not found"
Sep 10 19:15:45.088110 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.088087    2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 10 19:15:45.088196 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.088134    2579 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 10 19:15:45.088196 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.088145    2579 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 10 19:15:45.088266 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.088230    2579 reconstruct.go:97] "Volume reconstruction finished"
Sep 10 19:15:45.088266 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.088242    2579 reconciler.go:29] "Reconciler: start to sync state"
Sep 10 19:15:45.089460 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.089440    2579 factory.go:55] Registering systemd factory
Sep 10 19:15:45.089460 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.089458    2579 factory.go:223] Registration of the systemd container factory successfully
Sep 10 19:15:45.089674 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.089663    2579 factory.go:153] Registering CRI-O factory
Sep 10 19:15:45.089731 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.089677    2579 factory.go:223] Registration of the crio container factory successfully
Sep 10 19:15:45.089805 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.089730    2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 10 19:15:45.089805 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.089769    2579 factory.go:103] Registering Raw factory
Sep 10 19:15:45.089805 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.089783    2579 manager.go:1196] Started watching for new ooms in manager
Sep 10 19:15:45.090160 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.090148    2579 manager.go:349] Starting recovery of all containers
Sep 10 19:15:45.101553 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.101533    2579 manager.go:354] Recovery completed
Sep 10 19:15:45.102228 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:45.102192    2579 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-20.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 10 19:15:45.102228 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:45.102220    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 10 19:15:45.103385 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.103284    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lrbph"
Sep 10 19:15:45.106329 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.106316    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 19:15:45.108491 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.108463    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-20.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 19:15:45.108583 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.108517    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-20.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 19:15:45.108583 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.108534    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-20.ec2.internal" event="NodeHasSufficientPID"
Sep 10 19:15:45.109671 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.109651    2579 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 10 19:15:45.109671 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.109671    2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 10 19:15:45.109818 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.109688    2579 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 19:15:45.109979 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.109960    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 19:15:45.111188 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.111163    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-20.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 19:15:45.111286 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.111201    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-20.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 19:15:45.111286 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.111216    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-20.ec2.internal" event="NodeHasSufficientPID"
Sep 10 19:15:45.111819 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.111803    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lrbph"
Sep 10 19:15:45.112882 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.112871    2579 policy_none.go:49] "None policy: Start"
Sep 10 19:15:45.112943 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.112886    2579 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 10 19:15:45.112943 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.112897    2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 10 19:15:45.114943 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.114931    2579 policy_none.go:47] "Start"
Sep 10 19:15:45.117294 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:45.117233    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-20.ec2.internal.18d40ba2a24cf881  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-20.ec2.internal,UID:ip-10-0-142-20.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-20.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-20.ec2.internal,},FirstTimestamp:2026-09-10 19:15:45.108490369 +0000 UTC m=+0.328338842,LastTimestamp:2026-09-10 19:15:45.108490369 +0000 UTC m=+0.328338842,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-20.ec2.internal,}"
Sep 10 19:15:45.150351 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.150336    2579 manager.go:337] "Starting Device Plugin manager"
Sep 10 19:15:45.168924 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:45.150372    2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 10 19:15:45.168924 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.150382    2579 server.go:87] "Starting device plugin registration server"
Sep 10 19:15:45.168924 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.150664    2579 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 10 19:15:45.168924 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.150680    2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 10 19:15:45.168924 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.150789    2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 10 19:15:45.168924 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.150880    2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 10 19:15:45.168924 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.150889    2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 10 19:15:45.168924 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:45.151375    2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 10 19:15:45.168924 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:45.151413    2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-20.ec2.internal\" not found"
Sep 10 19:15:45.230851 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.230810    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 10 19:15:45.232301 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.232271    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 10 19:15:45.232462 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.232358    2579 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 10 19:15:45.232462 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.232395    2579 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 10 19:15:45.232550 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:45.232464    2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 10 19:15:45.235016 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.234992    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 19:15:45.251585 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.251567    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 19:15:45.252422 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.252404    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-20.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 19:15:45.252529 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.252439    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-20.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 19:15:45.252529 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.252451    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-20.ec2.internal" event="NodeHasSufficientPID"
Sep 10 19:15:45.252529 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.252474    2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-20.ec2.internal"
Sep 10 19:15:45.261212 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.261116    2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-20.ec2.internal"
Sep 10 19:15:45.261212 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:45.261145    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-20.ec2.internal\": node \"ip-10-0-142-20.ec2.internal\" not found"
Sep 10 19:15:45.276533 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:45.276508    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-20.ec2.internal\" not found"
Sep 10 19:15:45.332959 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.332922    2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-20.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-20.ec2.internal"]
Sep 10 19:15:45.333067 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.333023    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 19:15:45.334035 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.334017    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-20.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 19:15:45.334125 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.334048    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-20.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 19:15:45.334125 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.334058    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-20.ec2.internal" event="NodeHasSufficientPID"
Sep 10 19:15:45.336386 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.336373    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 19:15:45.336505 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.336491    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-20.ec2.internal"
Sep 10 19:15:45.336547 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.336522    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 19:15:45.337195 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.337177    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-20.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 19:15:45.337291 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.337206    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-20.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 19:15:45.337291 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.337208    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-20.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 19:15:45.337291 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.337218    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-20.ec2.internal" event="NodeHasSufficientPID"
Sep 10 19:15:45.337291 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.337226    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-20.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 19:15:45.337291 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.337237    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-20.ec2.internal" event="NodeHasSufficientPID"
Sep 10 19:15:45.339428 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.339412    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-20.ec2.internal"
Sep 10 19:15:45.339516 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.339437    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 19:15:45.340170 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.340157    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-20.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 19:15:45.340244 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.340183    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-20.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 19:15:45.340244 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.340194    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-20.ec2.internal" event="NodeHasSufficientPID"
Sep 10 19:15:45.359348 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:45.359321    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-20.ec2.internal\" not found" node="ip-10-0-142-20.ec2.internal"
Sep 10 19:15:45.362737 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:45.362721    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-20.ec2.internal\" not found" node="ip-10-0-142-20.ec2.internal"
Sep 10 19:15:45.376879 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:45.376854    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-20.ec2.internal\" not found"
Sep 10 19:15:45.389845 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.389816    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dc403cc24af90cbf61a5edf297685b42-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-20.ec2.internal\" (UID: \"dc403cc24af90cbf61a5edf297685b42\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-20.ec2.internal"
Sep 10 19:15:45.389941 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.389849    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc403cc24af90cbf61a5edf297685b42-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-20.ec2.internal\" (UID: \"dc403cc24af90cbf61a5edf297685b42\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-20.ec2.internal"
Sep 10 19:15:45.389941 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.389867    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e0bff5a7795653f306d187693d0b9c36-config\") pod \"kube-apiserver-proxy-ip-10-0-142-20.ec2.internal\" (UID: \"e0bff5a7795653f306d187693d0b9c36\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-20.ec2.internal"
Sep 10 19:15:45.477499 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:45.477470    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-20.ec2.internal\" not found"
Sep 10 19:15:45.490836 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.490808    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dc403cc24af90cbf61a5edf297685b42-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-20.ec2.internal\" (UID: \"dc403cc24af90cbf61a5edf297685b42\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-20.ec2.internal"
Sep 10 19:15:45.490888 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.490848    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc403cc24af90cbf61a5edf297685b42-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-20.ec2.internal\" (UID: \"dc403cc24af90cbf61a5edf297685b42\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-20.ec2.internal"
Sep 10 19:15:45.490888 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.490870    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e0bff5a7795653f306d187693d0b9c36-config\") pod \"kube-apiserver-proxy-ip-10-0-142-20.ec2.internal\" (UID: \"e0bff5a7795653f306d187693d0b9c36\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-20.ec2.internal"
Sep 10 19:15:45.490954 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.490921    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dc403cc24af90cbf61a5edf297685b42-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-20.ec2.internal\" (UID: \"dc403cc24af90cbf61a5edf297685b42\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-20.ec2.internal"
Sep 10 19:15:45.490954 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.490936    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e0bff5a7795653f306d187693d0b9c36-config\") pod \"kube-apiserver-proxy-ip-10-0-142-20.ec2.internal\" (UID: \"e0bff5a7795653f306d187693d0b9c36\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-20.ec2.internal"
Sep 10 19:15:45.490954 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.490944    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc403cc24af90cbf61a5edf297685b42-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-20.ec2.internal\" (UID: \"dc403cc24af90cbf61a5edf297685b42\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-20.ec2.internal"
Sep 10 19:15:45.578327 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:45.578233    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-20.ec2.internal\" not found"
Sep 10 19:15:45.661786 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.661738    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-20.ec2.internal"
Sep 10 19:15:45.667353 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:45.666681    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-20.ec2.internal"
Sep 10 19:15:45.678710 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:45.678689    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-20.ec2.internal\" not found"
Sep 10 19:15:45.779298 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:45.779247    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-20.ec2.internal\" not found"
Sep 10 19:15:45.879878 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:45.879791    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-20.ec2.internal\" not found"
Sep 10 19:15:45.980568 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:45.980513    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-20.ec2.internal\" not found"
Sep 10 19:15:46.031082 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:46.031047    2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 10 19:15:46.031552 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:46.031198    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 19:15:46.081311 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:46.081284    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-20.ec2.internal\" not found"
Sep 10 19:15:46.117261 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:46.117187    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 19:10:45 +0000 UTC" deadline="2028-03-18 06:43:37.990569258 +0000 UTC"
Sep 10 19:15:46.117261 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:46.117260    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13307h27m51.873315079s"
Sep 10 19:15:46.172569 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:46.172536    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 19:15:46.173764 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:46.173729    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-20.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 19:15:46.173873 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:46.173777    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-20.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 19:15:46.173873 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:46.173792    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-20.ec2.internal" event="NodeHasSufficientPID"
Sep 10 19:15:46.182084 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:46.182063    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-20.ec2.internal\" not found"
Sep 10 19:15:46.224966 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:46.224781    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc403cc24af90cbf61a5edf297685b42.slice/crio-d1f83d7dfa4fc843134b6e7323aa2fca9e7cb050ffb91cc55a2f4def80951784 WatchSource:0}: Error finding container d1f83d7dfa4fc843134b6e7323aa2fca9e7cb050ffb91cc55a2f4def80951784: Status 404 returned error can't find the container with id d1f83d7dfa4fc843134b6e7323aa2fca9e7cb050ffb91cc55a2f4def80951784
Sep 10 19:15:46.231094 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:46.230426    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 19:15:46.235786 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:46.235725    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-20.ec2.internal" event={"ID":"dc403cc24af90cbf61a5edf297685b42","Type":"ContainerStarted","Data":"d1f83d7dfa4fc843134b6e7323aa2fca9e7cb050ffb91cc55a2f4def80951784"}
Sep 10 19:15:46.282211 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:46.282171    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-20.ec2.internal\" not found"
Sep 10 19:15:46.382694 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:46.382660    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-20.ec2.internal\" not found"
Sep 10 19:15:46.407553 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:46.407522    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0bff5a7795653f306d187693d0b9c36.slice/crio-8f0aedb33dd27ba4efbf562bd02c7a0f75955fb7a943cf4fac67e2b27fe38823 WatchSource:0}: Error finding container 8f0aedb33dd27ba4efbf562bd02c7a0f75955fb7a943cf4fac67e2b27fe38823: Status 404 returned error can't find the container with id 8f0aedb33dd27ba4efbf562bd02c7a0f75955fb7a943cf4fac67e2b27fe38823
Sep 10 19:15:46.483766 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:46.483668    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-20.ec2.internal\" not found"
Sep 10 19:15:46.495665 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:46.495643    2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 19:15:46.587654 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:46.587622    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-20.ec2.internal"
Sep 10 19:15:46.599115 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:46.599088    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 19:15:46.600633 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:46.600617    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-20.ec2.internal"
Sep 10 19:15:46.609130 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:46.609110    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 19:15:46.652294 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:46.652270    2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 19:15:46.681434 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:46.681404    2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 19:15:46.967754 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:46.967580    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 19:15:47.068814 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.068726    2579 apiserver.go:52] "Watching apiserver"
Sep 10 19:15:47.074357 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.074323    2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 10 19:15:47.075538 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.075505    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-47rkn","openshift-network-diagnostics/network-check-target-dns2r","openshift-network-operator/iptables-alerter-89qpn","openshift-ovn-kubernetes/ovnkube-node-8r66s","kube-system/global-pull-secret-syncer-mkkzj","kube-system/kube-apiserver-proxy-ip-10-0-142-20.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwrt5","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-20.ec2.internal","openshift-multus/multus-additional-cni-plugins-qcw6c","openshift-multus/network-metrics-daemon-qjlbx","kube-system/konnectivity-agent-cllmw","openshift-cluster-node-tuning-operator/tuned-zd86j","openshift-dns/node-resolver-k8tfc","openshift-image-registry/node-ca-zdnmx"]
Sep 10 19:15:47.078332 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.078252    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwrt5"
Sep 10 19:15:47.080815 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.080788    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-k8tfc"
Sep 10 19:15:47.081639 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.081619    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 10 19:15:47.082059 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.082037    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 10 19:15:47.082290 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.082273    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-wnpqz\""
Sep 10 19:15:47.082424 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.082405    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 10 19:15:47.082835 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.082796    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 10 19:15:47.082992 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.082949    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 10 19:15:47.084877 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.083063    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-fs6r4\""
Sep 10 19:15:47.084877 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.083301    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-89qpn"
Sep 10 19:15:47.085887 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.085417    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 10 19:15:47.085887 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.085677    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 10 19:15:47.085887 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.085747    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5v2sv\""
Sep 10 19:15:47.085887 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.085888    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 10 19:15:47.086169 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.086140    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.088981 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.088165    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 10 19:15:47.088981 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.088213    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vdpbj\""
Sep 10 19:15:47.088981 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.088261    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 10 19:15:47.088981 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.088533    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 10 19:15:47.088981 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.088882    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 10 19:15:47.088981 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.088900    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 10 19:15:47.090736 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.089944    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 10 19:15:47.090736 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.090214    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 10 19:15:47.091528 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.091054    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkkzj"
Sep 10 19:15:47.096010 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:47.091650    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mkkzj" podUID="acd262af-057c-4bbf-93e3-faa020c1825c"
Sep 10 19:15:47.097891 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.097854    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zdnmx"
Sep 10 19:15:47.099620 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.099592    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2826f8ec-78f6-4cab-879e-47a21854ed5a-ovnkube-script-lib\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.099620 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.099614    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 19:15:47.099786 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.099640    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/89ea61c7-adeb-40e7-8a14-22c8d11a5262-registration-dir\") pod \"aws-ebs-csi-driver-node-fwrt5\" (UID: \"89ea61c7-adeb-40e7-8a14-22c8d11a5262\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwrt5"
Sep 10 19:15:47.099786 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.099670    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-run-systemd\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.099786 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.099695    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/89ea61c7-adeb-40e7-8a14-22c8d11a5262-device-dir\") pod \"aws-ebs-csi-driver-node-fwrt5\" (UID: \"89ea61c7-adeb-40e7-8a14-22c8d11a5262\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwrt5"
Sep 10 19:15:47.099786 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.099735    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0df00d14-a6d5-4b0f-895b-2f6cacbe131c-iptables-alerter-script\") pod \"iptables-alerter-89qpn\" (UID: \"0df00d14-a6d5-4b0f-895b-2f6cacbe131c\") " pod="openshift-network-operator/iptables-alerter-89qpn"
Sep 10 19:15:47.099786 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.099760    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2s6c2\" (UniqueName: \"kubernetes.io/projected/0df00d14-a6d5-4b0f-895b-2f6cacbe131c-kube-api-access-2s6c2\") pod \"iptables-alerter-89qpn\" (UID: \"0df00d14-a6d5-4b0f-895b-2f6cacbe131c\") " pod="openshift-network-operator/iptables-alerter-89qpn"
Sep 10 19:15:47.099786 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.099786    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-run-ovn\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.100001 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.099810    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-log-socket\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.100001 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.099855    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-host-cni-bin\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.100001 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.099881    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2826f8ec-78f6-4cab-879e-47a21854ed5a-ovnkube-config\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.100001 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.099922    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2826f8ec-78f6-4cab-879e-47a21854ed5a-ovn-node-metrics-cert\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.100001 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.099946    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 10 19:15:47.100001 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.099957    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-host-run-ovn-kubernetes\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.100260 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.099984    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/89ea61c7-adeb-40e7-8a14-22c8d11a5262-sys-fs\") pod \"aws-ebs-csi-driver-node-fwrt5\" (UID: \"89ea61c7-adeb-40e7-8a14-22c8d11a5262\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwrt5"
Sep 10 19:15:47.100260 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.100082    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lp2p\" (UniqueName: \"kubernetes.io/projected/89ea61c7-adeb-40e7-8a14-22c8d11a5262-kube-api-access-4lp2p\") pod \"aws-ebs-csi-driver-node-fwrt5\" (UID: \"89ea61c7-adeb-40e7-8a14-22c8d11a5262\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwrt5"
Sep 10 19:15:47.100260 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.100110    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-systemd-units\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.100260 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.100135    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-host-run-netns\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.100260 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.100162    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.100260 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.100187    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/acd262af-057c-4bbf-93e3-faa020c1825c-kubelet-config\") pod \"global-pull-secret-syncer-mkkzj\" (UID: \"acd262af-057c-4bbf-93e3-faa020c1825c\") " pod="kube-system/global-pull-secret-syncer-mkkzj"
Sep 10 19:15:47.100260 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.100214    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c357f393-afff-4968-aba1-3ef6f02620eb-tmp-dir\") pod \"node-resolver-k8tfc\" (UID: \"c357f393-afff-4968-aba1-3ef6f02620eb\") " pod="openshift-dns/node-resolver-k8tfc"
Sep 10 19:15:47.100260 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.100237    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vb4xk\" (UniqueName: \"kubernetes.io/projected/c357f393-afff-4968-aba1-3ef6f02620eb-kube-api-access-vb4xk\") pod \"node-resolver-k8tfc\" (UID: \"c357f393-afff-4968-aba1-3ef6f02620eb\") " pod="openshift-dns/node-resolver-k8tfc"
Sep 10 19:15:47.100260 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.100242    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-dvz6x\""
Sep 10 19:15:47.100653 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.100260    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-host-kubelet\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.100653 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.100306    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/89ea61c7-adeb-40e7-8a14-22c8d11a5262-socket-dir\") pod \"aws-ebs-csi-driver-node-fwrt5\" (UID: \"89ea61c7-adeb-40e7-8a14-22c8d11a5262\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwrt5"
Sep 10 19:15:47.100653 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.100311    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 10 19:15:47.100653 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.100330    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2826f8ec-78f6-4cab-879e-47a21854ed5a-env-overrides\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.100653 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.100353    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/acd262af-057c-4bbf-93e3-faa020c1825c-dbus\") pod \"global-pull-secret-syncer-mkkzj\" (UID: \"acd262af-057c-4bbf-93e3-faa020c1825c\") " pod="kube-system/global-pull-secret-syncer-mkkzj"
Sep 10 19:15:47.100653 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.100380    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 10 19:15:47.100653 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.100378    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/acd262af-057c-4bbf-93e3-faa020c1825c-original-pull-secret\") pod \"global-pull-secret-syncer-mkkzj\" (UID: \"acd262af-057c-4bbf-93e3-faa020c1825c\") " pod="kube-system/global-pull-secret-syncer-mkkzj"
Sep 10 19:15:47.100653 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.100465    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/89ea61c7-adeb-40e7-8a14-22c8d11a5262-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fwrt5\" (UID: \"89ea61c7-adeb-40e7-8a14-22c8d11a5262\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwrt5"
Sep 10 19:15:47.100653 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.100495    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/89ea61c7-adeb-40e7-8a14-22c8d11a5262-etc-selinux\") pod \"aws-ebs-csi-driver-node-fwrt5\" (UID: \"89ea61c7-adeb-40e7-8a14-22c8d11a5262\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwrt5"
Sep 10 19:15:47.100653 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.100522    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c357f393-afff-4968-aba1-3ef6f02620eb-hosts-file\") pod \"node-resolver-k8tfc\" (UID: \"c357f393-afff-4968-aba1-3ef6f02620eb\") " pod="openshift-dns/node-resolver-k8tfc"
Sep 10 19:15:47.100653 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.100549    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0df00d14-a6d5-4b0f-895b-2f6cacbe131c-host-slash\") pod \"iptables-alerter-89qpn\" (UID: \"0df00d14-a6d5-4b0f-895b-2f6cacbe131c\") " pod="openshift-network-operator/iptables-alerter-89qpn"
Sep 10 19:15:47.100653 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.100572    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-var-lib-openvswitch\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.100653 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.100588    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-run-openvswitch\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.100653 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.100604    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qb7k\" (UniqueName: \"kubernetes.io/projected/2826f8ec-78f6-4cab-879e-47a21854ed5a-kube-api-access-2qb7k\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.100653 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.100622    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-host-slash\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.100653 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.100637    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-etc-openvswitch\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.100653 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.100661    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-node-log\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.101394 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.100685    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-host-cni-netd\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.101453 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.101390    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qcw6c"
Sep 10 19:15:47.104150 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.103268    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 10 19:15:47.104150 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.103669    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-x9bc4\""
Sep 10 19:15:47.104150 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.103790    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 10 19:15:47.104150 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.103858    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 10 19:15:47.104150 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.104027    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 10 19:15:47.104150 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.104046    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 10 19:15:47.105209 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.104773    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.106498 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.106478    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 10 19:15:47.106751 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.106736    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-nr8q4\""
Sep 10 19:15:47.108425 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.108405    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dns2r"
Sep 10 19:15:47.108536 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:47.108495    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dns2r" podUID="6a81beec-be63-4b8b-9999-b92fcda80550"
Sep 10 19:15:47.111797 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.111645    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjlbx"
Sep 10 19:15:47.111797 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:47.111734    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qjlbx" podUID="af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0"
Sep 10 19:15:47.115385 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.115360    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cllmw"
Sep 10 19:15:47.118683 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.117660    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 10 19:15:47.118683 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.118054    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 10 19:15:47.118683 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.118384    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jg9j2\""
Sep 10 19:15:47.119029 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.118741    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.120858 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.120635    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5d8ss\""
Sep 10 19:15:47.121985 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.121196    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 10 19:15:47.121985 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.121414    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 10 19:15:47.153750 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.153707    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ghb8w"
Sep 10 19:15:47.162198 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.161750    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ghb8w"
Sep 10 19:15:47.189404 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.189375    2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 10 19:15:47.201107 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.201068    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0df00d14-a6d5-4b0f-895b-2f6cacbe131c-host-slash\") pod \"iptables-alerter-89qpn\" (UID: \"0df00d14-a6d5-4b0f-895b-2f6cacbe131c\") " pod="openshift-network-operator/iptables-alerter-89qpn"
Sep 10 19:15:47.201268 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.201120    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-var-lib-openvswitch\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.201268 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.201151    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-host-cni-netd\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.201268 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.201180    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b47634b2-779d-426a-8a6b-c74a246654d4-konnectivity-ca\") pod \"konnectivity-agent-cllmw\" (UID: \"b47634b2-779d-426a-8a6b-c74a246654d4\") " pod="kube-system/konnectivity-agent-cllmw"
Sep 10 19:15:47.201268 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.201205    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/affdbda0-e684-4e0d-a9e0-934646c95fe3-host\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.201268 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.201233    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/89ea61c7-adeb-40e7-8a14-22c8d11a5262-registration-dir\") pod \"aws-ebs-csi-driver-node-fwrt5\" (UID: \"89ea61c7-adeb-40e7-8a14-22c8d11a5262\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwrt5"
Sep 10 19:15:47.201268 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.201258    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-run-systemd\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.201450 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.201281    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-host-var-lib-cni-multus\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.201450 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.201308    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcbzc\" (UniqueName: \"kubernetes.io/projected/af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0-kube-api-access-dcbzc\") pod \"network-metrics-daemon-qjlbx\" (UID: \"af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0\") " pod="openshift-multus/network-metrics-daemon-qjlbx"
Sep 10 19:15:47.201450 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.201331    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/affdbda0-e684-4e0d-a9e0-934646c95fe3-etc-modprobe-d\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.201450 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.201358    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/89ea61c7-adeb-40e7-8a14-22c8d11a5262-device-dir\") pod \"aws-ebs-csi-driver-node-fwrt5\" (UID: \"89ea61c7-adeb-40e7-8a14-22c8d11a5262\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwrt5"
Sep 10 19:15:47.201450 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.201385    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-run-ovn\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.201450 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.201407    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-host-cni-bin\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.201450 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.201429    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2826f8ec-78f6-4cab-879e-47a21854ed5a-ovn-node-metrics-cert\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.201707 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.201452    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/acd262af-057c-4bbf-93e3-faa020c1825c-kubelet-config\") pod \"global-pull-secret-syncer-mkkzj\" (UID: \"acd262af-057c-4bbf-93e3-faa020c1825c\") " pod="kube-system/global-pull-secret-syncer-mkkzj"
Sep 10 19:15:47.201707 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.201519    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/affdbda0-e684-4e0d-a9e0-934646c95fe3-var-lib-kubelet\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.201707 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.201550    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-systemd-units\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.201707 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.201580    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.201707 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.201610    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmbxc\" (UniqueName: \"kubernetes.io/projected/6a81beec-be63-4b8b-9999-b92fcda80550-kube-api-access-fmbxc\") pod \"network-check-target-dns2r\" (UID: \"6a81beec-be63-4b8b-9999-b92fcda80550\") " pod="openshift-network-diagnostics/network-check-target-dns2r"
Sep 10 19:15:47.201707 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.201632    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0df00d14-a6d5-4b0f-895b-2f6cacbe131c-host-slash\") pod \"iptables-alerter-89qpn\" (UID: \"0df00d14-a6d5-4b0f-895b-2f6cacbe131c\") " pod="openshift-network-operator/iptables-alerter-89qpn"
Sep 10 19:15:47.201707 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.201636    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/affdbda0-e684-4e0d-a9e0-934646c95fe3-etc-sysctl-conf\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.201707 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.201663    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/affdbda0-e684-4e0d-a9e0-934646c95fe3-run\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.201707 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.201686    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-var-lib-openvswitch\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.202202 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.201750    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-host-cni-netd\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.202202 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.201759    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/89ea61c7-adeb-40e7-8a14-22c8d11a5262-device-dir\") pod \"aws-ebs-csi-driver-node-fwrt5\" (UID: \"89ea61c7-adeb-40e7-8a14-22c8d11a5262\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwrt5"
Sep 10 19:15:47.202202 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.201803    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-run-ovn\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.202202 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.201851    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/89ea61c7-adeb-40e7-8a14-22c8d11a5262-registration-dir\") pod \"aws-ebs-csi-driver-node-fwrt5\" (UID: \"89ea61c7-adeb-40e7-8a14-22c8d11a5262\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwrt5"
Sep 10 19:15:47.202202 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.201870    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-host-cni-bin\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.202202 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.201688    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2-system-cni-dir\") pod \"multus-additional-cni-plugins-qcw6c\" (UID: \"630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2\") " pod="openshift-multus/multus-additional-cni-plugins-qcw6c"
Sep 10 19:15:47.202202 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.201906    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-run-systemd\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.202202 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.201907    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-multus-cni-dir\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.202202 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.201953    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/808f5f6a-9f93-4350-af47-8d19c19b8658-cni-binary-copy\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.202202 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.201960    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-systemd-units\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.202202 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.201994    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c357f393-afff-4968-aba1-3ef6f02620eb-tmp-dir\") pod \"node-resolver-k8tfc\" (UID: \"c357f393-afff-4968-aba1-3ef6f02620eb\") " pod="openshift-dns/node-resolver-k8tfc"
Sep 10 19:15:47.202202 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.202033    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/98e70b32-791a-4a50-b558-5337fc78d24c-host\") pod \"node-ca-zdnmx\" (UID: \"98e70b32-791a-4a50-b558-5337fc78d24c\") " pod="openshift-image-registry/node-ca-zdnmx"
Sep 10 19:15:47.202202 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.202057    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2-cni-binary-copy\") pod \"multus-additional-cni-plugins-qcw6c\" (UID: \"630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2\") " pod="openshift-multus/multus-additional-cni-plugins-qcw6c"
Sep 10 19:15:47.202202 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.202079    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-host-run-netns\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.202202 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.202100    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-etc-kubernetes\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.202202 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.202137    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4wpc\" (UniqueName: \"kubernetes.io/projected/808f5f6a-9f93-4350-af47-8d19c19b8658-kube-api-access-l4wpc\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.202202 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.202159    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/89ea61c7-adeb-40e7-8a14-22c8d11a5262-socket-dir\") pod \"aws-ebs-csi-driver-node-fwrt5\" (UID: \"89ea61c7-adeb-40e7-8a14-22c8d11a5262\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwrt5"
Sep 10 19:15:47.203122 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.202197    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/89ea61c7-adeb-40e7-8a14-22c8d11a5262-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fwrt5\" (UID: \"89ea61c7-adeb-40e7-8a14-22c8d11a5262\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwrt5"
Sep 10 19:15:47.203122 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.202228    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/89ea61c7-adeb-40e7-8a14-22c8d11a5262-etc-selinux\") pod \"aws-ebs-csi-driver-node-fwrt5\" (UID: \"89ea61c7-adeb-40e7-8a14-22c8d11a5262\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwrt5"
Sep 10 19:15:47.203122 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.202268    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/affdbda0-e684-4e0d-a9e0-934646c95fe3-etc-sysconfig\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.203122 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.202291    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c357f393-afff-4968-aba1-3ef6f02620eb-hosts-file\") pod \"node-resolver-k8tfc\" (UID: \"c357f393-afff-4968-aba1-3ef6f02620eb\") " pod="openshift-dns/node-resolver-k8tfc"
Sep 10 19:15:47.203122 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.202314    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-run-openvswitch\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.203122 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.202354    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qb7k\" (UniqueName: \"kubernetes.io/projected/2826f8ec-78f6-4cab-879e-47a21854ed5a-kube-api-access-2qb7k\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.203122 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.202375    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqm27\" (UniqueName: \"kubernetes.io/projected/affdbda0-e684-4e0d-a9e0-934646c95fe3-kube-api-access-hqm27\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.203122 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.202410    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/98e70b32-791a-4a50-b558-5337fc78d24c-serviceca\") pod \"node-ca-zdnmx\" (UID: \"98e70b32-791a-4a50-b558-5337fc78d24c\") " pod="openshift-image-registry/node-ca-zdnmx"
Sep 10 19:15:47.203122 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.202431    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-host-slash\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.203122 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.202441    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.203122 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.202481    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/acd262af-057c-4bbf-93e3-faa020c1825c-kubelet-config\") pod \"global-pull-secret-syncer-mkkzj\" (UID: \"acd262af-057c-4bbf-93e3-faa020c1825c\") " pod="kube-system/global-pull-secret-syncer-mkkzj"
Sep 10 19:15:47.203122 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.202452    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-etc-openvswitch\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.203122 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.202541    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-node-log\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.203122 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.202572    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2826f8ec-78f6-4cab-879e-47a21854ed5a-ovnkube-script-lib\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.203122 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.202579    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/89ea61c7-adeb-40e7-8a14-22c8d11a5262-etc-selinux\") pod \"aws-ebs-csi-driver-node-fwrt5\" (UID: \"89ea61c7-adeb-40e7-8a14-22c8d11a5262\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwrt5"
Sep 10 19:15:47.203122 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.202587    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/89ea61c7-adeb-40e7-8a14-22c8d11a5262-socket-dir\") pod \"aws-ebs-csi-driver-node-fwrt5\" (UID: \"89ea61c7-adeb-40e7-8a14-22c8d11a5262\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwrt5"
Sep 10 19:15:47.203122 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.202488    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-etc-openvswitch\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.204030 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.202613    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/affdbda0-e684-4e0d-a9e0-934646c95fe3-etc-sysctl-d\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.204030 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.202683    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c357f393-afff-4968-aba1-3ef6f02620eb-hosts-file\") pod \"node-resolver-k8tfc\" (UID: \"c357f393-afff-4968-aba1-3ef6f02620eb\") " pod="openshift-dns/node-resolver-k8tfc"
Sep 10 19:15:47.204030 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.202734    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-run-openvswitch\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.204030 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.202743    2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 10 19:15:47.204030 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.202801    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c357f393-afff-4968-aba1-3ef6f02620eb-tmp-dir\") pod \"node-resolver-k8tfc\" (UID: \"c357f393-afff-4968-aba1-3ef6f02620eb\") " pod="openshift-dns/node-resolver-k8tfc"
Sep 10 19:15:47.204030 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.202816    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-host-slash\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.204030 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.202440    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/89ea61c7-adeb-40e7-8a14-22c8d11a5262-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fwrt5\" (UID: \"89ea61c7-adeb-40e7-8a14-22c8d11a5262\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwrt5"
Sep 10 19:15:47.204030 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.202923    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-node-log\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.204030 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.202924    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/affdbda0-e684-4e0d-a9e0-934646c95fe3-etc-systemd\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.204030 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.202963    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/affdbda0-e684-4e0d-a9e0-934646c95fe3-etc-tuned\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.204030 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.203039    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tl9kl\" (UniqueName: \"kubernetes.io/projected/98e70b32-791a-4a50-b558-5337fc78d24c-kube-api-access-tl9kl\") pod \"node-ca-zdnmx\" (UID: \"98e70b32-791a-4a50-b558-5337fc78d24c\") " pod="openshift-image-registry/node-ca-zdnmx"
Sep 10 19:15:47.204030 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.203113    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0-metrics-certs\") pod \"network-metrics-daemon-qjlbx\" (UID: \"af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0\") " pod="openshift-multus/network-metrics-daemon-qjlbx"
Sep 10 19:15:47.204030 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.203142    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qcw6c\" (UID: \"630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2\") " pod="openshift-multus/multus-additional-cni-plugins-qcw6c"
Sep 10 19:15:47.204030 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.203169    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-host-run-k8s-cni-cncf-io\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.204030 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.203192    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-host-var-lib-cni-bin\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.204030 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.203225    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-host-run-multus-certs\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.204030 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.203207    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2826f8ec-78f6-4cab-879e-47a21854ed5a-ovnkube-script-lib\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.204993 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.203245    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0df00d14-a6d5-4b0f-895b-2f6cacbe131c-iptables-alerter-script\") pod \"iptables-alerter-89qpn\" (UID: \"0df00d14-a6d5-4b0f-895b-2f6cacbe131c\") " pod="openshift-network-operator/iptables-alerter-89qpn"
Sep 10 19:15:47.204993 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.203282    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2s6c2\" (UniqueName: \"kubernetes.io/projected/0df00d14-a6d5-4b0f-895b-2f6cacbe131c-kube-api-access-2s6c2\") pod \"iptables-alerter-89qpn\" (UID: \"0df00d14-a6d5-4b0f-895b-2f6cacbe131c\") " pod="openshift-network-operator/iptables-alerter-89qpn"
Sep 10 19:15:47.204993 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.203317    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-log-socket\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.204993 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.203357    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2826f8ec-78f6-4cab-879e-47a21854ed5a-ovnkube-config\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.204993 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.203391    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-log-socket\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.204993 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.203410    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/affdbda0-e684-4e0d-a9e0-934646c95fe3-lib-modules\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.204993 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.203428    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-host-var-lib-kubelet\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.204993 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.203445    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-host-run-ovn-kubernetes\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.204993 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.203485    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/89ea61c7-adeb-40e7-8a14-22c8d11a5262-sys-fs\") pod \"aws-ebs-csi-driver-node-fwrt5\" (UID: \"89ea61c7-adeb-40e7-8a14-22c8d11a5262\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwrt5"
Sep 10 19:15:47.204993 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.203502    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lp2p\" (UniqueName: \"kubernetes.io/projected/89ea61c7-adeb-40e7-8a14-22c8d11a5262-kube-api-access-4lp2p\") pod \"aws-ebs-csi-driver-node-fwrt5\" (UID: \"89ea61c7-adeb-40e7-8a14-22c8d11a5262\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwrt5"
Sep 10 19:15:47.204993 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.203519    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-host-run-netns\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.204993 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.203537    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qcw6c\" (UID: \"630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2\") " pod="openshift-multus/multus-additional-cni-plugins-qcw6c"
Sep 10 19:15:47.204993 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.203553    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nzgq\" (UniqueName: \"kubernetes.io/projected/630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2-kube-api-access-6nzgq\") pod \"multus-additional-cni-plugins-qcw6c\" (UID: \"630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2\") " pod="openshift-multus/multus-additional-cni-plugins-qcw6c"
Sep 10 19:15:47.204993 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.203569    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-cnibin\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.204993 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.203605    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-hostroot\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.204993 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.203623    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vb4xk\" (UniqueName: \"kubernetes.io/projected/c357f393-afff-4968-aba1-3ef6f02620eb-kube-api-access-vb4xk\") pod \"node-resolver-k8tfc\" (UID: \"c357f393-afff-4968-aba1-3ef6f02620eb\") " pod="openshift-dns/node-resolver-k8tfc"
Sep 10 19:15:47.204993 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.203626    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0df00d14-a6d5-4b0f-895b-2f6cacbe131c-iptables-alerter-script\") pod \"iptables-alerter-89qpn\" (UID: \"0df00d14-a6d5-4b0f-895b-2f6cacbe131c\") " pod="openshift-network-operator/iptables-alerter-89qpn"
Sep 10 19:15:47.205935 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.203640    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-host-kubelet\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.205935 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.203657    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/affdbda0-e684-4e0d-a9e0-934646c95fe3-tmp\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.205935 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.203670    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-host-run-ovn-kubernetes\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.205935 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.203671    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-system-cni-dir\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.205935 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.203706    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/89ea61c7-adeb-40e7-8a14-22c8d11a5262-sys-fs\") pod \"aws-ebs-csi-driver-node-fwrt5\" (UID: \"89ea61c7-adeb-40e7-8a14-22c8d11a5262\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwrt5"
Sep 10 19:15:47.205935 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.203732    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-multus-socket-dir-parent\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.205935 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.203754    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-multus-conf-dir\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.205935 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.203782    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/808f5f6a-9f93-4350-af47-8d19c19b8658-multus-daemon-config\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.205935 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.203799    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2826f8ec-78f6-4cab-879e-47a21854ed5a-env-overrides\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.205935 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.203816    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/acd262af-057c-4bbf-93e3-faa020c1825c-dbus\") pod \"global-pull-secret-syncer-mkkzj\" (UID: \"acd262af-057c-4bbf-93e3-faa020c1825c\") " pod="kube-system/global-pull-secret-syncer-mkkzj"
Sep 10 19:15:47.205935 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.203961    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/acd262af-057c-4bbf-93e3-faa020c1825c-dbus\") pod \"global-pull-secret-syncer-mkkzj\" (UID: \"acd262af-057c-4bbf-93e3-faa020c1825c\") " pod="kube-system/global-pull-secret-syncer-mkkzj"
Sep 10 19:15:47.205935 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.203989    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-host-kubelet\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.205935 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.204308    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2826f8ec-78f6-4cab-879e-47a21854ed5a-ovnkube-config\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.205935 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.204348    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/acd262af-057c-4bbf-93e3-faa020c1825c-original-pull-secret\") pod \"global-pull-secret-syncer-mkkzj\" (UID: \"acd262af-057c-4bbf-93e3-faa020c1825c\") " pod="kube-system/global-pull-secret-syncer-mkkzj"
Sep 10 19:15:47.205935 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.204377    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/affdbda0-e684-4e0d-a9e0-934646c95fe3-etc-kubernetes\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.205935 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.204419    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/affdbda0-e684-4e0d-a9e0-934646c95fe3-sys\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.205935 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.204443    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2-cnibin\") pod \"multus-additional-cni-plugins-qcw6c\" (UID: \"630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2\") " pod="openshift-multus/multus-additional-cni-plugins-qcw6c"
Sep 10 19:15:47.206915 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.204470    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qcw6c\" (UID: \"630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2\") " pod="openshift-multus/multus-additional-cni-plugins-qcw6c"
Sep 10 19:15:47.206915 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.204498    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b47634b2-779d-426a-8a6b-c74a246654d4-agent-certs\") pod \"konnectivity-agent-cllmw\" (UID: \"b47634b2-779d-426a-8a6b-c74a246654d4\") " pod="kube-system/konnectivity-agent-cllmw"
Sep 10 19:15:47.206915 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.204580    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2826f8ec-78f6-4cab-879e-47a21854ed5a-env-overrides\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.206915 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:47.204679    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:47.206915 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:47.204763    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acd262af-057c-4bbf-93e3-faa020c1825c-original-pull-secret podName:acd262af-057c-4bbf-93e3-faa020c1825c nodeName:}" failed. No retries permitted until 2026-09-10 19:15:47.70474079 +0000 UTC m=+2.924589277 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/acd262af-057c-4bbf-93e3-faa020c1825c-original-pull-secret") pod "global-pull-secret-syncer-mkkzj" (UID: "acd262af-057c-4bbf-93e3-faa020c1825c") : object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:47.206915 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.204972    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2826f8ec-78f6-4cab-879e-47a21854ed5a-host-run-netns\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.208772 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.208430    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2826f8ec-78f6-4cab-879e-47a21854ed5a-ovn-node-metrics-cert\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.215210 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.213636    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vb4xk\" (UniqueName: \"kubernetes.io/projected/c357f393-afff-4968-aba1-3ef6f02620eb-kube-api-access-vb4xk\") pod \"node-resolver-k8tfc\" (UID: \"c357f393-afff-4968-aba1-3ef6f02620eb\") " pod="openshift-dns/node-resolver-k8tfc"
Sep 10 19:15:47.215210 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.214118    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2s6c2\" (UniqueName: \"kubernetes.io/projected/0df00d14-a6d5-4b0f-895b-2f6cacbe131c-kube-api-access-2s6c2\") pod \"iptables-alerter-89qpn\" (UID: \"0df00d14-a6d5-4b0f-895b-2f6cacbe131c\") " pod="openshift-network-operator/iptables-alerter-89qpn"
Sep 10 19:15:47.217193 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.217169    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lp2p\" (UniqueName: \"kubernetes.io/projected/89ea61c7-adeb-40e7-8a14-22c8d11a5262-kube-api-access-4lp2p\") pod \"aws-ebs-csi-driver-node-fwrt5\" (UID: \"89ea61c7-adeb-40e7-8a14-22c8d11a5262\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwrt5"
Sep 10 19:15:47.222266 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.222193    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qb7k\" (UniqueName: \"kubernetes.io/projected/2826f8ec-78f6-4cab-879e-47a21854ed5a-kube-api-access-2qb7k\") pod \"ovnkube-node-8r66s\" (UID: \"2826f8ec-78f6-4cab-879e-47a21854ed5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.239254 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.239210    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-20.ec2.internal" event={"ID":"e0bff5a7795653f306d187693d0b9c36","Type":"ContainerStarted","Data":"8f0aedb33dd27ba4efbf562bd02c7a0f75955fb7a943cf4fac67e2b27fe38823"}
Sep 10 19:15:47.305234 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.305200    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-host-run-k8s-cni-cncf-io\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.305234 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.305243    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-host-var-lib-cni-bin\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.305499 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.305261    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-host-run-multus-certs\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.305499 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.305464    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/affdbda0-e684-4e0d-a9e0-934646c95fe3-lib-modules\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.310063 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.310032    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-host-var-lib-kubelet\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.310253 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.310121    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qcw6c\" (UID: \"630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2\") " pod="openshift-multus/multus-additional-cni-plugins-qcw6c"
Sep 10 19:15:47.310253 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.310158    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nzgq\" (UniqueName: \"kubernetes.io/projected/630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2-kube-api-access-6nzgq\") pod \"multus-additional-cni-plugins-qcw6c\" (UID: \"630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2\") " pod="openshift-multus/multus-additional-cni-plugins-qcw6c"
Sep 10 19:15:47.310253 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.310190    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-cnibin\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.310253 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.310231    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-hostroot\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.310467 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.310264    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/affdbda0-e684-4e0d-a9e0-934646c95fe3-tmp\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.310467 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.310305    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-system-cni-dir\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.310467 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.310338    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-multus-socket-dir-parent\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.310467 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.310370    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-multus-conf-dir\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.310467 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.310398    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/808f5f6a-9f93-4350-af47-8d19c19b8658-multus-daemon-config\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.310467 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.310451    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/affdbda0-e684-4e0d-a9e0-934646c95fe3-etc-kubernetes\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.310737 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.310478    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/affdbda0-e684-4e0d-a9e0-934646c95fe3-sys\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.310737 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.310509    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2-cnibin\") pod \"multus-additional-cni-plugins-qcw6c\" (UID: \"630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2\") " pod="openshift-multus/multus-additional-cni-plugins-qcw6c"
Sep 10 19:15:47.310737 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.310538    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qcw6c\" (UID: \"630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2\") " pod="openshift-multus/multus-additional-cni-plugins-qcw6c"
Sep 10 19:15:47.310737 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.310574    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b47634b2-779d-426a-8a6b-c74a246654d4-agent-certs\") pod \"konnectivity-agent-cllmw\" (UID: \"b47634b2-779d-426a-8a6b-c74a246654d4\") " pod="kube-system/konnectivity-agent-cllmw"
Sep 10 19:15:47.310737 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.310612    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b47634b2-779d-426a-8a6b-c74a246654d4-konnectivity-ca\") pod \"konnectivity-agent-cllmw\" (UID: \"b47634b2-779d-426a-8a6b-c74a246654d4\") " pod="kube-system/konnectivity-agent-cllmw"
Sep 10 19:15:47.310737 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.310635    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/affdbda0-e684-4e0d-a9e0-934646c95fe3-host\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.310737 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.310668    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-host-var-lib-cni-multus\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.310737 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.310702    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcbzc\" (UniqueName: \"kubernetes.io/projected/af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0-kube-api-access-dcbzc\") pod \"network-metrics-daemon-qjlbx\" (UID: \"af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0\") " pod="openshift-multus/network-metrics-daemon-qjlbx"
Sep 10 19:15:47.310737 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.310734    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/affdbda0-e684-4e0d-a9e0-934646c95fe3-etc-modprobe-d\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.311274 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.310783    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/affdbda0-e684-4e0d-a9e0-934646c95fe3-var-lib-kubelet\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.311274 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.310813    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmbxc\" (UniqueName: \"kubernetes.io/projected/6a81beec-be63-4b8b-9999-b92fcda80550-kube-api-access-fmbxc\") pod \"network-check-target-dns2r\" (UID: \"6a81beec-be63-4b8b-9999-b92fcda80550\") " pod="openshift-network-diagnostics/network-check-target-dns2r"
Sep 10 19:15:47.311274 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.310873    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/affdbda0-e684-4e0d-a9e0-934646c95fe3-etc-sysctl-conf\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.311274 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.310904    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/affdbda0-e684-4e0d-a9e0-934646c95fe3-run\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.311274 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.310935    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2-system-cni-dir\") pod \"multus-additional-cni-plugins-qcw6c\" (UID: \"630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2\") " pod="openshift-multus/multus-additional-cni-plugins-qcw6c"
Sep 10 19:15:47.311274 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.310964    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-multus-cni-dir\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.311274 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.310995    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/808f5f6a-9f93-4350-af47-8d19c19b8658-cni-binary-copy\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.311274 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.311026    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/98e70b32-791a-4a50-b558-5337fc78d24c-host\") pod \"node-ca-zdnmx\" (UID: \"98e70b32-791a-4a50-b558-5337fc78d24c\") " pod="openshift-image-registry/node-ca-zdnmx"
Sep 10 19:15:47.311274 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.311056    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2-cni-binary-copy\") pod \"multus-additional-cni-plugins-qcw6c\" (UID: \"630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2\") " pod="openshift-multus/multus-additional-cni-plugins-qcw6c"
Sep 10 19:15:47.311274 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.311093    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-host-run-netns\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.311274 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.311122    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-etc-kubernetes\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.311274 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.311151    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4wpc\" (UniqueName: \"kubernetes.io/projected/808f5f6a-9f93-4350-af47-8d19c19b8658-kube-api-access-l4wpc\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.311274 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.311180    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/affdbda0-e684-4e0d-a9e0-934646c95fe3-etc-sysconfig\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.311274 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.311217    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqm27\" (UniqueName: \"kubernetes.io/projected/affdbda0-e684-4e0d-a9e0-934646c95fe3-kube-api-access-hqm27\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.311274 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.311247    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/98e70b32-791a-4a50-b558-5337fc78d24c-serviceca\") pod \"node-ca-zdnmx\" (UID: \"98e70b32-791a-4a50-b558-5337fc78d24c\") " pod="openshift-image-registry/node-ca-zdnmx"
Sep 10 19:15:47.312001 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.311286    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/affdbda0-e684-4e0d-a9e0-934646c95fe3-etc-sysctl-d\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.312001 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.311318    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/affdbda0-e684-4e0d-a9e0-934646c95fe3-etc-systemd\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.312001 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.311345    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/affdbda0-e684-4e0d-a9e0-934646c95fe3-etc-tuned\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.312001 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.311376    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tl9kl\" (UniqueName: \"kubernetes.io/projected/98e70b32-791a-4a50-b558-5337fc78d24c-kube-api-access-tl9kl\") pod \"node-ca-zdnmx\" (UID: \"98e70b32-791a-4a50-b558-5337fc78d24c\") " pod="openshift-image-registry/node-ca-zdnmx"
Sep 10 19:15:47.312001 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.311408    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0-metrics-certs\") pod \"network-metrics-daemon-qjlbx\" (UID: \"af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0\") " pod="openshift-multus/network-metrics-daemon-qjlbx"
Sep 10 19:15:47.312001 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.311440    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qcw6c\" (UID: \"630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2\") " pod="openshift-multus/multus-additional-cni-plugins-qcw6c"
Sep 10 19:15:47.312283 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.312135    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qcw6c\" (UID: \"630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2\") " pod="openshift-multus/multus-additional-cni-plugins-qcw6c"
Sep 10 19:15:47.312283 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.305785    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-host-var-lib-cni-bin\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.312283 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.312244    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-host-var-lib-kubelet\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.312283 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.309971    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/affdbda0-e684-4e0d-a9e0-934646c95fe3-lib-modules\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.312737 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.312713    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qcw6c\" (UID: \"630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2\") " pod="openshift-multus/multus-additional-cni-plugins-qcw6c"
Sep 10 19:15:47.312836 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.305366    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-host-run-multus-certs\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.313208 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.313184    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-cnibin\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.313310 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.313245    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-hostroot\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.316144 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.315133    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-host-run-k8s-cni-cncf-io\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.316144 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.315324    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/affdbda0-e684-4e0d-a9e0-934646c95fe3-etc-sysctl-conf\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.316144 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.315465    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-system-cni-dir\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.316144 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.315502    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2-system-cni-dir\") pod \"multus-additional-cni-plugins-qcw6c\" (UID: \"630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2\") " pod="openshift-multus/multus-additional-cni-plugins-qcw6c"
Sep 10 19:15:47.316144 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.315541    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-multus-socket-dir-parent\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.316144 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.315628    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-multus-cni-dir\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.316144 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.315654    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/affdbda0-e684-4e0d-a9e0-934646c95fe3-run\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.316144 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.315676    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/affdbda0-e684-4e0d-a9e0-934646c95fe3-host\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.316144 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.315722    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/affdbda0-e684-4e0d-a9e0-934646c95fe3-sys\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.316144 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.315733    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/affdbda0-e684-4e0d-a9e0-934646c95fe3-etc-kubernetes\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.316144 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.315914    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qcw6c\" (UID: \"630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2\") " pod="openshift-multus/multus-additional-cni-plugins-qcw6c"
Sep 10 19:15:47.316760 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.316210    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-etc-kubernetes\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.316760 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.316309    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2-cni-binary-copy\") pod \"multus-additional-cni-plugins-qcw6c\" (UID: \"630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2\") " pod="openshift-multus/multus-additional-cni-plugins-qcw6c"
Sep 10 19:15:47.316760 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.316338    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b47634b2-779d-426a-8a6b-c74a246654d4-konnectivity-ca\") pod \"konnectivity-agent-cllmw\" (UID: \"b47634b2-779d-426a-8a6b-c74a246654d4\") " pod="kube-system/konnectivity-agent-cllmw"
Sep 10 19:15:47.316760 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.316593    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2-cnibin\") pod \"multus-additional-cni-plugins-qcw6c\" (UID: \"630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2\") " pod="openshift-multus/multus-additional-cni-plugins-qcw6c"
Sep 10 19:15:47.316760 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.316626    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-host-run-netns\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.316760 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.316752    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/808f5f6a-9f93-4350-af47-8d19c19b8658-cni-binary-copy\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.317117 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.316780    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/affdbda0-e684-4e0d-a9e0-934646c95fe3-etc-modprobe-d\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.317117 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.317067    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/affdbda0-e684-4e0d-a9e0-934646c95fe3-tmp\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.317220 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.317120    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/affdbda0-e684-4e0d-a9e0-934646c95fe3-etc-systemd\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.317396 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.317353    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/affdbda0-e684-4e0d-a9e0-934646c95fe3-etc-sysctl-d\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.318612 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:47.317615    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:47.318612 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.317667    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-host-var-lib-cni-multus\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.318612 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.317786    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/affdbda0-e684-4e0d-a9e0-934646c95fe3-var-lib-kubelet\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.318612 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.317817    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/808f5f6a-9f93-4350-af47-8d19c19b8658-multus-conf-dir\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.318612 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:47.317942    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0-metrics-certs podName:af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0 nodeName:}" failed. No retries permitted until 2026-09-10 19:15:47.817909668 +0000 UTC m=+3.037758147 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0-metrics-certs") pod "network-metrics-daemon-qjlbx" (UID: "af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:47.318612 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.317939    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/affdbda0-e684-4e0d-a9e0-934646c95fe3-etc-sysconfig\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.318612 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.318025    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/98e70b32-791a-4a50-b558-5337fc78d24c-host\") pod \"node-ca-zdnmx\" (UID: \"98e70b32-791a-4a50-b558-5337fc78d24c\") " pod="openshift-image-registry/node-ca-zdnmx"
Sep 10 19:15:47.318612 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.318033    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/98e70b32-791a-4a50-b558-5337fc78d24c-serviceca\") pod \"node-ca-zdnmx\" (UID: \"98e70b32-791a-4a50-b558-5337fc78d24c\") " pod="openshift-image-registry/node-ca-zdnmx"
Sep 10 19:15:47.318612 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.318499    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/808f5f6a-9f93-4350-af47-8d19c19b8658-multus-daemon-config\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.320992 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.320962    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/affdbda0-e684-4e0d-a9e0-934646c95fe3-etc-tuned\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.322214 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.322156    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b47634b2-779d-426a-8a6b-c74a246654d4-agent-certs\") pod \"konnectivity-agent-cllmw\" (UID: \"b47634b2-779d-426a-8a6b-c74a246654d4\") " pod="kube-system/konnectivity-agent-cllmw"
Sep 10 19:15:47.323937 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.323908    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nzgq\" (UniqueName: \"kubernetes.io/projected/630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2-kube-api-access-6nzgq\") pod \"multus-additional-cni-plugins-qcw6c\" (UID: \"630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2\") " pod="openshift-multus/multus-additional-cni-plugins-qcw6c"
Sep 10 19:15:47.324421 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:47.324396    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 19:15:47.324514 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:47.324425    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 19:15:47.324514 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:47.324441    2579 projected.go:196] Error preparing data for projected volume kube-api-access-fmbxc for pod openshift-network-diagnostics/network-check-target-dns2r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:47.324631 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:47.324535    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a81beec-be63-4b8b-9999-b92fcda80550-kube-api-access-fmbxc podName:6a81beec-be63-4b8b-9999-b92fcda80550 nodeName:}" failed. No retries permitted until 2026-09-10 19:15:47.824511853 +0000 UTC m=+3.044360336 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-fmbxc" (UniqueName: "kubernetes.io/projected/6a81beec-be63-4b8b-9999-b92fcda80550-kube-api-access-fmbxc") pod "network-check-target-dns2r" (UID: "6a81beec-be63-4b8b-9999-b92fcda80550") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:47.327378 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.327301    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4wpc\" (UniqueName: \"kubernetes.io/projected/808f5f6a-9f93-4350-af47-8d19c19b8658-kube-api-access-l4wpc\") pod \"multus-47rkn\" (UID: \"808f5f6a-9f93-4350-af47-8d19c19b8658\") " pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.330134 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.330104    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqm27\" (UniqueName: \"kubernetes.io/projected/affdbda0-e684-4e0d-a9e0-934646c95fe3-kube-api-access-hqm27\") pod \"tuned-zd86j\" (UID: \"affdbda0-e684-4e0d-a9e0-934646c95fe3\") " pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.330242 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.330141    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tl9kl\" (UniqueName: \"kubernetes.io/projected/98e70b32-791a-4a50-b558-5337fc78d24c-kube-api-access-tl9kl\") pod \"node-ca-zdnmx\" (UID: \"98e70b32-791a-4a50-b558-5337fc78d24c\") " pod="openshift-image-registry/node-ca-zdnmx"
Sep 10 19:15:47.330242 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.330153    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcbzc\" (UniqueName: \"kubernetes.io/projected/af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0-kube-api-access-dcbzc\") pod \"network-metrics-daemon-qjlbx\" (UID: \"af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0\") " pod="openshift-multus/network-metrics-daemon-qjlbx"
Sep 10 19:15:47.403624 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.403589    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwrt5"
Sep 10 19:15:47.413606 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.413571    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-k8tfc"
Sep 10 19:15:47.426986 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.426928    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-89qpn"
Sep 10 19:15:47.434419 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.434389    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:15:47.444085 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.444053    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zdnmx"
Sep 10 19:15:47.454520 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.454314    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qcw6c"
Sep 10 19:15:47.466456 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.466417    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-47rkn"
Sep 10 19:15:47.478607 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.478532    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cllmw"
Sep 10 19:15:47.485721 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.485695    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zd86j"
Sep 10 19:15:47.715280 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.715242    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/acd262af-057c-4bbf-93e3-faa020c1825c-original-pull-secret\") pod \"global-pull-secret-syncer-mkkzj\" (UID: \"acd262af-057c-4bbf-93e3-faa020c1825c\") " pod="kube-system/global-pull-secret-syncer-mkkzj"
Sep 10 19:15:47.715465 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:47.715375    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:47.715465 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:47.715437    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acd262af-057c-4bbf-93e3-faa020c1825c-original-pull-secret podName:acd262af-057c-4bbf-93e3-faa020c1825c nodeName:}" failed. No retries permitted until 2026-09-10 19:15:48.71541916 +0000 UTC m=+3.935267643 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/acd262af-057c-4bbf-93e3-faa020c1825c-original-pull-secret") pod "global-pull-secret-syncer-mkkzj" (UID: "acd262af-057c-4bbf-93e3-faa020c1825c") : object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:47.916973 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.916930    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0-metrics-certs\") pod \"network-metrics-daemon-qjlbx\" (UID: \"af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0\") " pod="openshift-multus/network-metrics-daemon-qjlbx"
Sep 10 19:15:47.917149 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:47.917017    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmbxc\" (UniqueName: \"kubernetes.io/projected/6a81beec-be63-4b8b-9999-b92fcda80550-kube-api-access-fmbxc\") pod \"network-check-target-dns2r\" (UID: \"6a81beec-be63-4b8b-9999-b92fcda80550\") " pod="openshift-network-diagnostics/network-check-target-dns2r"
Sep 10 19:15:47.917149 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:47.917104    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:47.917149 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:47.917135    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 19:15:47.917149 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:47.917150    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 19:15:47.917325 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:47.917164    2579 projected.go:196] Error preparing data for projected volume kube-api-access-fmbxc for pod openshift-network-diagnostics/network-check-target-dns2r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:47.917325 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:47.917177    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0-metrics-certs podName:af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0 nodeName:}" failed. No retries permitted until 2026-09-10 19:15:48.917157487 +0000 UTC m=+4.137005948 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0-metrics-certs") pod "network-metrics-daemon-qjlbx" (UID: "af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:47.917325 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:47.917214    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a81beec-be63-4b8b-9999-b92fcda80550-kube-api-access-fmbxc podName:6a81beec-be63-4b8b-9999-b92fcda80550 nodeName:}" failed. No retries permitted until 2026-09-10 19:15:48.917188943 +0000 UTC m=+4.137037419 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-fmbxc" (UniqueName: "kubernetes.io/projected/6a81beec-be63-4b8b-9999-b92fcda80550-kube-api-access-fmbxc") pod "network-check-target-dns2r" (UID: "6a81beec-be63-4b8b-9999-b92fcda80550") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:48.170845 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:48.170735    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 19:10:47 +0000 UTC" deadline="2028-03-13 22:22:31.984867624 +0000 UTC"
Sep 10 19:15:48.170845 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:48.170788    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13203h6m43.814084428s"
Sep 10 19:15:48.233286 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:48.233242    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjlbx"
Sep 10 19:15:48.233479 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:48.233400    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qjlbx" podUID="af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0"
Sep 10 19:15:48.233899 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:48.233872    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dns2r"
Sep 10 19:15:48.233998 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:48.233967    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dns2r" podUID="6a81beec-be63-4b8b-9999-b92fcda80550"
Sep 10 19:15:48.325312 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:48.325275    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb47634b2_779d_426a_8a6b_c74a246654d4.slice/crio-8b6bed5170def2d6360503120dba66eb004f0053ef1ea0a8c61ff5e3205ac2ef WatchSource:0}: Error finding container 8b6bed5170def2d6360503120dba66eb004f0053ef1ea0a8c61ff5e3205ac2ef: Status 404 returned error can't find the container with id 8b6bed5170def2d6360503120dba66eb004f0053ef1ea0a8c61ff5e3205ac2ef
Sep 10 19:15:48.326065 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:48.325925    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0df00d14_a6d5_4b0f_895b_2f6cacbe131c.slice/crio-b1b3f9dd6e93f5376626b034a916a4def98aae4a7d73e4ba73140e04043cd05e WatchSource:0}: Error finding container b1b3f9dd6e93f5376626b034a916a4def98aae4a7d73e4ba73140e04043cd05e: Status 404 returned error can't find the container with id b1b3f9dd6e93f5376626b034a916a4def98aae4a7d73e4ba73140e04043cd05e
Sep 10 19:15:48.327137 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:48.326998    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc357f393_afff_4968_aba1_3ef6f02620eb.slice/crio-659da0d61423f60a990877987e51360a06dee5d45a66c55b194131976ecaf4ca WatchSource:0}: Error finding container 659da0d61423f60a990877987e51360a06dee5d45a66c55b194131976ecaf4ca: Status 404 returned error can't find the container with id 659da0d61423f60a990877987e51360a06dee5d45a66c55b194131976ecaf4ca
Sep 10 19:15:48.328041 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:48.327995    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98e70b32_791a_4a50_b558_5337fc78d24c.slice/crio-dbeb7d5295d849a3c5c8e7e9046750fd0d04eda22d29f559ae21aef6795fe6d4 WatchSource:0}: Error finding container dbeb7d5295d849a3c5c8e7e9046750fd0d04eda22d29f559ae21aef6795fe6d4: Status 404 returned error can't find the container with id dbeb7d5295d849a3c5c8e7e9046750fd0d04eda22d29f559ae21aef6795fe6d4
Sep 10 19:15:48.331541 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:48.331512    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod808f5f6a_9f93_4350_af47_8d19c19b8658.slice/crio-552c6cc5f1aaacdb4873a94ac21b9bf5711c0255e1e3eddb9e76319b216b8e28 WatchSource:0}: Error finding container 552c6cc5f1aaacdb4873a94ac21b9bf5711c0255e1e3eddb9e76319b216b8e28: Status 404 returned error can't find the container with id 552c6cc5f1aaacdb4873a94ac21b9bf5711c0255e1e3eddb9e76319b216b8e28
Sep 10 19:15:48.357486 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:48.356356    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaffdbda0_e684_4e0d_a9e0_934646c95fe3.slice/crio-25cd1d28d07828e3f90560719fcabf0ab504d2442d9922eb7fe0861073c796a8 WatchSource:0}: Error finding container 25cd1d28d07828e3f90560719fcabf0ab504d2442d9922eb7fe0861073c796a8: Status 404 returned error can't find the container with id 25cd1d28d07828e3f90560719fcabf0ab504d2442d9922eb7fe0861073c796a8
Sep 10 19:15:48.361835 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:15:48.361791    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2826f8ec_78f6_4cab_879e_47a21854ed5a.slice/crio-85ba1ad8e360c76d5fe3c7dc2df0b2a821c77a244ec88d12bac31d68ccfb256c WatchSource:0}: Error finding container 85ba1ad8e360c76d5fe3c7dc2df0b2a821c77a244ec88d12bac31d68ccfb256c: Status 404 returned error can't find the container with id 85ba1ad8e360c76d5fe3c7dc2df0b2a821c77a244ec88d12bac31d68ccfb256c
Sep 10 19:15:48.723329 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:48.723237    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/acd262af-057c-4bbf-93e3-faa020c1825c-original-pull-secret\") pod \"global-pull-secret-syncer-mkkzj\" (UID: \"acd262af-057c-4bbf-93e3-faa020c1825c\") " pod="kube-system/global-pull-secret-syncer-mkkzj"
Sep 10 19:15:48.723489 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:48.723387    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:48.723489 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:48.723469    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acd262af-057c-4bbf-93e3-faa020c1825c-original-pull-secret podName:acd262af-057c-4bbf-93e3-faa020c1825c nodeName:}" failed. No retries permitted until 2026-09-10 19:15:50.723450401 +0000 UTC m=+5.943298883 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/acd262af-057c-4bbf-93e3-faa020c1825c-original-pull-secret") pod "global-pull-secret-syncer-mkkzj" (UID: "acd262af-057c-4bbf-93e3-faa020c1825c") : object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:48.926403 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:48.925682    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0-metrics-certs\") pod \"network-metrics-daemon-qjlbx\" (UID: \"af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0\") " pod="openshift-multus/network-metrics-daemon-qjlbx"
Sep 10 19:15:48.926403 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:48.925803    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmbxc\" (UniqueName: \"kubernetes.io/projected/6a81beec-be63-4b8b-9999-b92fcda80550-kube-api-access-fmbxc\") pod \"network-check-target-dns2r\" (UID: \"6a81beec-be63-4b8b-9999-b92fcda80550\") " pod="openshift-network-diagnostics/network-check-target-dns2r"
Sep 10 19:15:48.926403 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:48.925967    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 19:15:48.926403 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:48.925988    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 19:15:48.926403 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:48.926001    2579 projected.go:196] Error preparing data for projected volume kube-api-access-fmbxc for pod openshift-network-diagnostics/network-check-target-dns2r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:48.926403 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:48.926072    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a81beec-be63-4b8b-9999-b92fcda80550-kube-api-access-fmbxc podName:6a81beec-be63-4b8b-9999-b92fcda80550 nodeName:}" failed. No retries permitted until 2026-09-10 19:15:50.926052959 +0000 UTC m=+6.145901434 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-fmbxc" (UniqueName: "kubernetes.io/projected/6a81beec-be63-4b8b-9999-b92fcda80550-kube-api-access-fmbxc") pod "network-check-target-dns2r" (UID: "6a81beec-be63-4b8b-9999-b92fcda80550") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:48.926403 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:48.926272    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:48.926403 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:48.926350    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0-metrics-certs podName:af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0 nodeName:}" failed. No retries permitted until 2026-09-10 19:15:50.926305966 +0000 UTC m=+6.146154450 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0-metrics-certs") pod "network-metrics-daemon-qjlbx" (UID: "af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:49.171517 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:49.171440    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 19:10:47 +0000 UTC" deadline="2028-05-15 15:15:29.359057306 +0000 UTC"
Sep 10 19:15:49.171517 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:49.171485    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14707h59m40.187577456s"
Sep 10 19:15:49.237255 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:49.236608    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkkzj"
Sep 10 19:15:49.237255 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:49.236761    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mkkzj" podUID="acd262af-057c-4bbf-93e3-faa020c1825c"
Sep 10 19:15:49.255782 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:49.255741    2579 generic.go:356] "Generic (PLEG): container finished" podID="dc403cc24af90cbf61a5edf297685b42" containerID="9d0ab9f247f07b2c33fbe23598038f6c32b83f95f366dddeeff2446f5044b652" exitCode=0
Sep 10 19:15:49.255990 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:49.255898    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-20.ec2.internal" event={"ID":"dc403cc24af90cbf61a5edf297685b42","Type":"ContainerDied","Data":"9d0ab9f247f07b2c33fbe23598038f6c32b83f95f366dddeeff2446f5044b652"}
Sep 10 19:15:49.264338 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:49.264265    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-20.ec2.internal" event={"ID":"e0bff5a7795653f306d187693d0b9c36","Type":"ContainerStarted","Data":"d216c5775115533a055eab1afcf285462e8535ddccdeae2cf770e417d2c20abc"}
Sep 10 19:15:49.268523 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:49.268495    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qcw6c" event={"ID":"630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2","Type":"ContainerStarted","Data":"fdd052c5620823d9dcb608d2d99aa90baec53bf71945af25c4ab4c5fe1df3679"}
Sep 10 19:15:49.271695 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:49.271656    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zd86j" event={"ID":"affdbda0-e684-4e0d-a9e0-934646c95fe3","Type":"ContainerStarted","Data":"25cd1d28d07828e3f90560719fcabf0ab504d2442d9922eb7fe0861073c796a8"}
Sep 10 19:15:49.286697 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:49.286659    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-47rkn" event={"ID":"808f5f6a-9f93-4350-af47-8d19c19b8658","Type":"ContainerStarted","Data":"552c6cc5f1aaacdb4873a94ac21b9bf5711c0255e1e3eddb9e76319b216b8e28"}
Sep 10 19:15:49.290238 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:49.290093    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-20.ec2.internal" podStartSLOduration=2.290058487 podStartE2EDuration="2.290058487s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:15:49.28846521 +0000 UTC m=+4.508313697" watchObservedRunningTime="2026-09-10 19:15:49.290058487 +0000 UTC m=+4.509906973"
Sep 10 19:15:49.300363 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:49.300091    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-89qpn" event={"ID":"0df00d14-a6d5-4b0f-895b-2f6cacbe131c","Type":"ContainerStarted","Data":"b1b3f9dd6e93f5376626b034a916a4def98aae4a7d73e4ba73140e04043cd05e"}
Sep 10 19:15:49.311319 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:49.311202    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8r66s" event={"ID":"2826f8ec-78f6-4cab-879e-47a21854ed5a","Type":"ContainerStarted","Data":"85ba1ad8e360c76d5fe3c7dc2df0b2a821c77a244ec88d12bac31d68ccfb256c"}
Sep 10 19:15:49.322925 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:49.322846    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwrt5" event={"ID":"89ea61c7-adeb-40e7-8a14-22c8d11a5262","Type":"ContainerStarted","Data":"ad0492d6c5161999d7da7dcbb48f1017256e4307dd3f0cb405bc686f01501872"}
Sep 10 19:15:49.327669 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:49.327570    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zdnmx" event={"ID":"98e70b32-791a-4a50-b558-5337fc78d24c","Type":"ContainerStarted","Data":"dbeb7d5295d849a3c5c8e7e9046750fd0d04eda22d29f559ae21aef6795fe6d4"}
Sep 10 19:15:49.333471 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:49.333411    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-k8tfc" event={"ID":"c357f393-afff-4968-aba1-3ef6f02620eb","Type":"ContainerStarted","Data":"659da0d61423f60a990877987e51360a06dee5d45a66c55b194131976ecaf4ca"}
Sep 10 19:15:49.338309 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:49.338281    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cllmw" event={"ID":"b47634b2-779d-426a-8a6b-c74a246654d4","Type":"ContainerStarted","Data":"8b6bed5170def2d6360503120dba66eb004f0053ef1ea0a8c61ff5e3205ac2ef"}
Sep 10 19:15:50.233675 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:50.233631    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjlbx"
Sep 10 19:15:50.234263 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:50.233827    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qjlbx" podUID="af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0"
Sep 10 19:15:50.234263 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:50.234241    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dns2r"
Sep 10 19:15:50.234423 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:50.234342    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dns2r" podUID="6a81beec-be63-4b8b-9999-b92fcda80550"
Sep 10 19:15:50.368046 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:50.368007    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-20.ec2.internal" event={"ID":"dc403cc24af90cbf61a5edf297685b42","Type":"ContainerStarted","Data":"1ce29fce40848dbe81638219ceee5ea8c8df66c96e887cde37a19bac9ce0fc5b"}
Sep 10 19:15:50.743193 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:50.742441    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/acd262af-057c-4bbf-93e3-faa020c1825c-original-pull-secret\") pod \"global-pull-secret-syncer-mkkzj\" (UID: \"acd262af-057c-4bbf-93e3-faa020c1825c\") " pod="kube-system/global-pull-secret-syncer-mkkzj"
Sep 10 19:15:50.743193 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:50.742660    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:50.743193 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:50.742738    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acd262af-057c-4bbf-93e3-faa020c1825c-original-pull-secret podName:acd262af-057c-4bbf-93e3-faa020c1825c nodeName:}" failed. No retries permitted until 2026-09-10 19:15:54.742715625 +0000 UTC m=+9.962564104 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/acd262af-057c-4bbf-93e3-faa020c1825c-original-pull-secret") pod "global-pull-secret-syncer-mkkzj" (UID: "acd262af-057c-4bbf-93e3-faa020c1825c") : object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:50.944625 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:50.944262    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmbxc\" (UniqueName: \"kubernetes.io/projected/6a81beec-be63-4b8b-9999-b92fcda80550-kube-api-access-fmbxc\") pod \"network-check-target-dns2r\" (UID: \"6a81beec-be63-4b8b-9999-b92fcda80550\") " pod="openshift-network-diagnostics/network-check-target-dns2r"
Sep 10 19:15:50.944625 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:50.944339    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0-metrics-certs\") pod \"network-metrics-daemon-qjlbx\" (UID: \"af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0\") " pod="openshift-multus/network-metrics-daemon-qjlbx"
Sep 10 19:15:50.944625 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:50.944487    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 19:15:50.944625 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:50.944496    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:50.944625 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:50.944520    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 19:15:50.944625 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:50.944535    2579 projected.go:196] Error preparing data for projected volume kube-api-access-fmbxc for pod openshift-network-diagnostics/network-check-target-dns2r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:50.944625 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:50.944573    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0-metrics-certs podName:af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0 nodeName:}" failed. No retries permitted until 2026-09-10 19:15:54.944550533 +0000 UTC m=+10.164399001 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0-metrics-certs") pod "network-metrics-daemon-qjlbx" (UID: "af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:50.944625 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:50.944597    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a81beec-be63-4b8b-9999-b92fcda80550-kube-api-access-fmbxc podName:6a81beec-be63-4b8b-9999-b92fcda80550 nodeName:}" failed. No retries permitted until 2026-09-10 19:15:54.94458482 +0000 UTC m=+10.164433290 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-fmbxc" (UniqueName: "kubernetes.io/projected/6a81beec-be63-4b8b-9999-b92fcda80550-kube-api-access-fmbxc") pod "network-check-target-dns2r" (UID: "6a81beec-be63-4b8b-9999-b92fcda80550") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:51.235009 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:51.234959    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkkzj"
Sep 10 19:15:51.235541 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:51.235107    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mkkzj" podUID="acd262af-057c-4bbf-93e3-faa020c1825c"
Sep 10 19:15:52.233421 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:52.233384    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjlbx"
Sep 10 19:15:52.233631 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:52.233553    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qjlbx" podUID="af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0"
Sep 10 19:15:52.233631 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:52.233600    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dns2r"
Sep 10 19:15:52.233762 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:52.233672    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dns2r" podUID="6a81beec-be63-4b8b-9999-b92fcda80550"
Sep 10 19:15:53.234069 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:53.233500    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkkzj"
Sep 10 19:15:53.234069 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:53.233663    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mkkzj" podUID="acd262af-057c-4bbf-93e3-faa020c1825c"
Sep 10 19:15:54.233281 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:54.233246    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjlbx"
Sep 10 19:15:54.233467 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:54.233246    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dns2r"
Sep 10 19:15:54.233467 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:54.233408    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qjlbx" podUID="af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0"
Sep 10 19:15:54.233611 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:54.233462    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dns2r" podUID="6a81beec-be63-4b8b-9999-b92fcda80550"
Sep 10 19:15:54.779279 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:54.779231    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/acd262af-057c-4bbf-93e3-faa020c1825c-original-pull-secret\") pod \"global-pull-secret-syncer-mkkzj\" (UID: \"acd262af-057c-4bbf-93e3-faa020c1825c\") " pod="kube-system/global-pull-secret-syncer-mkkzj"
Sep 10 19:15:54.779778 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:54.779379    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:54.779778 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:54.779455    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acd262af-057c-4bbf-93e3-faa020c1825c-original-pull-secret podName:acd262af-057c-4bbf-93e3-faa020c1825c nodeName:}" failed. No retries permitted until 2026-09-10 19:16:02.779435181 +0000 UTC m=+17.999283656 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/acd262af-057c-4bbf-93e3-faa020c1825c-original-pull-secret") pod "global-pull-secret-syncer-mkkzj" (UID: "acd262af-057c-4bbf-93e3-faa020c1825c") : object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:54.981287 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:54.981243    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0-metrics-certs\") pod \"network-metrics-daemon-qjlbx\" (UID: \"af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0\") " pod="openshift-multus/network-metrics-daemon-qjlbx"
Sep 10 19:15:54.981492 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:54.981314    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmbxc\" (UniqueName: \"kubernetes.io/projected/6a81beec-be63-4b8b-9999-b92fcda80550-kube-api-access-fmbxc\") pod \"network-check-target-dns2r\" (UID: \"6a81beec-be63-4b8b-9999-b92fcda80550\") " pod="openshift-network-diagnostics/network-check-target-dns2r"
Sep 10 19:15:54.981492 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:54.981437    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:54.981492 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:54.981460    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 19:15:54.981492 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:54.981483    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 19:15:54.981492 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:54.981493    2579 projected.go:196] Error preparing data for projected volume kube-api-access-fmbxc for pod openshift-network-diagnostics/network-check-target-dns2r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:54.981758 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:54.981519    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0-metrics-certs podName:af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:02.981497036 +0000 UTC m=+18.201345511 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0-metrics-certs") pod "network-metrics-daemon-qjlbx" (UID: "af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:54.981758 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:54.981542    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a81beec-be63-4b8b-9999-b92fcda80550-kube-api-access-fmbxc podName:6a81beec-be63-4b8b-9999-b92fcda80550 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:02.981530338 +0000 UTC m=+18.201378833 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-fmbxc" (UniqueName: "kubernetes.io/projected/6a81beec-be63-4b8b-9999-b92fcda80550-kube-api-access-fmbxc") pod "network-check-target-dns2r" (UID: "6a81beec-be63-4b8b-9999-b92fcda80550") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:55.234494 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:55.234446    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkkzj"
Sep 10 19:15:55.234709 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:55.234589    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mkkzj" podUID="acd262af-057c-4bbf-93e3-faa020c1825c"
Sep 10 19:15:56.233229 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:56.233189    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjlbx"
Sep 10 19:15:56.233723 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:56.233231    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dns2r"
Sep 10 19:15:56.233723 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:56.233352    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qjlbx" podUID="af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0"
Sep 10 19:15:56.233723 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:56.233476    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dns2r" podUID="6a81beec-be63-4b8b-9999-b92fcda80550"
Sep 10 19:15:57.236775 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:57.236739    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkkzj"
Sep 10 19:15:57.237270 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:57.236894    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mkkzj" podUID="acd262af-057c-4bbf-93e3-faa020c1825c"
Sep 10 19:15:58.234228 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:58.233476    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjlbx"
Sep 10 19:15:58.234228 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:58.233649    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qjlbx" podUID="af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0"
Sep 10 19:15:58.234228 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:58.234095    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dns2r"
Sep 10 19:15:58.234228 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:58.234180    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dns2r" podUID="6a81beec-be63-4b8b-9999-b92fcda80550"
Sep 10 19:15:59.233438 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:15:59.233406    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkkzj"
Sep 10 19:15:59.233829 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:15:59.233537    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mkkzj" podUID="acd262af-057c-4bbf-93e3-faa020c1825c"
Sep 10 19:16:00.232760 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:00.232721    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjlbx"
Sep 10 19:16:00.232934 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:00.232781    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dns2r"
Sep 10 19:16:00.232934 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:00.232891    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qjlbx" podUID="af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0"
Sep 10 19:16:00.233065 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:00.233040    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dns2r" podUID="6a81beec-be63-4b8b-9999-b92fcda80550"
Sep 10 19:16:01.233629 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:01.233595    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkkzj"
Sep 10 19:16:01.234083 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:01.233739    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mkkzj" podUID="acd262af-057c-4bbf-93e3-faa020c1825c"
Sep 10 19:16:02.233516 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:02.233479    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjlbx"
Sep 10 19:16:02.233698 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:02.233523    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dns2r"
Sep 10 19:16:02.233698 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:02.233615    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qjlbx" podUID="af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0"
Sep 10 19:16:02.234081 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:02.233728    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dns2r" podUID="6a81beec-be63-4b8b-9999-b92fcda80550"
Sep 10 19:16:02.843440 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:02.843394    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/acd262af-057c-4bbf-93e3-faa020c1825c-original-pull-secret\") pod \"global-pull-secret-syncer-mkkzj\" (UID: \"acd262af-057c-4bbf-93e3-faa020c1825c\") " pod="kube-system/global-pull-secret-syncer-mkkzj"
Sep 10 19:16:02.843595 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:02.843549    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 19:16:02.843631 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:02.843614    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acd262af-057c-4bbf-93e3-faa020c1825c-original-pull-secret podName:acd262af-057c-4bbf-93e3-faa020c1825c nodeName:}" failed. No retries permitted until 2026-09-10 19:16:18.843597723 +0000 UTC m=+34.063446183 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/acd262af-057c-4bbf-93e3-faa020c1825c-original-pull-secret") pod "global-pull-secret-syncer-mkkzj" (UID: "acd262af-057c-4bbf-93e3-faa020c1825c") : object "kube-system"/"original-pull-secret" not registered
Sep 10 19:16:03.044654 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:03.044614    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmbxc\" (UniqueName: \"kubernetes.io/projected/6a81beec-be63-4b8b-9999-b92fcda80550-kube-api-access-fmbxc\") pod \"network-check-target-dns2r\" (UID: \"6a81beec-be63-4b8b-9999-b92fcda80550\") " pod="openshift-network-diagnostics/network-check-target-dns2r"
Sep 10 19:16:03.044654 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:03.044661    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0-metrics-certs\") pod \"network-metrics-daemon-qjlbx\" (UID: \"af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0\") " pod="openshift-multus/network-metrics-daemon-qjlbx"
Sep 10 19:16:03.044848 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:03.044776    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:16:03.044848 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:03.044788    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 19:16:03.044848 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:03.044813    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 19:16:03.044848 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:03.044823    2579 projected.go:196] Error preparing data for projected volume kube-api-access-fmbxc for pod openshift-network-diagnostics/network-check-target-dns2r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:16:03.044848 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:03.044826    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0-metrics-certs podName:af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:19.044812114 +0000 UTC m=+34.264660573 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0-metrics-certs") pod "network-metrics-daemon-qjlbx" (UID: "af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:16:03.045000 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:03.044865    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a81beec-be63-4b8b-9999-b92fcda80550-kube-api-access-fmbxc podName:6a81beec-be63-4b8b-9999-b92fcda80550 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:19.044850924 +0000 UTC m=+34.264699397 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-fmbxc" (UniqueName: "kubernetes.io/projected/6a81beec-be63-4b8b-9999-b92fcda80550-kube-api-access-fmbxc") pod "network-check-target-dns2r" (UID: "6a81beec-be63-4b8b-9999-b92fcda80550") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:16:03.232810 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:03.232789    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkkzj"
Sep 10 19:16:03.232932 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:03.232914    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mkkzj" podUID="acd262af-057c-4bbf-93e3-faa020c1825c"
Sep 10 19:16:04.233088 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:04.233044    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjlbx"
Sep 10 19:16:04.233543 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:04.233193    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qjlbx" podUID="af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0"
Sep 10 19:16:04.233543 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:04.233232    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dns2r"
Sep 10 19:16:04.233543 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:04.233350    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dns2r" podUID="6a81beec-be63-4b8b-9999-b92fcda80550"
Sep 10 19:16:05.234122 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:05.234088    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkkzj"
Sep 10 19:16:05.234583 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:05.234185    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mkkzj" podUID="acd262af-057c-4bbf-93e3-faa020c1825c"
Sep 10 19:16:06.232814 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:06.232778    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjlbx"
Sep 10 19:16:06.233003 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:06.232788    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dns2r"
Sep 10 19:16:06.233003 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:06.232925    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qjlbx" podUID="af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0"
Sep 10 19:16:06.233098 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:06.232995    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dns2r" podUID="6a81beec-be63-4b8b-9999-b92fcda80550"
Sep 10 19:16:07.232778 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:07.232730    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkkzj"
Sep 10 19:16:07.233178 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:07.232849    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mkkzj" podUID="acd262af-057c-4bbf-93e3-faa020c1825c"
Sep 10 19:16:07.409584 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:07.409551    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zd86j" event={"ID":"affdbda0-e684-4e0d-a9e0-934646c95fe3","Type":"ContainerStarted","Data":"ec286a2ba21a57643d53e1f9ae7d780bd4da9b2b89ac09e241ae380b2ead2c10"}
Sep 10 19:16:07.412438 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:07.412408    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8r66s" event={"ID":"2826f8ec-78f6-4cab-879e-47a21854ed5a","Type":"ContainerStarted","Data":"bf9785b779836637f3d72624c13cb00415ecb47972087bd520d3c8d06cd11a88"}
Sep 10 19:16:07.414114 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:07.414053    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwrt5" event={"ID":"89ea61c7-adeb-40e7-8a14-22c8d11a5262","Type":"ContainerStarted","Data":"44c64129255176971398b3e5a4ec9903a6309c876815071dddafe23dd06d4e1b"}
Sep 10 19:16:07.415308 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:07.415280    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cllmw" event={"ID":"b47634b2-779d-426a-8a6b-c74a246654d4","Type":"ContainerStarted","Data":"75f4dba531b6d7cdba2b187568955f2c9ebf232857b029c84895c50df63da5d7"}
Sep 10 19:16:07.431155 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:07.431112    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-20.ec2.internal" podStartSLOduration=20.431098035 podStartE2EDuration="20.431098035s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:15:50.386350846 +0000 UTC m=+5.606199358" watchObservedRunningTime="2026-09-10 19:16:07.431098035 +0000 UTC m=+22.650946516"
Sep 10 19:16:08.233215 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:08.232958    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dns2r"
Sep 10 19:16:08.234044 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:08.233314    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dns2r" podUID="6a81beec-be63-4b8b-9999-b92fcda80550"
Sep 10 19:16:08.234044 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:08.233031    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjlbx"
Sep 10 19:16:08.234044 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:08.233778    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qjlbx" podUID="af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0"
Sep 10 19:16:08.419017 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:08.418981    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zdnmx" event={"ID":"98e70b32-791a-4a50-b558-5337fc78d24c","Type":"ContainerStarted","Data":"23df2b077691d1da018e22ac6e5c6239c73cd9b7d388644d0ac59482c8a8f560"}
Sep 10 19:16:08.420454 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:08.420431    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-k8tfc" event={"ID":"c357f393-afff-4968-aba1-3ef6f02620eb","Type":"ContainerStarted","Data":"d7086e35bd5a46f9b01d1af564606b084a29fee4adffb721eba3aa84be9816cf"}
Sep 10 19:16:08.424639 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:08.423642    2579 generic.go:356] "Generic (PLEG): container finished" podID="630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2" containerID="0e7a51a360ccccdce5ce2ecffa0437510b7ac04ec2ddabe3d69d61c8d0cacd18" exitCode=0
Sep 10 19:16:08.424639 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:08.423765    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qcw6c" event={"ID":"630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2","Type":"ContainerDied","Data":"0e7a51a360ccccdce5ce2ecffa0437510b7ac04ec2ddabe3d69d61c8d0cacd18"}
Sep 10 19:16:08.426352 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:08.426333    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-47rkn" event={"ID":"808f5f6a-9f93-4350-af47-8d19c19b8658","Type":"ContainerStarted","Data":"9403a0b46046dbeff36c5b2e52d717b9b95d2a76d294536b2768a95888fa32c2"}
Sep 10 19:16:08.427766 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:08.427731    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-89qpn" event={"ID":"0df00d14-a6d5-4b0f-895b-2f6cacbe131c","Type":"ContainerStarted","Data":"84a37c787bc20dde02b7638bf9057daad95d6ce713c661bfb38df19a9152aabe"}
Sep 10 19:16:08.430181 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:08.430164    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8r66s_2826f8ec-78f6-4cab-879e-47a21854ed5a/ovn-acl-logging/0.log"
Sep 10 19:16:08.430450 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:08.430433    2579 generic.go:356] "Generic (PLEG): container finished" podID="2826f8ec-78f6-4cab-879e-47a21854ed5a" containerID="a20ac1fe72a0d41b1de0922337592c6508a47fe3e39e51261abb6772480aefa6" exitCode=1
Sep 10 19:16:08.430523 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:08.430503    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8r66s" event={"ID":"2826f8ec-78f6-4cab-879e-47a21854ed5a","Type":"ContainerStarted","Data":"8903dae8a46801d24d9a80f180ff3df6a4b8b6ac5d4c67f1ee6516aecfe4125f"}
Sep 10 19:16:08.430587 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:08.430573    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8r66s" event={"ID":"2826f8ec-78f6-4cab-879e-47a21854ed5a","Type":"ContainerStarted","Data":"3649d4df5cb7737c7b333cd8a23042aa25e307f5aefc65607b39ecac0be20560"}
Sep 10 19:16:08.430639 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:08.430626    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8r66s" event={"ID":"2826f8ec-78f6-4cab-879e-47a21854ed5a","Type":"ContainerStarted","Data":"fe25d69e7b25451cba72c2d5c77d3bcbc0c8144a57dcf23a37f7737ea0039b07"}
Sep 10 19:16:08.430672 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:08.430662    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8r66s" event={"ID":"2826f8ec-78f6-4cab-879e-47a21854ed5a","Type":"ContainerStarted","Data":"b4a8c0b1031f9cf2888516bdbe623e5cc4db4d0599d0a9a5a3f1f2d47690a12f"}
Sep 10 19:16:08.430712 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:08.430698    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8r66s" event={"ID":"2826f8ec-78f6-4cab-879e-47a21854ed5a","Type":"ContainerDied","Data":"a20ac1fe72a0d41b1de0922337592c6508a47fe3e39e51261abb6772480aefa6"}
Sep 10 19:16:08.451295 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:08.451255    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-cllmw" podStartSLOduration=3.791650378 podStartE2EDuration="22.451228536s" totalImagesPullingTime="18.659578158s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:15:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:07.43097022 +0000 UTC m=+22.650818703" watchObservedRunningTime="2026-09-10 19:16:08.451228536 +0000 UTC m=+23.671077046"
Sep 10 19:16:08.451430 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:08.451411    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-zdnmx" podStartSLOduration=3.755636537 podStartE2EDuration="22.451405339s" totalImagesPullingTime="18.695768802s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:08.45086694 +0000 UTC m=+23.670715422" watchObservedRunningTime="2026-09-10 19:16:08.451405339 +0000 UTC m=+23.671253819"
Sep 10 19:16:08.486594 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:08.486527    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-89qpn" podStartSLOduration=3.789587429 podStartE2EDuration="22.486514655s" totalImagesPullingTime="18.696927226s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:08.465244099 +0000 UTC m=+23.685092579" watchObservedRunningTime="2026-09-10 19:16:08.486514655 +0000 UTC m=+23.706363135"
Sep 10 19:16:08.524684 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:08.524643    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-zd86j" podStartSLOduration=3.846392438 podStartE2EDuration="22.524629801s" totalImagesPullingTime="18.678237363s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:08.524373511 +0000 UTC m=+23.744221993" watchObservedRunningTime="2026-09-10 19:16:08.524629801 +0000 UTC m=+23.744478283"
Sep 10 19:16:08.524853 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:08.524822    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-47rkn" podStartSLOduration=3.541913709 podStartE2EDuration="22.524814773s" totalImagesPullingTime="18.982901064s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:08.504182804 +0000 UTC m=+23.724031285" watchObservedRunningTime="2026-09-10 19:16:08.524814773 +0000 UTC m=+23.744663254"
Sep 10 19:16:08.718108 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:08.718085    2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 10 19:16:09.166107 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:09.166011    2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T19:16:08.718101721Z","UUID":"d25c9007-1140-4aee-b2c4-7f775762b068","Handler":null,"Name":"","Endpoint":""}
Sep 10 19:16:09.167857 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:09.167789    2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 10 19:16:09.167857 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:09.167822    2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 10 19:16:09.232849 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:09.232817    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkkzj"
Sep 10 19:16:09.233010 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:09.232964    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mkkzj" podUID="acd262af-057c-4bbf-93e3-faa020c1825c"
Sep 10 19:16:09.435279 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:09.435194    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwrt5" event={"ID":"89ea61c7-adeb-40e7-8a14-22c8d11a5262","Type":"ContainerStarted","Data":"f268bf92654c581b8b7401efb43499804399b7767b807ba73b870c7693a2db77"}
Sep 10 19:16:10.233362 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:10.233328    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dns2r"
Sep 10 19:16:10.233362 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:10.233351    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjlbx"
Sep 10 19:16:10.233598 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:10.233446    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dns2r" podUID="6a81beec-be63-4b8b-9999-b92fcda80550"
Sep 10 19:16:10.233650 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:10.233606    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qjlbx" podUID="af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0"
Sep 10 19:16:10.439799 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:10.439766    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwrt5" event={"ID":"89ea61c7-adeb-40e7-8a14-22c8d11a5262","Type":"ContainerStarted","Data":"64f1aefcad1af47035c7dcc9cf085d0f7a7a0970bb3266a15b030598a57fb539"}
Sep 10 19:16:10.443155 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:10.443133    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8r66s_2826f8ec-78f6-4cab-879e-47a21854ed5a/ovn-acl-logging/0.log"
Sep 10 19:16:10.443568 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:10.443544    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8r66s" event={"ID":"2826f8ec-78f6-4cab-879e-47a21854ed5a","Type":"ContainerStarted","Data":"9411c15338c346b2dc5368b256644099daad284d7257e66d8af10cfe89be22a3"}
Sep 10 19:16:10.458076 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:10.458022    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-k8tfc" podStartSLOduration=5.741252269 podStartE2EDuration="24.458006552s" totalImagesPullingTime="18.716754283s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:08.546132059 +0000 UTC m=+23.765980540" watchObservedRunningTime="2026-09-10 19:16:10.458006552 +0000 UTC m=+25.677855034"
Sep 10 19:16:10.458519 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:10.458491    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwrt5" podStartSLOduration=3.5191271459999998 podStartE2EDuration="24.458482852s" totalImagesPullingTime="20.939355706s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:10.457264804 +0000 UTC m=+25.677113286" watchObservedRunningTime="2026-09-10 19:16:10.458482852 +0000 UTC m=+25.678331332"
Sep 10 19:16:11.233490 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:11.233458    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkkzj"
Sep 10 19:16:11.233655 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:11.233592    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mkkzj" podUID="acd262af-057c-4bbf-93e3-faa020c1825c"
Sep 10 19:16:11.412323 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:11.412289    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-cllmw"
Sep 10 19:16:11.412910 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:11.412885    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-cllmw"
Sep 10 19:16:11.445825 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:11.445795    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-cllmw"
Sep 10 19:16:11.446231 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:11.446124    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-cllmw"
Sep 10 19:16:12.233067 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:12.233036    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjlbx"
Sep 10 19:16:12.233233 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:12.233080    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dns2r"
Sep 10 19:16:12.233233 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:12.233158    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qjlbx" podUID="af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0"
Sep 10 19:16:12.233330 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:12.233235    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dns2r" podUID="6a81beec-be63-4b8b-9999-b92fcda80550"
Sep 10 19:16:13.233242 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:13.233004    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkkzj"
Sep 10 19:16:13.233997 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:13.233280    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mkkzj" podUID="acd262af-057c-4bbf-93e3-faa020c1825c"
Sep 10 19:16:13.451287 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:13.451247    2579 generic.go:356] "Generic (PLEG): container finished" podID="630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2" containerID="556e59e93b20331c16f3785b7181010031a723c81a8ae087505111df09a2d1e1" exitCode=0
Sep 10 19:16:13.451434 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:13.451331    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qcw6c" event={"ID":"630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2","Type":"ContainerDied","Data":"556e59e93b20331c16f3785b7181010031a723c81a8ae087505111df09a2d1e1"}
Sep 10 19:16:13.455087 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:13.455072    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8r66s_2826f8ec-78f6-4cab-879e-47a21854ed5a/ovn-acl-logging/0.log"
Sep 10 19:16:13.455496 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:13.455470    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8r66s" event={"ID":"2826f8ec-78f6-4cab-879e-47a21854ed5a","Type":"ContainerStarted","Data":"2f755814dbc6d02e33fbe9437881de16d5c00e6b77608067cb7c8103f4d4fd29"}
Sep 10 19:16:13.455794 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:13.455773    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:16:13.455875 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:13.455845    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:16:13.455919 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:13.455876    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:16:13.455954 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:13.455916    2579 scope.go:117] "RemoveContainer" containerID="a20ac1fe72a0d41b1de0922337592c6508a47fe3e39e51261abb6772480aefa6"
Sep 10 19:16:13.472426 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:13.472405    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:16:13.476188 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:13.476170    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:16:14.232934 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:14.232730    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjlbx"
Sep 10 19:16:14.233094 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:14.232786    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dns2r"
Sep 10 19:16:14.233094 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:14.233048    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qjlbx" podUID="af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0"
Sep 10 19:16:14.233180 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:14.233111    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dns2r" podUID="6a81beec-be63-4b8b-9999-b92fcda80550"
Sep 10 19:16:14.463142 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:14.463041    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qcw6c" event={"ID":"630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2","Type":"ContainerStarted","Data":"73b70c9885f46c22ae647b475724b366a27f7bde0dc88cb48fc38bd19d4032cc"}
Sep 10 19:16:14.467585 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:14.467566    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8r66s_2826f8ec-78f6-4cab-879e-47a21854ed5a/ovn-acl-logging/0.log"
Sep 10 19:16:14.467949 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:14.467925    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8r66s" event={"ID":"2826f8ec-78f6-4cab-879e-47a21854ed5a","Type":"ContainerStarted","Data":"7177298c65f5132469b12cec2f9f93405e83688e9478138d467cf889417e0664"}
Sep 10 19:16:14.508649 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:14.508559    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-8r66s" podStartSLOduration=9.816209965 podStartE2EDuration="28.508540888s" totalImagesPullingTime="18.692330923s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:14.50794549 +0000 UTC m=+29.727793994" watchObservedRunningTime="2026-09-10 19:16:14.508540888 +0000 UTC m=+29.728389371"
Sep 10 19:16:14.587396 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:14.587365    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dns2r"]
Sep 10 19:16:14.587535 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:14.587468    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dns2r"
Sep 10 19:16:14.587576 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:14.587548    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dns2r" podUID="6a81beec-be63-4b8b-9999-b92fcda80550"
Sep 10 19:16:14.590344 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:14.590320    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mkkzj"]
Sep 10 19:16:14.590465 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:14.590445    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkkzj"
Sep 10 19:16:14.590569 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:14.590549    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mkkzj" podUID="acd262af-057c-4bbf-93e3-faa020c1825c"
Sep 10 19:16:14.590876 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:14.590859    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qjlbx"]
Sep 10 19:16:14.590948 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:14.590932    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjlbx"
Sep 10 19:16:14.591020 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:14.591005    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qjlbx" podUID="af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0"
Sep 10 19:16:15.472776 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:15.472725    2579 generic.go:356] "Generic (PLEG): container finished" podID="630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2" containerID="73b70c9885f46c22ae647b475724b366a27f7bde0dc88cb48fc38bd19d4032cc" exitCode=0
Sep 10 19:16:15.473142 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:15.472815    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qcw6c" event={"ID":"630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2","Type":"ContainerDied","Data":"73b70c9885f46c22ae647b475724b366a27f7bde0dc88cb48fc38bd19d4032cc"}
Sep 10 19:16:16.233340 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:16.233303    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dns2r"
Sep 10 19:16:16.233531 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:16.233304    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjlbx"
Sep 10 19:16:16.233531 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:16.233445    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dns2r" podUID="6a81beec-be63-4b8b-9999-b92fcda80550"
Sep 10 19:16:16.233531 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:16.233303    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkkzj"
Sep 10 19:16:16.233653 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:16.233532    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qjlbx" podUID="af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0"
Sep 10 19:16:16.233653 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:16.233564    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mkkzj" podUID="acd262af-057c-4bbf-93e3-faa020c1825c"
Sep 10 19:16:16.477536 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:16.477451    2579 generic.go:356] "Generic (PLEG): container finished" podID="630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2" containerID="a921ed104a935e43f30ac18d10b35638c183c739a74f2a9399e5820b0ed91b96" exitCode=0
Sep 10 19:16:16.477536 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:16.477489    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qcw6c" event={"ID":"630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2","Type":"ContainerDied","Data":"a921ed104a935e43f30ac18d10b35638c183c739a74f2a9399e5820b0ed91b96"}
Sep 10 19:16:18.232551 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:18.232520    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjlbx"
Sep 10 19:16:18.233061 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:18.232612    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkkzj"
Sep 10 19:16:18.233061 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:18.232618    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qjlbx" podUID="af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0"
Sep 10 19:16:18.233061 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:18.232625    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dns2r"
Sep 10 19:16:18.233061 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:18.232695    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mkkzj" podUID="acd262af-057c-4bbf-93e3-faa020c1825c"
Sep 10 19:16:18.233061 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:18.232766    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dns2r" podUID="6a81beec-be63-4b8b-9999-b92fcda80550"
Sep 10 19:16:18.868446 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:18.868408    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/acd262af-057c-4bbf-93e3-faa020c1825c-original-pull-secret\") pod \"global-pull-secret-syncer-mkkzj\" (UID: \"acd262af-057c-4bbf-93e3-faa020c1825c\") " pod="kube-system/global-pull-secret-syncer-mkkzj"
Sep 10 19:16:18.868630 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:18.868574    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 19:16:18.868687 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:18.868645    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acd262af-057c-4bbf-93e3-faa020c1825c-original-pull-secret podName:acd262af-057c-4bbf-93e3-faa020c1825c nodeName:}" failed. No retries permitted until 2026-09-10 19:16:50.868629662 +0000 UTC m=+66.088478137 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/acd262af-057c-4bbf-93e3-faa020c1825c-original-pull-secret") pod "global-pull-secret-syncer-mkkzj" (UID: "acd262af-057c-4bbf-93e3-faa020c1825c") : object "kube-system"/"original-pull-secret" not registered
Sep 10 19:16:19.070376 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:19.070333    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0-metrics-certs\") pod \"network-metrics-daemon-qjlbx\" (UID: \"af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0\") " pod="openshift-multus/network-metrics-daemon-qjlbx"
Sep 10 19:16:19.070563 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:19.070428    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmbxc\" (UniqueName: \"kubernetes.io/projected/6a81beec-be63-4b8b-9999-b92fcda80550-kube-api-access-fmbxc\") pod \"network-check-target-dns2r\" (UID: \"6a81beec-be63-4b8b-9999-b92fcda80550\") " pod="openshift-network-diagnostics/network-check-target-dns2r"
Sep 10 19:16:19.070563 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:19.070511    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:16:19.070563 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:19.070555    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 19:16:19.070721 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:19.070572    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 19:16:19.070721 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:19.070585    2579 projected.go:196] Error preparing data for projected volume kube-api-access-fmbxc for pod openshift-network-diagnostics/network-check-target-dns2r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:16:19.070721 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:19.070586    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0-metrics-certs podName:af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:51.070568285 +0000 UTC m=+66.290416761 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0-metrics-certs") pod "network-metrics-daemon-qjlbx" (UID: "af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:16:19.070721 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:19.070637    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a81beec-be63-4b8b-9999-b92fcda80550-kube-api-access-fmbxc podName:6a81beec-be63-4b8b-9999-b92fcda80550 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:51.07062142 +0000 UTC m=+66.290469894 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-fmbxc" (UniqueName: "kubernetes.io/projected/6a81beec-be63-4b8b-9999-b92fcda80550-kube-api-access-fmbxc") pod "network-check-target-dns2r" (UID: "6a81beec-be63-4b8b-9999-b92fcda80550") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:16:20.064284 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.064070    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-20.ec2.internal" event="NodeReady"
Sep 10 19:16:20.064775 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.064434    2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 10 19:16:20.111813 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.111775    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vv7rr"]
Sep 10 19:16:20.130522 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.130490    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gdvkn"]
Sep 10 19:16:20.130702 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.130632    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vv7rr"
Sep 10 19:16:20.133055 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.133034    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 10 19:16:20.133055 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.133051    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 10 19:16:20.134608 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.133351    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wks57\""
Sep 10 19:16:20.146882 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.146857    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vv7rr"]
Sep 10 19:16:20.147015 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.146904    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gdvkn"]
Sep 10 19:16:20.147015 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.146985    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gdvkn"
Sep 10 19:16:20.148998 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.148979    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 10 19:16:20.149120 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.148986    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 10 19:16:20.149120 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.149052    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rxhx6\""
Sep 10 19:16:20.149298 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.149279    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 10 19:16:20.232996 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.232908    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkkzj"
Sep 10 19:16:20.232996 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.232939    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjlbx"
Sep 10 19:16:20.232996 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.232939    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dns2r"
Sep 10 19:16:20.235394 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.235372    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 19:16:20.235394 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.235383    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-q8jbb\""
Sep 10 19:16:20.235592 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.235474    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 19:16:20.235592 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.235572    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 19:16:20.235699 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.235603    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tsrfs\""
Sep 10 19:16:20.236033 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.235822    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 19:16:20.279123 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.279094    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmcbf\" (UniqueName: \"kubernetes.io/projected/27b0fcb8-4e92-4753-8562-6e1e8bfa9e97-kube-api-access-tmcbf\") pod \"ingress-canary-gdvkn\" (UID: \"27b0fcb8-4e92-4753-8562-6e1e8bfa9e97\") " pod="openshift-ingress-canary/ingress-canary-gdvkn"
Sep 10 19:16:20.279294 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.279133    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rm6q\" (UniqueName: \"kubernetes.io/projected/feafd719-ab13-485e-b195-ecbd0900e222-kube-api-access-5rm6q\") pod \"dns-default-vv7rr\" (UID: \"feafd719-ab13-485e-b195-ecbd0900e222\") " pod="openshift-dns/dns-default-vv7rr"
Sep 10 19:16:20.279294 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.279163    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/feafd719-ab13-485e-b195-ecbd0900e222-metrics-tls\") pod \"dns-default-vv7rr\" (UID: \"feafd719-ab13-485e-b195-ecbd0900e222\") " pod="openshift-dns/dns-default-vv7rr"
Sep 10 19:16:20.279294 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.279193    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/feafd719-ab13-485e-b195-ecbd0900e222-tmp-dir\") pod \"dns-default-vv7rr\" (UID: \"feafd719-ab13-485e-b195-ecbd0900e222\") " pod="openshift-dns/dns-default-vv7rr"
Sep 10 19:16:20.279294 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.279281    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/feafd719-ab13-485e-b195-ecbd0900e222-config-volume\") pod \"dns-default-vv7rr\" (UID: \"feafd719-ab13-485e-b195-ecbd0900e222\") " pod="openshift-dns/dns-default-vv7rr"
Sep 10 19:16:20.279481 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.279338    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/27b0fcb8-4e92-4753-8562-6e1e8bfa9e97-cert\") pod \"ingress-canary-gdvkn\" (UID: \"27b0fcb8-4e92-4753-8562-6e1e8bfa9e97\") " pod="openshift-ingress-canary/ingress-canary-gdvkn"
Sep 10 19:16:20.380428 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.380395    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/27b0fcb8-4e92-4753-8562-6e1e8bfa9e97-cert\") pod \"ingress-canary-gdvkn\" (UID: \"27b0fcb8-4e92-4753-8562-6e1e8bfa9e97\") " pod="openshift-ingress-canary/ingress-canary-gdvkn"
Sep 10 19:16:20.380617 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.380499    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmcbf\" (UniqueName: \"kubernetes.io/projected/27b0fcb8-4e92-4753-8562-6e1e8bfa9e97-kube-api-access-tmcbf\") pod \"ingress-canary-gdvkn\" (UID: \"27b0fcb8-4e92-4753-8562-6e1e8bfa9e97\") " pod="openshift-ingress-canary/ingress-canary-gdvkn"
Sep 10 19:16:20.380617 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.380530    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rm6q\" (UniqueName: \"kubernetes.io/projected/feafd719-ab13-485e-b195-ecbd0900e222-kube-api-access-5rm6q\") pod \"dns-default-vv7rr\" (UID: \"feafd719-ab13-485e-b195-ecbd0900e222\") " pod="openshift-dns/dns-default-vv7rr"
Sep 10 19:16:20.380617 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.380552    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/feafd719-ab13-485e-b195-ecbd0900e222-metrics-tls\") pod \"dns-default-vv7rr\" (UID: \"feafd719-ab13-485e-b195-ecbd0900e222\") " pod="openshift-dns/dns-default-vv7rr"
Sep 10 19:16:20.380617 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.380580    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/feafd719-ab13-485e-b195-ecbd0900e222-tmp-dir\") pod \"dns-default-vv7rr\" (UID: \"feafd719-ab13-485e-b195-ecbd0900e222\") " pod="openshift-dns/dns-default-vv7rr"
Sep 10 19:16:20.380845 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.380646    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/feafd719-ab13-485e-b195-ecbd0900e222-config-volume\") pod \"dns-default-vv7rr\" (UID: \"feafd719-ab13-485e-b195-ecbd0900e222\") " pod="openshift-dns/dns-default-vv7rr"
Sep 10 19:16:20.380898 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:20.380866    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 19:16:20.380953 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:20.380929    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feafd719-ab13-485e-b195-ecbd0900e222-metrics-tls podName:feafd719-ab13-485e-b195-ecbd0900e222 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:20.880910938 +0000 UTC m=+36.100759410 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/feafd719-ab13-485e-b195-ecbd0900e222-metrics-tls") pod "dns-default-vv7rr" (UID: "feafd719-ab13-485e-b195-ecbd0900e222") : secret "dns-default-metrics-tls" not found
Sep 10 19:16:20.381316 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.381293    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/feafd719-ab13-485e-b195-ecbd0900e222-tmp-dir\") pod \"dns-default-vv7rr\" (UID: \"feafd719-ab13-485e-b195-ecbd0900e222\") " pod="openshift-dns/dns-default-vv7rr"
Sep 10 19:16:20.381679 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.381646    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/feafd719-ab13-485e-b195-ecbd0900e222-config-volume\") pod \"dns-default-vv7rr\" (UID: \"feafd719-ab13-485e-b195-ecbd0900e222\") " pod="openshift-dns/dns-default-vv7rr"
Sep 10 19:16:20.385906 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.385883    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/27b0fcb8-4e92-4753-8562-6e1e8bfa9e97-cert\") pod \"ingress-canary-gdvkn\" (UID: \"27b0fcb8-4e92-4753-8562-6e1e8bfa9e97\") " pod="openshift-ingress-canary/ingress-canary-gdvkn"
Sep 10 19:16:20.392112 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.392080    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmcbf\" (UniqueName: \"kubernetes.io/projected/27b0fcb8-4e92-4753-8562-6e1e8bfa9e97-kube-api-access-tmcbf\") pod \"ingress-canary-gdvkn\" (UID: \"27b0fcb8-4e92-4753-8562-6e1e8bfa9e97\") " pod="openshift-ingress-canary/ingress-canary-gdvkn"
Sep 10 19:16:20.392209 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.392143    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rm6q\" (UniqueName: \"kubernetes.io/projected/feafd719-ab13-485e-b195-ecbd0900e222-kube-api-access-5rm6q\") pod \"dns-default-vv7rr\" (UID: \"feafd719-ab13-485e-b195-ecbd0900e222\") " pod="openshift-dns/dns-default-vv7rr"
Sep 10 19:16:20.456869 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.456826    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gdvkn"
Sep 10 19:16:20.884790 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:20.884762    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/feafd719-ab13-485e-b195-ecbd0900e222-metrics-tls\") pod \"dns-default-vv7rr\" (UID: \"feafd719-ab13-485e-b195-ecbd0900e222\") " pod="openshift-dns/dns-default-vv7rr"
Sep 10 19:16:20.884965 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:20.884936    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 19:16:20.885014 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:20.885006    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feafd719-ab13-485e-b195-ecbd0900e222-metrics-tls podName:feafd719-ab13-485e-b195-ecbd0900e222 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:21.884991761 +0000 UTC m=+37.104840220 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/feafd719-ab13-485e-b195-ecbd0900e222-metrics-tls") pod "dns-default-vv7rr" (UID: "feafd719-ab13-485e-b195-ecbd0900e222") : secret "dns-default-metrics-tls" not found
Sep 10 19:16:21.705159 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:21.705080    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gdvkn"]
Sep 10 19:16:21.753035 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:16:21.753000    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27b0fcb8_4e92_4753_8562_6e1e8bfa9e97.slice/crio-a59e9ad43875dd8b000e9017c2c5ca4c6d599b410cc4878808924a7ece35b7df WatchSource:0}: Error finding container a59e9ad43875dd8b000e9017c2c5ca4c6d599b410cc4878808924a7ece35b7df: Status 404 returned error can't find the container with id a59e9ad43875dd8b000e9017c2c5ca4c6d599b410cc4878808924a7ece35b7df
Sep 10 19:16:21.891341 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:21.891314    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/feafd719-ab13-485e-b195-ecbd0900e222-metrics-tls\") pod \"dns-default-vv7rr\" (UID: \"feafd719-ab13-485e-b195-ecbd0900e222\") " pod="openshift-dns/dns-default-vv7rr"
Sep 10 19:16:21.891473 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:21.891458    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 19:16:21.891527 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:21.891518    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feafd719-ab13-485e-b195-ecbd0900e222-metrics-tls podName:feafd719-ab13-485e-b195-ecbd0900e222 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:23.891503061 +0000 UTC m=+39.111351521 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/feafd719-ab13-485e-b195-ecbd0900e222-metrics-tls") pod "dns-default-vv7rr" (UID: "feafd719-ab13-485e-b195-ecbd0900e222") : secret "dns-default-metrics-tls" not found
Sep 10 19:16:22.493428 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:22.493393    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gdvkn" event={"ID":"27b0fcb8-4e92-4753-8562-6e1e8bfa9e97","Type":"ContainerStarted","Data":"a59e9ad43875dd8b000e9017c2c5ca4c6d599b410cc4878808924a7ece35b7df"}
Sep 10 19:16:22.495823 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:22.495796    2579 generic.go:356] "Generic (PLEG): container finished" podID="630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2" containerID="9f976a4b2de00d16395040fba932d74619c22cc43ebd40fb75ae9fe68ddc3017" exitCode=0
Sep 10 19:16:22.495931 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:22.495833    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qcw6c" event={"ID":"630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2","Type":"ContainerDied","Data":"9f976a4b2de00d16395040fba932d74619c22cc43ebd40fb75ae9fe68ddc3017"}
Sep 10 19:16:23.502278 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:23.502044    2579 generic.go:356] "Generic (PLEG): container finished" podID="630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2" containerID="563b1cdd96b6127dd757d5b2ed13290500818c84efdc4789bebceee107da2d1a" exitCode=0
Sep 10 19:16:23.502278 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:23.502139    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qcw6c" event={"ID":"630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2","Type":"ContainerDied","Data":"563b1cdd96b6127dd757d5b2ed13290500818c84efdc4789bebceee107da2d1a"}
Sep 10 19:16:23.906476 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:23.906443    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/feafd719-ab13-485e-b195-ecbd0900e222-metrics-tls\") pod \"dns-default-vv7rr\" (UID: \"feafd719-ab13-485e-b195-ecbd0900e222\") " pod="openshift-dns/dns-default-vv7rr"
Sep 10 19:16:23.906629 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:23.906578    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 19:16:23.906673 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:23.906644    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feafd719-ab13-485e-b195-ecbd0900e222-metrics-tls podName:feafd719-ab13-485e-b195-ecbd0900e222 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:27.906621853 +0000 UTC m=+43.126470332 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/feafd719-ab13-485e-b195-ecbd0900e222-metrics-tls") pod "dns-default-vv7rr" (UID: "feafd719-ab13-485e-b195-ecbd0900e222") : secret "dns-default-metrics-tls" not found
Sep 10 19:16:24.508142 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:24.508071    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qcw6c" event={"ID":"630fe5e7-4636-4c1e-ab69-b4a6ddfdaeb2","Type":"ContainerStarted","Data":"4b9dd18c858b60416257a0447d205adb3e0491dbcf94cf8901cf9057e9c00423"}
Sep 10 19:16:24.538363 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:24.538302    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qcw6c" podStartSLOduration=8.687849327 podStartE2EDuration="38.538285748s" totalImagesPullingTime="29.850436421s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:46 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:24.537223159 +0000 UTC m=+39.757071639" watchObservedRunningTime="2026-09-10 19:16:24.538285748 +0000 UTC m=+39.758134228"
Sep 10 19:16:25.511805 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:25.511769    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gdvkn" event={"ID":"27b0fcb8-4e92-4753-8562-6e1e8bfa9e97","Type":"ContainerStarted","Data":"b95030f95f6feb70c5fe91b93efa21ffbffad14ba4932defbeabe89654615d4f"}
Sep 10 19:16:25.526976 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:25.526928    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gdvkn" podStartSLOduration=2.381790617 podStartE2EDuration="5.526912147s" totalImagesPullingTime="3.14512153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:16:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:25.526176891 +0000 UTC m=+40.746025373" watchObservedRunningTime="2026-09-10 19:16:25.526912147 +0000 UTC m=+40.746760622"
Sep 10 19:16:27.078962 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.078928    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5psdd"]
Sep 10 19:16:27.103514 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.103447    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5psdd"]
Sep 10 19:16:27.103727 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.103588    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5psdd"
Sep 10 19:16:27.105651 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.105626    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 10 19:16:27.105801 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.105666    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 10 19:16:27.105801 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.105667    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 10 19:16:27.105973 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.105961    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 10 19:16:27.106214 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.106201    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-wmnlr\""
Sep 10 19:16:27.193209 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.193178    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-665c8cd58b-t27cc"]
Sep 10 19:16:27.213293 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.213262    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-bmdg9"]
Sep 10 19:16:27.213449 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.213393    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-665c8cd58b-t27cc"
Sep 10 19:16:27.215443 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.215422    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 10 19:16:27.215564 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.215521    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 10 19:16:27.215564 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.215550    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 10 19:16:27.215685 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.215573    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-n7kcr\""
Sep 10 19:16:27.225536 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.225512    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 10 19:16:27.229341 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.229322    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-bmdg9"]
Sep 10 19:16:27.229416 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.229355    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-665c8cd58b-t27cc"]
Sep 10 19:16:27.229454 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.229436    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-bmdg9"
Sep 10 19:16:27.229494 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.229481    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/77f7916d-5334-4e57-848f-afedd9e63d29-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-5psdd\" (UID: \"77f7916d-5334-4e57-848f-afedd9e63d29\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5psdd"
Sep 10 19:16:27.229558 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.229543    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/77f7916d-5334-4e57-848f-afedd9e63d29-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-5psdd\" (UID: \"77f7916d-5334-4e57-848f-afedd9e63d29\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5psdd"
Sep 10 19:16:27.229684 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.229663    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z99lb\" (UniqueName: \"kubernetes.io/projected/77f7916d-5334-4e57-848f-afedd9e63d29-kube-api-access-z99lb\") pod \"kube-storage-version-migrator-operator-8d7dd4675-5psdd\" (UID: \"77f7916d-5334-4e57-848f-afedd9e63d29\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5psdd"
Sep 10 19:16:27.231808 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.231784    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 10 19:16:27.231930 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.231841    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 10 19:16:27.231930 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.231854    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-6lffx\""
Sep 10 19:16:27.231930 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.231870    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 10 19:16:27.232102 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.232089    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 10 19:16:27.237491 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.237472    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 10 19:16:27.331123 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.331031    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-ca-trust-extracted\") pod \"image-registry-665c8cd58b-t27cc\" (UID: \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\") " pod="openshift-image-registry/image-registry-665c8cd58b-t27cc"
Sep 10 19:16:27.331123 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.331068    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-registry-certificates\") pod \"image-registry-665c8cd58b-t27cc\" (UID: \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\") " pod="openshift-image-registry/image-registry-665c8cd58b-t27cc"
Sep 10 19:16:27.331123 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.331093    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-bound-sa-token\") pod \"image-registry-665c8cd58b-t27cc\" (UID: \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\") " pod="openshift-image-registry/image-registry-665c8cd58b-t27cc"
Sep 10 19:16:27.331123 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.331108    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4j42k\" (UniqueName: \"kubernetes.io/projected/9817270c-8629-44c4-ab8f-f3982e1f0e2f-kube-api-access-4j42k\") pod \"console-operator-85b454874b-bmdg9\" (UID: \"9817270c-8629-44c4-ab8f-f3982e1f0e2f\") " pod="openshift-console-operator/console-operator-85b454874b-bmdg9"
Sep 10 19:16:27.331416 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.331190    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/77f7916d-5334-4e57-848f-afedd9e63d29-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-5psdd\" (UID: \"77f7916d-5334-4e57-848f-afedd9e63d29\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5psdd"
Sep 10 19:16:27.331416 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.331227    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-registry-tls\") pod \"image-registry-665c8cd58b-t27cc\" (UID: \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\") " pod="openshift-image-registry/image-registry-665c8cd58b-t27cc"
Sep 10 19:16:27.331416 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.331257    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sgtx\" (UniqueName: \"kubernetes.io/projected/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-kube-api-access-9sgtx\") pod \"image-registry-665c8cd58b-t27cc\" (UID: \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\") " pod="openshift-image-registry/image-registry-665c8cd58b-t27cc"
Sep 10 19:16:27.331416 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.331273    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9817270c-8629-44c4-ab8f-f3982e1f0e2f-trusted-ca\") pod \"console-operator-85b454874b-bmdg9\" (UID: \"9817270c-8629-44c4-ab8f-f3982e1f0e2f\") " pod="openshift-console-operator/console-operator-85b454874b-bmdg9"
Sep 10 19:16:27.331416 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.331303    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-image-registry-private-configuration\") pod \"image-registry-665c8cd58b-t27cc\" (UID: \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\") " pod="openshift-image-registry/image-registry-665c8cd58b-t27cc"
Sep 10 19:16:27.331416 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.331329    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-trusted-ca\") pod \"image-registry-665c8cd58b-t27cc\" (UID: \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\") " pod="openshift-image-registry/image-registry-665c8cd58b-t27cc"
Sep 10 19:16:27.331416 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.331378    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9817270c-8629-44c4-ab8f-f3982e1f0e2f-serving-cert\") pod \"console-operator-85b454874b-bmdg9\" (UID: \"9817270c-8629-44c4-ab8f-f3982e1f0e2f\") " pod="openshift-console-operator/console-operator-85b454874b-bmdg9"
Sep 10 19:16:27.331680 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.331422    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9817270c-8629-44c4-ab8f-f3982e1f0e2f-config\") pod \"console-operator-85b454874b-bmdg9\" (UID: \"9817270c-8629-44c4-ab8f-f3982e1f0e2f\") " pod="openshift-console-operator/console-operator-85b454874b-bmdg9"
Sep 10 19:16:27.331680 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.331468    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z99lb\" (UniqueName: \"kubernetes.io/projected/77f7916d-5334-4e57-848f-afedd9e63d29-kube-api-access-z99lb\") pod \"kube-storage-version-migrator-operator-8d7dd4675-5psdd\" (UID: \"77f7916d-5334-4e57-848f-afedd9e63d29\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5psdd"
Sep 10 19:16:27.331680 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.331485    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/77f7916d-5334-4e57-848f-afedd9e63d29-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-5psdd\" (UID: \"77f7916d-5334-4e57-848f-afedd9e63d29\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5psdd"
Sep 10 19:16:27.331680 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.331499    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-installation-pull-secrets\") pod \"image-registry-665c8cd58b-t27cc\" (UID: \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\") " pod="openshift-image-registry/image-registry-665c8cd58b-t27cc"
Sep 10 19:16:27.332227 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.332202    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/77f7916d-5334-4e57-848f-afedd9e63d29-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-5psdd\" (UID: \"77f7916d-5334-4e57-848f-afedd9e63d29\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5psdd"
Sep 10 19:16:27.334795 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.334774    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/77f7916d-5334-4e57-848f-afedd9e63d29-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-5psdd\" (UID: \"77f7916d-5334-4e57-848f-afedd9e63d29\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5psdd"
Sep 10 19:16:27.341877 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.341855    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z99lb\" (UniqueName: \"kubernetes.io/projected/77f7916d-5334-4e57-848f-afedd9e63d29-kube-api-access-z99lb\") pod \"kube-storage-version-migrator-operator-8d7dd4675-5psdd\" (UID: \"77f7916d-5334-4e57-848f-afedd9e63d29\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5psdd"
Sep 10 19:16:27.413565 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.413520    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5psdd"
Sep 10 19:16:27.432898 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.432862    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-registry-tls\") pod \"image-registry-665c8cd58b-t27cc\" (UID: \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\") " pod="openshift-image-registry/image-registry-665c8cd58b-t27cc"
Sep 10 19:16:27.432898 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.432898    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sgtx\" (UniqueName: \"kubernetes.io/projected/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-kube-api-access-9sgtx\") pod \"image-registry-665c8cd58b-t27cc\" (UID: \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\") " pod="openshift-image-registry/image-registry-665c8cd58b-t27cc"
Sep 10 19:16:27.433150 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.432916    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9817270c-8629-44c4-ab8f-f3982e1f0e2f-trusted-ca\") pod \"console-operator-85b454874b-bmdg9\" (UID: \"9817270c-8629-44c4-ab8f-f3982e1f0e2f\") " pod="openshift-console-operator/console-operator-85b454874b-bmdg9"
Sep 10 19:16:27.433150 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.432944    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-image-registry-private-configuration\") pod \"image-registry-665c8cd58b-t27cc\" (UID: \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\") " pod="openshift-image-registry/image-registry-665c8cd58b-t27cc"
Sep 10 19:16:27.433150 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.432968    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-trusted-ca\") pod \"image-registry-665c8cd58b-t27cc\" (UID: \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\") " pod="openshift-image-registry/image-registry-665c8cd58b-t27cc"
Sep 10 19:16:27.433150 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.432990    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9817270c-8629-44c4-ab8f-f3982e1f0e2f-serving-cert\") pod \"console-operator-85b454874b-bmdg9\" (UID: \"9817270c-8629-44c4-ab8f-f3982e1f0e2f\") " pod="openshift-console-operator/console-operator-85b454874b-bmdg9"
Sep 10 19:16:27.433150 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.433019    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9817270c-8629-44c4-ab8f-f3982e1f0e2f-config\") pod \"console-operator-85b454874b-bmdg9\" (UID: \"9817270c-8629-44c4-ab8f-f3982e1f0e2f\") " pod="openshift-console-operator/console-operator-85b454874b-bmdg9"
Sep 10 19:16:27.433150 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:27.433055    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 19:16:27.433150 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:27.433084    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-665c8cd58b-t27cc: secret "image-registry-tls" not found
Sep 10 19:16:27.433150 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.433086    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-installation-pull-secrets\") pod \"image-registry-665c8cd58b-t27cc\" (UID: \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\") " pod="openshift-image-registry/image-registry-665c8cd58b-t27cc"
Sep 10 19:16:27.433150 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:27.433143    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-registry-tls podName:20fcd10a-d90b-4eaf-96a0-06b5ec96437a nodeName:}" failed. No retries permitted until 2026-09-10 19:16:27.933122991 +0000 UTC m=+43.152971475 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-registry-tls") pod "image-registry-665c8cd58b-t27cc" (UID: "20fcd10a-d90b-4eaf-96a0-06b5ec96437a") : secret "image-registry-tls" not found
Sep 10 19:16:27.433584 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.433270    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-ca-trust-extracted\") pod \"image-registry-665c8cd58b-t27cc\" (UID: \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\") " pod="openshift-image-registry/image-registry-665c8cd58b-t27cc"
Sep 10 19:16:27.433584 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.433301    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-registry-certificates\") pod \"image-registry-665c8cd58b-t27cc\" (UID: \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\") " pod="openshift-image-registry/image-registry-665c8cd58b-t27cc"
Sep 10 19:16:27.433584 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.433336    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-bound-sa-token\") pod \"image-registry-665c8cd58b-t27cc\" (UID: \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\") " pod="openshift-image-registry/image-registry-665c8cd58b-t27cc"
Sep 10 19:16:27.433584 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.433363    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4j42k\" (UniqueName: \"kubernetes.io/projected/9817270c-8629-44c4-ab8f-f3982e1f0e2f-kube-api-access-4j42k\") pod \"console-operator-85b454874b-bmdg9\" (UID: \"9817270c-8629-44c4-ab8f-f3982e1f0e2f\") " pod="openshift-console-operator/console-operator-85b454874b-bmdg9"
Sep 10 19:16:27.434488 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.434075    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-ca-trust-extracted\") pod \"image-registry-665c8cd58b-t27cc\" (UID: \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\") " pod="openshift-image-registry/image-registry-665c8cd58b-t27cc"
Sep 10 19:16:27.434488 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.434192    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9817270c-8629-44c4-ab8f-f3982e1f0e2f-trusted-ca\") pod \"console-operator-85b454874b-bmdg9\" (UID: \"9817270c-8629-44c4-ab8f-f3982e1f0e2f\") " pod="openshift-console-operator/console-operator-85b454874b-bmdg9"
Sep 10 19:16:27.434488 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.434245    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9817270c-8629-44c4-ab8f-f3982e1f0e2f-config\") pod \"console-operator-85b454874b-bmdg9\" (UID: \"9817270c-8629-44c4-ab8f-f3982e1f0e2f\") " pod="openshift-console-operator/console-operator-85b454874b-bmdg9"
Sep 10 19:16:27.434488 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.434394    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-trusted-ca\") pod \"image-registry-665c8cd58b-t27cc\" (UID: \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\") " pod="openshift-image-registry/image-registry-665c8cd58b-t27cc"
Sep 10 19:16:27.434688 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.434532    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-registry-certificates\") pod \"image-registry-665c8cd58b-t27cc\" (UID: \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\") " pod="openshift-image-registry/image-registry-665c8cd58b-t27cc"
Sep 10 19:16:27.435647 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.435621    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9817270c-8629-44c4-ab8f-f3982e1f0e2f-serving-cert\") pod \"console-operator-85b454874b-bmdg9\" (UID: \"9817270c-8629-44c4-ab8f-f3982e1f0e2f\") " pod="openshift-console-operator/console-operator-85b454874b-bmdg9"
Sep 10 19:16:27.436353 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.436326    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-image-registry-private-configuration\") pod \"image-registry-665c8cd58b-t27cc\" (UID: \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\") " pod="openshift-image-registry/image-registry-665c8cd58b-t27cc"
Sep 10 19:16:27.436404 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.436373    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-installation-pull-secrets\") pod \"image-registry-665c8cd58b-t27cc\" (UID: \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\") " pod="openshift-image-registry/image-registry-665c8cd58b-t27cc"
Sep 10 19:16:27.445270 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.445237    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-bound-sa-token\") pod \"image-registry-665c8cd58b-t27cc\" (UID: \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\") " pod="openshift-image-registry/image-registry-665c8cd58b-t27cc"
Sep 10 19:16:27.445423 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.445393    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4j42k\" (UniqueName: \"kubernetes.io/projected/9817270c-8629-44c4-ab8f-f3982e1f0e2f-kube-api-access-4j42k\") pod \"console-operator-85b454874b-bmdg9\" (UID: \"9817270c-8629-44c4-ab8f-f3982e1f0e2f\") " pod="openshift-console-operator/console-operator-85b454874b-bmdg9"
Sep 10 19:16:27.445796 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.445772    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sgtx\" (UniqueName: \"kubernetes.io/projected/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-kube-api-access-9sgtx\") pod \"image-registry-665c8cd58b-t27cc\" (UID: \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\") " pod="openshift-image-registry/image-registry-665c8cd58b-t27cc"
Sep 10 19:16:27.539185 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.539143    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-bmdg9"
Sep 10 19:16:27.709345 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.709309    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5psdd"]
Sep 10 19:16:27.712222 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:16:27.712192    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77f7916d_5334_4e57_848f_afedd9e63d29.slice/crio-472389fc18e4c5181bdd1a1524a66ef67cf75dbd23f817ec2dec04d3dbfa27e6 WatchSource:0}: Error finding container 472389fc18e4c5181bdd1a1524a66ef67cf75dbd23f817ec2dec04d3dbfa27e6: Status 404 returned error can't find the container with id 472389fc18e4c5181bdd1a1524a66ef67cf75dbd23f817ec2dec04d3dbfa27e6
Sep 10 19:16:27.840683 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.840652    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-bmdg9"]
Sep 10 19:16:27.843842 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:16:27.843816    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9817270c_8629_44c4_ab8f_f3982e1f0e2f.slice/crio-672daf413fbd75703f180f3ca8b0906c3fb6073e83b7e5c125836456e5fc35d5 WatchSource:0}: Error finding container 672daf413fbd75703f180f3ca8b0906c3fb6073e83b7e5c125836456e5fc35d5: Status 404 returned error can't find the container with id 672daf413fbd75703f180f3ca8b0906c3fb6073e83b7e5c125836456e5fc35d5
Sep 10 19:16:27.937982 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.937947    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-registry-tls\") pod \"image-registry-665c8cd58b-t27cc\" (UID: \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\") " pod="openshift-image-registry/image-registry-665c8cd58b-t27cc"
Sep 10 19:16:27.937982 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:27.937985    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/feafd719-ab13-485e-b195-ecbd0900e222-metrics-tls\") pod \"dns-default-vv7rr\" (UID: \"feafd719-ab13-485e-b195-ecbd0900e222\") " pod="openshift-dns/dns-default-vv7rr"
Sep 10 19:16:27.938210 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:27.938105    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 19:16:27.938210 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:27.938114    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 19:16:27.938210 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:27.938136    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-665c8cd58b-t27cc: secret "image-registry-tls" not found
Sep 10 19:16:27.938210 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:27.938158    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feafd719-ab13-485e-b195-ecbd0900e222-metrics-tls podName:feafd719-ab13-485e-b195-ecbd0900e222 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:35.938144049 +0000 UTC m=+51.157992507 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/feafd719-ab13-485e-b195-ecbd0900e222-metrics-tls") pod "dns-default-vv7rr" (UID: "feafd719-ab13-485e-b195-ecbd0900e222") : secret "dns-default-metrics-tls" not found
Sep 10 19:16:27.938210 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:27.938182    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-registry-tls podName:20fcd10a-d90b-4eaf-96a0-06b5ec96437a nodeName:}" failed. No retries permitted until 2026-09-10 19:16:28.938166541 +0000 UTC m=+44.158015043 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-registry-tls") pod "image-registry-665c8cd58b-t27cc" (UID: "20fcd10a-d90b-4eaf-96a0-06b5ec96437a") : secret "image-registry-tls" not found
Sep 10 19:16:28.521422 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:28.521382    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-bmdg9" event={"ID":"9817270c-8629-44c4-ab8f-f3982e1f0e2f","Type":"ContainerStarted","Data":"672daf413fbd75703f180f3ca8b0906c3fb6073e83b7e5c125836456e5fc35d5"}
Sep 10 19:16:28.522641 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:28.522601    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5psdd" event={"ID":"77f7916d-5334-4e57-848f-afedd9e63d29","Type":"ContainerStarted","Data":"472389fc18e4c5181bdd1a1524a66ef67cf75dbd23f817ec2dec04d3dbfa27e6"}
Sep 10 19:16:28.945166 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:28.945132    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-registry-tls\") pod \"image-registry-665c8cd58b-t27cc\" (UID: \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\") " pod="openshift-image-registry/image-registry-665c8cd58b-t27cc"
Sep 10 19:16:28.945349 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:28.945310    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 19:16:28.945349 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:28.945332    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-665c8cd58b-t27cc: secret "image-registry-tls" not found
Sep 10 19:16:28.945467 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:28.945404    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-registry-tls podName:20fcd10a-d90b-4eaf-96a0-06b5ec96437a nodeName:}" failed. No retries permitted until 2026-09-10 19:16:30.94538223 +0000 UTC m=+46.165230707 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-registry-tls") pod "image-registry-665c8cd58b-t27cc" (UID: "20fcd10a-d90b-4eaf-96a0-06b5ec96437a") : secret "image-registry-tls" not found
Sep 10 19:16:30.963614 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:30.963572    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-registry-tls\") pod \"image-registry-665c8cd58b-t27cc\" (UID: \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\") " pod="openshift-image-registry/image-registry-665c8cd58b-t27cc"
Sep 10 19:16:30.964087 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:30.963772    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 19:16:30.964087 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:30.963798    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-665c8cd58b-t27cc: secret "image-registry-tls" not found
Sep 10 19:16:30.964087 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:30.963889    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-registry-tls podName:20fcd10a-d90b-4eaf-96a0-06b5ec96437a nodeName:}" failed. No retries permitted until 2026-09-10 19:16:34.96386482 +0000 UTC m=+50.183713298 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-registry-tls") pod "image-registry-665c8cd58b-t27cc" (UID: "20fcd10a-d90b-4eaf-96a0-06b5ec96437a") : secret "image-registry-tls" not found
Sep 10 19:16:31.531549 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:31.531509    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5psdd" event={"ID":"77f7916d-5334-4e57-848f-afedd9e63d29","Type":"ContainerStarted","Data":"25dab80f84ad35f6532c3e3c31b1a928a297be3129c20ca319ec39a58c0ca89b"}
Sep 10 19:16:31.533163 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:31.533139    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bmdg9_9817270c-8629-44c4-ab8f-f3982e1f0e2f/console-operator/0.log"
Sep 10 19:16:31.533298 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:31.533185    2579 generic.go:356] "Generic (PLEG): container finished" podID="9817270c-8629-44c4-ab8f-f3982e1f0e2f" containerID="0c6c0a30296209ec4f0c186c29050715d7c393a8c30734a03337a7ad93c8998e" exitCode=255
Sep 10 19:16:31.533298 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:31.533227    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-bmdg9" event={"ID":"9817270c-8629-44c4-ab8f-f3982e1f0e2f","Type":"ContainerDied","Data":"0c6c0a30296209ec4f0c186c29050715d7c393a8c30734a03337a7ad93c8998e"}
Sep 10 19:16:31.533473 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:31.533456    2579 scope.go:117] "RemoveContainer" containerID="0c6c0a30296209ec4f0c186c29050715d7c393a8c30734a03337a7ad93c8998e"
Sep 10 19:16:31.555487 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:31.555439    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5psdd" podStartSLOduration=1.5040571310000002 podStartE2EDuration="4.555423541s" totalImagesPullingTime="3.05136641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:16:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:31.555250041 +0000 UTC m=+46.775098576" watchObservedRunningTime="2026-09-10 19:16:31.555423541 +0000 UTC m=+46.775272022"
Sep 10 19:16:32.537453 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:32.537376    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bmdg9_9817270c-8629-44c4-ab8f-f3982e1f0e2f/console-operator/1.log"
Sep 10 19:16:32.537829 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:32.537800    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bmdg9_9817270c-8629-44c4-ab8f-f3982e1f0e2f/console-operator/0.log"
Sep 10 19:16:32.537898 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:32.537835    2579 generic.go:356] "Generic (PLEG): container finished" podID="9817270c-8629-44c4-ab8f-f3982e1f0e2f" containerID="90f8d3a514d0fe91e04980d561bdddbb3f3216d0f2a31002cb7e368fd6c96ff3" exitCode=255
Sep 10 19:16:32.537954 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:32.537930    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-bmdg9" event={"ID":"9817270c-8629-44c4-ab8f-f3982e1f0e2f","Type":"ContainerDied","Data":"90f8d3a514d0fe91e04980d561bdddbb3f3216d0f2a31002cb7e368fd6c96ff3"}
Sep 10 19:16:32.537998 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:32.537988    2579 scope.go:117] "RemoveContainer" containerID="0c6c0a30296209ec4f0c186c29050715d7c393a8c30734a03337a7ad93c8998e"
Sep 10 19:16:32.538265 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:32.538249    2579 scope.go:117] "RemoveContainer" containerID="90f8d3a514d0fe91e04980d561bdddbb3f3216d0f2a31002cb7e368fd6c96ff3"
Sep 10 19:16:32.538441 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:32.538419    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-bmdg9_openshift-console-operator(9817270c-8629-44c4-ab8f-f3982e1f0e2f)\"" pod="openshift-console-operator/console-operator-85b454874b-bmdg9" podUID="9817270c-8629-44c4-ab8f-f3982e1f0e2f"
Sep 10 19:16:32.755663 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:32.755631    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ssqgh"]
Sep 10 19:16:32.785463 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:32.785420    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ssqgh"]
Sep 10 19:16:32.785624 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:32.785526    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ssqgh"
Sep 10 19:16:32.787237 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:32.787210    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 10 19:16:32.787373 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:32.787332    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 10 19:16:32.787579 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:32.787531    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 10 19:16:32.787579 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:32.787565    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 10 19:16:32.787696 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:32.787609    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-q7pt8\""
Sep 10 19:16:32.880664 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:32.880619    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d1058bfb-6fa3-4f17-bc23-87860236c43a-crio-socket\") pod \"insights-runtime-extractor-ssqgh\" (UID: \"d1058bfb-6fa3-4f17-bc23-87860236c43a\") " pod="openshift-insights/insights-runtime-extractor-ssqgh"
Sep 10 19:16:32.880860 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:32.880770    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d1058bfb-6fa3-4f17-bc23-87860236c43a-data-volume\") pod \"insights-runtime-extractor-ssqgh\" (UID: \"d1058bfb-6fa3-4f17-bc23-87860236c43a\") " pod="openshift-insights/insights-runtime-extractor-ssqgh"
Sep 10 19:16:32.880860 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:32.880798    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d1058bfb-6fa3-4f17-bc23-87860236c43a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ssqgh\" (UID: \"d1058bfb-6fa3-4f17-bc23-87860236c43a\") " pod="openshift-insights/insights-runtime-extractor-ssqgh"
Sep 10 19:16:32.880860 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:32.880825    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtqbk\" (UniqueName: \"kubernetes.io/projected/d1058bfb-6fa3-4f17-bc23-87860236c43a-kube-api-access-xtqbk\") pod \"insights-runtime-extractor-ssqgh\" (UID: \"d1058bfb-6fa3-4f17-bc23-87860236c43a\") " pod="openshift-insights/insights-runtime-extractor-ssqgh"
Sep 10 19:16:32.880986 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:32.880898    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d1058bfb-6fa3-4f17-bc23-87860236c43a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ssqgh\" (UID: \"d1058bfb-6fa3-4f17-bc23-87860236c43a\") " pod="openshift-insights/insights-runtime-extractor-ssqgh"
Sep 10 19:16:32.982161 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:32.982130    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d1058bfb-6fa3-4f17-bc23-87860236c43a-data-volume\") pod \"insights-runtime-extractor-ssqgh\" (UID: \"d1058bfb-6fa3-4f17-bc23-87860236c43a\") " pod="openshift-insights/insights-runtime-extractor-ssqgh"
Sep 10 19:16:32.982161 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:32.982164    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d1058bfb-6fa3-4f17-bc23-87860236c43a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ssqgh\" (UID: \"d1058bfb-6fa3-4f17-bc23-87860236c43a\") " pod="openshift-insights/insights-runtime-extractor-ssqgh"
Sep 10 19:16:32.982381 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:32.982191    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtqbk\" (UniqueName: \"kubernetes.io/projected/d1058bfb-6fa3-4f17-bc23-87860236c43a-kube-api-access-xtqbk\") pod \"insights-runtime-extractor-ssqgh\" (UID: \"d1058bfb-6fa3-4f17-bc23-87860236c43a\") " pod="openshift-insights/insights-runtime-extractor-ssqgh"
Sep 10 19:16:32.982381 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:32.982213    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d1058bfb-6fa3-4f17-bc23-87860236c43a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ssqgh\" (UID: \"d1058bfb-6fa3-4f17-bc23-87860236c43a\") " pod="openshift-insights/insights-runtime-extractor-ssqgh"
Sep 10 19:16:32.982381 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:32.982258    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d1058bfb-6fa3-4f17-bc23-87860236c43a-crio-socket\") pod \"insights-runtime-extractor-ssqgh\" (UID: \"d1058bfb-6fa3-4f17-bc23-87860236c43a\") " pod="openshift-insights/insights-runtime-extractor-ssqgh"
Sep 10 19:16:32.982381 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:32.982295    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 19:16:32.982381 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:32.982343    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:32.982613 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:32.982390    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d1058bfb-6fa3-4f17-bc23-87860236c43a-kube-rbac-proxy-cm podName:d1058bfb-6fa3-4f17-bc23-87860236c43a nodeName:}" failed. No retries permitted until 2026-09-10 19:16:33.482367916 +0000 UTC m=+48.702216400 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d1058bfb-6fa3-4f17-bc23-87860236c43a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ssqgh" (UID: "d1058bfb-6fa3-4f17-bc23-87860236c43a") : configmap "kube-rbac-proxy" not found
Sep 10 19:16:32.982613 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:32.982423    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1058bfb-6fa3-4f17-bc23-87860236c43a-insights-runtime-extractor-tls podName:d1058bfb-6fa3-4f17-bc23-87860236c43a nodeName:}" failed. No retries permitted until 2026-09-10 19:16:33.482406358 +0000 UTC m=+48.702254827 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d1058bfb-6fa3-4f17-bc23-87860236c43a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ssqgh" (UID: "d1058bfb-6fa3-4f17-bc23-87860236c43a") : secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:32.982613 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:32.982489    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d1058bfb-6fa3-4f17-bc23-87860236c43a-crio-socket\") pod \"insights-runtime-extractor-ssqgh\" (UID: \"d1058bfb-6fa3-4f17-bc23-87860236c43a\") " pod="openshift-insights/insights-runtime-extractor-ssqgh"
Sep 10 19:16:32.982613 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:32.982501    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d1058bfb-6fa3-4f17-bc23-87860236c43a-data-volume\") pod \"insights-runtime-extractor-ssqgh\" (UID: \"d1058bfb-6fa3-4f17-bc23-87860236c43a\") " pod="openshift-insights/insights-runtime-extractor-ssqgh"
Sep 10 19:16:32.993633 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:32.993608    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtqbk\" (UniqueName: \"kubernetes.io/projected/d1058bfb-6fa3-4f17-bc23-87860236c43a-kube-api-access-xtqbk\") pod \"insights-runtime-extractor-ssqgh\" (UID: \"d1058bfb-6fa3-4f17-bc23-87860236c43a\") " pod="openshift-insights/insights-runtime-extractor-ssqgh"
Sep 10 19:16:33.487221 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:33.487179    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d1058bfb-6fa3-4f17-bc23-87860236c43a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ssqgh\" (UID: \"d1058bfb-6fa3-4f17-bc23-87860236c43a\") " pod="openshift-insights/insights-runtime-extractor-ssqgh"
Sep 10 19:16:33.487382 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:33.487281    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d1058bfb-6fa3-4f17-bc23-87860236c43a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ssqgh\" (UID: \"d1058bfb-6fa3-4f17-bc23-87860236c43a\") " pod="openshift-insights/insights-runtime-extractor-ssqgh"
Sep 10 19:16:33.487382 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:33.487345    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:33.487382 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:33.487366    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 19:16:33.487484 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:33.487423    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1058bfb-6fa3-4f17-bc23-87860236c43a-insights-runtime-extractor-tls podName:d1058bfb-6fa3-4f17-bc23-87860236c43a nodeName:}" failed. No retries permitted until 2026-09-10 19:16:34.487399307 +0000 UTC m=+49.707247782 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d1058bfb-6fa3-4f17-bc23-87860236c43a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ssqgh" (UID: "d1058bfb-6fa3-4f17-bc23-87860236c43a") : secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:33.487484 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:33.487451    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d1058bfb-6fa3-4f17-bc23-87860236c43a-kube-rbac-proxy-cm podName:d1058bfb-6fa3-4f17-bc23-87860236c43a nodeName:}" failed. No retries permitted until 2026-09-10 19:16:34.487439314 +0000 UTC m=+49.707287791 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d1058bfb-6fa3-4f17-bc23-87860236c43a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ssqgh" (UID: "d1058bfb-6fa3-4f17-bc23-87860236c43a") : configmap "kube-rbac-proxy" not found
Sep 10 19:16:33.542235 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:33.542212    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bmdg9_9817270c-8629-44c4-ab8f-f3982e1f0e2f/console-operator/1.log"
Sep 10 19:16:33.542633 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:33.542502    2579 scope.go:117] "RemoveContainer" containerID="90f8d3a514d0fe91e04980d561bdddbb3f3216d0f2a31002cb7e368fd6c96ff3"
Sep 10 19:16:33.542690 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:33.542666    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-bmdg9_openshift-console-operator(9817270c-8629-44c4-ab8f-f3982e1f0e2f)\"" pod="openshift-console-operator/console-operator-85b454874b-bmdg9" podUID="9817270c-8629-44c4-ab8f-f3982e1f0e2f"
Sep 10 19:16:33.987673 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:33.987640    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-87jkz"]
Sep 10 19:16:34.007970 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:34.007932    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-87jkz"]
Sep 10 19:16:34.008111 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:34.008031    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-87jkz"
Sep 10 19:16:34.010149 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:34.010126    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 10 19:16:34.010312 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:34.010132    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 10 19:16:34.010649 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:34.010634    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-lqw9p\""
Sep 10 19:16:34.092607 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:34.092565    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9x4zz\" (UniqueName: \"kubernetes.io/projected/9a81694c-77e4-4fe2-9c3a-739d14ebfdbe-kube-api-access-9x4zz\") pod \"migrator-fdbb8fdff-87jkz\" (UID: \"9a81694c-77e4-4fe2-9c3a-739d14ebfdbe\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-87jkz"
Sep 10 19:16:34.194013 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:34.193975    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x4zz\" (UniqueName: \"kubernetes.io/projected/9a81694c-77e4-4fe2-9c3a-739d14ebfdbe-kube-api-access-9x4zz\") pod \"migrator-fdbb8fdff-87jkz\" (UID: \"9a81694c-77e4-4fe2-9c3a-739d14ebfdbe\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-87jkz"
Sep 10 19:16:34.203186 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:34.203163    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9x4zz\" (UniqueName: \"kubernetes.io/projected/9a81694c-77e4-4fe2-9c3a-739d14ebfdbe-kube-api-access-9x4zz\") pod \"migrator-fdbb8fdff-87jkz\" (UID: \"9a81694c-77e4-4fe2-9c3a-739d14ebfdbe\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-87jkz"
Sep 10 19:16:34.317650 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:34.317572    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-87jkz"
Sep 10 19:16:34.496514 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:34.496475    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d1058bfb-6fa3-4f17-bc23-87860236c43a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ssqgh\" (UID: \"d1058bfb-6fa3-4f17-bc23-87860236c43a\") " pod="openshift-insights/insights-runtime-extractor-ssqgh"
Sep 10 19:16:34.496690 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:34.496527    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d1058bfb-6fa3-4f17-bc23-87860236c43a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ssqgh\" (UID: \"d1058bfb-6fa3-4f17-bc23-87860236c43a\") " pod="openshift-insights/insights-runtime-extractor-ssqgh"
Sep 10 19:16:34.496690 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:34.496603    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 19:16:34.496690 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:34.496626    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:34.496690 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:34.496672    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1058bfb-6fa3-4f17-bc23-87860236c43a-insights-runtime-extractor-tls podName:d1058bfb-6fa3-4f17-bc23-87860236c43a nodeName:}" failed. No retries permitted until 2026-09-10 19:16:36.496658621 +0000 UTC m=+51.716507079 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d1058bfb-6fa3-4f17-bc23-87860236c43a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ssqgh" (UID: "d1058bfb-6fa3-4f17-bc23-87860236c43a") : secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:34.496690 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:34.496686    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d1058bfb-6fa3-4f17-bc23-87860236c43a-kube-rbac-proxy-cm podName:d1058bfb-6fa3-4f17-bc23-87860236c43a nodeName:}" failed. No retries permitted until 2026-09-10 19:16:36.496678726 +0000 UTC m=+51.716527184 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d1058bfb-6fa3-4f17-bc23-87860236c43a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ssqgh" (UID: "d1058bfb-6fa3-4f17-bc23-87860236c43a") : configmap "kube-rbac-proxy" not found
Sep 10 19:16:34.498021 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:34.498003    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-k8tfc_c357f393-afff-4968-aba1-3ef6f02620eb/dns-node-resolver/0.log"
Sep 10 19:16:34.599895 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:34.599823    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-87jkz"]
Sep 10 19:16:34.603282 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:16:34.603257    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a81694c_77e4_4fe2_9c3a_739d14ebfdbe.slice/crio-196971e5c49c082ae7c84e29d461017f6a59984851fa66f82391d6d20f178c5c WatchSource:0}: Error finding container 196971e5c49c082ae7c84e29d461017f6a59984851fa66f82391d6d20f178c5c: Status 404 returned error can't find the container with id 196971e5c49c082ae7c84e29d461017f6a59984851fa66f82391d6d20f178c5c
Sep 10 19:16:35.001133 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:35.001092    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-registry-tls\") pod \"image-registry-665c8cd58b-t27cc\" (UID: \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\") " pod="openshift-image-registry/image-registry-665c8cd58b-t27cc"
Sep 10 19:16:35.001363 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:35.001254    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 19:16:35.001363 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:35.001276    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-665c8cd58b-t27cc: secret "image-registry-tls" not found
Sep 10 19:16:35.001363 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:35.001331    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-registry-tls podName:20fcd10a-d90b-4eaf-96a0-06b5ec96437a nodeName:}" failed. No retries permitted until 2026-09-10 19:16:43.001314764 +0000 UTC m=+58.221163247 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-registry-tls") pod "image-registry-665c8cd58b-t27cc" (UID: "20fcd10a-d90b-4eaf-96a0-06b5ec96437a") : secret "image-registry-tls" not found
Sep 10 19:16:35.549558 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:35.549518    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-87jkz" event={"ID":"9a81694c-77e4-4fe2-9c3a-739d14ebfdbe","Type":"ContainerStarted","Data":"196971e5c49c082ae7c84e29d461017f6a59984851fa66f82391d6d20f178c5c"}
Sep 10 19:16:35.901268 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:35.901236    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zdnmx_98e70b32-791a-4a50-b558-5337fc78d24c/node-ca/0.log"
Sep 10 19:16:36.009921 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:36.009888    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/feafd719-ab13-485e-b195-ecbd0900e222-metrics-tls\") pod \"dns-default-vv7rr\" (UID: \"feafd719-ab13-485e-b195-ecbd0900e222\") " pod="openshift-dns/dns-default-vv7rr"
Sep 10 19:16:36.010069 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:36.010015    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 19:16:36.010108 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:36.010072    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feafd719-ab13-485e-b195-ecbd0900e222-metrics-tls podName:feafd719-ab13-485e-b195-ecbd0900e222 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:52.010056176 +0000 UTC m=+67.229904654 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/feafd719-ab13-485e-b195-ecbd0900e222-metrics-tls") pod "dns-default-vv7rr" (UID: "feafd719-ab13-485e-b195-ecbd0900e222") : secret "dns-default-metrics-tls" not found
Sep 10 19:16:36.501466 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:36.501446    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gdvkn_27b0fcb8-4e92-4753-8562-6e1e8bfa9e97/serve-healthcheck-canary/0.log"
Sep 10 19:16:36.515306 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:36.515279    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d1058bfb-6fa3-4f17-bc23-87860236c43a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ssqgh\" (UID: \"d1058bfb-6fa3-4f17-bc23-87860236c43a\") " pod="openshift-insights/insights-runtime-extractor-ssqgh"
Sep 10 19:16:36.515456 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:36.515319    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d1058bfb-6fa3-4f17-bc23-87860236c43a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ssqgh\" (UID: \"d1058bfb-6fa3-4f17-bc23-87860236c43a\") " pod="openshift-insights/insights-runtime-extractor-ssqgh"
Sep 10 19:16:36.515456 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:36.515393    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 19:16:36.515456 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:36.515415    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:36.515456 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:36.515450    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d1058bfb-6fa3-4f17-bc23-87860236c43a-kube-rbac-proxy-cm podName:d1058bfb-6fa3-4f17-bc23-87860236c43a nodeName:}" failed. No retries permitted until 2026-09-10 19:16:40.515437456 +0000 UTC m=+55.735285928 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d1058bfb-6fa3-4f17-bc23-87860236c43a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ssqgh" (UID: "d1058bfb-6fa3-4f17-bc23-87860236c43a") : configmap "kube-rbac-proxy" not found
Sep 10 19:16:36.515609 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:36.515465    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1058bfb-6fa3-4f17-bc23-87860236c43a-insights-runtime-extractor-tls podName:d1058bfb-6fa3-4f17-bc23-87860236c43a nodeName:}" failed. No retries permitted until 2026-09-10 19:16:40.515458394 +0000 UTC m=+55.735306853 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d1058bfb-6fa3-4f17-bc23-87860236c43a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ssqgh" (UID: "d1058bfb-6fa3-4f17-bc23-87860236c43a") : secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:36.557959 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:36.557928    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-87jkz" event={"ID":"9a81694c-77e4-4fe2-9c3a-739d14ebfdbe","Type":"ContainerStarted","Data":"7d93b57885d78aa8a57715dd2b82ada6454729b14a9bdc3b23dea2a81619860c"}
Sep 10 19:16:37.539461 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:37.539417    2579 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-bmdg9"
Sep 10 19:16:37.539927 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:37.539488    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-bmdg9"
Sep 10 19:16:37.539927 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:37.539847    2579 scope.go:117] "RemoveContainer" containerID="90f8d3a514d0fe91e04980d561bdddbb3f3216d0f2a31002cb7e368fd6c96ff3"
Sep 10 19:16:37.540018 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:37.539997    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-bmdg9_openshift-console-operator(9817270c-8629-44c4-ab8f-f3982e1f0e2f)\"" pod="openshift-console-operator/console-operator-85b454874b-bmdg9" podUID="9817270c-8629-44c4-ab8f-f3982e1f0e2f"
Sep 10 19:16:37.562456 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:37.562422    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-87jkz" event={"ID":"9a81694c-77e4-4fe2-9c3a-739d14ebfdbe","Type":"ContainerStarted","Data":"b5827656fd50167eeb71607b9026be26c8e8100956f311daad93b268eab59a7d"}
Sep 10 19:16:37.579448 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:37.579375    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-87jkz" podStartSLOduration=2.760238009 podStartE2EDuration="4.579362173s" totalImagesPullingTime="1.819124164s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:16:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:37.578798853 +0000 UTC m=+52.798647333" watchObservedRunningTime="2026-09-10 19:16:37.579362173 +0000 UTC m=+52.799210654"
Sep 10 19:16:40.547235 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:40.547189    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d1058bfb-6fa3-4f17-bc23-87860236c43a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ssqgh\" (UID: \"d1058bfb-6fa3-4f17-bc23-87860236c43a\") " pod="openshift-insights/insights-runtime-extractor-ssqgh"
Sep 10 19:16:40.547684 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:40.547247    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d1058bfb-6fa3-4f17-bc23-87860236c43a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ssqgh\" (UID: \"d1058bfb-6fa3-4f17-bc23-87860236c43a\") " pod="openshift-insights/insights-runtime-extractor-ssqgh"
Sep 10 19:16:40.547684 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:40.547315    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 19:16:40.547684 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:40.547383    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d1058bfb-6fa3-4f17-bc23-87860236c43a-kube-rbac-proxy-cm podName:d1058bfb-6fa3-4f17-bc23-87860236c43a nodeName:}" failed. No retries permitted until 2026-09-10 19:16:48.547365991 +0000 UTC m=+63.767214454 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d1058bfb-6fa3-4f17-bc23-87860236c43a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ssqgh" (UID: "d1058bfb-6fa3-4f17-bc23-87860236c43a") : configmap "kube-rbac-proxy" not found
Sep 10 19:16:40.547684 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:40.547406    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:40.547684 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:40.547469    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1058bfb-6fa3-4f17-bc23-87860236c43a-insights-runtime-extractor-tls podName:d1058bfb-6fa3-4f17-bc23-87860236c43a nodeName:}" failed. No retries permitted until 2026-09-10 19:16:48.547452205 +0000 UTC m=+63.767300682 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d1058bfb-6fa3-4f17-bc23-87860236c43a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ssqgh" (UID: "d1058bfb-6fa3-4f17-bc23-87860236c43a") : secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:43.064692 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:43.064638    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-registry-tls\") pod \"image-registry-665c8cd58b-t27cc\" (UID: \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\") " pod="openshift-image-registry/image-registry-665c8cd58b-t27cc"
Sep 10 19:16:43.067150 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:43.067121    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-registry-tls\") pod \"image-registry-665c8cd58b-t27cc\" (UID: \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\") " pod="openshift-image-registry/image-registry-665c8cd58b-t27cc"
Sep 10 19:16:43.124879 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:43.124841    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-665c8cd58b-t27cc"
Sep 10 19:16:43.398007 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:43.397978    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-665c8cd58b-t27cc"]
Sep 10 19:16:43.401243 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:16:43.401213    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20fcd10a_d90b_4eaf_96a0_06b5ec96437a.slice/crio-1e1a80b0f63b959068ff383c7e093a02be2f1d6ab87d0c7e8895036bd3348ab2 WatchSource:0}: Error finding container 1e1a80b0f63b959068ff383c7e093a02be2f1d6ab87d0c7e8895036bd3348ab2: Status 404 returned error can't find the container with id 1e1a80b0f63b959068ff383c7e093a02be2f1d6ab87d0c7e8895036bd3348ab2
Sep 10 19:16:43.582789 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:43.582741    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-665c8cd58b-t27cc" event={"ID":"20fcd10a-d90b-4eaf-96a0-06b5ec96437a","Type":"ContainerStarted","Data":"e97bcd6689239462bfc4a8ac78a148196b7a2673e9eff17831ca5b8e41202325"}
Sep 10 19:16:43.582979 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:43.582802    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-665c8cd58b-t27cc" event={"ID":"20fcd10a-d90b-4eaf-96a0-06b5ec96437a","Type":"ContainerStarted","Data":"1e1a80b0f63b959068ff383c7e093a02be2f1d6ab87d0c7e8895036bd3348ab2"}
Sep 10 19:16:43.582979 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:43.582879    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-665c8cd58b-t27cc"
Sep 10 19:16:43.601291 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:43.601240    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-665c8cd58b-t27cc" podStartSLOduration=16.601224455 podStartE2EDuration="16.601224455s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:16:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:43.600776979 +0000 UTC m=+58.820625460" watchObservedRunningTime="2026-09-10 19:16:43.601224455 +0000 UTC m=+58.821073347"
Sep 10 19:16:45.483875 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:45.483844    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8r66s"
Sep 10 19:16:48.609722 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:48.609687    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d1058bfb-6fa3-4f17-bc23-87860236c43a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ssqgh\" (UID: \"d1058bfb-6fa3-4f17-bc23-87860236c43a\") " pod="openshift-insights/insights-runtime-extractor-ssqgh"
Sep 10 19:16:48.610141 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:48.609799    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d1058bfb-6fa3-4f17-bc23-87860236c43a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ssqgh\" (UID: \"d1058bfb-6fa3-4f17-bc23-87860236c43a\") " pod="openshift-insights/insights-runtime-extractor-ssqgh"
Sep 10 19:16:48.610141 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:48.609859    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:48.610141 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:16:48.609917    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1058bfb-6fa3-4f17-bc23-87860236c43a-insights-runtime-extractor-tls podName:d1058bfb-6fa3-4f17-bc23-87860236c43a nodeName:}" failed. No retries permitted until 2026-09-10 19:17:04.609903049 +0000 UTC m=+79.829751508 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d1058bfb-6fa3-4f17-bc23-87860236c43a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ssqgh" (UID: "d1058bfb-6fa3-4f17-bc23-87860236c43a") : secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:48.610274 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:48.610251    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d1058bfb-6fa3-4f17-bc23-87860236c43a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ssqgh\" (UID: \"d1058bfb-6fa3-4f17-bc23-87860236c43a\") " pod="openshift-insights/insights-runtime-extractor-ssqgh"
Sep 10 19:16:50.233841 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:50.233808    2579 scope.go:117] "RemoveContainer" containerID="90f8d3a514d0fe91e04980d561bdddbb3f3216d0f2a31002cb7e368fd6c96ff3"
Sep 10 19:16:50.608050 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:50.607976    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bmdg9_9817270c-8629-44c4-ab8f-f3982e1f0e2f/console-operator/1.log"
Sep 10 19:16:50.608050 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:50.608030    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-bmdg9" event={"ID":"9817270c-8629-44c4-ab8f-f3982e1f0e2f","Type":"ContainerStarted","Data":"f7644735a36b1bd26d51b1933fe94cd38c5cc976ea82d05561f35d8807529996"}
Sep 10 19:16:50.608336 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:50.608314    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-bmdg9"
Sep 10 19:16:50.631184 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:50.631131    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-bmdg9" podStartSLOduration=20.392433061 podStartE2EDuration="23.631117418s" totalImagesPullingTime="3.238684357s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:16:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:50.630915355 +0000 UTC m=+65.850763835" watchObservedRunningTime="2026-09-10 19:16:50.631117418 +0000 UTC m=+65.850965899"
Sep 10 19:16:50.899516 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:50.899489    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-bmdg9"
Sep 10 19:16:50.926463 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:50.926428    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/acd262af-057c-4bbf-93e3-faa020c1825c-original-pull-secret\") pod \"global-pull-secret-syncer-mkkzj\" (UID: \"acd262af-057c-4bbf-93e3-faa020c1825c\") " pod="kube-system/global-pull-secret-syncer-mkkzj"
Sep 10 19:16:50.928658 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:50.928627    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 19:16:50.940417 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:50.940382    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/acd262af-057c-4bbf-93e3-faa020c1825c-original-pull-secret\") pod \"global-pull-secret-syncer-mkkzj\" (UID: \"acd262af-057c-4bbf-93e3-faa020c1825c\") " pod="kube-system/global-pull-secret-syncer-mkkzj"
Sep 10 19:16:51.128436 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:51.128396    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmbxc\" (UniqueName: \"kubernetes.io/projected/6a81beec-be63-4b8b-9999-b92fcda80550-kube-api-access-fmbxc\") pod \"network-check-target-dns2r\" (UID: \"6a81beec-be63-4b8b-9999-b92fcda80550\") " pod="openshift-network-diagnostics/network-check-target-dns2r"
Sep 10 19:16:51.128628 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:51.128454    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0-metrics-certs\") pod \"network-metrics-daemon-qjlbx\" (UID: \"af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0\") " pod="openshift-multus/network-metrics-daemon-qjlbx"
Sep 10 19:16:51.130803 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:51.130773    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 19:16:51.130803 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:51.130742    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 19:16:51.141624 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:51.141602    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 19:16:51.142199 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:51.142173    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0-metrics-certs\") pod \"network-metrics-daemon-qjlbx\" (UID: \"af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0\") " pod="openshift-multus/network-metrics-daemon-qjlbx"
Sep 10 19:16:51.144289 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:51.144270    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mkkzj"
Sep 10 19:16:51.152259 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:51.152235    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-q8jbb\""
Sep 10 19:16:51.152714 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:51.152696    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmbxc\" (UniqueName: \"kubernetes.io/projected/6a81beec-be63-4b8b-9999-b92fcda80550-kube-api-access-fmbxc\") pod \"network-check-target-dns2r\" (UID: \"6a81beec-be63-4b8b-9999-b92fcda80550\") " pod="openshift-network-diagnostics/network-check-target-dns2r"
Sep 10 19:16:51.159149 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:51.159129    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tsrfs\""
Sep 10 19:16:51.161158 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:51.161133    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjlbx"
Sep 10 19:16:51.168022 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:51.168000    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dns2r"
Sep 10 19:16:51.413934 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:51.413672    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mkkzj"]
Sep 10 19:16:51.416183 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:16:51.416157    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacd262af_057c_4bbf_93e3_faa020c1825c.slice/crio-d6681b5be2cf0e1d84a66c75cbea152c81e4ea676f6b388ee6ec706e2f3b4970 WatchSource:0}: Error finding container d6681b5be2cf0e1d84a66c75cbea152c81e4ea676f6b388ee6ec706e2f3b4970: Status 404 returned error can't find the container with id d6681b5be2cf0e1d84a66c75cbea152c81e4ea676f6b388ee6ec706e2f3b4970
Sep 10 19:16:51.440977 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:51.440946    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qjlbx"]
Sep 10 19:16:51.443814 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:16:51.443786    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf9a4f3f_e1b3_41e5_b930_fa5a3f49e5b0.slice/crio-e7c5ba83f1f6e277b6017016e605ac1ef7c2ef87e46158292c5e892e01f477f2 WatchSource:0}: Error finding container e7c5ba83f1f6e277b6017016e605ac1ef7c2ef87e46158292c5e892e01f477f2: Status 404 returned error can't find the container with id e7c5ba83f1f6e277b6017016e605ac1ef7c2ef87e46158292c5e892e01f477f2
Sep 10 19:16:51.463203 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:51.463171    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dns2r"]
Sep 10 19:16:51.466482 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:16:51.466458    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a81beec_be63_4b8b_9999_b92fcda80550.slice/crio-db27e08b664d1e392549a376448f4d3dd36ec8c9c11e1f7c19150c5cbba485e5 WatchSource:0}: Error finding container db27e08b664d1e392549a376448f4d3dd36ec8c9c11e1f7c19150c5cbba485e5: Status 404 returned error can't find the container with id db27e08b664d1e392549a376448f4d3dd36ec8c9c11e1f7c19150c5cbba485e5
Sep 10 19:16:51.612216 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:51.612183    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dns2r" event={"ID":"6a81beec-be63-4b8b-9999-b92fcda80550","Type":"ContainerStarted","Data":"db27e08b664d1e392549a376448f4d3dd36ec8c9c11e1f7c19150c5cbba485e5"}
Sep 10 19:16:51.613273 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:51.613246    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mkkzj" event={"ID":"acd262af-057c-4bbf-93e3-faa020c1825c","Type":"ContainerStarted","Data":"d6681b5be2cf0e1d84a66c75cbea152c81e4ea676f6b388ee6ec706e2f3b4970"}
Sep 10 19:16:51.614289 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:51.614265    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qjlbx" event={"ID":"af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0","Type":"ContainerStarted","Data":"e7c5ba83f1f6e277b6017016e605ac1ef7c2ef87e46158292c5e892e01f477f2"}
Sep 10 19:16:52.036499 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:52.036459    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/feafd719-ab13-485e-b195-ecbd0900e222-metrics-tls\") pod \"dns-default-vv7rr\" (UID: \"feafd719-ab13-485e-b195-ecbd0900e222\") " pod="openshift-dns/dns-default-vv7rr"
Sep 10 19:16:52.039768 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:52.039717    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/feafd719-ab13-485e-b195-ecbd0900e222-metrics-tls\") pod \"dns-default-vv7rr\" (UID: \"feafd719-ab13-485e-b195-ecbd0900e222\") " pod="openshift-dns/dns-default-vv7rr"
Sep 10 19:16:52.245203 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:52.245166    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wks57\""
Sep 10 19:16:52.253548 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:52.253519    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vv7rr"
Sep 10 19:16:52.731104 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:52.731075    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vv7rr"]
Sep 10 19:16:52.799893 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:16:52.799851    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeafd719_ab13_485e_b195_ecbd0900e222.slice/crio-8812a8e36f595bd6dbc074dcb5d076a1daf3f415590eb5fa2c0a7aa8fcb1ba73 WatchSource:0}: Error finding container 8812a8e36f595bd6dbc074dcb5d076a1daf3f415590eb5fa2c0a7aa8fcb1ba73: Status 404 returned error can't find the container with id 8812a8e36f595bd6dbc074dcb5d076a1daf3f415590eb5fa2c0a7aa8fcb1ba73
Sep 10 19:16:53.625415 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:53.625371    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qjlbx" event={"ID":"af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0","Type":"ContainerStarted","Data":"2e6da02f9d5ecba22eecd5895f0616e3348849e374588d381ba1aba25ce0073e"}
Sep 10 19:16:53.625585 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:53.625434    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qjlbx" event={"ID":"af9a4f3f-e1b3-41e5-b930-fa5a3f49e5b0","Type":"ContainerStarted","Data":"8f36d0a3db604502a65ad77128fd6f1e53792c65433667596ca7f363429e8435"}
Sep 10 19:16:53.626913 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:53.626883    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vv7rr" event={"ID":"feafd719-ab13-485e-b195-ecbd0900e222","Type":"ContainerStarted","Data":"8812a8e36f595bd6dbc074dcb5d076a1daf3f415590eb5fa2c0a7aa8fcb1ba73"}
Sep 10 19:16:53.642195 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:53.642138    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-qjlbx" podStartSLOduration=66.259382657 podStartE2EDuration="1m7.642117497s" totalImagesPullingTime="1.38273484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:15:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:53.641500776 +0000 UTC m=+68.861349257" watchObservedRunningTime="2026-09-10 19:16:53.642117497 +0000 UTC m=+68.861965979"
Sep 10 19:16:57.643423 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:57.643385    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dns2r" event={"ID":"6a81beec-be63-4b8b-9999-b92fcda80550","Type":"ContainerStarted","Data":"ef9fe78167014e3f88af9c8bc4c7fcff4c664a8d17dfae9e498b78dd1c657457"}
Sep 10 19:16:57.643885 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:57.643543    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-dns2r"
Sep 10 19:16:57.644931 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:57.644908    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mkkzj" event={"ID":"acd262af-057c-4bbf-93e3-faa020c1825c","Type":"ContainerStarted","Data":"8703ca0debeb4e8beb13dc5bc5adddeffeb7f116955bb071ec45bea369cc0315"}
Sep 10 19:16:57.646876 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:57.646857    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vv7rr" event={"ID":"feafd719-ab13-485e-b195-ecbd0900e222","Type":"ContainerStarted","Data":"9cbf506dae0b7a0462b35c85824f36b52eb4891f4e7dbcd98f46515869c4a831"}
Sep 10 19:16:57.646970 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:57.646897    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vv7rr" event={"ID":"feafd719-ab13-485e-b195-ecbd0900e222","Type":"ContainerStarted","Data":"d774da5905dca115e3f5aa463bbb4976715476fa1ef6abd22af4cc798d21441c"}
Sep 10 19:16:57.647035 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:57.646989    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vv7rr"
Sep 10 19:16:57.658454 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:57.658414    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-dns2r" podStartSLOduration=66.04949265 podStartE2EDuration="1m11.658400156s" totalImagesPullingTime="5.608907506s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:15:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:57.658138524 +0000 UTC m=+72.877987005" watchObservedRunningTime="2026-09-10 19:16:57.658400156 +0000 UTC m=+72.878248666"
Sep 10 19:16:57.673552 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:57.673510    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-mkkzj" podStartSLOduration=66.025870965 podStartE2EDuration="1m11.673494438s" totalImagesPullingTime="5.647623473s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:57.673004537 +0000 UTC m=+72.892853018" watchObservedRunningTime="2026-09-10 19:16:57.673494438 +0000 UTC m=+72.893342909"
Sep 10 19:16:57.697856 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:57.697792    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vv7rr" podStartSLOduration=33.454392481 podStartE2EDuration="37.697770556s" totalImagesPullingTime="4.243378075s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:16:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:57.696123266 +0000 UTC m=+72.915971747" watchObservedRunningTime="2026-09-10 19:16:57.697770556 +0000 UTC m=+72.917619038"
Sep 10 19:16:58.256080 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.256042    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-554589bb78-drh2c"]
Sep 10 19:16:58.258988 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.258966    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-554589bb78-drh2c"
Sep 10 19:16:58.263542 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.263517    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-7tt5s\""
Sep 10 19:16:58.264644 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.264614    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 10 19:16:58.264791 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.264777    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 10 19:16:58.266040 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.266022    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 10 19:16:58.266262 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.266242    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 10 19:16:58.287281 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.287252    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-554589bb78-drh2c"]
Sep 10 19:16:58.292652 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.292628    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-665c8cd58b-t27cc"]
Sep 10 19:16:58.297235 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.297205    2579 patch_prober.go:28] interesting pod/image-registry-665c8cd58b-t27cc container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 10 19:16:58.297378 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.297266    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-665c8cd58b-t27cc" podUID="20fcd10a-d90b-4eaf-96a0-06b5ec96437a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 19:16:58.344768 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.344718    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-757bf78f57-kszf7"]
Sep 10 19:16:58.347620 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.347603    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-757bf78f57-kszf7"
Sep 10 19:16:58.367032 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.367002    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-757bf78f57-kszf7"]
Sep 10 19:16:58.390819 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.390780    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgxcs\" (UniqueName: \"kubernetes.io/projected/f9528bef-58a1-4f36-9ece-185c3881c62c-kube-api-access-dgxcs\") pod \"managed-serviceaccount-addon-agent-554589bb78-drh2c\" (UID: \"f9528bef-58a1-4f36-9ece-185c3881c62c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-554589bb78-drh2c"
Sep 10 19:16:58.391004 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.390831    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f9528bef-58a1-4f36-9ece-185c3881c62c-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-554589bb78-drh2c\" (UID: \"f9528bef-58a1-4f36-9ece-185c3881c62c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-554589bb78-drh2c"
Sep 10 19:16:58.492028 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.491990    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgxcs\" (UniqueName: \"kubernetes.io/projected/f9528bef-58a1-4f36-9ece-185c3881c62c-kube-api-access-dgxcs\") pod \"managed-serviceaccount-addon-agent-554589bb78-drh2c\" (UID: \"f9528bef-58a1-4f36-9ece-185c3881c62c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-554589bb78-drh2c"
Sep 10 19:16:58.492028 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.492030    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdb58\" (UniqueName: \"kubernetes.io/projected/db963baa-0057-405d-b1b1-2307a554d4c3-kube-api-access-zdb58\") pod \"image-registry-757bf78f57-kszf7\" (UID: \"db963baa-0057-405d-b1b1-2307a554d4c3\") " pod="openshift-image-registry/image-registry-757bf78f57-kszf7"
Sep 10 19:16:58.492253 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.492059    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f9528bef-58a1-4f36-9ece-185c3881c62c-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-554589bb78-drh2c\" (UID: \"f9528bef-58a1-4f36-9ece-185c3881c62c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-554589bb78-drh2c"
Sep 10 19:16:58.492253 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.492086    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/db963baa-0057-405d-b1b1-2307a554d4c3-image-registry-private-configuration\") pod \"image-registry-757bf78f57-kszf7\" (UID: \"db963baa-0057-405d-b1b1-2307a554d4c3\") " pod="openshift-image-registry/image-registry-757bf78f57-kszf7"
Sep 10 19:16:58.492253 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.492111    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/db963baa-0057-405d-b1b1-2307a554d4c3-bound-sa-token\") pod \"image-registry-757bf78f57-kszf7\" (UID: \"db963baa-0057-405d-b1b1-2307a554d4c3\") " pod="openshift-image-registry/image-registry-757bf78f57-kszf7"
Sep 10 19:16:58.492253 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.492137    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/db963baa-0057-405d-b1b1-2307a554d4c3-registry-tls\") pod \"image-registry-757bf78f57-kszf7\" (UID: \"db963baa-0057-405d-b1b1-2307a554d4c3\") " pod="openshift-image-registry/image-registry-757bf78f57-kszf7"
Sep 10 19:16:58.492253 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.492212    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/db963baa-0057-405d-b1b1-2307a554d4c3-registry-certificates\") pod \"image-registry-757bf78f57-kszf7\" (UID: \"db963baa-0057-405d-b1b1-2307a554d4c3\") " pod="openshift-image-registry/image-registry-757bf78f57-kszf7"
Sep 10 19:16:58.492253 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.492234    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/db963baa-0057-405d-b1b1-2307a554d4c3-installation-pull-secrets\") pod \"image-registry-757bf78f57-kszf7\" (UID: \"db963baa-0057-405d-b1b1-2307a554d4c3\") " pod="openshift-image-registry/image-registry-757bf78f57-kszf7"
Sep 10 19:16:58.492462 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.492284    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/db963baa-0057-405d-b1b1-2307a554d4c3-trusted-ca\") pod \"image-registry-757bf78f57-kszf7\" (UID: \"db963baa-0057-405d-b1b1-2307a554d4c3\") " pod="openshift-image-registry/image-registry-757bf78f57-kszf7"
Sep 10 19:16:58.492462 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.492327    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/db963baa-0057-405d-b1b1-2307a554d4c3-ca-trust-extracted\") pod \"image-registry-757bf78f57-kszf7\" (UID: \"db963baa-0057-405d-b1b1-2307a554d4c3\") " pod="openshift-image-registry/image-registry-757bf78f57-kszf7"
Sep 10 19:16:58.494732 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.494709    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f9528bef-58a1-4f36-9ece-185c3881c62c-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-554589bb78-drh2c\" (UID: \"f9528bef-58a1-4f36-9ece-185c3881c62c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-554589bb78-drh2c"
Sep 10 19:16:58.504082 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.504050    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgxcs\" (UniqueName: \"kubernetes.io/projected/f9528bef-58a1-4f36-9ece-185c3881c62c-kube-api-access-dgxcs\") pod \"managed-serviceaccount-addon-agent-554589bb78-drh2c\" (UID: \"f9528bef-58a1-4f36-9ece-185c3881c62c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-554589bb78-drh2c"
Sep 10 19:16:58.573830 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.573718    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-554589bb78-drh2c"
Sep 10 19:16:58.592859 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.592817    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/db963baa-0057-405d-b1b1-2307a554d4c3-registry-tls\") pod \"image-registry-757bf78f57-kszf7\" (UID: \"db963baa-0057-405d-b1b1-2307a554d4c3\") " pod="openshift-image-registry/image-registry-757bf78f57-kszf7"
Sep 10 19:16:58.592987 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.592873    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/db963baa-0057-405d-b1b1-2307a554d4c3-registry-certificates\") pod \"image-registry-757bf78f57-kszf7\" (UID: \"db963baa-0057-405d-b1b1-2307a554d4c3\") " pod="openshift-image-registry/image-registry-757bf78f57-kszf7"
Sep 10 19:16:58.592987 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.592904    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/db963baa-0057-405d-b1b1-2307a554d4c3-installation-pull-secrets\") pod \"image-registry-757bf78f57-kszf7\" (UID: \"db963baa-0057-405d-b1b1-2307a554d4c3\") " pod="openshift-image-registry/image-registry-757bf78f57-kszf7"
Sep 10 19:16:58.592987 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.592963    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/db963baa-0057-405d-b1b1-2307a554d4c3-trusted-ca\") pod \"image-registry-757bf78f57-kszf7\" (UID: \"db963baa-0057-405d-b1b1-2307a554d4c3\") " pod="openshift-image-registry/image-registry-757bf78f57-kszf7"
Sep 10 19:16:58.593118 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.593006    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/db963baa-0057-405d-b1b1-2307a554d4c3-ca-trust-extracted\") pod \"image-registry-757bf78f57-kszf7\" (UID: \"db963baa-0057-405d-b1b1-2307a554d4c3\") " pod="openshift-image-registry/image-registry-757bf78f57-kszf7"
Sep 10 19:16:58.593118 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.593049    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdb58\" (UniqueName: \"kubernetes.io/projected/db963baa-0057-405d-b1b1-2307a554d4c3-kube-api-access-zdb58\") pod \"image-registry-757bf78f57-kszf7\" (UID: \"db963baa-0057-405d-b1b1-2307a554d4c3\") " pod="openshift-image-registry/image-registry-757bf78f57-kszf7"
Sep 10 19:16:58.593118 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.593079    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/db963baa-0057-405d-b1b1-2307a554d4c3-image-registry-private-configuration\") pod \"image-registry-757bf78f57-kszf7\" (UID: \"db963baa-0057-405d-b1b1-2307a554d4c3\") " pod="openshift-image-registry/image-registry-757bf78f57-kszf7"
Sep 10 19:16:58.593118 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.593103    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/db963baa-0057-405d-b1b1-2307a554d4c3-bound-sa-token\") pod \"image-registry-757bf78f57-kszf7\" (UID: \"db963baa-0057-405d-b1b1-2307a554d4c3\") " pod="openshift-image-registry/image-registry-757bf78f57-kszf7"
Sep 10 19:16:58.593544 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.593465    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/db963baa-0057-405d-b1b1-2307a554d4c3-ca-trust-extracted\") pod \"image-registry-757bf78f57-kszf7\" (UID: \"db963baa-0057-405d-b1b1-2307a554d4c3\") " pod="openshift-image-registry/image-registry-757bf78f57-kszf7"
Sep 10 19:16:58.593820 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.593797    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/db963baa-0057-405d-b1b1-2307a554d4c3-registry-certificates\") pod \"image-registry-757bf78f57-kszf7\" (UID: \"db963baa-0057-405d-b1b1-2307a554d4c3\") " pod="openshift-image-registry/image-registry-757bf78f57-kszf7"
Sep 10 19:16:58.594023 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.594000    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/db963baa-0057-405d-b1b1-2307a554d4c3-trusted-ca\") pod \"image-registry-757bf78f57-kszf7\" (UID: \"db963baa-0057-405d-b1b1-2307a554d4c3\") " pod="openshift-image-registry/image-registry-757bf78f57-kszf7"
Sep 10 19:16:58.596011 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.595984    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/db963baa-0057-405d-b1b1-2307a554d4c3-image-registry-private-configuration\") pod \"image-registry-757bf78f57-kszf7\" (UID: \"db963baa-0057-405d-b1b1-2307a554d4c3\") " pod="openshift-image-registry/image-registry-757bf78f57-kszf7"
Sep 10 19:16:58.596385 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.596360    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/db963baa-0057-405d-b1b1-2307a554d4c3-installation-pull-secrets\") pod \"image-registry-757bf78f57-kszf7\" (UID: \"db963baa-0057-405d-b1b1-2307a554d4c3\") " pod="openshift-image-registry/image-registry-757bf78f57-kszf7"
Sep 10 19:16:58.596484 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.596394    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/db963baa-0057-405d-b1b1-2307a554d4c3-registry-tls\") pod \"image-registry-757bf78f57-kszf7\" (UID: \"db963baa-0057-405d-b1b1-2307a554d4c3\") " pod="openshift-image-registry/image-registry-757bf78f57-kszf7"
Sep 10 19:16:58.603859 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.603839    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdb58\" (UniqueName: \"kubernetes.io/projected/db963baa-0057-405d-b1b1-2307a554d4c3-kube-api-access-zdb58\") pod \"image-registry-757bf78f57-kszf7\" (UID: \"db963baa-0057-405d-b1b1-2307a554d4c3\") " pod="openshift-image-registry/image-registry-757bf78f57-kszf7"
Sep 10 19:16:58.604613 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.604594    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/db963baa-0057-405d-b1b1-2307a554d4c3-bound-sa-token\") pod \"image-registry-757bf78f57-kszf7\" (UID: \"db963baa-0057-405d-b1b1-2307a554d4c3\") " pod="openshift-image-registry/image-registry-757bf78f57-kszf7"
Sep 10 19:16:58.657037 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.657009    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-757bf78f57-kszf7"
Sep 10 19:16:58.843475 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.843402    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-554589bb78-drh2c"]
Sep 10 19:16:58.847097 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:16:58.847068    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf9528bef_58a1_4f36_9ece_185c3881c62c.slice/crio-d50dc663d23f284e5fc1cb8317350b5c93059d35ffe5cafaf195086c232f8af0 WatchSource:0}: Error finding container d50dc663d23f284e5fc1cb8317350b5c93059d35ffe5cafaf195086c232f8af0: Status 404 returned error can't find the container with id d50dc663d23f284e5fc1cb8317350b5c93059d35ffe5cafaf195086c232f8af0
Sep 10 19:16:58.935661 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:58.935551    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-757bf78f57-kszf7"]
Sep 10 19:16:58.938092 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:16:58.938069    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb963baa_0057_405d_b1b1_2307a554d4c3.slice/crio-2f2201b7ac568b50cf9ae5f6d77e3565349cc8d0315eb0ad95dea28d87ec41cd WatchSource:0}: Error finding container 2f2201b7ac568b50cf9ae5f6d77e3565349cc8d0315eb0ad95dea28d87ec41cd: Status 404 returned error can't find the container with id 2f2201b7ac568b50cf9ae5f6d77e3565349cc8d0315eb0ad95dea28d87ec41cd
Sep 10 19:16:59.657208 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:59.657165    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-554589bb78-drh2c" event={"ID":"f9528bef-58a1-4f36-9ece-185c3881c62c","Type":"ContainerStarted","Data":"d50dc663d23f284e5fc1cb8317350b5c93059d35ffe5cafaf195086c232f8af0"}
Sep 10 19:16:59.659600 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:59.659103    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-757bf78f57-kszf7" event={"ID":"db963baa-0057-405d-b1b1-2307a554d4c3","Type":"ContainerStarted","Data":"90111fea3b59507164d12e35c4c6d0e23f5f6f25320184ad1de7aec07fc8ab8d"}
Sep 10 19:16:59.659600 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:59.659151    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-757bf78f57-kszf7" event={"ID":"db963baa-0057-405d-b1b1-2307a554d4c3","Type":"ContainerStarted","Data":"2f2201b7ac568b50cf9ae5f6d77e3565349cc8d0315eb0ad95dea28d87ec41cd"}
Sep 10 19:16:59.659600 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:59.659231    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-757bf78f57-kszf7"
Sep 10 19:16:59.683764 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:16:59.683668    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-757bf78f57-kszf7" podStartSLOduration=1.683645904 podStartE2EDuration="1.683645904s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:16:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:59.682657506 +0000 UTC m=+74.902505990" watchObservedRunningTime="2026-09-10 19:16:59.683645904 +0000 UTC m=+74.903494387"
Sep 10 19:17:01.668662 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:01.668620    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-554589bb78-drh2c" event={"ID":"f9528bef-58a1-4f36-9ece-185c3881c62c","Type":"ContainerStarted","Data":"33b00bd80cbdfcd44e5315c1b3f0e990f231bc8913ee7bc4742faf4b4d67a2c4"}
Sep 10 19:17:01.684765 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:01.684688    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-554589bb78-drh2c" podStartSLOduration=1.070881842 podStartE2EDuration="3.68467085s" totalImagesPullingTime="2.613789008s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:16:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:17:01.684507038 +0000 UTC m=+76.904355518" watchObservedRunningTime="2026-09-10 19:17:01.68467085 +0000 UTC m=+76.904519392"
Sep 10 19:17:04.644386 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:04.644349    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d1058bfb-6fa3-4f17-bc23-87860236c43a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ssqgh\" (UID: \"d1058bfb-6fa3-4f17-bc23-87860236c43a\") " pod="openshift-insights/insights-runtime-extractor-ssqgh"
Sep 10 19:17:04.646838 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:04.646817    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d1058bfb-6fa3-4f17-bc23-87860236c43a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ssqgh\" (UID: \"d1058bfb-6fa3-4f17-bc23-87860236c43a\") " pod="openshift-insights/insights-runtime-extractor-ssqgh"
Sep 10 19:17:04.896072 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:04.895991    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-q7pt8\""
Sep 10 19:17:04.904466 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:04.904447    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ssqgh"
Sep 10 19:17:05.174962 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:05.174883    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ssqgh"]
Sep 10 19:17:05.177990 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:17:05.177966    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1058bfb_6fa3_4f17_bc23_87860236c43a.slice/crio-1b3b5dbf30a89616e3c5edda4ed2d1c8145fed29dd3e30c55d0a11722851c921 WatchSource:0}: Error finding container 1b3b5dbf30a89616e3c5edda4ed2d1c8145fed29dd3e30c55d0a11722851c921: Status 404 returned error can't find the container with id 1b3b5dbf30a89616e3c5edda4ed2d1c8145fed29dd3e30c55d0a11722851c921
Sep 10 19:17:05.683793 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:05.683734    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ssqgh" event={"ID":"d1058bfb-6fa3-4f17-bc23-87860236c43a","Type":"ContainerStarted","Data":"a95647261ae3be44f382e6ce52e21514412c7250693aa51d97f905ff135685b3"}
Sep 10 19:17:05.684173 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:05.683815    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ssqgh" event={"ID":"d1058bfb-6fa3-4f17-bc23-87860236c43a","Type":"ContainerStarted","Data":"1b3b5dbf30a89616e3c5edda4ed2d1c8145fed29dd3e30c55d0a11722851c921"}
Sep 10 19:17:06.691122 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:06.691053    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ssqgh" event={"ID":"d1058bfb-6fa3-4f17-bc23-87860236c43a","Type":"ContainerStarted","Data":"b421f1e4e1c88b8e939933b1bfd5c79c64715c6c700da3ef91d4dcb977fdbe7d"}
Sep 10 19:17:07.019869 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:07.019778    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7959dbb847-5lm7n"]
Sep 10 19:17:07.023438 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:07.023417    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7959dbb847-5lm7n"
Sep 10 19:17:07.026798 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:07.026774    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 10 19:17:07.027062 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:07.027047    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 10 19:17:07.027497 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:07.027476    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 10 19:17:07.027663 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:07.027481    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 10 19:17:07.027804 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:07.027781    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-kgnp2\""
Sep 10 19:17:07.027804 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:07.027795    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 10 19:17:07.028127 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:07.027953    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 10 19:17:07.028127 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:07.027968    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 10 19:17:07.032597 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:07.032281    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 10 19:17:07.036324 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:07.036298    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7959dbb847-5lm7n"]
Sep 10 19:17:07.164258 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:07.164210    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2e41614d-6b68-4fd4-a424-e2185a53e6cd-oauth-serving-cert\") pod \"console-7959dbb847-5lm7n\" (UID: \"2e41614d-6b68-4fd4-a424-e2185a53e6cd\") " pod="openshift-console/console-7959dbb847-5lm7n"
Sep 10 19:17:07.164449 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:07.164270    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e41614d-6b68-4fd4-a424-e2185a53e6cd-console-serving-cert\") pod \"console-7959dbb847-5lm7n\" (UID: \"2e41614d-6b68-4fd4-a424-e2185a53e6cd\") " pod="openshift-console/console-7959dbb847-5lm7n"
Sep 10 19:17:07.164449 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:07.164297    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2e41614d-6b68-4fd4-a424-e2185a53e6cd-console-config\") pod \"console-7959dbb847-5lm7n\" (UID: \"2e41614d-6b68-4fd4-a424-e2185a53e6cd\") " pod="openshift-console/console-7959dbb847-5lm7n"
Sep 10 19:17:07.164449 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:07.164408    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4km7\" (UniqueName: \"kubernetes.io/projected/2e41614d-6b68-4fd4-a424-e2185a53e6cd-kube-api-access-f4km7\") pod \"console-7959dbb847-5lm7n\" (UID: \"2e41614d-6b68-4fd4-a424-e2185a53e6cd\") " pod="openshift-console/console-7959dbb847-5lm7n"
Sep 10 19:17:07.164449 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:07.164441    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2e41614d-6b68-4fd4-a424-e2185a53e6cd-console-oauth-config\") pod \"console-7959dbb847-5lm7n\" (UID: \"2e41614d-6b68-4fd4-a424-e2185a53e6cd\") " pod="openshift-console/console-7959dbb847-5lm7n"
Sep 10 19:17:07.164683 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:07.164501    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e41614d-6b68-4fd4-a424-e2185a53e6cd-service-ca\") pod \"console-7959dbb847-5lm7n\" (UID: \"2e41614d-6b68-4fd4-a424-e2185a53e6cd\") " pod="openshift-console/console-7959dbb847-5lm7n"
Sep 10 19:17:07.164683 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:07.164528    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e41614d-6b68-4fd4-a424-e2185a53e6cd-trusted-ca-bundle\") pod \"console-7959dbb847-5lm7n\" (UID: \"2e41614d-6b68-4fd4-a424-e2185a53e6cd\") " pod="openshift-console/console-7959dbb847-5lm7n"
Sep 10 19:17:07.265091 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:07.265046    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e41614d-6b68-4fd4-a424-e2185a53e6cd-service-ca\") pod \"console-7959dbb847-5lm7n\" (UID: \"2e41614d-6b68-4fd4-a424-e2185a53e6cd\") " pod="openshift-console/console-7959dbb847-5lm7n"
Sep 10 19:17:07.265283 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:07.265106    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e41614d-6b68-4fd4-a424-e2185a53e6cd-trusted-ca-bundle\") pod \"console-7959dbb847-5lm7n\" (UID: \"2e41614d-6b68-4fd4-a424-e2185a53e6cd\") " pod="openshift-console/console-7959dbb847-5lm7n"
Sep 10 19:17:07.265283 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:07.265145    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2e41614d-6b68-4fd4-a424-e2185a53e6cd-oauth-serving-cert\") pod \"console-7959dbb847-5lm7n\" (UID: \"2e41614d-6b68-4fd4-a424-e2185a53e6cd\") " pod="openshift-console/console-7959dbb847-5lm7n"
Sep 10 19:17:07.265283 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:07.265182    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e41614d-6b68-4fd4-a424-e2185a53e6cd-console-serving-cert\") pod \"console-7959dbb847-5lm7n\" (UID: \"2e41614d-6b68-4fd4-a424-e2185a53e6cd\") " pod="openshift-console/console-7959dbb847-5lm7n"
Sep 10 19:17:07.265283 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:07.265207    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2e41614d-6b68-4fd4-a424-e2185a53e6cd-console-config\") pod \"console-7959dbb847-5lm7n\" (UID: \"2e41614d-6b68-4fd4-a424-e2185a53e6cd\") " pod="openshift-console/console-7959dbb847-5lm7n"
Sep 10 19:17:07.265283 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:07.265238    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4km7\" (UniqueName: \"kubernetes.io/projected/2e41614d-6b68-4fd4-a424-e2185a53e6cd-kube-api-access-f4km7\") pod \"console-7959dbb847-5lm7n\" (UID: \"2e41614d-6b68-4fd4-a424-e2185a53e6cd\") " pod="openshift-console/console-7959dbb847-5lm7n"
Sep 10 19:17:07.265283 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:07.265264    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2e41614d-6b68-4fd4-a424-e2185a53e6cd-console-oauth-config\") pod \"console-7959dbb847-5lm7n\" (UID: \"2e41614d-6b68-4fd4-a424-e2185a53e6cd\") " pod="openshift-console/console-7959dbb847-5lm7n"
Sep 10 19:17:07.265866 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:07.265817    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e41614d-6b68-4fd4-a424-e2185a53e6cd-service-ca\") pod \"console-7959dbb847-5lm7n\" (UID: \"2e41614d-6b68-4fd4-a424-e2185a53e6cd\") " pod="openshift-console/console-7959dbb847-5lm7n"
Sep 10 19:17:07.266419 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:07.266368    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e41614d-6b68-4fd4-a424-e2185a53e6cd-trusted-ca-bundle\") pod \"console-7959dbb847-5lm7n\" (UID: \"2e41614d-6b68-4fd4-a424-e2185a53e6cd\") " pod="openshift-console/console-7959dbb847-5lm7n"
Sep 10 19:17:07.266627 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:07.266586    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2e41614d-6b68-4fd4-a424-e2185a53e6cd-console-config\") pod \"console-7959dbb847-5lm7n\" (UID: \"2e41614d-6b68-4fd4-a424-e2185a53e6cd\") " pod="openshift-console/console-7959dbb847-5lm7n"
Sep 10 19:17:07.266973 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:07.266936    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2e41614d-6b68-4fd4-a424-e2185a53e6cd-oauth-serving-cert\") pod \"console-7959dbb847-5lm7n\" (UID: \"2e41614d-6b68-4fd4-a424-e2185a53e6cd\") " pod="openshift-console/console-7959dbb847-5lm7n"
Sep 10 19:17:07.268928 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:07.268911    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e41614d-6b68-4fd4-a424-e2185a53e6cd-console-serving-cert\") pod \"console-7959dbb847-5lm7n\" (UID: \"2e41614d-6b68-4fd4-a424-e2185a53e6cd\") " pod="openshift-console/console-7959dbb847-5lm7n"
Sep 10 19:17:07.269187 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:07.269163    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2e41614d-6b68-4fd4-a424-e2185a53e6cd-console-oauth-config\") pod \"console-7959dbb847-5lm7n\" (UID: \"2e41614d-6b68-4fd4-a424-e2185a53e6cd\") " pod="openshift-console/console-7959dbb847-5lm7n"
Sep 10 19:17:07.278882 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:07.278827    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4km7\" (UniqueName: \"kubernetes.io/projected/2e41614d-6b68-4fd4-a424-e2185a53e6cd-kube-api-access-f4km7\") pod \"console-7959dbb847-5lm7n\" (UID: \"2e41614d-6b68-4fd4-a424-e2185a53e6cd\") " pod="openshift-console/console-7959dbb847-5lm7n"
Sep 10 19:17:07.337345 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:07.337305    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7959dbb847-5lm7n"
Sep 10 19:17:07.614134 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:07.614055    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7959dbb847-5lm7n"]
Sep 10 19:17:07.653456 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:07.653421    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vv7rr"
Sep 10 19:17:07.748438 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:17:07.748402    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e41614d_6b68_4fd4_a424_e2185a53e6cd.slice/crio-ee57aaa52c6b71b68dfa59d1e07d054591ca016b104f99b32113f5d0069d80e2 WatchSource:0}: Error finding container ee57aaa52c6b71b68dfa59d1e07d054591ca016b104f99b32113f5d0069d80e2: Status 404 returned error can't find the container with id ee57aaa52c6b71b68dfa59d1e07d054591ca016b104f99b32113f5d0069d80e2
Sep 10 19:17:08.297088 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:08.297052    2579 patch_prober.go:28] interesting pod/image-registry-665c8cd58b-t27cc container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 10 19:17:08.297246 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:08.297105    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-665c8cd58b-t27cc" podUID="20fcd10a-d90b-4eaf-96a0-06b5ec96437a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 19:17:08.701449 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:08.701409    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7959dbb847-5lm7n" event={"ID":"2e41614d-6b68-4fd4-a424-e2185a53e6cd","Type":"ContainerStarted","Data":"ee57aaa52c6b71b68dfa59d1e07d054591ca016b104f99b32113f5d0069d80e2"}
Sep 10 19:17:08.703818 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:08.703787    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ssqgh" event={"ID":"d1058bfb-6fa3-4f17-bc23-87860236c43a","Type":"ContainerStarted","Data":"c928b8cb7b6fda8fdbdbf16080d2f07b9abf2dcb6293cd265fd4056ec9977be7"}
Sep 10 19:17:08.725563 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:08.725502    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ssqgh" podStartSLOduration=34.236991956 podStartE2EDuration="36.725483306s" totalImagesPullingTime="2.48849135s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:16:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:17:08.72544552 +0000 UTC m=+83.945294001" watchObservedRunningTime="2026-09-10 19:17:08.725483306 +0000 UTC m=+83.945331789"
Sep 10 19:17:10.712800 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:10.712768    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7959dbb847-5lm7n" event={"ID":"2e41614d-6b68-4fd4-a424-e2185a53e6cd","Type":"ContainerStarted","Data":"4438b481b73bc6b0a938be92b4d0bf259cd62c204dab60e3b0f7baae680170b4"}
Sep 10 19:17:10.732506 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:10.732414    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7959dbb847-5lm7n" podStartSLOduration=1.999038338 podStartE2EDuration="4.732398842s" totalImagesPullingTime="2.733360504s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:17:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:17:10.731289872 +0000 UTC m=+85.951138353" watchObservedRunningTime="2026-09-10 19:17:10.732398842 +0000 UTC m=+85.952247399"
Sep 10 19:17:17.338479 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:17.338433    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7959dbb847-5lm7n"
Sep 10 19:17:17.339098 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:17.338498    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7959dbb847-5lm7n"
Sep 10 19:17:17.344235 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:17.344209    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7959dbb847-5lm7n"
Sep 10 19:17:17.738170 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:17.738142    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7959dbb847-5lm7n"
Sep 10 19:17:18.297440 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:18.297414    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-665c8cd58b-t27cc"
Sep 10 19:17:20.668072 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:20.668045    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-757bf78f57-kszf7"
Sep 10 19:17:23.313004 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:23.312964    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-665c8cd58b-t27cc" podUID="20fcd10a-d90b-4eaf-96a0-06b5ec96437a" containerName="registry" containerID="cri-o://e97bcd6689239462bfc4a8ac78a148196b7a2673e9eff17831ca5b8e41202325" gracePeriod=30
Sep 10 19:17:23.555447 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:23.555423    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-665c8cd58b-t27cc"
Sep 10 19:17:23.688374 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:23.688349    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-installation-pull-secrets\") pod \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\" (UID: \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\") "
Sep 10 19:17:23.688544 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:23.688417    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-bound-sa-token\") pod \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\" (UID: \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\") "
Sep 10 19:17:23.688544 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:23.688463    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-registry-tls\") pod \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\" (UID: \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\") "
Sep 10 19:17:23.688544 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:23.688492    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-registry-certificates\") pod \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\" (UID: \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\") "
Sep 10 19:17:23.688544 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:23.688520    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9sgtx\" (UniqueName: \"kubernetes.io/projected/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-kube-api-access-9sgtx\") pod \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\" (UID: \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\") "
Sep 10 19:17:23.688731 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:23.688545    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-image-registry-private-configuration\") pod \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\" (UID: \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\") "
Sep 10 19:17:23.688731 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:23.688595    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-trusted-ca\") pod \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\" (UID: \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\") "
Sep 10 19:17:23.688731 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:23.688637    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-ca-trust-extracted\") pod \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\" (UID: \"20fcd10a-d90b-4eaf-96a0-06b5ec96437a\") "
Sep 10 19:17:23.688959 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:23.688920    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "20fcd10a-d90b-4eaf-96a0-06b5ec96437a" (UID: "20fcd10a-d90b-4eaf-96a0-06b5ec96437a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:17:23.689237 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:23.689193    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "20fcd10a-d90b-4eaf-96a0-06b5ec96437a" (UID: "20fcd10a-d90b-4eaf-96a0-06b5ec96437a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:17:23.691237 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:23.691195    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "20fcd10a-d90b-4eaf-96a0-06b5ec96437a" (UID: "20fcd10a-d90b-4eaf-96a0-06b5ec96437a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:17:23.691415 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:23.691385    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-kube-api-access-9sgtx" (OuterVolumeSpecName: "kube-api-access-9sgtx") pod "20fcd10a-d90b-4eaf-96a0-06b5ec96437a" (UID: "20fcd10a-d90b-4eaf-96a0-06b5ec96437a"). InnerVolumeSpecName "kube-api-access-9sgtx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:17:23.691536 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:23.691398    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "20fcd10a-d90b-4eaf-96a0-06b5ec96437a" (UID: "20fcd10a-d90b-4eaf-96a0-06b5ec96437a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:17:23.691536 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:23.691464    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "20fcd10a-d90b-4eaf-96a0-06b5ec96437a" (UID: "20fcd10a-d90b-4eaf-96a0-06b5ec96437a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:17:23.691649 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:23.691564    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "20fcd10a-d90b-4eaf-96a0-06b5ec96437a" (UID: "20fcd10a-d90b-4eaf-96a0-06b5ec96437a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:17:23.697030 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:23.697009    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "20fcd10a-d90b-4eaf-96a0-06b5ec96437a" (UID: "20fcd10a-d90b-4eaf-96a0-06b5ec96437a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:17:23.756902 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:23.756867    2579 generic.go:356] "Generic (PLEG): container finished" podID="20fcd10a-d90b-4eaf-96a0-06b5ec96437a" containerID="e97bcd6689239462bfc4a8ac78a148196b7a2673e9eff17831ca5b8e41202325" exitCode=0
Sep 10 19:17:23.757079 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:23.756912    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-665c8cd58b-t27cc"
Sep 10 19:17:23.757079 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:23.756951    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-665c8cd58b-t27cc" event={"ID":"20fcd10a-d90b-4eaf-96a0-06b5ec96437a","Type":"ContainerDied","Data":"e97bcd6689239462bfc4a8ac78a148196b7a2673e9eff17831ca5b8e41202325"}
Sep 10 19:17:23.757079 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:23.757010    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-665c8cd58b-t27cc" event={"ID":"20fcd10a-d90b-4eaf-96a0-06b5ec96437a","Type":"ContainerDied","Data":"1e1a80b0f63b959068ff383c7e093a02be2f1d6ab87d0c7e8895036bd3348ab2"}
Sep 10 19:17:23.757079 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:23.757041    2579 scope.go:117] "RemoveContainer" containerID="e97bcd6689239462bfc4a8ac78a148196b7a2673e9eff17831ca5b8e41202325"
Sep 10 19:17:23.765673 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:23.765656    2579 scope.go:117] "RemoveContainer" containerID="e97bcd6689239462bfc4a8ac78a148196b7a2673e9eff17831ca5b8e41202325"
Sep 10 19:17:23.765971 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:17:23.765951    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e97bcd6689239462bfc4a8ac78a148196b7a2673e9eff17831ca5b8e41202325\": container with ID starting with e97bcd6689239462bfc4a8ac78a148196b7a2673e9eff17831ca5b8e41202325 not found: ID does not exist" containerID="e97bcd6689239462bfc4a8ac78a148196b7a2673e9eff17831ca5b8e41202325"
Sep 10 19:17:23.766034 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:23.765980    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e97bcd6689239462bfc4a8ac78a148196b7a2673e9eff17831ca5b8e41202325"} err="failed to get container status \"e97bcd6689239462bfc4a8ac78a148196b7a2673e9eff17831ca5b8e41202325\": rpc error: code = NotFound desc = could not find container \"e97bcd6689239462bfc4a8ac78a148196b7a2673e9eff17831ca5b8e41202325\": container with ID starting with e97bcd6689239462bfc4a8ac78a148196b7a2673e9eff17831ca5b8e41202325 not found: ID does not exist"
Sep 10 19:17:23.783453 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:23.783422    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-665c8cd58b-t27cc"]
Sep 10 19:17:23.789537 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:23.789518    2579 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-registry-tls\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:17:23.789597 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:23.789539    2579 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-registry-certificates\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:17:23.789597 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:23.789552    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9sgtx\" (UniqueName: \"kubernetes.io/projected/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-kube-api-access-9sgtx\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:17:23.789597 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:23.789562    2579 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-image-registry-private-configuration\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:17:23.789597 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:23.789571    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-trusted-ca\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:17:23.789597 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:23.789579    2579 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-ca-trust-extracted\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:17:23.789597 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:23.789587    2579 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-installation-pull-secrets\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:17:23.789597 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:23.789595    2579 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/20fcd10a-d90b-4eaf-96a0-06b5ec96437a-bound-sa-token\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:17:23.796584 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:23.796307    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-665c8cd58b-t27cc"]
Sep 10 19:17:25.199486 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.199457    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-d679v"]
Sep 10 19:17:25.199902 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.199697    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20fcd10a-d90b-4eaf-96a0-06b5ec96437a" containerName="registry"
Sep 10 19:17:25.199902 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.199709    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="20fcd10a-d90b-4eaf-96a0-06b5ec96437a" containerName="registry"
Sep 10 19:17:25.199902 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.199764    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="20fcd10a-d90b-4eaf-96a0-06b5ec96437a" containerName="registry"
Sep 10 19:17:25.204436 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.204417    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-d679v"
Sep 10 19:17:25.207699 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.207668    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 10 19:17:25.208628 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.208599    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 10 19:17:25.208741 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.208660    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 10 19:17:25.208833 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.208788    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 10 19:17:25.208896 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.208838    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 10 19:17:25.208946 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.208897    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 10 19:17:25.208946 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.208911    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-9h527\""
Sep 10 19:17:25.217701 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.217679    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-d679v"]
Sep 10 19:17:25.238196 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.238167    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20fcd10a-d90b-4eaf-96a0-06b5ec96437a" path="/var/lib/kubelet/pods/20fcd10a-d90b-4eaf-96a0-06b5ec96437a/volumes"
Sep 10 19:17:25.238407 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.238394    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-tphzt"]
Sep 10 19:17:25.241570 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.241552    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tphzt"
Sep 10 19:17:25.243944 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.243925    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-gfqql\""
Sep 10 19:17:25.244290 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.244275    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 10 19:17:25.244376 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.244299    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 10 19:17:25.244376 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.244364    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 10 19:17:25.299739 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.299703    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/acc7de42-5736-472c-958d-63e300039ca9-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-d679v\" (UID: \"acc7de42-5736-472c-958d-63e300039ca9\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-d679v"
Sep 10 19:17:25.299912 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.299785    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/acc7de42-5736-472c-958d-63e300039ca9-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-d679v\" (UID: \"acc7de42-5736-472c-958d-63e300039ca9\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-d679v"
Sep 10 19:17:25.299912 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.299832    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/acc7de42-5736-472c-958d-63e300039ca9-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-d679v\" (UID: \"acc7de42-5736-472c-958d-63e300039ca9\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-d679v"
Sep 10 19:17:25.299912 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.299859    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/acc7de42-5736-472c-958d-63e300039ca9-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-d679v\" (UID: \"acc7de42-5736-472c-958d-63e300039ca9\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-d679v"
Sep 10 19:17:25.300041 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.299914    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/acc7de42-5736-472c-958d-63e300039ca9-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-d679v\" (UID: \"acc7de42-5736-472c-958d-63e300039ca9\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-d679v"
Sep 10 19:17:25.300041 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.299955    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2h4j\" (UniqueName: \"kubernetes.io/projected/acc7de42-5736-472c-958d-63e300039ca9-kube-api-access-t2h4j\") pod \"kube-state-metrics-74cb57df7-d679v\" (UID: \"acc7de42-5736-472c-958d-63e300039ca9\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-d679v"
Sep 10 19:17:25.400466 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.400429    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/acc7de42-5736-472c-958d-63e300039ca9-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-d679v\" (UID: \"acc7de42-5736-472c-958d-63e300039ca9\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-d679v"
Sep 10 19:17:25.400652 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.400482    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8b85fa77-814d-412c-a98e-e424d077a16c-root\") pod \"node-exporter-tphzt\" (UID: \"8b85fa77-814d-412c-a98e-e424d077a16c\") " pod="openshift-monitoring/node-exporter-tphzt"
Sep 10 19:17:25.400652 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.400500    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8b85fa77-814d-412c-a98e-e424d077a16c-node-exporter-accelerators-collector-config\") pod \"node-exporter-tphzt\" (UID: \"8b85fa77-814d-412c-a98e-e424d077a16c\") " pod="openshift-monitoring/node-exporter-tphzt"
Sep 10 19:17:25.400652 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.400559    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/acc7de42-5736-472c-958d-63e300039ca9-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-d679v\" (UID: \"acc7de42-5736-472c-958d-63e300039ca9\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-d679v"
Sep 10 19:17:25.400652 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.400596    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/acc7de42-5736-472c-958d-63e300039ca9-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-d679v\" (UID: \"acc7de42-5736-472c-958d-63e300039ca9\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-d679v"
Sep 10 19:17:25.400652 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.400623    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/acc7de42-5736-472c-958d-63e300039ca9-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-d679v\" (UID: \"acc7de42-5736-472c-958d-63e300039ca9\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-d679v"
Sep 10 19:17:25.400652 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.400648    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8b85fa77-814d-412c-a98e-e424d077a16c-node-exporter-textfile\") pod \"node-exporter-tphzt\" (UID: \"8b85fa77-814d-412c-a98e-e424d077a16c\") " pod="openshift-monitoring/node-exporter-tphzt"
Sep 10 19:17:25.400962 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.400663    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2h4j\" (UniqueName: \"kubernetes.io/projected/acc7de42-5736-472c-958d-63e300039ca9-kube-api-access-t2h4j\") pod \"kube-state-metrics-74cb57df7-d679v\" (UID: \"acc7de42-5736-472c-958d-63e300039ca9\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-d679v"
Sep 10 19:17:25.400962 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.400685    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlg5t\" (UniqueName: \"kubernetes.io/projected/8b85fa77-814d-412c-a98e-e424d077a16c-kube-api-access-rlg5t\") pod \"node-exporter-tphzt\" (UID: \"8b85fa77-814d-412c-a98e-e424d077a16c\") " pod="openshift-monitoring/node-exporter-tphzt"
Sep 10 19:17:25.400962 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.400709    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8b85fa77-814d-412c-a98e-e424d077a16c-node-exporter-tls\") pod \"node-exporter-tphzt\" (UID: \"8b85fa77-814d-412c-a98e-e424d077a16c\") " pod="openshift-monitoring/node-exporter-tphzt"
Sep 10 19:17:25.400962 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.400736    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b85fa77-814d-412c-a98e-e424d077a16c-sys\") pod \"node-exporter-tphzt\" (UID: \"8b85fa77-814d-412c-a98e-e424d077a16c\") " pod="openshift-monitoring/node-exporter-tphzt"
Sep 10 19:17:25.400962 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.400908    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8b85fa77-814d-412c-a98e-e424d077a16c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tphzt\" (UID: \"8b85fa77-814d-412c-a98e-e424d077a16c\") " pod="openshift-monitoring/node-exporter-tphzt"
Sep 10 19:17:25.400962 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.400956    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8b85fa77-814d-412c-a98e-e424d077a16c-node-exporter-wtmp\") pod \"node-exporter-tphzt\" (UID: \"8b85fa77-814d-412c-a98e-e424d077a16c\") " pod="openshift-monitoring/node-exporter-tphzt"
Sep 10 19:17:25.401266 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.400983    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/acc7de42-5736-472c-958d-63e300039ca9-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-d679v\" (UID: \"acc7de42-5736-472c-958d-63e300039ca9\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-d679v"
Sep 10 19:17:25.401266 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.401001    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b85fa77-814d-412c-a98e-e424d077a16c-metrics-client-ca\") pod \"node-exporter-tphzt\" (UID: \"8b85fa77-814d-412c-a98e-e424d077a16c\") " pod="openshift-monitoring/node-exporter-tphzt"
Sep 10 19:17:25.401266 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.401033    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/acc7de42-5736-472c-958d-63e300039ca9-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-d679v\" (UID: \"acc7de42-5736-472c-958d-63e300039ca9\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-d679v"
Sep 10 19:17:25.401426 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.401306    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/acc7de42-5736-472c-958d-63e300039ca9-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-d679v\" (UID: \"acc7de42-5736-472c-958d-63e300039ca9\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-d679v"
Sep 10 19:17:25.401485 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.401459    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/acc7de42-5736-472c-958d-63e300039ca9-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-d679v\" (UID: \"acc7de42-5736-472c-958d-63e300039ca9\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-d679v"
Sep 10 19:17:25.403204 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.403185    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/acc7de42-5736-472c-958d-63e300039ca9-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-d679v\" (UID: \"acc7de42-5736-472c-958d-63e300039ca9\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-d679v"
Sep 10 19:17:25.403377 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.403360    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/acc7de42-5736-472c-958d-63e300039ca9-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-d679v\" (UID: \"acc7de42-5736-472c-958d-63e300039ca9\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-d679v"
Sep 10 19:17:25.411400 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.411372    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2h4j\" (UniqueName: \"kubernetes.io/projected/acc7de42-5736-472c-958d-63e300039ca9-kube-api-access-t2h4j\") pod \"kube-state-metrics-74cb57df7-d679v\" (UID: \"acc7de42-5736-472c-958d-63e300039ca9\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-d679v"
Sep 10 19:17:25.501541 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.501443    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8b85fa77-814d-412c-a98e-e424d077a16c-root\") pod \"node-exporter-tphzt\" (UID: \"8b85fa77-814d-412c-a98e-e424d077a16c\") " pod="openshift-monitoring/node-exporter-tphzt"
Sep 10 19:17:25.501541 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.501483    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8b85fa77-814d-412c-a98e-e424d077a16c-node-exporter-accelerators-collector-config\") pod \"node-exporter-tphzt\" (UID: \"8b85fa77-814d-412c-a98e-e424d077a16c\") " pod="openshift-monitoring/node-exporter-tphzt"
Sep 10 19:17:25.501541 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.501507    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8b85fa77-814d-412c-a98e-e424d077a16c-node-exporter-textfile\") pod \"node-exporter-tphzt\" (UID: \"8b85fa77-814d-412c-a98e-e424d077a16c\") " pod="openshift-monitoring/node-exporter-tphzt"
Sep 10 19:17:25.501541 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.501533    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlg5t\" (UniqueName: \"kubernetes.io/projected/8b85fa77-814d-412c-a98e-e424d077a16c-kube-api-access-rlg5t\") pod \"node-exporter-tphzt\" (UID: \"8b85fa77-814d-412c-a98e-e424d077a16c\") " pod="openshift-monitoring/node-exporter-tphzt"
Sep 10 19:17:25.501897 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.501554    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8b85fa77-814d-412c-a98e-e424d077a16c-root\") pod \"node-exporter-tphzt\" (UID: \"8b85fa77-814d-412c-a98e-e424d077a16c\") " pod="openshift-monitoring/node-exporter-tphzt"
Sep 10 19:17:25.501897 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.501559    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8b85fa77-814d-412c-a98e-e424d077a16c-node-exporter-tls\") pod \"node-exporter-tphzt\" (UID: \"8b85fa77-814d-412c-a98e-e424d077a16c\") " pod="openshift-monitoring/node-exporter-tphzt"
Sep 10 19:17:25.501897 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.501619    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b85fa77-814d-412c-a98e-e424d077a16c-sys\") pod \"node-exporter-tphzt\" (UID: \"8b85fa77-814d-412c-a98e-e424d077a16c\") " pod="openshift-monitoring/node-exporter-tphzt"
Sep 10 19:17:25.501897 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.501660    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8b85fa77-814d-412c-a98e-e424d077a16c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tphzt\" (UID: \"8b85fa77-814d-412c-a98e-e424d077a16c\") " pod="openshift-monitoring/node-exporter-tphzt"
Sep 10 19:17:25.501897 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.501694    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8b85fa77-814d-412c-a98e-e424d077a16c-node-exporter-wtmp\") pod \"node-exporter-tphzt\" (UID: \"8b85fa77-814d-412c-a98e-e424d077a16c\") " pod="openshift-monitoring/node-exporter-tphzt"
Sep 10 19:17:25.501897 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.501730    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b85fa77-814d-412c-a98e-e424d077a16c-sys\") pod \"node-exporter-tphzt\" (UID: \"8b85fa77-814d-412c-a98e-e424d077a16c\") " pod="openshift-monitoring/node-exporter-tphzt"
Sep 10 19:17:25.501897 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.501794    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b85fa77-814d-412c-a98e-e424d077a16c-metrics-client-ca\") pod \"node-exporter-tphzt\" (UID: \"8b85fa77-814d-412c-a98e-e424d077a16c\") " pod="openshift-monitoring/node-exporter-tphzt"
Sep 10 19:17:25.501897 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.501839    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8b85fa77-814d-412c-a98e-e424d077a16c-node-exporter-wtmp\") pod \"node-exporter-tphzt\" (UID: \"8b85fa77-814d-412c-a98e-e424d077a16c\") " pod="openshift-monitoring/node-exporter-tphzt"
Sep 10 19:17:25.502265 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.501897    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8b85fa77-814d-412c-a98e-e424d077a16c-node-exporter-textfile\") pod \"node-exporter-tphzt\" (UID: \"8b85fa77-814d-412c-a98e-e424d077a16c\") " pod="openshift-monitoring/node-exporter-tphzt"
Sep 10 19:17:25.502265 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.502206    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8b85fa77-814d-412c-a98e-e424d077a16c-node-exporter-accelerators-collector-config\") pod \"node-exporter-tphzt\" (UID: \"8b85fa77-814d-412c-a98e-e424d077a16c\") " pod="openshift-monitoring/node-exporter-tphzt"
Sep 10 19:17:25.502343 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.502333    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b85fa77-814d-412c-a98e-e424d077a16c-metrics-client-ca\") pod \"node-exporter-tphzt\" (UID: \"8b85fa77-814d-412c-a98e-e424d077a16c\") " pod="openshift-monitoring/node-exporter-tphzt"
Sep 10 19:17:25.504149 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.504119    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8b85fa77-814d-412c-a98e-e424d077a16c-node-exporter-tls\") pod \"node-exporter-tphzt\" (UID: \"8b85fa77-814d-412c-a98e-e424d077a16c\") " pod="openshift-monitoring/node-exporter-tphzt"
Sep 10 19:17:25.504254 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.504174    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8b85fa77-814d-412c-a98e-e424d077a16c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tphzt\" (UID: \"8b85fa77-814d-412c-a98e-e424d077a16c\") " pod="openshift-monitoring/node-exporter-tphzt"
Sep 10 19:17:25.511597 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.511575    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlg5t\" (UniqueName: \"kubernetes.io/projected/8b85fa77-814d-412c-a98e-e424d077a16c-kube-api-access-rlg5t\") pod \"node-exporter-tphzt\" (UID: \"8b85fa77-814d-412c-a98e-e424d077a16c\") " pod="openshift-monitoring/node-exporter-tphzt"
Sep 10 19:17:25.513531 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.513517    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-d679v"
Sep 10 19:17:25.550894 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.550600    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tphzt"
Sep 10 19:17:25.562276 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:17:25.562245    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b85fa77_814d_412c_a98e_e424d077a16c.slice/crio-44a5fad8f85f291082247a81920ed390c08c7c364b7a4f9f015f2548c0b94c17 WatchSource:0}: Error finding container 44a5fad8f85f291082247a81920ed390c08c7c364b7a4f9f015f2548c0b94c17: Status 404 returned error can't find the container with id 44a5fad8f85f291082247a81920ed390c08c7c364b7a4f9f015f2548c0b94c17
Sep 10 19:17:25.766709 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.766635    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tphzt" event={"ID":"8b85fa77-814d-412c-a98e-e424d077a16c","Type":"ContainerStarted","Data":"44a5fad8f85f291082247a81920ed390c08c7c364b7a4f9f015f2548c0b94c17"}
Sep 10 19:17:25.788365 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:25.788333    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-d679v"]
Sep 10 19:17:25.791057 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:17:25.791023    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacc7de42_5736_472c_958d_63e300039ca9.slice/crio-4752f2581fd7a9c9f48c5a19bab833a79f0ebe92b2de0d1d69b53351132b5092 WatchSource:0}: Error finding container 4752f2581fd7a9c9f48c5a19bab833a79f0ebe92b2de0d1d69b53351132b5092: Status 404 returned error can't find the container with id 4752f2581fd7a9c9f48c5a19bab833a79f0ebe92b2de0d1d69b53351132b5092
Sep 10 19:17:26.771598 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:26.771565    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-d679v" event={"ID":"acc7de42-5736-472c-958d-63e300039ca9","Type":"ContainerStarted","Data":"4752f2581fd7a9c9f48c5a19bab833a79f0ebe92b2de0d1d69b53351132b5092"}
Sep 10 19:17:27.279072 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.279038    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7967dd7f95-fzkhg"]
Sep 10 19:17:27.285118 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.285095    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7967dd7f95-fzkhg"
Sep 10 19:17:27.288571 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.288058    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 10 19:17:27.288571 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.288171    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 10 19:17:27.288571 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.288322    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 10 19:17:27.288571 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.288435    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 10 19:17:27.288571 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.288465    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 10 19:17:27.289047 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.288588    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-7j9pv\""
Sep 10 19:17:27.289047 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.288640    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-htofjih8b52i\""
Sep 10 19:17:27.296412 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.296383    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7967dd7f95-fzkhg"]
Sep 10 19:17:27.418510 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.418455    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f1d1a41d-eb53-4873-852d-50ceb50156e2-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7967dd7f95-fzkhg\" (UID: \"f1d1a41d-eb53-4873-852d-50ceb50156e2\") " pod="openshift-monitoring/thanos-querier-7967dd7f95-fzkhg"
Sep 10 19:17:27.418633 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.418537    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f1d1a41d-eb53-4873-852d-50ceb50156e2-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7967dd7f95-fzkhg\" (UID: \"f1d1a41d-eb53-4873-852d-50ceb50156e2\") " pod="openshift-monitoring/thanos-querier-7967dd7f95-fzkhg"
Sep 10 19:17:27.418633 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.418571    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k52dm\" (UniqueName: \"kubernetes.io/projected/f1d1a41d-eb53-4873-852d-50ceb50156e2-kube-api-access-k52dm\") pod \"thanos-querier-7967dd7f95-fzkhg\" (UID: \"f1d1a41d-eb53-4873-852d-50ceb50156e2\") " pod="openshift-monitoring/thanos-querier-7967dd7f95-fzkhg"
Sep 10 19:17:27.418779 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.418651    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f1d1a41d-eb53-4873-852d-50ceb50156e2-secret-thanos-querier-tls\") pod \"thanos-querier-7967dd7f95-fzkhg\" (UID: \"f1d1a41d-eb53-4873-852d-50ceb50156e2\") " pod="openshift-monitoring/thanos-querier-7967dd7f95-fzkhg"
Sep 10 19:17:27.418779 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.418680    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f1d1a41d-eb53-4873-852d-50ceb50156e2-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7967dd7f95-fzkhg\" (UID: \"f1d1a41d-eb53-4873-852d-50ceb50156e2\") " pod="openshift-monitoring/thanos-querier-7967dd7f95-fzkhg"
Sep 10 19:17:27.418779 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.418703    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f1d1a41d-eb53-4873-852d-50ceb50156e2-secret-grpc-tls\") pod \"thanos-querier-7967dd7f95-fzkhg\" (UID: \"f1d1a41d-eb53-4873-852d-50ceb50156e2\") " pod="openshift-monitoring/thanos-querier-7967dd7f95-fzkhg"
Sep 10 19:17:27.418779 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.418728    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1d1a41d-eb53-4873-852d-50ceb50156e2-metrics-client-ca\") pod \"thanos-querier-7967dd7f95-fzkhg\" (UID: \"f1d1a41d-eb53-4873-852d-50ceb50156e2\") " pod="openshift-monitoring/thanos-querier-7967dd7f95-fzkhg"
Sep 10 19:17:27.418996 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.418803    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f1d1a41d-eb53-4873-852d-50ceb50156e2-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7967dd7f95-fzkhg\" (UID: \"f1d1a41d-eb53-4873-852d-50ceb50156e2\") " pod="openshift-monitoring/thanos-querier-7967dd7f95-fzkhg"
Sep 10 19:17:27.519412 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.519374    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f1d1a41d-eb53-4873-852d-50ceb50156e2-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7967dd7f95-fzkhg\" (UID: \"f1d1a41d-eb53-4873-852d-50ceb50156e2\") " pod="openshift-monitoring/thanos-querier-7967dd7f95-fzkhg"
Sep 10 19:17:27.519412 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.519419    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f1d1a41d-eb53-4873-852d-50ceb50156e2-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7967dd7f95-fzkhg\" (UID: \"f1d1a41d-eb53-4873-852d-50ceb50156e2\") " pod="openshift-monitoring/thanos-querier-7967dd7f95-fzkhg"
Sep 10 19:17:27.519664 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.519557    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f1d1a41d-eb53-4873-852d-50ceb50156e2-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7967dd7f95-fzkhg\" (UID: \"f1d1a41d-eb53-4873-852d-50ceb50156e2\") " pod="openshift-monitoring/thanos-querier-7967dd7f95-fzkhg"
Sep 10 19:17:27.519664 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.519594    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k52dm\" (UniqueName: \"kubernetes.io/projected/f1d1a41d-eb53-4873-852d-50ceb50156e2-kube-api-access-k52dm\") pod \"thanos-querier-7967dd7f95-fzkhg\" (UID: \"f1d1a41d-eb53-4873-852d-50ceb50156e2\") " pod="openshift-monitoring/thanos-querier-7967dd7f95-fzkhg"
Sep 10 19:17:27.519805 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.519665    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f1d1a41d-eb53-4873-852d-50ceb50156e2-secret-thanos-querier-tls\") pod \"thanos-querier-7967dd7f95-fzkhg\" (UID: \"f1d1a41d-eb53-4873-852d-50ceb50156e2\") " pod="openshift-monitoring/thanos-querier-7967dd7f95-fzkhg"
Sep 10 19:17:27.519805 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.519696    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f1d1a41d-eb53-4873-852d-50ceb50156e2-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7967dd7f95-fzkhg\" (UID: \"f1d1a41d-eb53-4873-852d-50ceb50156e2\") " pod="openshift-monitoring/thanos-querier-7967dd7f95-fzkhg"
Sep 10 19:17:27.519805 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.519723    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f1d1a41d-eb53-4873-852d-50ceb50156e2-secret-grpc-tls\") pod \"thanos-querier-7967dd7f95-fzkhg\" (UID: \"f1d1a41d-eb53-4873-852d-50ceb50156e2\") " pod="openshift-monitoring/thanos-querier-7967dd7f95-fzkhg"
Sep 10 19:17:27.519805 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.519770    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1d1a41d-eb53-4873-852d-50ceb50156e2-metrics-client-ca\") pod \"thanos-querier-7967dd7f95-fzkhg\" (UID: \"f1d1a41d-eb53-4873-852d-50ceb50156e2\") " pod="openshift-monitoring/thanos-querier-7967dd7f95-fzkhg"
Sep 10 19:17:27.520806 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.520733    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1d1a41d-eb53-4873-852d-50ceb50156e2-metrics-client-ca\") pod \"thanos-querier-7967dd7f95-fzkhg\" (UID: \"f1d1a41d-eb53-4873-852d-50ceb50156e2\") " pod="openshift-monitoring/thanos-querier-7967dd7f95-fzkhg"
Sep 10 19:17:27.522545 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.522488    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f1d1a41d-eb53-4873-852d-50ceb50156e2-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7967dd7f95-fzkhg\" (UID: \"f1d1a41d-eb53-4873-852d-50ceb50156e2\") " pod="openshift-monitoring/thanos-querier-7967dd7f95-fzkhg"
Sep 10 19:17:27.522789 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.522727    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f1d1a41d-eb53-4873-852d-50ceb50156e2-secret-thanos-querier-tls\") pod \"thanos-querier-7967dd7f95-fzkhg\" (UID: \"f1d1a41d-eb53-4873-852d-50ceb50156e2\") " pod="openshift-monitoring/thanos-querier-7967dd7f95-fzkhg"
Sep 10 19:17:27.522880 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.522852    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f1d1a41d-eb53-4873-852d-50ceb50156e2-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7967dd7f95-fzkhg\" (UID: \"f1d1a41d-eb53-4873-852d-50ceb50156e2\") " pod="openshift-monitoring/thanos-querier-7967dd7f95-fzkhg"
Sep 10 19:17:27.522936 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.522886    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f1d1a41d-eb53-4873-852d-50ceb50156e2-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7967dd7f95-fzkhg\" (UID: \"f1d1a41d-eb53-4873-852d-50ceb50156e2\") " pod="openshift-monitoring/thanos-querier-7967dd7f95-fzkhg"
Sep 10 19:17:27.522986 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.522940    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f1d1a41d-eb53-4873-852d-50ceb50156e2-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7967dd7f95-fzkhg\" (UID: \"f1d1a41d-eb53-4873-852d-50ceb50156e2\") " pod="openshift-monitoring/thanos-querier-7967dd7f95-fzkhg"
Sep 10 19:17:27.523332 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.523314    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f1d1a41d-eb53-4873-852d-50ceb50156e2-secret-grpc-tls\") pod \"thanos-querier-7967dd7f95-fzkhg\" (UID: \"f1d1a41d-eb53-4873-852d-50ceb50156e2\") " pod="openshift-monitoring/thanos-querier-7967dd7f95-fzkhg"
Sep 10 19:17:27.531415 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.528557    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k52dm\" (UniqueName: \"kubernetes.io/projected/f1d1a41d-eb53-4873-852d-50ceb50156e2-kube-api-access-k52dm\") pod \"thanos-querier-7967dd7f95-fzkhg\" (UID: \"f1d1a41d-eb53-4873-852d-50ceb50156e2\") " pod="openshift-monitoring/thanos-querier-7967dd7f95-fzkhg"
Sep 10 19:17:27.599680 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.599635    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7967dd7f95-fzkhg"
Sep 10 19:17:27.776544 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.776510    2579 generic.go:356] "Generic (PLEG): container finished" podID="8b85fa77-814d-412c-a98e-e424d077a16c" containerID="53f807b2ad5a22764443f6a3ad5fe27c6cd72190406e11c8c657d43b44a2dd8c" exitCode=0
Sep 10 19:17:27.776992 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.776573    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tphzt" event={"ID":"8b85fa77-814d-412c-a98e-e424d077a16c","Type":"ContainerDied","Data":"53f807b2ad5a22764443f6a3ad5fe27c6cd72190406e11c8c657d43b44a2dd8c"}
Sep 10 19:17:27.779055 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.779029    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-d679v" event={"ID":"acc7de42-5736-472c-958d-63e300039ca9","Type":"ContainerStarted","Data":"772ab51b16cdbcb6455f14d4f804f424a79c88f9497786ee0bb6ea2d744dac92"}
Sep 10 19:17:27.779176 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.779089    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-d679v" event={"ID":"acc7de42-5736-472c-958d-63e300039ca9","Type":"ContainerStarted","Data":"fc390dba25969d0dfacc2137fc79e2243ee31b0a29e51e4c38b9f29fd29abed9"}
Sep 10 19:17:27.779176 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.779118    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-d679v" event={"ID":"acc7de42-5736-472c-958d-63e300039ca9","Type":"ContainerStarted","Data":"5d1b65597b07a88c8cdc20523025bb63424b9efae70d849899eb6861083222d1"}
Sep 10 19:17:27.811133 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.811080    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-d679v" podStartSLOduration=1.442943165 podStartE2EDuration="2.811058842s" totalImagesPullingTime="1.368115677s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:17:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:17:27.809839945 +0000 UTC m=+103.029688426" watchObservedRunningTime="2026-09-10 19:17:27.811058842 +0000 UTC m=+103.030907324"
Sep 10 19:17:27.909374 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:27.909351    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7967dd7f95-fzkhg"]
Sep 10 19:17:27.910068 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:17:27.910047    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1d1a41d_eb53_4873_852d_50ceb50156e2.slice/crio-5101e7579be71df8dd21c9c2a53289d9257ad13ff0970bffae0f0fb5cb3b93c6 WatchSource:0}: Error finding container 5101e7579be71df8dd21c9c2a53289d9257ad13ff0970bffae0f0fb5cb3b93c6: Status 404 returned error can't find the container with id 5101e7579be71df8dd21c9c2a53289d9257ad13ff0970bffae0f0fb5cb3b93c6
Sep 10 19:17:28.653057 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:28.653024    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-dns2r"
Sep 10 19:17:28.785941 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:28.785901    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tphzt" event={"ID":"8b85fa77-814d-412c-a98e-e424d077a16c","Type":"ContainerStarted","Data":"0a6cfeae2cc877378da2e95d03e7f2fcb66e4b1778bd68d14619bbedcbb193ed"}
Sep 10 19:17:28.786463 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:28.785973    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tphzt" event={"ID":"8b85fa77-814d-412c-a98e-e424d077a16c","Type":"ContainerStarted","Data":"e3bda3c18f2c0749e31db8a56b8dfd61d72bdbceeba5b5ddb7b25019f14ea95f"}
Sep 10 19:17:28.787379 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:28.787352    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7967dd7f95-fzkhg" event={"ID":"f1d1a41d-eb53-4873-852d-50ceb50156e2","Type":"ContainerStarted","Data":"5101e7579be71df8dd21c9c2a53289d9257ad13ff0970bffae0f0fb5cb3b93c6"}
Sep 10 19:17:28.805429 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:28.805371    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-tphzt" podStartSLOduration=2.1514060600000002 podStartE2EDuration="3.805351952s" totalImagesPullingTime="1.554787292s" totalInitContainerRuntime="99.1586ms" isStatefulPod=true podCreationTimestamp="2026-09-10 19:17:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:17:28.804469521 +0000 UTC m=+104.024318003" watchObservedRunningTime="2026-09-10 19:17:28.805351952 +0000 UTC m=+104.025200434"
Sep 10 19:17:29.495411 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:29.495378    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-55f6cbdc4b-6s7ds"]
Sep 10 19:17:29.521959 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:29.521921    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-55f6cbdc4b-6s7ds"]
Sep 10 19:17:29.522140 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:29.522061    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-55f6cbdc4b-6s7ds"
Sep 10 19:17:29.525129 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:29.525102    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 10 19:17:29.525285 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:29.525154    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 10 19:17:29.525285 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:29.525178    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-wg2fr\""
Sep 10 19:17:29.525285 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:29.525187    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-djok2ot77ok8q\""
Sep 10 19:17:29.525285 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:29.525252    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 10 19:17:29.525462 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:29.525388    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 10 19:17:29.639324 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:29.639286    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/6071f2a7-2810-4cd3-93e2-f18ac96d1950-secret-metrics-server-client-certs\") pod \"metrics-server-55f6cbdc4b-6s7ds\" (UID: \"6071f2a7-2810-4cd3-93e2-f18ac96d1950\") " pod="openshift-monitoring/metrics-server-55f6cbdc4b-6s7ds"
Sep 10 19:17:29.639524 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:29.639339    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6071f2a7-2810-4cd3-93e2-f18ac96d1950-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-55f6cbdc4b-6s7ds\" (UID: \"6071f2a7-2810-4cd3-93e2-f18ac96d1950\") " pod="openshift-monitoring/metrics-server-55f6cbdc4b-6s7ds"
Sep 10 19:17:29.639524 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:29.639415    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/6071f2a7-2810-4cd3-93e2-f18ac96d1950-secret-metrics-server-tls\") pod \"metrics-server-55f6cbdc4b-6s7ds\" (UID: \"6071f2a7-2810-4cd3-93e2-f18ac96d1950\") " pod="openshift-monitoring/metrics-server-55f6cbdc4b-6s7ds"
Sep 10 19:17:29.639524 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:29.639507    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7qp4\" (UniqueName: \"kubernetes.io/projected/6071f2a7-2810-4cd3-93e2-f18ac96d1950-kube-api-access-h7qp4\") pod \"metrics-server-55f6cbdc4b-6s7ds\" (UID: \"6071f2a7-2810-4cd3-93e2-f18ac96d1950\") " pod="openshift-monitoring/metrics-server-55f6cbdc4b-6s7ds"
Sep 10 19:17:29.639699 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:29.639567    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6071f2a7-2810-4cd3-93e2-f18ac96d1950-client-ca-bundle\") pod \"metrics-server-55f6cbdc4b-6s7ds\" (UID: \"6071f2a7-2810-4cd3-93e2-f18ac96d1950\") " pod="openshift-monitoring/metrics-server-55f6cbdc4b-6s7ds"
Sep 10 19:17:29.639699 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:29.639586    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/6071f2a7-2810-4cd3-93e2-f18ac96d1950-audit-log\") pod \"metrics-server-55f6cbdc4b-6s7ds\" (UID: \"6071f2a7-2810-4cd3-93e2-f18ac96d1950\") " pod="openshift-monitoring/metrics-server-55f6cbdc4b-6s7ds"
Sep 10 19:17:29.639699 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:29.639620    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/6071f2a7-2810-4cd3-93e2-f18ac96d1950-metrics-server-audit-profiles\") pod \"metrics-server-55f6cbdc4b-6s7ds\" (UID: \"6071f2a7-2810-4cd3-93e2-f18ac96d1950\") " pod="openshift-monitoring/metrics-server-55f6cbdc4b-6s7ds"
Sep 10 19:17:29.740636 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:29.740588    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/6071f2a7-2810-4cd3-93e2-f18ac96d1950-metrics-server-audit-profiles\") pod \"metrics-server-55f6cbdc4b-6s7ds\" (UID: \"6071f2a7-2810-4cd3-93e2-f18ac96d1950\") " pod="openshift-monitoring/metrics-server-55f6cbdc4b-6s7ds"
Sep 10 19:17:29.740865 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:29.740671    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/6071f2a7-2810-4cd3-93e2-f18ac96d1950-secret-metrics-server-client-certs\") pod \"metrics-server-55f6cbdc4b-6s7ds\" (UID: \"6071f2a7-2810-4cd3-93e2-f18ac96d1950\") " pod="openshift-monitoring/metrics-server-55f6cbdc4b-6s7ds"
Sep 10 19:17:29.740865 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:29.740701    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6071f2a7-2810-4cd3-93e2-f18ac96d1950-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-55f6cbdc4b-6s7ds\" (UID: \"6071f2a7-2810-4cd3-93e2-f18ac96d1950\") " pod="openshift-monitoring/metrics-server-55f6cbdc4b-6s7ds"
Sep 10 19:17:29.740865 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:29.740733    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/6071f2a7-2810-4cd3-93e2-f18ac96d1950-secret-metrics-server-tls\") pod \"metrics-server-55f6cbdc4b-6s7ds\" (UID: \"6071f2a7-2810-4cd3-93e2-f18ac96d1950\") " pod="openshift-monitoring/metrics-server-55f6cbdc4b-6s7ds"
Sep 10 19:17:29.740865 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:29.740799    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7qp4\" (UniqueName: \"kubernetes.io/projected/6071f2a7-2810-4cd3-93e2-f18ac96d1950-kube-api-access-h7qp4\") pod \"metrics-server-55f6cbdc4b-6s7ds\" (UID: \"6071f2a7-2810-4cd3-93e2-f18ac96d1950\") " pod="openshift-monitoring/metrics-server-55f6cbdc4b-6s7ds"
Sep 10 19:17:29.740865 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:29.740860    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6071f2a7-2810-4cd3-93e2-f18ac96d1950-client-ca-bundle\") pod \"metrics-server-55f6cbdc4b-6s7ds\" (UID: \"6071f2a7-2810-4cd3-93e2-f18ac96d1950\") " pod="openshift-monitoring/metrics-server-55f6cbdc4b-6s7ds"
Sep 10 19:17:29.741069 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:29.740931    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/6071f2a7-2810-4cd3-93e2-f18ac96d1950-audit-log\") pod \"metrics-server-55f6cbdc4b-6s7ds\" (UID: \"6071f2a7-2810-4cd3-93e2-f18ac96d1950\") " pod="openshift-monitoring/metrics-server-55f6cbdc4b-6s7ds"
Sep 10 19:17:29.741392 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:29.741349    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/6071f2a7-2810-4cd3-93e2-f18ac96d1950-audit-log\") pod \"metrics-server-55f6cbdc4b-6s7ds\" (UID: \"6071f2a7-2810-4cd3-93e2-f18ac96d1950\") " pod="openshift-monitoring/metrics-server-55f6cbdc4b-6s7ds"
Sep 10 19:17:29.741516 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:29.741411    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6071f2a7-2810-4cd3-93e2-f18ac96d1950-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-55f6cbdc4b-6s7ds\" (UID: \"6071f2a7-2810-4cd3-93e2-f18ac96d1950\") " pod="openshift-monitoring/metrics-server-55f6cbdc4b-6s7ds"
Sep 10 19:17:29.741693 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:29.741669    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/6071f2a7-2810-4cd3-93e2-f18ac96d1950-metrics-server-audit-profiles\") pod \"metrics-server-55f6cbdc4b-6s7ds\" (UID: \"6071f2a7-2810-4cd3-93e2-f18ac96d1950\") " pod="openshift-monitoring/metrics-server-55f6cbdc4b-6s7ds"
Sep 10 19:17:29.744012 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:29.743991    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/6071f2a7-2810-4cd3-93e2-f18ac96d1950-secret-metrics-server-client-certs\") pod \"metrics-server-55f6cbdc4b-6s7ds\" (UID: \"6071f2a7-2810-4cd3-93e2-f18ac96d1950\") " pod="openshift-monitoring/metrics-server-55f6cbdc4b-6s7ds"
Sep 10 19:17:29.744488 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:29.744469    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6071f2a7-2810-4cd3-93e2-f18ac96d1950-client-ca-bundle\") pod \"metrics-server-55f6cbdc4b-6s7ds\" (UID: \"6071f2a7-2810-4cd3-93e2-f18ac96d1950\") " pod="openshift-monitoring/metrics-server-55f6cbdc4b-6s7ds"
Sep 10 19:17:29.744923 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:29.744904    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/6071f2a7-2810-4cd3-93e2-f18ac96d1950-secret-metrics-server-tls\") pod \"metrics-server-55f6cbdc4b-6s7ds\" (UID: \"6071f2a7-2810-4cd3-93e2-f18ac96d1950\") " pod="openshift-monitoring/metrics-server-55f6cbdc4b-6s7ds"
Sep 10 19:17:29.750183 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:29.750134    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7qp4\" (UniqueName: \"kubernetes.io/projected/6071f2a7-2810-4cd3-93e2-f18ac96d1950-kube-api-access-h7qp4\") pod \"metrics-server-55f6cbdc4b-6s7ds\" (UID: \"6071f2a7-2810-4cd3-93e2-f18ac96d1950\") " pod="openshift-monitoring/metrics-server-55f6cbdc4b-6s7ds"
Sep 10 19:17:29.832905 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:29.832871    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-55f6cbdc4b-6s7ds"
Sep 10 19:17:29.978425 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:29.978391    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-t9rm6"]
Sep 10 19:17:29.996077 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:29.996040    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-t9rm6"]
Sep 10 19:17:29.996077 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:29.996082    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8cf859848-hfmz8"]
Sep 10 19:17:29.996299 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:29.996175    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-t9rm6"
Sep 10 19:17:30.001152 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:29.998361    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-rxm4j\""
Sep 10 19:17:30.001152 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:29.998631    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 10 19:17:30.020575 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.020545    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8cf859848-hfmz8"]
Sep 10 19:17:30.020727 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.020701    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8cf859848-hfmz8"
Sep 10 19:17:30.144204 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.144169    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-console-serving-cert\") pod \"console-8cf859848-hfmz8\" (UID: \"1f07ec2b-f1d6-41fb-af14-95bdebb52a94\") " pod="openshift-console/console-8cf859848-hfmz8"
Sep 10 19:17:30.144363 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.144211    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-trusted-ca-bundle\") pod \"console-8cf859848-hfmz8\" (UID: \"1f07ec2b-f1d6-41fb-af14-95bdebb52a94\") " pod="openshift-console/console-8cf859848-hfmz8"
Sep 10 19:17:30.144363 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.144239    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-service-ca\") pod \"console-8cf859848-hfmz8\" (UID: \"1f07ec2b-f1d6-41fb-af14-95bdebb52a94\") " pod="openshift-console/console-8cf859848-hfmz8"
Sep 10 19:17:30.144363 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.144268    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3fe9e5b0-7ea7-4b64-b60b-b098a1a2aec2-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-t9rm6\" (UID: \"3fe9e5b0-7ea7-4b64-b60b-b098a1a2aec2\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-t9rm6"
Sep 10 19:17:30.144363 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.144296    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnnm9\" (UniqueName: \"kubernetes.io/projected/3fe9e5b0-7ea7-4b64-b60b-b098a1a2aec2-kube-api-access-mnnm9\") pod \"monitoring-plugin-85bbbbbf4-t9rm6\" (UID: \"3fe9e5b0-7ea7-4b64-b60b-b098a1a2aec2\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-t9rm6"
Sep 10 19:17:30.144493 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.144371    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-console-config\") pod \"console-8cf859848-hfmz8\" (UID: \"1f07ec2b-f1d6-41fb-af14-95bdebb52a94\") " pod="openshift-console/console-8cf859848-hfmz8"
Sep 10 19:17:30.144493 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.144401    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-oauth-serving-cert\") pod \"console-8cf859848-hfmz8\" (UID: \"1f07ec2b-f1d6-41fb-af14-95bdebb52a94\") " pod="openshift-console/console-8cf859848-hfmz8"
Sep 10 19:17:30.144493 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.144418    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8xgq\" (UniqueName: \"kubernetes.io/projected/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-kube-api-access-m8xgq\") pod \"console-8cf859848-hfmz8\" (UID: \"1f07ec2b-f1d6-41fb-af14-95bdebb52a94\") " pod="openshift-console/console-8cf859848-hfmz8"
Sep 10 19:17:30.144493 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.144452    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-console-oauth-config\") pod \"console-8cf859848-hfmz8\" (UID: \"1f07ec2b-f1d6-41fb-af14-95bdebb52a94\") " pod="openshift-console/console-8cf859848-hfmz8"
Sep 10 19:17:30.245291 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.245253    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-console-oauth-config\") pod \"console-8cf859848-hfmz8\" (UID: \"1f07ec2b-f1d6-41fb-af14-95bdebb52a94\") " pod="openshift-console/console-8cf859848-hfmz8"
Sep 10 19:17:30.245462 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.245311    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-console-serving-cert\") pod \"console-8cf859848-hfmz8\" (UID: \"1f07ec2b-f1d6-41fb-af14-95bdebb52a94\") " pod="openshift-console/console-8cf859848-hfmz8"
Sep 10 19:17:30.245462 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.245334    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-trusted-ca-bundle\") pod \"console-8cf859848-hfmz8\" (UID: \"1f07ec2b-f1d6-41fb-af14-95bdebb52a94\") " pod="openshift-console/console-8cf859848-hfmz8"
Sep 10 19:17:30.245609 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.245505    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-service-ca\") pod \"console-8cf859848-hfmz8\" (UID: \"1f07ec2b-f1d6-41fb-af14-95bdebb52a94\") " pod="openshift-console/console-8cf859848-hfmz8"
Sep 10 19:17:30.245609 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.245569    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3fe9e5b0-7ea7-4b64-b60b-b098a1a2aec2-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-t9rm6\" (UID: \"3fe9e5b0-7ea7-4b64-b60b-b098a1a2aec2\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-t9rm6"
Sep 10 19:17:30.245609 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.245606    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnnm9\" (UniqueName: \"kubernetes.io/projected/3fe9e5b0-7ea7-4b64-b60b-b098a1a2aec2-kube-api-access-mnnm9\") pod \"monitoring-plugin-85bbbbbf4-t9rm6\" (UID: \"3fe9e5b0-7ea7-4b64-b60b-b098a1a2aec2\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-t9rm6"
Sep 10 19:17:30.245793 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.245655    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-console-config\") pod \"console-8cf859848-hfmz8\" (UID: \"1f07ec2b-f1d6-41fb-af14-95bdebb52a94\") " pod="openshift-console/console-8cf859848-hfmz8"
Sep 10 19:17:30.245793 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:17:30.245765    2579 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 10 19:17:30.245793 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.245785    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-oauth-serving-cert\") pod \"console-8cf859848-hfmz8\" (UID: \"1f07ec2b-f1d6-41fb-af14-95bdebb52a94\") " pod="openshift-console/console-8cf859848-hfmz8"
Sep 10 19:17:30.245953 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.245820    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8xgq\" (UniqueName: \"kubernetes.io/projected/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-kube-api-access-m8xgq\") pod \"console-8cf859848-hfmz8\" (UID: \"1f07ec2b-f1d6-41fb-af14-95bdebb52a94\") " pod="openshift-console/console-8cf859848-hfmz8"
Sep 10 19:17:30.245953 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:17:30.245850    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fe9e5b0-7ea7-4b64-b60b-b098a1a2aec2-monitoring-plugin-cert podName:3fe9e5b0-7ea7-4b64-b60b-b098a1a2aec2 nodeName:}" failed. No retries permitted until 2026-09-10 19:17:30.745826986 +0000 UTC m=+105.965675460 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/3fe9e5b0-7ea7-4b64-b60b-b098a1a2aec2-monitoring-plugin-cert") pod "monitoring-plugin-85bbbbbf4-t9rm6" (UID: "3fe9e5b0-7ea7-4b64-b60b-b098a1a2aec2") : secret "monitoring-plugin-cert" not found
Sep 10 19:17:30.246374 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.246346    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-service-ca\") pod \"console-8cf859848-hfmz8\" (UID: \"1f07ec2b-f1d6-41fb-af14-95bdebb52a94\") " pod="openshift-console/console-8cf859848-hfmz8"
Sep 10 19:17:30.246470 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.246382    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-trusted-ca-bundle\") pod \"console-8cf859848-hfmz8\" (UID: \"1f07ec2b-f1d6-41fb-af14-95bdebb52a94\") " pod="openshift-console/console-8cf859848-hfmz8"
Sep 10 19:17:30.246523 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.246498    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-console-config\") pod \"console-8cf859848-hfmz8\" (UID: \"1f07ec2b-f1d6-41fb-af14-95bdebb52a94\") " pod="openshift-console/console-8cf859848-hfmz8"
Sep 10 19:17:30.246569 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.246553    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-oauth-serving-cert\") pod \"console-8cf859848-hfmz8\" (UID: \"1f07ec2b-f1d6-41fb-af14-95bdebb52a94\") " pod="openshift-console/console-8cf859848-hfmz8"
Sep 10 19:17:30.248549 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.248524    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-console-oauth-config\") pod \"console-8cf859848-hfmz8\" (UID: \"1f07ec2b-f1d6-41fb-af14-95bdebb52a94\") " pod="openshift-console/console-8cf859848-hfmz8"
Sep 10 19:17:30.248670 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.248649    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-console-serving-cert\") pod \"console-8cf859848-hfmz8\" (UID: \"1f07ec2b-f1d6-41fb-af14-95bdebb52a94\") " pod="openshift-console/console-8cf859848-hfmz8"
Sep 10 19:17:30.256441 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.256386    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8xgq\" (UniqueName: \"kubernetes.io/projected/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-kube-api-access-m8xgq\") pod \"console-8cf859848-hfmz8\" (UID: \"1f07ec2b-f1d6-41fb-af14-95bdebb52a94\") " pod="openshift-console/console-8cf859848-hfmz8"
Sep 10 19:17:30.256863 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.256845    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnnm9\" (UniqueName: \"kubernetes.io/projected/3fe9e5b0-7ea7-4b64-b60b-b098a1a2aec2-kube-api-access-mnnm9\") pod \"monitoring-plugin-85bbbbbf4-t9rm6\" (UID: \"3fe9e5b0-7ea7-4b64-b60b-b098a1a2aec2\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-t9rm6"
Sep 10 19:17:30.288866 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.288790    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-55f6cbdc4b-6s7ds"]
Sep 10 19:17:30.338232 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.338189    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8cf859848-hfmz8"
Sep 10 19:17:30.390319 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:17:30.390278    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6071f2a7_2810_4cd3_93e2_f18ac96d1950.slice/crio-3e1a4c8230f6c7173d713ba08b8623daccfec6df2989e364d4d12702efe83c74 WatchSource:0}: Error finding container 3e1a4c8230f6c7173d713ba08b8623daccfec6df2989e364d4d12702efe83c74: Status 404 returned error can't find the container with id 3e1a4c8230f6c7173d713ba08b8623daccfec6df2989e364d4d12702efe83c74
Sep 10 19:17:30.682773 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.682720    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8cf859848-hfmz8"]
Sep 10 19:17:30.685525 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:17:30.685491    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f07ec2b_f1d6_41fb_af14_95bdebb52a94.slice/crio-6786a053a40afc93b80de06332fac59ffb03de4a9054d049d1b21d8cacdce579 WatchSource:0}: Error finding container 6786a053a40afc93b80de06332fac59ffb03de4a9054d049d1b21d8cacdce579: Status 404 returned error can't find the container with id 6786a053a40afc93b80de06332fac59ffb03de4a9054d049d1b21d8cacdce579
Sep 10 19:17:30.751121 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.751053    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3fe9e5b0-7ea7-4b64-b60b-b098a1a2aec2-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-t9rm6\" (UID: \"3fe9e5b0-7ea7-4b64-b60b-b098a1a2aec2\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-t9rm6"
Sep 10 19:17:30.754116 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.754091    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3fe9e5b0-7ea7-4b64-b60b-b098a1a2aec2-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-t9rm6\" (UID: \"3fe9e5b0-7ea7-4b64-b60b-b098a1a2aec2\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-t9rm6"
Sep 10 19:17:30.798310 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.798220    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8cf859848-hfmz8" event={"ID":"1f07ec2b-f1d6-41fb-af14-95bdebb52a94","Type":"ContainerStarted","Data":"482ddf964a0689a517393c4454294a19dc62d2bd6f91605e82afeae023fa55fc"}
Sep 10 19:17:30.798310 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.798270    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8cf859848-hfmz8" event={"ID":"1f07ec2b-f1d6-41fb-af14-95bdebb52a94","Type":"ContainerStarted","Data":"6786a053a40afc93b80de06332fac59ffb03de4a9054d049d1b21d8cacdce579"}
Sep 10 19:17:30.800039 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.800015    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-55f6cbdc4b-6s7ds" event={"ID":"6071f2a7-2810-4cd3-93e2-f18ac96d1950","Type":"ContainerStarted","Data":"3e1a4c8230f6c7173d713ba08b8623daccfec6df2989e364d4d12702efe83c74"}
Sep 10 19:17:30.802373 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.802352    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7967dd7f95-fzkhg" event={"ID":"f1d1a41d-eb53-4873-852d-50ceb50156e2","Type":"ContainerStarted","Data":"5104fbe28025a0cc1389af61fb897eacfbd131a7f33e2a7eeaf3736c63f21ec0"}
Sep 10 19:17:30.802473 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.802401    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7967dd7f95-fzkhg" event={"ID":"f1d1a41d-eb53-4873-852d-50ceb50156e2","Type":"ContainerStarted","Data":"a9efb676fc5a104222546064b449f6d0bef5d02ae97492025b15ea80968fddd2"}
Sep 10 19:17:30.802473 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.802426    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7967dd7f95-fzkhg" event={"ID":"f1d1a41d-eb53-4873-852d-50ceb50156e2","Type":"ContainerStarted","Data":"184a3984b1c93a1c82d17488cd6921877a1cfe430659130f8fc4846559ea604a"}
Sep 10 19:17:30.814896 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.814603    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8cf859848-hfmz8" podStartSLOduration=1.814585703 podStartE2EDuration="1.814585703s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:17:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:17:30.814095299 +0000 UTC m=+106.033943780" watchObservedRunningTime="2026-09-10 19:17:30.814585703 +0000 UTC m=+106.034434186"
Sep 10 19:17:30.908658 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:30.908625    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-t9rm6"
Sep 10 19:17:31.198819 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:31.198788    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-t9rm6"]
Sep 10 19:17:31.201449 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:17:31.201416    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fe9e5b0_7ea7_4b64_b60b_b098a1a2aec2.slice/crio-bc1fc0bd4c15a8f18aa1da3d3db077b722615b44116923649823cffe03db4ecb WatchSource:0}: Error finding container bc1fc0bd4c15a8f18aa1da3d3db077b722615b44116923649823cffe03db4ecb: Status 404 returned error can't find the container with id bc1fc0bd4c15a8f18aa1da3d3db077b722615b44116923649823cffe03db4ecb
Sep 10 19:17:31.808821 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:31.808784    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-t9rm6" event={"ID":"3fe9e5b0-7ea7-4b64-b60b-b098a1a2aec2","Type":"ContainerStarted","Data":"bc1fc0bd4c15a8f18aa1da3d3db077b722615b44116923649823cffe03db4ecb"}
Sep 10 19:17:33.819352 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:33.819246    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-55f6cbdc4b-6s7ds" event={"ID":"6071f2a7-2810-4cd3-93e2-f18ac96d1950","Type":"ContainerStarted","Data":"070216d8b6312f358287253acd9f4bff5d60f5c47f9f05cc7386047c665598d9"}
Sep 10 19:17:33.820986 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:33.820959    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-t9rm6" event={"ID":"3fe9e5b0-7ea7-4b64-b60b-b098a1a2aec2","Type":"ContainerStarted","Data":"1524349b6ac81b7393d3d501cdacc481b6610b72411abb9ee0a2c4da9c57cf30"}
Sep 10 19:17:33.821127 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:33.821014    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-t9rm6"
Sep 10 19:17:33.824913 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:33.824880    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7967dd7f95-fzkhg" event={"ID":"f1d1a41d-eb53-4873-852d-50ceb50156e2","Type":"ContainerStarted","Data":"4f5f6f14949d804ba72b638ce2229b4d3f623dc679589b28e7d006f29ac26569"}
Sep 10 19:17:33.825030 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:33.824946    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7967dd7f95-fzkhg" event={"ID":"f1d1a41d-eb53-4873-852d-50ceb50156e2","Type":"ContainerStarted","Data":"47a47df2a31aa650d883659dde6b6c0d65f39e93b68b690b1f8ec7bf104393f7"}
Sep 10 19:17:33.825030 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:33.824994    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7967dd7f95-fzkhg"
Sep 10 19:17:33.825161 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:33.825031    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7967dd7f95-fzkhg" event={"ID":"f1d1a41d-eb53-4873-852d-50ceb50156e2","Type":"ContainerStarted","Data":"bb8bce72392a0e7d1684c6487ee01bfda57f11ce75df7dfdd0aa8ff7142bdd68"}
Sep 10 19:17:33.826735 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:33.826714    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-t9rm6"
Sep 10 19:17:33.841875 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:33.841826    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-55f6cbdc4b-6s7ds" podStartSLOduration=1.745113707 podStartE2EDuration="4.841814005s" totalImagesPullingTime="3.096700298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:17:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:17:33.841632716 +0000 UTC m=+109.061481199" watchObservedRunningTime="2026-09-10 19:17:33.841814005 +0000 UTC m=+109.061662486"
Sep 10 19:17:33.858237 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:33.858191    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-t9rm6" podStartSLOduration=2.546013868 podStartE2EDuration="4.85818146s" totalImagesPullingTime="2.312167592s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:17:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:17:33.857016153 +0000 UTC m=+109.076864633" watchObservedRunningTime="2026-09-10 19:17:33.85818146 +0000 UTC m=+109.078029941"
Sep 10 19:17:33.881038 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:33.880981    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7967dd7f95-fzkhg" podStartSLOduration=1.5914642620000001 podStartE2EDuration="6.880960133s" totalImagesPullingTime="5.289495871s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:17:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:17:33.880127129 +0000 UTC m=+109.099975611" watchObservedRunningTime="2026-09-10 19:17:33.880960133 +0000 UTC m=+109.100808614"
Sep 10 19:17:35.947013 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:35.946970    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8cf859848-hfmz8"]
Sep 10 19:17:35.986491 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:35.986457    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c658d4d8c-vmzq9"]
Sep 10 19:17:35.989737 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:35.989720    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c658d4d8c-vmzq9"
Sep 10 19:17:35.999856 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:35.999832    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c658d4d8c-vmzq9"]
Sep 10 19:17:36.096462 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:36.096407    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/037a1789-f3a1-40b1-9e35-d9716af544d7-console-oauth-config\") pod \"console-c658d4d8c-vmzq9\" (UID: \"037a1789-f3a1-40b1-9e35-d9716af544d7\") " pod="openshift-console/console-c658d4d8c-vmzq9"
Sep 10 19:17:36.096462 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:36.096471    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/037a1789-f3a1-40b1-9e35-d9716af544d7-trusted-ca-bundle\") pod \"console-c658d4d8c-vmzq9\" (UID: \"037a1789-f3a1-40b1-9e35-d9716af544d7\") " pod="openshift-console/console-c658d4d8c-vmzq9"
Sep 10 19:17:36.096681 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:36.096538    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/037a1789-f3a1-40b1-9e35-d9716af544d7-console-config\") pod \"console-c658d4d8c-vmzq9\" (UID: \"037a1789-f3a1-40b1-9e35-d9716af544d7\") " pod="openshift-console/console-c658d4d8c-vmzq9"
Sep 10 19:17:36.096681 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:36.096582    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/037a1789-f3a1-40b1-9e35-d9716af544d7-oauth-serving-cert\") pod \"console-c658d4d8c-vmzq9\" (UID: \"037a1789-f3a1-40b1-9e35-d9716af544d7\") " pod="openshift-console/console-c658d4d8c-vmzq9"
Sep 10 19:17:36.096681 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:36.096617    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/037a1789-f3a1-40b1-9e35-d9716af544d7-service-ca\") pod \"console-c658d4d8c-vmzq9\" (UID: \"037a1789-f3a1-40b1-9e35-d9716af544d7\") " pod="openshift-console/console-c658d4d8c-vmzq9"
Sep 10 19:17:36.096807 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:36.096688    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/037a1789-f3a1-40b1-9e35-d9716af544d7-console-serving-cert\") pod \"console-c658d4d8c-vmzq9\" (UID: \"037a1789-f3a1-40b1-9e35-d9716af544d7\") " pod="openshift-console/console-c658d4d8c-vmzq9"
Sep 10 19:17:36.096807 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:36.096714    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgc29\" (UniqueName: \"kubernetes.io/projected/037a1789-f3a1-40b1-9e35-d9716af544d7-kube-api-access-fgc29\") pod \"console-c658d4d8c-vmzq9\" (UID: \"037a1789-f3a1-40b1-9e35-d9716af544d7\") " pod="openshift-console/console-c658d4d8c-vmzq9"
Sep 10 19:17:36.198110 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:36.198010    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/037a1789-f3a1-40b1-9e35-d9716af544d7-service-ca\") pod \"console-c658d4d8c-vmzq9\" (UID: \"037a1789-f3a1-40b1-9e35-d9716af544d7\") " pod="openshift-console/console-c658d4d8c-vmzq9"
Sep 10 19:17:36.198110 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:36.198075    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/037a1789-f3a1-40b1-9e35-d9716af544d7-console-serving-cert\") pod \"console-c658d4d8c-vmzq9\" (UID: \"037a1789-f3a1-40b1-9e35-d9716af544d7\") " pod="openshift-console/console-c658d4d8c-vmzq9"
Sep 10 19:17:36.198110 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:36.198093    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgc29\" (UniqueName: \"kubernetes.io/projected/037a1789-f3a1-40b1-9e35-d9716af544d7-kube-api-access-fgc29\") pod \"console-c658d4d8c-vmzq9\" (UID: \"037a1789-f3a1-40b1-9e35-d9716af544d7\") " pod="openshift-console/console-c658d4d8c-vmzq9"
Sep 10 19:17:36.198375 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:36.198133    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/037a1789-f3a1-40b1-9e35-d9716af544d7-console-oauth-config\") pod \"console-c658d4d8c-vmzq9\" (UID: \"037a1789-f3a1-40b1-9e35-d9716af544d7\") " pod="openshift-console/console-c658d4d8c-vmzq9"
Sep 10 19:17:36.198375 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:36.198155    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/037a1789-f3a1-40b1-9e35-d9716af544d7-trusted-ca-bundle\") pod \"console-c658d4d8c-vmzq9\" (UID: \"037a1789-f3a1-40b1-9e35-d9716af544d7\") " pod="openshift-console/console-c658d4d8c-vmzq9"
Sep 10 19:17:36.198375 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:36.198172    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/037a1789-f3a1-40b1-9e35-d9716af544d7-console-config\") pod \"console-c658d4d8c-vmzq9\" (UID: \"037a1789-f3a1-40b1-9e35-d9716af544d7\") " pod="openshift-console/console-c658d4d8c-vmzq9"
Sep 10 19:17:36.198375 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:36.198343    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/037a1789-f3a1-40b1-9e35-d9716af544d7-oauth-serving-cert\") pod \"console-c658d4d8c-vmzq9\" (UID: \"037a1789-f3a1-40b1-9e35-d9716af544d7\") " pod="openshift-console/console-c658d4d8c-vmzq9"
Sep 10 19:17:36.198885 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:36.198861    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/037a1789-f3a1-40b1-9e35-d9716af544d7-console-config\") pod \"console-c658d4d8c-vmzq9\" (UID: \"037a1789-f3a1-40b1-9e35-d9716af544d7\") " pod="openshift-console/console-c658d4d8c-vmzq9"
Sep 10 19:17:36.198988 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:36.198953    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/037a1789-f3a1-40b1-9e35-d9716af544d7-oauth-serving-cert\") pod \"console-c658d4d8c-vmzq9\" (UID: \"037a1789-f3a1-40b1-9e35-d9716af544d7\") " pod="openshift-console/console-c658d4d8c-vmzq9"
Sep 10 19:17:36.199039 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:36.198918    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/037a1789-f3a1-40b1-9e35-d9716af544d7-service-ca\") pod \"console-c658d4d8c-vmzq9\" (UID: \"037a1789-f3a1-40b1-9e35-d9716af544d7\") " pod="openshift-console/console-c658d4d8c-vmzq9"
Sep 10 19:17:36.199289 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:36.199265    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/037a1789-f3a1-40b1-9e35-d9716af544d7-trusted-ca-bundle\") pod \"console-c658d4d8c-vmzq9\" (UID: \"037a1789-f3a1-40b1-9e35-d9716af544d7\") " pod="openshift-console/console-c658d4d8c-vmzq9"
Sep 10 19:17:36.200694 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:36.200677    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/037a1789-f3a1-40b1-9e35-d9716af544d7-console-serving-cert\") pod \"console-c658d4d8c-vmzq9\" (UID: \"037a1789-f3a1-40b1-9e35-d9716af544d7\") " pod="openshift-console/console-c658d4d8c-vmzq9"
Sep 10 19:17:36.200796 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:36.200778    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/037a1789-f3a1-40b1-9e35-d9716af544d7-console-oauth-config\") pod \"console-c658d4d8c-vmzq9\" (UID: \"037a1789-f3a1-40b1-9e35-d9716af544d7\") " pod="openshift-console/console-c658d4d8c-vmzq9"
Sep 10 19:17:36.207202 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:36.207185    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgc29\" (UniqueName: \"kubernetes.io/projected/037a1789-f3a1-40b1-9e35-d9716af544d7-kube-api-access-fgc29\") pod \"console-c658d4d8c-vmzq9\" (UID: \"037a1789-f3a1-40b1-9e35-d9716af544d7\") " pod="openshift-console/console-c658d4d8c-vmzq9"
Sep 10 19:17:36.298534 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:36.298497    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c658d4d8c-vmzq9"
Sep 10 19:17:36.572532 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:36.572508    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c658d4d8c-vmzq9"]
Sep 10 19:17:36.574973 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:17:36.574944    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod037a1789_f3a1_40b1_9e35_d9716af544d7.slice/crio-b39fdbc1b5320cad8294ca3da3e506d58f098cc78c4317657da0fe0b47393919 WatchSource:0}: Error finding container b39fdbc1b5320cad8294ca3da3e506d58f098cc78c4317657da0fe0b47393919: Status 404 returned error can't find the container with id b39fdbc1b5320cad8294ca3da3e506d58f098cc78c4317657da0fe0b47393919
Sep 10 19:17:36.838347 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:36.838266    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c658d4d8c-vmzq9" event={"ID":"037a1789-f3a1-40b1-9e35-d9716af544d7","Type":"ContainerStarted","Data":"1a33ebdc358b7dc8835dbd41ba6fa84cda67732f9ab9766cb332b7e43bb61c88"}
Sep 10 19:17:36.838347 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:36.838311    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c658d4d8c-vmzq9" event={"ID":"037a1789-f3a1-40b1-9e35-d9716af544d7","Type":"ContainerStarted","Data":"b39fdbc1b5320cad8294ca3da3e506d58f098cc78c4317657da0fe0b47393919"}
Sep 10 19:17:36.857625 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:36.857569    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c658d4d8c-vmzq9" podStartSLOduration=1.8575514119999998 podStartE2EDuration="1.857551412s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:17:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:17:36.856255444 +0000 UTC m=+112.076103926" watchObservedRunningTime="2026-09-10 19:17:36.857551412 +0000 UTC m=+112.077399893"
Sep 10 19:17:39.834422 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:39.834393    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7967dd7f95-fzkhg"
Sep 10 19:17:40.338539 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:40.338498    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8cf859848-hfmz8"
Sep 10 19:17:43.713388 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:43.713355    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gdvkn_27b0fcb8-4e92-4753-8562-6e1e8bfa9e97/serve-healthcheck-canary/0.log"
Sep 10 19:17:46.299664 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:46.299609    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-c658d4d8c-vmzq9"
Sep 10 19:17:46.300129 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:46.299693    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c658d4d8c-vmzq9"
Sep 10 19:17:46.304606 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:46.304576    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-c658d4d8c-vmzq9"
Sep 10 19:17:46.881205 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:46.881176    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-c658d4d8c-vmzq9"
Sep 10 19:17:46.937526 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:46.937484    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7959dbb847-5lm7n"]
Sep 10 19:17:49.833900 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:49.833863    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-55f6cbdc4b-6s7ds"
Sep 10 19:17:49.834287 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:17:49.833931    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-55f6cbdc4b-6s7ds"
Sep 10 19:18:00.967533 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:00.967498    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8cf859848-hfmz8" podUID="1f07ec2b-f1d6-41fb-af14-95bdebb52a94" containerName="console" containerID="cri-o://482ddf964a0689a517393c4454294a19dc62d2bd6f91605e82afeae023fa55fc" gracePeriod=15
Sep 10 19:18:01.201986 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:01.201963    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8cf859848-hfmz8"
Sep 10 19:18:01.321424 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:01.321345    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-trusted-ca-bundle\") pod \"1f07ec2b-f1d6-41fb-af14-95bdebb52a94\" (UID: \"1f07ec2b-f1d6-41fb-af14-95bdebb52a94\") "
Sep 10 19:18:01.321424 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:01.321411    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-console-config\") pod \"1f07ec2b-f1d6-41fb-af14-95bdebb52a94\" (UID: \"1f07ec2b-f1d6-41fb-af14-95bdebb52a94\") "
Sep 10 19:18:01.321793 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:01.321445    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-console-oauth-config\") pod \"1f07ec2b-f1d6-41fb-af14-95bdebb52a94\" (UID: \"1f07ec2b-f1d6-41fb-af14-95bdebb52a94\") "
Sep 10 19:18:01.321793 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:01.321518    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m8xgq\" (UniqueName: \"kubernetes.io/projected/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-kube-api-access-m8xgq\") pod \"1f07ec2b-f1d6-41fb-af14-95bdebb52a94\" (UID: \"1f07ec2b-f1d6-41fb-af14-95bdebb52a94\") "
Sep 10 19:18:01.321793 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:01.321554    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-console-serving-cert\") pod \"1f07ec2b-f1d6-41fb-af14-95bdebb52a94\" (UID: \"1f07ec2b-f1d6-41fb-af14-95bdebb52a94\") "
Sep 10 19:18:01.321793 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:01.321620    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-service-ca\") pod \"1f07ec2b-f1d6-41fb-af14-95bdebb52a94\" (UID: \"1f07ec2b-f1d6-41fb-af14-95bdebb52a94\") "
Sep 10 19:18:01.321793 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:01.321656    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-oauth-serving-cert\") pod \"1f07ec2b-f1d6-41fb-af14-95bdebb52a94\" (UID: \"1f07ec2b-f1d6-41fb-af14-95bdebb52a94\") "
Sep 10 19:18:01.322038 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:01.321822    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1f07ec2b-f1d6-41fb-af14-95bdebb52a94" (UID: "1f07ec2b-f1d6-41fb-af14-95bdebb52a94"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:18:01.322038 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:01.321980    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-console-config" (OuterVolumeSpecName: "console-config") pod "1f07ec2b-f1d6-41fb-af14-95bdebb52a94" (UID: "1f07ec2b-f1d6-41fb-af14-95bdebb52a94"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:18:01.322038 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:01.322035    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-trusted-ca-bundle\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:18:01.322183 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:01.322058    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1f07ec2b-f1d6-41fb-af14-95bdebb52a94" (UID: "1f07ec2b-f1d6-41fb-af14-95bdebb52a94"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:18:01.322183 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:01.322117    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-service-ca" (OuterVolumeSpecName: "service-ca") pod "1f07ec2b-f1d6-41fb-af14-95bdebb52a94" (UID: "1f07ec2b-f1d6-41fb-af14-95bdebb52a94"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:18:01.323956 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:01.323925    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1f07ec2b-f1d6-41fb-af14-95bdebb52a94" (UID: "1f07ec2b-f1d6-41fb-af14-95bdebb52a94"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:18:01.324074 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:01.323995    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-kube-api-access-m8xgq" (OuterVolumeSpecName: "kube-api-access-m8xgq") pod "1f07ec2b-f1d6-41fb-af14-95bdebb52a94" (UID: "1f07ec2b-f1d6-41fb-af14-95bdebb52a94"). InnerVolumeSpecName "kube-api-access-m8xgq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:18:01.324074 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:01.324030    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1f07ec2b-f1d6-41fb-af14-95bdebb52a94" (UID: "1f07ec2b-f1d6-41fb-af14-95bdebb52a94"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:18:01.423359 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:01.423330    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-console-config\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:18:01.423359 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:01.423356    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-console-oauth-config\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:18:01.423359 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:01.423365    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m8xgq\" (UniqueName: \"kubernetes.io/projected/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-kube-api-access-m8xgq\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:18:01.423571 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:01.423375    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-console-serving-cert\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:18:01.423571 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:01.423384    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-service-ca\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:18:01.423571 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:01.423392    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f07ec2b-f1d6-41fb-af14-95bdebb52a94-oauth-serving-cert\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:18:01.940912 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:01.940876    2579 generic.go:356] "Generic (PLEG): container finished" podID="1f07ec2b-f1d6-41fb-af14-95bdebb52a94" containerID="482ddf964a0689a517393c4454294a19dc62d2bd6f91605e82afeae023fa55fc" exitCode=0
Sep 10 19:18:01.941101 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:01.940919    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8cf859848-hfmz8"
Sep 10 19:18:01.941101 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:01.940975    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8cf859848-hfmz8" event={"ID":"1f07ec2b-f1d6-41fb-af14-95bdebb52a94","Type":"ContainerDied","Data":"482ddf964a0689a517393c4454294a19dc62d2bd6f91605e82afeae023fa55fc"}
Sep 10 19:18:01.941101 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:01.941031    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8cf859848-hfmz8" event={"ID":"1f07ec2b-f1d6-41fb-af14-95bdebb52a94","Type":"ContainerDied","Data":"6786a053a40afc93b80de06332fac59ffb03de4a9054d049d1b21d8cacdce579"}
Sep 10 19:18:01.941101 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:01.941057    2579 scope.go:117] "RemoveContainer" containerID="482ddf964a0689a517393c4454294a19dc62d2bd6f91605e82afeae023fa55fc"
Sep 10 19:18:01.942600 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:01.942580    2579 generic.go:356] "Generic (PLEG): container finished" podID="77f7916d-5334-4e57-848f-afedd9e63d29" containerID="25dab80f84ad35f6532c3e3c31b1a928a297be3129c20ca319ec39a58c0ca89b" exitCode=0
Sep 10 19:18:01.942687 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:01.942637    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5psdd" event={"ID":"77f7916d-5334-4e57-848f-afedd9e63d29","Type":"ContainerDied","Data":"25dab80f84ad35f6532c3e3c31b1a928a297be3129c20ca319ec39a58c0ca89b"}
Sep 10 19:18:01.942967 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:01.942946    2579 scope.go:117] "RemoveContainer" containerID="25dab80f84ad35f6532c3e3c31b1a928a297be3129c20ca319ec39a58c0ca89b"
Sep 10 19:18:01.952282 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:01.952261    2579 scope.go:117] "RemoveContainer" containerID="482ddf964a0689a517393c4454294a19dc62d2bd6f91605e82afeae023fa55fc"
Sep 10 19:18:01.952543 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:18:01.952524    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"482ddf964a0689a517393c4454294a19dc62d2bd6f91605e82afeae023fa55fc\": container with ID starting with 482ddf964a0689a517393c4454294a19dc62d2bd6f91605e82afeae023fa55fc not found: ID does not exist" containerID="482ddf964a0689a517393c4454294a19dc62d2bd6f91605e82afeae023fa55fc"
Sep 10 19:18:01.952614 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:01.952549    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"482ddf964a0689a517393c4454294a19dc62d2bd6f91605e82afeae023fa55fc"} err="failed to get container status \"482ddf964a0689a517393c4454294a19dc62d2bd6f91605e82afeae023fa55fc\": rpc error: code = NotFound desc = could not find container \"482ddf964a0689a517393c4454294a19dc62d2bd6f91605e82afeae023fa55fc\": container with ID starting with 482ddf964a0689a517393c4454294a19dc62d2bd6f91605e82afeae023fa55fc not found: ID does not exist"
Sep 10 19:18:01.983399 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:01.983369    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8cf859848-hfmz8"]
Sep 10 19:18:01.987928 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:01.987901    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8cf859848-hfmz8"]
Sep 10 19:18:02.949586 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:02.949555    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5psdd" event={"ID":"77f7916d-5334-4e57-848f-afedd9e63d29","Type":"ContainerStarted","Data":"1341bcfe9174621659a7bc6d5c2b788c4b2ecc94d9fabc3744f11babd3af2db6"}
Sep 10 19:18:03.238450 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:03.238371    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f07ec2b-f1d6-41fb-af14-95bdebb52a94" path="/var/lib/kubelet/pods/1f07ec2b-f1d6-41fb-af14-95bdebb52a94/volumes"
Sep 10 19:18:09.838573 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:09.838543    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-55f6cbdc4b-6s7ds"
Sep 10 19:18:09.842443 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:09.842419    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-55f6cbdc4b-6s7ds"
Sep 10 19:18:11.962048 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:11.962007    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7959dbb847-5lm7n" podUID="2e41614d-6b68-4fd4-a424-e2185a53e6cd" containerName="console" containerID="cri-o://4438b481b73bc6b0a938be92b4d0bf259cd62c204dab60e3b0f7baae680170b4" gracePeriod=15
Sep 10 19:18:12.209400 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:12.209378    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7959dbb847-5lm7n"
Sep 10 19:18:12.314732 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:12.314631    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e41614d-6b68-4fd4-a424-e2185a53e6cd-console-serving-cert\") pod \"2e41614d-6b68-4fd4-a424-e2185a53e6cd\" (UID: \"2e41614d-6b68-4fd4-a424-e2185a53e6cd\") "
Sep 10 19:18:12.314732 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:12.314679    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f4km7\" (UniqueName: \"kubernetes.io/projected/2e41614d-6b68-4fd4-a424-e2185a53e6cd-kube-api-access-f4km7\") pod \"2e41614d-6b68-4fd4-a424-e2185a53e6cd\" (UID: \"2e41614d-6b68-4fd4-a424-e2185a53e6cd\") "
Sep 10 19:18:12.314987 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:12.314742    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2e41614d-6b68-4fd4-a424-e2185a53e6cd-console-config\") pod \"2e41614d-6b68-4fd4-a424-e2185a53e6cd\" (UID: \"2e41614d-6b68-4fd4-a424-e2185a53e6cd\") "
Sep 10 19:18:12.314987 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:12.314796    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e41614d-6b68-4fd4-a424-e2185a53e6cd-trusted-ca-bundle\") pod \"2e41614d-6b68-4fd4-a424-e2185a53e6cd\" (UID: \"2e41614d-6b68-4fd4-a424-e2185a53e6cd\") "
Sep 10 19:18:12.314987 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:12.314820    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2e41614d-6b68-4fd4-a424-e2185a53e6cd-oauth-serving-cert\") pod \"2e41614d-6b68-4fd4-a424-e2185a53e6cd\" (UID: \"2e41614d-6b68-4fd4-a424-e2185a53e6cd\") "
Sep 10 19:18:12.314987 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:12.314853    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2e41614d-6b68-4fd4-a424-e2185a53e6cd-console-oauth-config\") pod \"2e41614d-6b68-4fd4-a424-e2185a53e6cd\" (UID: \"2e41614d-6b68-4fd4-a424-e2185a53e6cd\") "
Sep 10 19:18:12.314987 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:12.314874    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e41614d-6b68-4fd4-a424-e2185a53e6cd-service-ca\") pod \"2e41614d-6b68-4fd4-a424-e2185a53e6cd\" (UID: \"2e41614d-6b68-4fd4-a424-e2185a53e6cd\") "
Sep 10 19:18:12.315265 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:12.315232    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e41614d-6b68-4fd4-a424-e2185a53e6cd-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2e41614d-6b68-4fd4-a424-e2185a53e6cd" (UID: "2e41614d-6b68-4fd4-a424-e2185a53e6cd"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:18:12.315265 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:12.315251    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e41614d-6b68-4fd4-a424-e2185a53e6cd-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "2e41614d-6b68-4fd4-a424-e2185a53e6cd" (UID: "2e41614d-6b68-4fd4-a424-e2185a53e6cd"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:18:12.315381 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:12.315259    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e41614d-6b68-4fd4-a424-e2185a53e6cd-console-config" (OuterVolumeSpecName: "console-config") pod "2e41614d-6b68-4fd4-a424-e2185a53e6cd" (UID: "2e41614d-6b68-4fd4-a424-e2185a53e6cd"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:18:12.315548 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:12.315442    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e41614d-6b68-4fd4-a424-e2185a53e6cd-service-ca" (OuterVolumeSpecName: "service-ca") pod "2e41614d-6b68-4fd4-a424-e2185a53e6cd" (UID: "2e41614d-6b68-4fd4-a424-e2185a53e6cd"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:18:12.317079 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:12.317047    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e41614d-6b68-4fd4-a424-e2185a53e6cd-kube-api-access-f4km7" (OuterVolumeSpecName: "kube-api-access-f4km7") pod "2e41614d-6b68-4fd4-a424-e2185a53e6cd" (UID: "2e41614d-6b68-4fd4-a424-e2185a53e6cd"). InnerVolumeSpecName "kube-api-access-f4km7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:18:12.317162 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:12.317074    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e41614d-6b68-4fd4-a424-e2185a53e6cd-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2e41614d-6b68-4fd4-a424-e2185a53e6cd" (UID: "2e41614d-6b68-4fd4-a424-e2185a53e6cd"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:18:12.317162 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:12.317100    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e41614d-6b68-4fd4-a424-e2185a53e6cd-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2e41614d-6b68-4fd4-a424-e2185a53e6cd" (UID: "2e41614d-6b68-4fd4-a424-e2185a53e6cd"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:18:12.415896 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:12.415862    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2e41614d-6b68-4fd4-a424-e2185a53e6cd-console-config\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:18:12.415896 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:12.415888    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e41614d-6b68-4fd4-a424-e2185a53e6cd-trusted-ca-bundle\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:18:12.415896 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:12.415898    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2e41614d-6b68-4fd4-a424-e2185a53e6cd-oauth-serving-cert\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:18:12.416118 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:12.415906    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2e41614d-6b68-4fd4-a424-e2185a53e6cd-console-oauth-config\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:18:12.416118 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:12.415916    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e41614d-6b68-4fd4-a424-e2185a53e6cd-service-ca\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:18:12.416118 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:12.415924    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e41614d-6b68-4fd4-a424-e2185a53e6cd-console-serving-cert\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:18:12.416118 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:12.415932    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4km7\" (UniqueName: \"kubernetes.io/projected/2e41614d-6b68-4fd4-a424-e2185a53e6cd-kube-api-access-f4km7\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:18:12.989417 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:12.989379    2579 generic.go:356] "Generic (PLEG): container finished" podID="2e41614d-6b68-4fd4-a424-e2185a53e6cd" containerID="4438b481b73bc6b0a938be92b4d0bf259cd62c204dab60e3b0f7baae680170b4" exitCode=0
Sep 10 19:18:12.989843 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:12.989425    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7959dbb847-5lm7n"
Sep 10 19:18:12.989843 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:12.989465    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7959dbb847-5lm7n" event={"ID":"2e41614d-6b68-4fd4-a424-e2185a53e6cd","Type":"ContainerDied","Data":"4438b481b73bc6b0a938be92b4d0bf259cd62c204dab60e3b0f7baae680170b4"}
Sep 10 19:18:12.989843 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:12.989520    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7959dbb847-5lm7n" event={"ID":"2e41614d-6b68-4fd4-a424-e2185a53e6cd","Type":"ContainerDied","Data":"ee57aaa52c6b71b68dfa59d1e07d054591ca016b104f99b32113f5d0069d80e2"}
Sep 10 19:18:12.989843 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:12.989540    2579 scope.go:117] "RemoveContainer" containerID="4438b481b73bc6b0a938be92b4d0bf259cd62c204dab60e3b0f7baae680170b4"
Sep 10 19:18:13.003522 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:13.003501    2579 scope.go:117] "RemoveContainer" containerID="4438b481b73bc6b0a938be92b4d0bf259cd62c204dab60e3b0f7baae680170b4"
Sep 10 19:18:13.003827 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:18:13.003804    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4438b481b73bc6b0a938be92b4d0bf259cd62c204dab60e3b0f7baae680170b4\": container with ID starting with 4438b481b73bc6b0a938be92b4d0bf259cd62c204dab60e3b0f7baae680170b4 not found: ID does not exist" containerID="4438b481b73bc6b0a938be92b4d0bf259cd62c204dab60e3b0f7baae680170b4"
Sep 10 19:18:13.003904 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:13.003833    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4438b481b73bc6b0a938be92b4d0bf259cd62c204dab60e3b0f7baae680170b4"} err="failed to get container status \"4438b481b73bc6b0a938be92b4d0bf259cd62c204dab60e3b0f7baae680170b4\": rpc error: code = NotFound desc = could not find container \"4438b481b73bc6b0a938be92b4d0bf259cd62c204dab60e3b0f7baae680170b4\": container with ID starting with 4438b481b73bc6b0a938be92b4d0bf259cd62c204dab60e3b0f7baae680170b4 not found: ID does not exist"
Sep 10 19:18:13.013991 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:13.013966    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7959dbb847-5lm7n"]
Sep 10 19:18:13.017969 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:13.017948    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7959dbb847-5lm7n"]
Sep 10 19:18:13.237694 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:13.237659    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e41614d-6b68-4fd4-a424-e2185a53e6cd" path="/var/lib/kubelet/pods/2e41614d-6b68-4fd4-a424-e2185a53e6cd/volumes"
Sep 10 19:18:51.099456 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:51.099418    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f8ddf84ff-x9544"]
Sep 10 19:18:51.099914 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:51.099723    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f07ec2b-f1d6-41fb-af14-95bdebb52a94" containerName="console"
Sep 10 19:18:51.099914 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:51.099735    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f07ec2b-f1d6-41fb-af14-95bdebb52a94" containerName="console"
Sep 10 19:18:51.099914 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:51.099774    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e41614d-6b68-4fd4-a424-e2185a53e6cd" containerName="console"
Sep 10 19:18:51.099914 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:51.099780    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e41614d-6b68-4fd4-a424-e2185a53e6cd" containerName="console"
Sep 10 19:18:51.099914 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:51.099822    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e41614d-6b68-4fd4-a424-e2185a53e6cd" containerName="console"
Sep 10 19:18:51.099914 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:51.099829    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f07ec2b-f1d6-41fb-af14-95bdebb52a94" containerName="console"
Sep 10 19:18:51.104123 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:51.104096    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f8ddf84ff-x9544"
Sep 10 19:18:51.113234 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:51.113213    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f8ddf84ff-x9544"]
Sep 10 19:18:51.233944 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:51.233908    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ba2017b0-d836-41df-abd0-eef93715adc6-console-serving-cert\") pod \"console-7f8ddf84ff-x9544\" (UID: \"ba2017b0-d836-41df-abd0-eef93715adc6\") " pod="openshift-console/console-7f8ddf84ff-x9544"
Sep 10 19:18:51.234135 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:51.233954    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ba2017b0-d836-41df-abd0-eef93715adc6-service-ca\") pod \"console-7f8ddf84ff-x9544\" (UID: \"ba2017b0-d836-41df-abd0-eef93715adc6\") " pod="openshift-console/console-7f8ddf84ff-x9544"
Sep 10 19:18:51.234135 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:51.233975    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ba2017b0-d836-41df-abd0-eef93715adc6-oauth-serving-cert\") pod \"console-7f8ddf84ff-x9544\" (UID: \"ba2017b0-d836-41df-abd0-eef93715adc6\") " pod="openshift-console/console-7f8ddf84ff-x9544"
Sep 10 19:18:51.234135 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:51.234054    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ms4rj\" (UniqueName: \"kubernetes.io/projected/ba2017b0-d836-41df-abd0-eef93715adc6-kube-api-access-ms4rj\") pod \"console-7f8ddf84ff-x9544\" (UID: \"ba2017b0-d836-41df-abd0-eef93715adc6\") " pod="openshift-console/console-7f8ddf84ff-x9544"
Sep 10 19:18:51.234135 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:51.234115    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ba2017b0-d836-41df-abd0-eef93715adc6-console-oauth-config\") pod \"console-7f8ddf84ff-x9544\" (UID: \"ba2017b0-d836-41df-abd0-eef93715adc6\") " pod="openshift-console/console-7f8ddf84ff-x9544"
Sep 10 19:18:51.234135 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:51.234131    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ba2017b0-d836-41df-abd0-eef93715adc6-console-config\") pod \"console-7f8ddf84ff-x9544\" (UID: \"ba2017b0-d836-41df-abd0-eef93715adc6\") " pod="openshift-console/console-7f8ddf84ff-x9544"
Sep 10 19:18:51.234317 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:51.234152    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba2017b0-d836-41df-abd0-eef93715adc6-trusted-ca-bundle\") pod \"console-7f8ddf84ff-x9544\" (UID: \"ba2017b0-d836-41df-abd0-eef93715adc6\") " pod="openshift-console/console-7f8ddf84ff-x9544"
Sep 10 19:18:51.334702 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:51.334660    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba2017b0-d836-41df-abd0-eef93715adc6-trusted-ca-bundle\") pod \"console-7f8ddf84ff-x9544\" (UID: \"ba2017b0-d836-41df-abd0-eef93715adc6\") " pod="openshift-console/console-7f8ddf84ff-x9544"
Sep 10 19:18:51.334702 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:51.334705    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ba2017b0-d836-41df-abd0-eef93715adc6-console-serving-cert\") pod \"console-7f8ddf84ff-x9544\" (UID: \"ba2017b0-d836-41df-abd0-eef93715adc6\") " pod="openshift-console/console-7f8ddf84ff-x9544"
Sep 10 19:18:51.334975 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:51.334744    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ba2017b0-d836-41df-abd0-eef93715adc6-service-ca\") pod \"console-7f8ddf84ff-x9544\" (UID: \"ba2017b0-d836-41df-abd0-eef93715adc6\") " pod="openshift-console/console-7f8ddf84ff-x9544"
Sep 10 19:18:51.334975 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:51.334796    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ba2017b0-d836-41df-abd0-eef93715adc6-oauth-serving-cert\") pod \"console-7f8ddf84ff-x9544\" (UID: \"ba2017b0-d836-41df-abd0-eef93715adc6\") " pod="openshift-console/console-7f8ddf84ff-x9544"
Sep 10 19:18:51.334975 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:51.334834    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms4rj\" (UniqueName: \"kubernetes.io/projected/ba2017b0-d836-41df-abd0-eef93715adc6-kube-api-access-ms4rj\") pod \"console-7f8ddf84ff-x9544\" (UID: \"ba2017b0-d836-41df-abd0-eef93715adc6\") " pod="openshift-console/console-7f8ddf84ff-x9544"
Sep 10 19:18:51.334975 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:51.334880    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ba2017b0-d836-41df-abd0-eef93715adc6-console-oauth-config\") pod \"console-7f8ddf84ff-x9544\" (UID: \"ba2017b0-d836-41df-abd0-eef93715adc6\") " pod="openshift-console/console-7f8ddf84ff-x9544"
Sep 10 19:18:51.334975 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:51.334897    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ba2017b0-d836-41df-abd0-eef93715adc6-console-config\") pod \"console-7f8ddf84ff-x9544\" (UID: \"ba2017b0-d836-41df-abd0-eef93715adc6\") " pod="openshift-console/console-7f8ddf84ff-x9544"
Sep 10 19:18:51.335634 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:51.335607    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ba2017b0-d836-41df-abd0-eef93715adc6-console-config\") pod \"console-7f8ddf84ff-x9544\" (UID: \"ba2017b0-d836-41df-abd0-eef93715adc6\") " pod="openshift-console/console-7f8ddf84ff-x9544"
Sep 10 19:18:51.335775 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:51.335644    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ba2017b0-d836-41df-abd0-eef93715adc6-oauth-serving-cert\") pod \"console-7f8ddf84ff-x9544\" (UID: \"ba2017b0-d836-41df-abd0-eef93715adc6\") " pod="openshift-console/console-7f8ddf84ff-x9544"
Sep 10 19:18:51.335775 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:51.335660    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ba2017b0-d836-41df-abd0-eef93715adc6-service-ca\") pod \"console-7f8ddf84ff-x9544\" (UID: \"ba2017b0-d836-41df-abd0-eef93715adc6\") " pod="openshift-console/console-7f8ddf84ff-x9544"
Sep 10 19:18:51.335869 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:51.335724    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba2017b0-d836-41df-abd0-eef93715adc6-trusted-ca-bundle\") pod \"console-7f8ddf84ff-x9544\" (UID: \"ba2017b0-d836-41df-abd0-eef93715adc6\") " pod="openshift-console/console-7f8ddf84ff-x9544"
Sep 10 19:18:51.337407 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:51.337386    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ba2017b0-d836-41df-abd0-eef93715adc6-console-serving-cert\") pod \"console-7f8ddf84ff-x9544\" (UID: \"ba2017b0-d836-41df-abd0-eef93715adc6\") " pod="openshift-console/console-7f8ddf84ff-x9544"
Sep 10 19:18:51.337519 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:51.337496    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ba2017b0-d836-41df-abd0-eef93715adc6-console-oauth-config\") pod \"console-7f8ddf84ff-x9544\" (UID: \"ba2017b0-d836-41df-abd0-eef93715adc6\") " pod="openshift-console/console-7f8ddf84ff-x9544"
Sep 10 19:18:51.343631 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:51.343610    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms4rj\" (UniqueName: \"kubernetes.io/projected/ba2017b0-d836-41df-abd0-eef93715adc6-kube-api-access-ms4rj\") pod \"console-7f8ddf84ff-x9544\" (UID: \"ba2017b0-d836-41df-abd0-eef93715adc6\") " pod="openshift-console/console-7f8ddf84ff-x9544"
Sep 10 19:18:51.415410 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:51.415379    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f8ddf84ff-x9544"
Sep 10 19:18:51.690914 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:51.690836    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f8ddf84ff-x9544"]
Sep 10 19:18:51.694113 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:18:51.694087    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba2017b0_d836_41df_abd0_eef93715adc6.slice/crio-bee010a5d9fbcd8b0d8e9de93ac6dedef1cbeb9362a317e80d38c7628c547f9c WatchSource:0}: Error finding container bee010a5d9fbcd8b0d8e9de93ac6dedef1cbeb9362a317e80d38c7628c547f9c: Status 404 returned error can't find the container with id bee010a5d9fbcd8b0d8e9de93ac6dedef1cbeb9362a317e80d38c7628c547f9c
Sep 10 19:18:52.144295 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:52.144261    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f8ddf84ff-x9544" event={"ID":"ba2017b0-d836-41df-abd0-eef93715adc6","Type":"ContainerStarted","Data":"fd5f625d6c67a315c2a44dbedb2fd0fab78b62f804dca17e3659724f21b7eb82"}
Sep 10 19:18:52.144725 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:52.144313    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f8ddf84ff-x9544" event={"ID":"ba2017b0-d836-41df-abd0-eef93715adc6","Type":"ContainerStarted","Data":"bee010a5d9fbcd8b0d8e9de93ac6dedef1cbeb9362a317e80d38c7628c547f9c"}
Sep 10 19:18:52.162725 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:18:52.162645    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7f8ddf84ff-x9544" podStartSLOduration=1.162630615 podStartE2EDuration="1.162630615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:18:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:18:52.162139383 +0000 UTC m=+187.381987863" watchObservedRunningTime="2026-09-10 19:18:52.162630615 +0000 UTC m=+187.382479096"
Sep 10 19:19:01.415735 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:01.415691    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f8ddf84ff-x9544"
Sep 10 19:19:01.416228 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:01.415779    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7f8ddf84ff-x9544"
Sep 10 19:19:01.420675 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:01.420653    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7f8ddf84ff-x9544"
Sep 10 19:19:02.185374 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:02.185339    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7f8ddf84ff-x9544"
Sep 10 19:19:02.235939 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:02.235906    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c658d4d8c-vmzq9"]
Sep 10 19:19:27.257512 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:27.257469    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-c658d4d8c-vmzq9" podUID="037a1789-f3a1-40b1-9e35-d9716af544d7" containerName="console" containerID="cri-o://1a33ebdc358b7dc8835dbd41ba6fa84cda67732f9ab9766cb332b7e43bb61c88" gracePeriod=15
Sep 10 19:19:27.499526 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:27.499501    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c658d4d8c-vmzq9"
Sep 10 19:19:27.527536 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:27.527465    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/037a1789-f3a1-40b1-9e35-d9716af544d7-console-serving-cert\") pod \"037a1789-f3a1-40b1-9e35-d9716af544d7\" (UID: \"037a1789-f3a1-40b1-9e35-d9716af544d7\") "
Sep 10 19:19:27.527536 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:27.527508    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fgc29\" (UniqueName: \"kubernetes.io/projected/037a1789-f3a1-40b1-9e35-d9716af544d7-kube-api-access-fgc29\") pod \"037a1789-f3a1-40b1-9e35-d9716af544d7\" (UID: \"037a1789-f3a1-40b1-9e35-d9716af544d7\") "
Sep 10 19:19:27.527536 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:27.527538    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/037a1789-f3a1-40b1-9e35-d9716af544d7-service-ca\") pod \"037a1789-f3a1-40b1-9e35-d9716af544d7\" (UID: \"037a1789-f3a1-40b1-9e35-d9716af544d7\") "
Sep 10 19:19:27.527819 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:27.527557    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/037a1789-f3a1-40b1-9e35-d9716af544d7-trusted-ca-bundle\") pod \"037a1789-f3a1-40b1-9e35-d9716af544d7\" (UID: \"037a1789-f3a1-40b1-9e35-d9716af544d7\") "
Sep 10 19:19:27.527819 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:27.527592    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/037a1789-f3a1-40b1-9e35-d9716af544d7-oauth-serving-cert\") pod \"037a1789-f3a1-40b1-9e35-d9716af544d7\" (UID: \"037a1789-f3a1-40b1-9e35-d9716af544d7\") "
Sep 10 19:19:27.527819 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:27.527641    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/037a1789-f3a1-40b1-9e35-d9716af544d7-console-config\") pod \"037a1789-f3a1-40b1-9e35-d9716af544d7\" (UID: \"037a1789-f3a1-40b1-9e35-d9716af544d7\") "
Sep 10 19:19:27.527819 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:27.527666    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/037a1789-f3a1-40b1-9e35-d9716af544d7-console-oauth-config\") pod \"037a1789-f3a1-40b1-9e35-d9716af544d7\" (UID: \"037a1789-f3a1-40b1-9e35-d9716af544d7\") "
Sep 10 19:19:27.528102 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:27.528050    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/037a1789-f3a1-40b1-9e35-d9716af544d7-service-ca" (OuterVolumeSpecName: "service-ca") pod "037a1789-f3a1-40b1-9e35-d9716af544d7" (UID: "037a1789-f3a1-40b1-9e35-d9716af544d7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:19:27.528186 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:27.528156    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/037a1789-f3a1-40b1-9e35-d9716af544d7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "037a1789-f3a1-40b1-9e35-d9716af544d7" (UID: "037a1789-f3a1-40b1-9e35-d9716af544d7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:19:27.528298 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:27.528270    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/037a1789-f3a1-40b1-9e35-d9716af544d7-console-config" (OuterVolumeSpecName: "console-config") pod "037a1789-f3a1-40b1-9e35-d9716af544d7" (UID: "037a1789-f3a1-40b1-9e35-d9716af544d7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:19:27.528361 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:27.528296    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/037a1789-f3a1-40b1-9e35-d9716af544d7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "037a1789-f3a1-40b1-9e35-d9716af544d7" (UID: "037a1789-f3a1-40b1-9e35-d9716af544d7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:19:27.530011 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:27.529992    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/037a1789-f3a1-40b1-9e35-d9716af544d7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "037a1789-f3a1-40b1-9e35-d9716af544d7" (UID: "037a1789-f3a1-40b1-9e35-d9716af544d7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:19:27.530435 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:27.530412    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/037a1789-f3a1-40b1-9e35-d9716af544d7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "037a1789-f3a1-40b1-9e35-d9716af544d7" (UID: "037a1789-f3a1-40b1-9e35-d9716af544d7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:19:27.530505 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:27.530413    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/037a1789-f3a1-40b1-9e35-d9716af544d7-kube-api-access-fgc29" (OuterVolumeSpecName: "kube-api-access-fgc29") pod "037a1789-f3a1-40b1-9e35-d9716af544d7" (UID: "037a1789-f3a1-40b1-9e35-d9716af544d7"). InnerVolumeSpecName "kube-api-access-fgc29". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:19:27.628984 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:27.628945    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fgc29\" (UniqueName: \"kubernetes.io/projected/037a1789-f3a1-40b1-9e35-d9716af544d7-kube-api-access-fgc29\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:19:27.628984 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:27.628973    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/037a1789-f3a1-40b1-9e35-d9716af544d7-service-ca\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:19:27.628984 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:27.628982    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/037a1789-f3a1-40b1-9e35-d9716af544d7-trusted-ca-bundle\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:19:27.628984 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:27.628992    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/037a1789-f3a1-40b1-9e35-d9716af544d7-oauth-serving-cert\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:19:27.629245 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:27.629001    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/037a1789-f3a1-40b1-9e35-d9716af544d7-console-config\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:19:27.629245 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:27.629009    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/037a1789-f3a1-40b1-9e35-d9716af544d7-console-oauth-config\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:19:27.629245 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:27.629017    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/037a1789-f3a1-40b1-9e35-d9716af544d7-console-serving-cert\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:19:28.282314 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:28.282275    2579 generic.go:356] "Generic (PLEG): container finished" podID="037a1789-f3a1-40b1-9e35-d9716af544d7" containerID="1a33ebdc358b7dc8835dbd41ba6fa84cda67732f9ab9766cb332b7e43bb61c88" exitCode=0
Sep 10 19:19:28.282724 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:28.282324    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c658d4d8c-vmzq9"
Sep 10 19:19:28.282724 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:28.282366    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c658d4d8c-vmzq9" event={"ID":"037a1789-f3a1-40b1-9e35-d9716af544d7","Type":"ContainerDied","Data":"1a33ebdc358b7dc8835dbd41ba6fa84cda67732f9ab9766cb332b7e43bb61c88"}
Sep 10 19:19:28.282724 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:28.282419    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c658d4d8c-vmzq9" event={"ID":"037a1789-f3a1-40b1-9e35-d9716af544d7","Type":"ContainerDied","Data":"b39fdbc1b5320cad8294ca3da3e506d58f098cc78c4317657da0fe0b47393919"}
Sep 10 19:19:28.282724 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:28.282439    2579 scope.go:117] "RemoveContainer" containerID="1a33ebdc358b7dc8835dbd41ba6fa84cda67732f9ab9766cb332b7e43bb61c88"
Sep 10 19:19:28.291180 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:28.291163    2579 scope.go:117] "RemoveContainer" containerID="1a33ebdc358b7dc8835dbd41ba6fa84cda67732f9ab9766cb332b7e43bb61c88"
Sep 10 19:19:28.291488 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:19:28.291469    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a33ebdc358b7dc8835dbd41ba6fa84cda67732f9ab9766cb332b7e43bb61c88\": container with ID starting with 1a33ebdc358b7dc8835dbd41ba6fa84cda67732f9ab9766cb332b7e43bb61c88 not found: ID does not exist" containerID="1a33ebdc358b7dc8835dbd41ba6fa84cda67732f9ab9766cb332b7e43bb61c88"
Sep 10 19:19:28.291533 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:28.291497    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a33ebdc358b7dc8835dbd41ba6fa84cda67732f9ab9766cb332b7e43bb61c88"} err="failed to get container status \"1a33ebdc358b7dc8835dbd41ba6fa84cda67732f9ab9766cb332b7e43bb61c88\": rpc error: code = NotFound desc = could not find container \"1a33ebdc358b7dc8835dbd41ba6fa84cda67732f9ab9766cb332b7e43bb61c88\": container with ID starting with 1a33ebdc358b7dc8835dbd41ba6fa84cda67732f9ab9766cb332b7e43bb61c88 not found: ID does not exist"
Sep 10 19:19:28.302597 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:28.302574    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c658d4d8c-vmzq9"]
Sep 10 19:19:28.304065 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:28.304043    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-c658d4d8c-vmzq9"]
Sep 10 19:19:29.237235 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:29.237202    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="037a1789-f3a1-40b1-9e35-d9716af544d7" path="/var/lib/kubelet/pods/037a1789-f3a1-40b1-9e35-d9716af544d7/volumes"
Sep 10 19:19:30.454994 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:30.454957    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404qchs"]
Sep 10 19:19:30.455372 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:30.455253    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="037a1789-f3a1-40b1-9e35-d9716af544d7" containerName="console"
Sep 10 19:19:30.455372 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:30.455265    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="037a1789-f3a1-40b1-9e35-d9716af544d7" containerName="console"
Sep 10 19:19:30.455372 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:30.455333    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="037a1789-f3a1-40b1-9e35-d9716af544d7" containerName="console"
Sep 10 19:19:30.459766 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:30.459734    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404qchs"
Sep 10 19:19:30.461903 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:30.461882    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 19:19:30.462011 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:30.461930    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ccnqb\""
Sep 10 19:19:30.462191 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:30.462178    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 19:19:30.468060 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:30.468036    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404qchs"]
Sep 10 19:19:30.550564 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:30.550525    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/acc3c23c-8196-4216-81bc-b0eee4b396f7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404qchs\" (UID: \"acc3c23c-8196-4216-81bc-b0eee4b396f7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404qchs"
Sep 10 19:19:30.550564 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:30.550570    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/acc3c23c-8196-4216-81bc-b0eee4b396f7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404qchs\" (UID: \"acc3c23c-8196-4216-81bc-b0eee4b396f7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404qchs"
Sep 10 19:19:30.550797 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:30.550603    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfxwx\" (UniqueName: \"kubernetes.io/projected/acc3c23c-8196-4216-81bc-b0eee4b396f7-kube-api-access-kfxwx\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404qchs\" (UID: \"acc3c23c-8196-4216-81bc-b0eee4b396f7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404qchs"
Sep 10 19:19:30.651461 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:30.651425    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/acc3c23c-8196-4216-81bc-b0eee4b396f7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404qchs\" (UID: \"acc3c23c-8196-4216-81bc-b0eee4b396f7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404qchs"
Sep 10 19:19:30.651461 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:30.651468    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfxwx\" (UniqueName: \"kubernetes.io/projected/acc3c23c-8196-4216-81bc-b0eee4b396f7-kube-api-access-kfxwx\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404qchs\" (UID: \"acc3c23c-8196-4216-81bc-b0eee4b396f7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404qchs"
Sep 10 19:19:30.651672 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:30.651528    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/acc3c23c-8196-4216-81bc-b0eee4b396f7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404qchs\" (UID: \"acc3c23c-8196-4216-81bc-b0eee4b396f7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404qchs"
Sep 10 19:19:30.651941 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:30.651923    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/acc3c23c-8196-4216-81bc-b0eee4b396f7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404qchs\" (UID: \"acc3c23c-8196-4216-81bc-b0eee4b396f7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404qchs"
Sep 10 19:19:30.651992 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:30.651921    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/acc3c23c-8196-4216-81bc-b0eee4b396f7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404qchs\" (UID: \"acc3c23c-8196-4216-81bc-b0eee4b396f7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404qchs"
Sep 10 19:19:30.659661 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:30.659624    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfxwx\" (UniqueName: \"kubernetes.io/projected/acc3c23c-8196-4216-81bc-b0eee4b396f7-kube-api-access-kfxwx\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404qchs\" (UID: \"acc3c23c-8196-4216-81bc-b0eee4b396f7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404qchs"
Sep 10 19:19:30.769987 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:30.769900    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404qchs"
Sep 10 19:19:31.042134 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:31.042106    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404qchs"]
Sep 10 19:19:31.044521 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:19:31.044490    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacc3c23c_8196_4216_81bc_b0eee4b396f7.slice/crio-b95ae00d99a63fc12b9a5c44ba490f5fdf977d926ec3b256630e51aeba70b0a0 WatchSource:0}: Error finding container b95ae00d99a63fc12b9a5c44ba490f5fdf977d926ec3b256630e51aeba70b0a0: Status 404 returned error can't find the container with id b95ae00d99a63fc12b9a5c44ba490f5fdf977d926ec3b256630e51aeba70b0a0
Sep 10 19:19:31.296412 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:31.296323    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404qchs" event={"ID":"acc3c23c-8196-4216-81bc-b0eee4b396f7","Type":"ContainerStarted","Data":"b95ae00d99a63fc12b9a5c44ba490f5fdf977d926ec3b256630e51aeba70b0a0"}
Sep 10 19:19:37.324362 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:37.324269    2579 generic.go:356] "Generic (PLEG): container finished" podID="acc3c23c-8196-4216-81bc-b0eee4b396f7" containerID="8cb6c1b20271e702396746726f1e3536da668b0894057b292dce503c5ef04aa4" exitCode=0
Sep 10 19:19:37.324362 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:37.324311    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404qchs" event={"ID":"acc3c23c-8196-4216-81bc-b0eee4b396f7","Type":"ContainerDied","Data":"8cb6c1b20271e702396746726f1e3536da668b0894057b292dce503c5ef04aa4"}
Sep 10 19:19:40.337271 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:40.337237    2579 generic.go:356] "Generic (PLEG): container finished" podID="acc3c23c-8196-4216-81bc-b0eee4b396f7" containerID="c566fad2ffb17eb35ef5c81e315e935353bb111a14302f72b437ee5c0d99a856" exitCode=0
Sep 10 19:19:40.337621 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:40.337308    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404qchs" event={"ID":"acc3c23c-8196-4216-81bc-b0eee4b396f7","Type":"ContainerDied","Data":"c566fad2ffb17eb35ef5c81e315e935353bb111a14302f72b437ee5c0d99a856"}
Sep 10 19:19:48.374310 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:48.374278    2579 generic.go:356] "Generic (PLEG): container finished" podID="acc3c23c-8196-4216-81bc-b0eee4b396f7" containerID="65649c0cfc46cd54036aabb8c55e3efc592610df633a57777d41ad5adff1765a" exitCode=0
Sep 10 19:19:48.374776 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:48.374367    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404qchs" event={"ID":"acc3c23c-8196-4216-81bc-b0eee4b396f7","Type":"ContainerDied","Data":"65649c0cfc46cd54036aabb8c55e3efc592610df633a57777d41ad5adff1765a"}
Sep 10 19:19:49.548940 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:49.548910    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404qchs"
Sep 10 19:19:49.620016 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:49.619981    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/acc3c23c-8196-4216-81bc-b0eee4b396f7-bundle\") pod \"acc3c23c-8196-4216-81bc-b0eee4b396f7\" (UID: \"acc3c23c-8196-4216-81bc-b0eee4b396f7\") "
Sep 10 19:19:49.620188 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:49.620060    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kfxwx\" (UniqueName: \"kubernetes.io/projected/acc3c23c-8196-4216-81bc-b0eee4b396f7-kube-api-access-kfxwx\") pod \"acc3c23c-8196-4216-81bc-b0eee4b396f7\" (UID: \"acc3c23c-8196-4216-81bc-b0eee4b396f7\") "
Sep 10 19:19:49.620188 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:49.620089    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/acc3c23c-8196-4216-81bc-b0eee4b396f7-util\") pod \"acc3c23c-8196-4216-81bc-b0eee4b396f7\" (UID: \"acc3c23c-8196-4216-81bc-b0eee4b396f7\") "
Sep 10 19:19:49.620763 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:49.620710    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acc3c23c-8196-4216-81bc-b0eee4b396f7-bundle" (OuterVolumeSpecName: "bundle") pod "acc3c23c-8196-4216-81bc-b0eee4b396f7" (UID: "acc3c23c-8196-4216-81bc-b0eee4b396f7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:19:49.622771 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:49.622731    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acc3c23c-8196-4216-81bc-b0eee4b396f7-kube-api-access-kfxwx" (OuterVolumeSpecName: "kube-api-access-kfxwx") pod "acc3c23c-8196-4216-81bc-b0eee4b396f7" (UID: "acc3c23c-8196-4216-81bc-b0eee4b396f7"). InnerVolumeSpecName "kube-api-access-kfxwx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:19:49.624287 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:49.624259    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acc3c23c-8196-4216-81bc-b0eee4b396f7-util" (OuterVolumeSpecName: "util") pod "acc3c23c-8196-4216-81bc-b0eee4b396f7" (UID: "acc3c23c-8196-4216-81bc-b0eee4b396f7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:19:49.721451 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:49.721417    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kfxwx\" (UniqueName: \"kubernetes.io/projected/acc3c23c-8196-4216-81bc-b0eee4b396f7-kube-api-access-kfxwx\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:19:49.721451 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:49.721446    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/acc3c23c-8196-4216-81bc-b0eee4b396f7-util\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:19:49.721451 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:49.721456    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/acc3c23c-8196-4216-81bc-b0eee4b396f7-bundle\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:19:50.383468 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:50.383442    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404qchs"
Sep 10 19:19:50.383640 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:50.383440    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404qchs" event={"ID":"acc3c23c-8196-4216-81bc-b0eee4b396f7","Type":"ContainerDied","Data":"b95ae00d99a63fc12b9a5c44ba490f5fdf977d926ec3b256630e51aeba70b0a0"}
Sep 10 19:19:50.383640 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:50.383555    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b95ae00d99a63fc12b9a5c44ba490f5fdf977d926ec3b256630e51aeba70b0a0"
Sep 10 19:19:57.894623 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:57.894582    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmspl"]
Sep 10 19:19:57.895185 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:57.894878    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acc3c23c-8196-4216-81bc-b0eee4b396f7" containerName="util"
Sep 10 19:19:57.895185 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:57.894889    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="acc3c23c-8196-4216-81bc-b0eee4b396f7" containerName="util"
Sep 10 19:19:57.895185 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:57.894897    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acc3c23c-8196-4216-81bc-b0eee4b396f7" containerName="extract"
Sep 10 19:19:57.895185 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:57.894910    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="acc3c23c-8196-4216-81bc-b0eee4b396f7" containerName="extract"
Sep 10 19:19:57.895185 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:57.894921    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acc3c23c-8196-4216-81bc-b0eee4b396f7" containerName="pull"
Sep 10 19:19:57.895185 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:57.894927    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="acc3c23c-8196-4216-81bc-b0eee4b396f7" containerName="pull"
Sep 10 19:19:57.895185 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:57.894980    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="acc3c23c-8196-4216-81bc-b0eee4b396f7" containerName="extract"
Sep 10 19:19:57.899195 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:57.899178    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmspl"
Sep 10 19:19:57.908076 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:57.908060    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 19:19:57.911562 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:57.911547    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ccnqb\""
Sep 10 19:19:57.911639 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:57.911620    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 19:19:57.923754 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:57.923730    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmspl"]
Sep 10 19:19:57.989590 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:57.989550    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5phj5\" (UniqueName: \"kubernetes.io/projected/462a2a7a-3493-44b0-a4fc-9a261c226ea9-kube-api-access-5phj5\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmspl\" (UID: \"462a2a7a-3493-44b0-a4fc-9a261c226ea9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmspl"
Sep 10 19:19:57.989590 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:57.989593    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/462a2a7a-3493-44b0-a4fc-9a261c226ea9-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmspl\" (UID: \"462a2a7a-3493-44b0-a4fc-9a261c226ea9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmspl"
Sep 10 19:19:57.989851 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:57.989712    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/462a2a7a-3493-44b0-a4fc-9a261c226ea9-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmspl\" (UID: \"462a2a7a-3493-44b0-a4fc-9a261c226ea9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmspl"
Sep 10 19:19:58.090175 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:58.090138    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/462a2a7a-3493-44b0-a4fc-9a261c226ea9-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmspl\" (UID: \"462a2a7a-3493-44b0-a4fc-9a261c226ea9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmspl"
Sep 10 19:19:58.090330 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:58.090210    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5phj5\" (UniqueName: \"kubernetes.io/projected/462a2a7a-3493-44b0-a4fc-9a261c226ea9-kube-api-access-5phj5\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmspl\" (UID: \"462a2a7a-3493-44b0-a4fc-9a261c226ea9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmspl"
Sep 10 19:19:58.090330 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:58.090239    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/462a2a7a-3493-44b0-a4fc-9a261c226ea9-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmspl\" (UID: \"462a2a7a-3493-44b0-a4fc-9a261c226ea9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmspl"
Sep 10 19:19:58.090501 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:58.090482    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/462a2a7a-3493-44b0-a4fc-9a261c226ea9-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmspl\" (UID: \"462a2a7a-3493-44b0-a4fc-9a261c226ea9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmspl"
Sep 10 19:19:58.090557 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:58.090539    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/462a2a7a-3493-44b0-a4fc-9a261c226ea9-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmspl\" (UID: \"462a2a7a-3493-44b0-a4fc-9a261c226ea9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmspl"
Sep 10 19:19:58.098561 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:58.098530    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5phj5\" (UniqueName: \"kubernetes.io/projected/462a2a7a-3493-44b0-a4fc-9a261c226ea9-kube-api-access-5phj5\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmspl\" (UID: \"462a2a7a-3493-44b0-a4fc-9a261c226ea9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmspl"
Sep 10 19:19:58.207548 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:58.207465    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmspl"
Sep 10 19:19:58.450661 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:58.450631    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-bwrzl"]
Sep 10 19:19:58.453984 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:58.453964    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-bwrzl"
Sep 10 19:19:58.456053 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:58.456023    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 10 19:19:58.456160 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:58.456064    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-xkssz\""
Sep 10 19:19:58.456286 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:58.456273    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 10 19:19:58.464027 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:58.464000    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-bwrzl"]
Sep 10 19:19:58.485082 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:58.485051    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmspl"]
Sep 10 19:19:58.488328 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:19:58.488295    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod462a2a7a_3493_44b0_a4fc_9a261c226ea9.slice/crio-57bf8a8f22cd3a5d85e4c49c5f6dd2df40d23e6bb0135e4a0a7ea86d370b18bf WatchSource:0}: Error finding container 57bf8a8f22cd3a5d85e4c49c5f6dd2df40d23e6bb0135e4a0a7ea86d370b18bf: Status 404 returned error can't find the container with id 57bf8a8f22cd3a5d85e4c49c5f6dd2df40d23e6bb0135e4a0a7ea86d370b18bf
Sep 10 19:19:58.595079 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:58.595048    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75sb7\" (UniqueName: \"kubernetes.io/projected/c29ef078-2088-4f33-994c-de20567e9cdb-kube-api-access-75sb7\") pod \"cert-manager-webhook-6486ffd997-bwrzl\" (UID: \"c29ef078-2088-4f33-994c-de20567e9cdb\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bwrzl"
Sep 10 19:19:58.595216 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:58.595102    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c29ef078-2088-4f33-994c-de20567e9cdb-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-bwrzl\" (UID: \"c29ef078-2088-4f33-994c-de20567e9cdb\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bwrzl"
Sep 10 19:19:58.695968 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:58.695933    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75sb7\" (UniqueName: \"kubernetes.io/projected/c29ef078-2088-4f33-994c-de20567e9cdb-kube-api-access-75sb7\") pod \"cert-manager-webhook-6486ffd997-bwrzl\" (UID: \"c29ef078-2088-4f33-994c-de20567e9cdb\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bwrzl"
Sep 10 19:19:58.695968 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:58.695987    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c29ef078-2088-4f33-994c-de20567e9cdb-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-bwrzl\" (UID: \"c29ef078-2088-4f33-994c-de20567e9cdb\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bwrzl"
Sep 10 19:19:58.707949 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:58.707924    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75sb7\" (UniqueName: \"kubernetes.io/projected/c29ef078-2088-4f33-994c-de20567e9cdb-kube-api-access-75sb7\") pod \"cert-manager-webhook-6486ffd997-bwrzl\" (UID: \"c29ef078-2088-4f33-994c-de20567e9cdb\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bwrzl"
Sep 10 19:19:58.708077 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:58.708017    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c29ef078-2088-4f33-994c-de20567e9cdb-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-bwrzl\" (UID: \"c29ef078-2088-4f33-994c-de20567e9cdb\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bwrzl"
Sep 10 19:19:58.764251 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:58.764171    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-bwrzl"
Sep 10 19:19:59.037777 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:59.037709    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-bwrzl"]
Sep 10 19:19:59.039808 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:19:59.039780    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc29ef078_2088_4f33_994c_de20567e9cdb.slice/crio-1e75643c455c4135c2f7f47a300bd8b1a1d7e49a6372dfcb3f88e11361574ca0 WatchSource:0}: Error finding container 1e75643c455c4135c2f7f47a300bd8b1a1d7e49a6372dfcb3f88e11361574ca0: Status 404 returned error can't find the container with id 1e75643c455c4135c2f7f47a300bd8b1a1d7e49a6372dfcb3f88e11361574ca0
Sep 10 19:19:59.418153 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:59.418115    2579 generic.go:356] "Generic (PLEG): container finished" podID="462a2a7a-3493-44b0-a4fc-9a261c226ea9" containerID="6d29f2f456c64f3d9d0376925aaa5e90553d9f623ea849a1f82fab8bc262aee1" exitCode=0
Sep 10 19:19:59.418341 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:59.418206    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmspl" event={"ID":"462a2a7a-3493-44b0-a4fc-9a261c226ea9","Type":"ContainerDied","Data":"6d29f2f456c64f3d9d0376925aaa5e90553d9f623ea849a1f82fab8bc262aee1"}
Sep 10 19:19:59.418341 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:59.418260    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmspl" event={"ID":"462a2a7a-3493-44b0-a4fc-9a261c226ea9","Type":"ContainerStarted","Data":"57bf8a8f22cd3a5d85e4c49c5f6dd2df40d23e6bb0135e4a0a7ea86d370b18bf"}
Sep 10 19:19:59.419513 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:59.419483    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-bwrzl" event={"ID":"c29ef078-2088-4f33-994c-de20567e9cdb","Type":"ContainerStarted","Data":"1e75643c455c4135c2f7f47a300bd8b1a1d7e49a6372dfcb3f88e11361574ca0"}
Sep 10 19:19:59.860218 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:59.860136    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-f5qll"]
Sep 10 19:19:59.863384 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:59.863365    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f5qll"
Sep 10 19:19:59.867120 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:59.867097    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-6qbmt\""
Sep 10 19:19:59.875786 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:19:59.875743    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-f5qll"]
Sep 10 19:20:00.005813 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:00.005777    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z89kf\" (UniqueName: \"kubernetes.io/projected/6fba7b42-cfe9-4f8b-9e98-236279fb1c00-kube-api-access-z89kf\") pod \"cert-manager-cainjector-559cdd8c87-f5qll\" (UID: \"6fba7b42-cfe9-4f8b-9e98-236279fb1c00\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f5qll"
Sep 10 19:20:00.005813 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:00.005816    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6fba7b42-cfe9-4f8b-9e98-236279fb1c00-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-f5qll\" (UID: \"6fba7b42-cfe9-4f8b-9e98-236279fb1c00\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f5qll"
Sep 10 19:20:00.107235 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:00.107197    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z89kf\" (UniqueName: \"kubernetes.io/projected/6fba7b42-cfe9-4f8b-9e98-236279fb1c00-kube-api-access-z89kf\") pod \"cert-manager-cainjector-559cdd8c87-f5qll\" (UID: \"6fba7b42-cfe9-4f8b-9e98-236279fb1c00\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f5qll"
Sep 10 19:20:00.107235 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:00.107237    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6fba7b42-cfe9-4f8b-9e98-236279fb1c00-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-f5qll\" (UID: \"6fba7b42-cfe9-4f8b-9e98-236279fb1c00\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f5qll"
Sep 10 19:20:00.116054 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:00.115975    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6fba7b42-cfe9-4f8b-9e98-236279fb1c00-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-f5qll\" (UID: \"6fba7b42-cfe9-4f8b-9e98-236279fb1c00\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f5qll"
Sep 10 19:20:00.116475 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:00.116440    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z89kf\" (UniqueName: \"kubernetes.io/projected/6fba7b42-cfe9-4f8b-9e98-236279fb1c00-kube-api-access-z89kf\") pod \"cert-manager-cainjector-559cdd8c87-f5qll\" (UID: \"6fba7b42-cfe9-4f8b-9e98-236279fb1c00\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f5qll"
Sep 10 19:20:00.173479 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:00.173443    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f5qll"
Sep 10 19:20:00.455048 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:00.454985    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-f5qll"]
Sep 10 19:20:00.457784 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:20:00.457728    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6fba7b42_cfe9_4f8b_9e98_236279fb1c00.slice/crio-e702b274fca516f1444417c6292c122e46ebf12a6978936c1d1735c14c0d47cc WatchSource:0}: Error finding container e702b274fca516f1444417c6292c122e46ebf12a6978936c1d1735c14c0d47cc: Status 404 returned error can't find the container with id e702b274fca516f1444417c6292c122e46ebf12a6978936c1d1735c14c0d47cc
Sep 10 19:20:01.431743 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:01.431710    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f5qll" event={"ID":"6fba7b42-cfe9-4f8b-9e98-236279fb1c00","Type":"ContainerStarted","Data":"e702b274fca516f1444417c6292c122e46ebf12a6978936c1d1735c14c0d47cc"}
Sep 10 19:20:02.440103 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:02.440064    2579 generic.go:356] "Generic (PLEG): container finished" podID="462a2a7a-3493-44b0-a4fc-9a261c226ea9" containerID="2f2b66df11394c46aebefc790a19c37fa7c22f419756ec16e5bf5a844bdf0227" exitCode=0
Sep 10 19:20:02.440578 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:02.440146    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmspl" event={"ID":"462a2a7a-3493-44b0-a4fc-9a261c226ea9","Type":"ContainerDied","Data":"2f2b66df11394c46aebefc790a19c37fa7c22f419756ec16e5bf5a844bdf0227"}
Sep 10 19:20:03.446978 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:03.446934    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f5qll" event={"ID":"6fba7b42-cfe9-4f8b-9e98-236279fb1c00","Type":"ContainerStarted","Data":"675334391c775cc60ce851b92668250708b7c1b6d44adf38451cbb8e1eb37987"}
Sep 10 19:20:03.449297 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:03.449269    2579 generic.go:356] "Generic (PLEG): container finished" podID="462a2a7a-3493-44b0-a4fc-9a261c226ea9" containerID="5235b6d2f24189966d67158cf3d5a5f6d5ffe857edbe7f0260dca22aa9202919" exitCode=0
Sep 10 19:20:03.449455 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:03.449347    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmspl" event={"ID":"462a2a7a-3493-44b0-a4fc-9a261c226ea9","Type":"ContainerDied","Data":"5235b6d2f24189966d67158cf3d5a5f6d5ffe857edbe7f0260dca22aa9202919"}
Sep 10 19:20:03.450905 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:03.450885    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-bwrzl" event={"ID":"c29ef078-2088-4f33-994c-de20567e9cdb","Type":"ContainerStarted","Data":"3eaabfbf683acc8499d6d41e01ded68a3851af35e3bb2c286959b106e5bcb565"}
Sep 10 19:20:03.451026 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:03.451013    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-bwrzl"
Sep 10 19:20:03.476876 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:03.476826    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f5qll" podStartSLOduration=1.844251936 podStartE2EDuration="4.476809191s" totalImagesPullingTime="2.632557255s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:19:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:20:03.474934638 +0000 UTC m=+258.694783118" watchObservedRunningTime="2026-09-10 19:20:03.476809191 +0000 UTC m=+258.696657672"
Sep 10 19:20:03.499879 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:03.499825    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-bwrzl" podStartSLOduration=1.45999074 podStartE2EDuration="5.499808949s" totalImagesPullingTime="4.039818209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:19:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:20:03.497938301 +0000 UTC m=+258.717786786" watchObservedRunningTime="2026-09-10 19:20:03.499808949 +0000 UTC m=+258.719657429"
Sep 10 19:20:04.584942 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:04.584919    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmspl"
Sep 10 19:20:04.651871 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:04.651834    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5phj5\" (UniqueName: \"kubernetes.io/projected/462a2a7a-3493-44b0-a4fc-9a261c226ea9-kube-api-access-5phj5\") pod \"462a2a7a-3493-44b0-a4fc-9a261c226ea9\" (UID: \"462a2a7a-3493-44b0-a4fc-9a261c226ea9\") "
Sep 10 19:20:04.652031 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:04.651901    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/462a2a7a-3493-44b0-a4fc-9a261c226ea9-util\") pod \"462a2a7a-3493-44b0-a4fc-9a261c226ea9\" (UID: \"462a2a7a-3493-44b0-a4fc-9a261c226ea9\") "
Sep 10 19:20:04.652031 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:04.651927    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/462a2a7a-3493-44b0-a4fc-9a261c226ea9-bundle\") pod \"462a2a7a-3493-44b0-a4fc-9a261c226ea9\" (UID: \"462a2a7a-3493-44b0-a4fc-9a261c226ea9\") "
Sep 10 19:20:04.652384 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:04.652349    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/462a2a7a-3493-44b0-a4fc-9a261c226ea9-bundle" (OuterVolumeSpecName: "bundle") pod "462a2a7a-3493-44b0-a4fc-9a261c226ea9" (UID: "462a2a7a-3493-44b0-a4fc-9a261c226ea9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:20:04.654446 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:04.654423    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/462a2a7a-3493-44b0-a4fc-9a261c226ea9-kube-api-access-5phj5" (OuterVolumeSpecName: "kube-api-access-5phj5") pod "462a2a7a-3493-44b0-a4fc-9a261c226ea9" (UID: "462a2a7a-3493-44b0-a4fc-9a261c226ea9"). InnerVolumeSpecName "kube-api-access-5phj5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:20:04.657681 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:04.657659    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/462a2a7a-3493-44b0-a4fc-9a261c226ea9-util" (OuterVolumeSpecName: "util") pod "462a2a7a-3493-44b0-a4fc-9a261c226ea9" (UID: "462a2a7a-3493-44b0-a4fc-9a261c226ea9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:20:04.752967 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:04.752871    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/462a2a7a-3493-44b0-a4fc-9a261c226ea9-util\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:20:04.752967 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:04.752912    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/462a2a7a-3493-44b0-a4fc-9a261c226ea9-bundle\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:20:04.752967 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:04.752921    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5phj5\" (UniqueName: \"kubernetes.io/projected/462a2a7a-3493-44b0-a4fc-9a261c226ea9-kube-api-access-5phj5\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:20:05.460425 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:05.460388    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmspl" event={"ID":"462a2a7a-3493-44b0-a4fc-9a261c226ea9","Type":"ContainerDied","Data":"57bf8a8f22cd3a5d85e4c49c5f6dd2df40d23e6bb0135e4a0a7ea86d370b18bf"}
Sep 10 19:20:05.460596 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:05.460442    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="57bf8a8f22cd3a5d85e4c49c5f6dd2df40d23e6bb0135e4a0a7ea86d370b18bf"
Sep 10 19:20:05.460596 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:05.460404    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhmspl"
Sep 10 19:20:09.457578 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:09.457506    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-bwrzl"
Sep 10 19:20:18.077172 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:18.077131    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-hcrh9"]
Sep 10 19:20:18.077678 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:18.077558    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="462a2a7a-3493-44b0-a4fc-9a261c226ea9" containerName="extract"
Sep 10 19:20:18.077678 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:18.077577    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="462a2a7a-3493-44b0-a4fc-9a261c226ea9" containerName="extract"
Sep 10 19:20:18.077678 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:18.077593    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="462a2a7a-3493-44b0-a4fc-9a261c226ea9" containerName="pull"
Sep 10 19:20:18.077678 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:18.077601    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="462a2a7a-3493-44b0-a4fc-9a261c226ea9" containerName="pull"
Sep 10 19:20:18.077678 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:18.077620    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="462a2a7a-3493-44b0-a4fc-9a261c226ea9" containerName="util"
Sep 10 19:20:18.077678 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:18.077628    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="462a2a7a-3493-44b0-a4fc-9a261c226ea9" containerName="util"
Sep 10 19:20:18.078018 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:18.077694    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="462a2a7a-3493-44b0-a4fc-9a261c226ea9" containerName="extract"
Sep 10 19:20:18.082636 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:18.082614    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-hcrh9"
Sep 10 19:20:18.084846 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:18.084822    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-ks7d2\""
Sep 10 19:20:18.087508 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:18.087485    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-hcrh9"]
Sep 10 19:20:18.159846 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:18.159805    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5b5vg\" (UniqueName: \"kubernetes.io/projected/4a485742-4eb9-4e77-a757-af361d1eb86e-kube-api-access-5b5vg\") pod \"cert-manager-55b4d954d5-hcrh9\" (UID: \"4a485742-4eb9-4e77-a757-af361d1eb86e\") " pod="cert-manager/cert-manager-55b4d954d5-hcrh9"
Sep 10 19:20:18.160018 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:18.159852    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4a485742-4eb9-4e77-a757-af361d1eb86e-bound-sa-token\") pod \"cert-manager-55b4d954d5-hcrh9\" (UID: \"4a485742-4eb9-4e77-a757-af361d1eb86e\") " pod="cert-manager/cert-manager-55b4d954d5-hcrh9"
Sep 10 19:20:18.260859 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:18.260817    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5b5vg\" (UniqueName: \"kubernetes.io/projected/4a485742-4eb9-4e77-a757-af361d1eb86e-kube-api-access-5b5vg\") pod \"cert-manager-55b4d954d5-hcrh9\" (UID: \"4a485742-4eb9-4e77-a757-af361d1eb86e\") " pod="cert-manager/cert-manager-55b4d954d5-hcrh9"
Sep 10 19:20:18.260859 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:18.260867    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4a485742-4eb9-4e77-a757-af361d1eb86e-bound-sa-token\") pod \"cert-manager-55b4d954d5-hcrh9\" (UID: \"4a485742-4eb9-4e77-a757-af361d1eb86e\") " pod="cert-manager/cert-manager-55b4d954d5-hcrh9"
Sep 10 19:20:18.268597 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:18.268560    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4a485742-4eb9-4e77-a757-af361d1eb86e-bound-sa-token\") pod \"cert-manager-55b4d954d5-hcrh9\" (UID: \"4a485742-4eb9-4e77-a757-af361d1eb86e\") " pod="cert-manager/cert-manager-55b4d954d5-hcrh9"
Sep 10 19:20:18.268734 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:18.268657    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5b5vg\" (UniqueName: \"kubernetes.io/projected/4a485742-4eb9-4e77-a757-af361d1eb86e-kube-api-access-5b5vg\") pod \"cert-manager-55b4d954d5-hcrh9\" (UID: \"4a485742-4eb9-4e77-a757-af361d1eb86e\") " pod="cert-manager/cert-manager-55b4d954d5-hcrh9"
Sep 10 19:20:18.393665 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:18.393635    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-hcrh9"
Sep 10 19:20:18.668327 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:18.668254    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-hcrh9"]
Sep 10 19:20:18.673830 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:20:18.673801    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4a485742_4eb9_4e77_a757_af361d1eb86e.slice/crio-aa0beccc98aa7de86a1deb6df89df2cc0c736ccc278c6d6a29aaa8c8eed71b9c WatchSource:0}: Error finding container aa0beccc98aa7de86a1deb6df89df2cc0c736ccc278c6d6a29aaa8c8eed71b9c: Status 404 returned error can't find the container with id aa0beccc98aa7de86a1deb6df89df2cc0c736ccc278c6d6a29aaa8c8eed71b9c
Sep 10 19:20:19.516866 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:19.516828    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-hcrh9" event={"ID":"4a485742-4eb9-4e77-a757-af361d1eb86e","Type":"ContainerStarted","Data":"a1364ef81c94e3af545f46df9311a94be6afe859ecea8c1a778d3eaa5fcbd6d6"}
Sep 10 19:20:19.516866 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:19.516871    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-hcrh9" event={"ID":"4a485742-4eb9-4e77-a757-af361d1eb86e","Type":"ContainerStarted","Data":"aa0beccc98aa7de86a1deb6df89df2cc0c736ccc278c6d6a29aaa8c8eed71b9c"}
Sep 10 19:20:19.533815 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:19.533737    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-hcrh9" podStartSLOduration=1.5337230750000002 podStartE2EDuration="1.533723075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:20:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:20:19.532366901 +0000 UTC m=+274.752215381" watchObservedRunningTime="2026-09-10 19:20:19.533723075 +0000 UTC m=+274.753571557"
Sep 10 19:20:22.281168 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:22.281131    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnqqxd"]
Sep 10 19:20:22.284393 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:22.284374    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnqqxd"
Sep 10 19:20:22.286473 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:22.286454    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 19:20:22.286473 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:22.286462    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 19:20:22.287134 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:22.287113    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ccnqb\""
Sep 10 19:20:22.293430 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:22.293409    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnqqxd"]
Sep 10 19:20:22.392609 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:22.392575    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ac648067-efca-4d85-8287-6fb6c548ea14-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnqqxd\" (UID: \"ac648067-efca-4d85-8287-6fb6c548ea14\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnqqxd"
Sep 10 19:20:22.392787 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:22.392658    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68tgx\" (UniqueName: \"kubernetes.io/projected/ac648067-efca-4d85-8287-6fb6c548ea14-kube-api-access-68tgx\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnqqxd\" (UID: \"ac648067-efca-4d85-8287-6fb6c548ea14\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnqqxd"
Sep 10 19:20:22.392787 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:22.392680    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ac648067-efca-4d85-8287-6fb6c548ea14-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnqqxd\" (UID: \"ac648067-efca-4d85-8287-6fb6c548ea14\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnqqxd"
Sep 10 19:20:22.493741 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:22.493705    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68tgx\" (UniqueName: \"kubernetes.io/projected/ac648067-efca-4d85-8287-6fb6c548ea14-kube-api-access-68tgx\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnqqxd\" (UID: \"ac648067-efca-4d85-8287-6fb6c548ea14\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnqqxd"
Sep 10 19:20:22.493872 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:22.493745    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ac648067-efca-4d85-8287-6fb6c548ea14-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnqqxd\" (UID: \"ac648067-efca-4d85-8287-6fb6c548ea14\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnqqxd"
Sep 10 19:20:22.493872 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:22.493793    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ac648067-efca-4d85-8287-6fb6c548ea14-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnqqxd\" (UID: \"ac648067-efca-4d85-8287-6fb6c548ea14\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnqqxd"
Sep 10 19:20:22.494120 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:22.494105    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ac648067-efca-4d85-8287-6fb6c548ea14-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnqqxd\" (UID: \"ac648067-efca-4d85-8287-6fb6c548ea14\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnqqxd"
Sep 10 19:20:22.494160 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:22.494137    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ac648067-efca-4d85-8287-6fb6c548ea14-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnqqxd\" (UID: \"ac648067-efca-4d85-8287-6fb6c548ea14\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnqqxd"
Sep 10 19:20:22.503321 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:22.503294    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68tgx\" (UniqueName: \"kubernetes.io/projected/ac648067-efca-4d85-8287-6fb6c548ea14-kube-api-access-68tgx\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnqqxd\" (UID: \"ac648067-efca-4d85-8287-6fb6c548ea14\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnqqxd"
Sep 10 19:20:22.594236 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:22.594142    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnqqxd"
Sep 10 19:20:22.866367 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:22.866338    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnqqxd"]
Sep 10 19:20:22.868514 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:20:22.868490    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac648067_efca_4d85_8287_6fb6c548ea14.slice/crio-90bfdf01628a0497b57cd24c3257979fc04260dfbd3b43b96cd29107352f22e6 WatchSource:0}: Error finding container 90bfdf01628a0497b57cd24c3257979fc04260dfbd3b43b96cd29107352f22e6: Status 404 returned error can't find the container with id 90bfdf01628a0497b57cd24c3257979fc04260dfbd3b43b96cd29107352f22e6
Sep 10 19:20:23.537705 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:23.537669    2579 generic.go:356] "Generic (PLEG): container finished" podID="ac648067-efca-4d85-8287-6fb6c548ea14" containerID="9e7af6553b8c8b8067a955f5a95a7e46298008a92b00c7b54fd0724a93412fb6" exitCode=0
Sep 10 19:20:23.538191 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:23.537728    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnqqxd" event={"ID":"ac648067-efca-4d85-8287-6fb6c548ea14","Type":"ContainerDied","Data":"9e7af6553b8c8b8067a955f5a95a7e46298008a92b00c7b54fd0724a93412fb6"}
Sep 10 19:20:23.538191 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:23.537808    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnqqxd" event={"ID":"ac648067-efca-4d85-8287-6fb6c548ea14","Type":"ContainerStarted","Data":"90bfdf01628a0497b57cd24c3257979fc04260dfbd3b43b96cd29107352f22e6"}
Sep 10 19:20:24.544052 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:24.544016    2579 generic.go:356] "Generic (PLEG): container finished" podID="ac648067-efca-4d85-8287-6fb6c548ea14" containerID="dd53d740eea40f51d5b66fe293c64c44fe74b08dadf3a98a8d88d3cf40ae73b9" exitCode=0
Sep 10 19:20:24.544436 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:24.544070    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnqqxd" event={"ID":"ac648067-efca-4d85-8287-6fb6c548ea14","Type":"ContainerDied","Data":"dd53d740eea40f51d5b66fe293c64c44fe74b08dadf3a98a8d88d3cf40ae73b9"}
Sep 10 19:20:25.550876 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:25.550843    2579 generic.go:356] "Generic (PLEG): container finished" podID="ac648067-efca-4d85-8287-6fb6c548ea14" containerID="285aa23d4de8787be6816ed1c87adcece68c0d5123ded39aca55d63461228057" exitCode=0
Sep 10 19:20:25.551366 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:25.550927    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnqqxd" event={"ID":"ac648067-efca-4d85-8287-6fb6c548ea14","Type":"ContainerDied","Data":"285aa23d4de8787be6816ed1c87adcece68c0d5123ded39aca55d63461228057"}
Sep 10 19:20:26.680292 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:26.680269    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnqqxd"
Sep 10 19:20:26.730374 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:26.730338    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ac648067-efca-4d85-8287-6fb6c548ea14-bundle\") pod \"ac648067-efca-4d85-8287-6fb6c548ea14\" (UID: \"ac648067-efca-4d85-8287-6fb6c548ea14\") "
Sep 10 19:20:26.730537 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:26.730447    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ac648067-efca-4d85-8287-6fb6c548ea14-util\") pod \"ac648067-efca-4d85-8287-6fb6c548ea14\" (UID: \"ac648067-efca-4d85-8287-6fb6c548ea14\") "
Sep 10 19:20:26.730537 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:26.730488    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68tgx\" (UniqueName: \"kubernetes.io/projected/ac648067-efca-4d85-8287-6fb6c548ea14-kube-api-access-68tgx\") pod \"ac648067-efca-4d85-8287-6fb6c548ea14\" (UID: \"ac648067-efca-4d85-8287-6fb6c548ea14\") "
Sep 10 19:20:26.731162 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:26.731135    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac648067-efca-4d85-8287-6fb6c548ea14-bundle" (OuterVolumeSpecName: "bundle") pod "ac648067-efca-4d85-8287-6fb6c548ea14" (UID: "ac648067-efca-4d85-8287-6fb6c548ea14"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:20:26.733035 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:26.733001    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac648067-efca-4d85-8287-6fb6c548ea14-kube-api-access-68tgx" (OuterVolumeSpecName: "kube-api-access-68tgx") pod "ac648067-efca-4d85-8287-6fb6c548ea14" (UID: "ac648067-efca-4d85-8287-6fb6c548ea14"). InnerVolumeSpecName "kube-api-access-68tgx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:20:26.736261 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:26.736241    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac648067-efca-4d85-8287-6fb6c548ea14-util" (OuterVolumeSpecName: "util") pod "ac648067-efca-4d85-8287-6fb6c548ea14" (UID: "ac648067-efca-4d85-8287-6fb6c548ea14"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:20:26.831302 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:26.831197    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ac648067-efca-4d85-8287-6fb6c548ea14-bundle\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:20:26.831302 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:26.831240    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ac648067-efca-4d85-8287-6fb6c548ea14-util\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:20:26.831302 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:26.831250    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68tgx\" (UniqueName: \"kubernetes.io/projected/ac648067-efca-4d85-8287-6fb6c548ea14-kube-api-access-68tgx\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:20:27.562991 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:27.562904    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnqqxd"
Sep 10 19:20:27.563134 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:27.562907    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnqqxd" event={"ID":"ac648067-efca-4d85-8287-6fb6c548ea14","Type":"ContainerDied","Data":"90bfdf01628a0497b57cd24c3257979fc04260dfbd3b43b96cd29107352f22e6"}
Sep 10 19:20:27.563134 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:27.563031    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="90bfdf01628a0497b57cd24c3257979fc04260dfbd3b43b96cd29107352f22e6"
Sep 10 19:20:32.690626 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:32.690587    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z4s9f"]
Sep 10 19:20:32.691043 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:32.690899    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac648067-efca-4d85-8287-6fb6c548ea14" containerName="extract"
Sep 10 19:20:32.691043 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:32.690910    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac648067-efca-4d85-8287-6fb6c548ea14" containerName="extract"
Sep 10 19:20:32.691043 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:32.690918    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac648067-efca-4d85-8287-6fb6c548ea14" containerName="util"
Sep 10 19:20:32.691043 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:32.690924    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac648067-efca-4d85-8287-6fb6c548ea14" containerName="util"
Sep 10 19:20:32.691043 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:32.690931    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac648067-efca-4d85-8287-6fb6c548ea14" containerName="pull"
Sep 10 19:20:32.691043 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:32.690937    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac648067-efca-4d85-8287-6fb6c548ea14" containerName="pull"
Sep 10 19:20:32.691043 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:32.690989    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac648067-efca-4d85-8287-6fb6c548ea14" containerName="extract"
Sep 10 19:20:32.694255 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:32.694239    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z4s9f"
Sep 10 19:20:32.696309 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:32.696283    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 19:20:32.696487 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:32.696429    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 19:20:32.696943 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:32.696922    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ccnqb\""
Sep 10 19:20:32.707954 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:32.707933    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z4s9f"]
Sep 10 19:20:32.781489 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:32.781452    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/daa200da-6daa-49ad-addd-2907c0455a4b-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z4s9f\" (UID: \"daa200da-6daa-49ad-addd-2907c0455a4b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z4s9f"
Sep 10 19:20:32.781682 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:32.781499    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jg65c\" (UniqueName: \"kubernetes.io/projected/daa200da-6daa-49ad-addd-2907c0455a4b-kube-api-access-jg65c\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z4s9f\" (UID: \"daa200da-6daa-49ad-addd-2907c0455a4b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z4s9f"
Sep 10 19:20:32.781682 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:32.781568    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/daa200da-6daa-49ad-addd-2907c0455a4b-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z4s9f\" (UID: \"daa200da-6daa-49ad-addd-2907c0455a4b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z4s9f"
Sep 10 19:20:32.882763 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:32.882711    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jg65c\" (UniqueName: \"kubernetes.io/projected/daa200da-6daa-49ad-addd-2907c0455a4b-kube-api-access-jg65c\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z4s9f\" (UID: \"daa200da-6daa-49ad-addd-2907c0455a4b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z4s9f"
Sep 10 19:20:32.882936 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:32.882830    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/daa200da-6daa-49ad-addd-2907c0455a4b-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z4s9f\" (UID: \"daa200da-6daa-49ad-addd-2907c0455a4b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z4s9f"
Sep 10 19:20:32.882936 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:32.882871    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/daa200da-6daa-49ad-addd-2907c0455a4b-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z4s9f\" (UID: \"daa200da-6daa-49ad-addd-2907c0455a4b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z4s9f"
Sep 10 19:20:32.883289 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:32.883266    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/daa200da-6daa-49ad-addd-2907c0455a4b-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z4s9f\" (UID: \"daa200da-6daa-49ad-addd-2907c0455a4b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z4s9f"
Sep 10 19:20:32.883350 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:32.883278    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/daa200da-6daa-49ad-addd-2907c0455a4b-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z4s9f\" (UID: \"daa200da-6daa-49ad-addd-2907c0455a4b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z4s9f"
Sep 10 19:20:32.893394 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:32.893367    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jg65c\" (UniqueName: \"kubernetes.io/projected/daa200da-6daa-49ad-addd-2907c0455a4b-kube-api-access-jg65c\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z4s9f\" (UID: \"daa200da-6daa-49ad-addd-2907c0455a4b\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z4s9f"
Sep 10 19:20:33.003261 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:33.003171    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z4s9f"
Sep 10 19:20:33.299547 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:33.299383    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z4s9f"]
Sep 10 19:20:33.302521 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:20:33.302494    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaa200da_6daa_49ad_addd_2907c0455a4b.slice/crio-08255c6208897f292c6083a228c18d3cb3f4600e07cc2bfb8a45e9fc5c1eb7f0 WatchSource:0}: Error finding container 08255c6208897f292c6083a228c18d3cb3f4600e07cc2bfb8a45e9fc5c1eb7f0: Status 404 returned error can't find the container with id 08255c6208897f292c6083a228c18d3cb3f4600e07cc2bfb8a45e9fc5c1eb7f0
Sep 10 19:20:33.590433 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:33.590347    2579 generic.go:356] "Generic (PLEG): container finished" podID="daa200da-6daa-49ad-addd-2907c0455a4b" containerID="c75f7765d32f9a2555ec17e8bbdafb17e12ac2f7422255e5c04a1aa508e9d830" exitCode=0
Sep 10 19:20:33.590588 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:33.590438    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z4s9f" event={"ID":"daa200da-6daa-49ad-addd-2907c0455a4b","Type":"ContainerDied","Data":"c75f7765d32f9a2555ec17e8bbdafb17e12ac2f7422255e5c04a1aa508e9d830"}
Sep 10 19:20:33.590588 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:33.590491    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z4s9f" event={"ID":"daa200da-6daa-49ad-addd-2907c0455a4b","Type":"ContainerStarted","Data":"08255c6208897f292c6083a228c18d3cb3f4600e07cc2bfb8a45e9fc5c1eb7f0"}
Sep 10 19:20:34.601944 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:34.601849    2579 generic.go:356] "Generic (PLEG): container finished" podID="daa200da-6daa-49ad-addd-2907c0455a4b" containerID="c8661d4c0d93a309e73bc92eb94616e235b745e23bed1c410170ccd4055bc659" exitCode=0
Sep 10 19:20:34.601944 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:34.601929    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z4s9f" event={"ID":"daa200da-6daa-49ad-addd-2907c0455a4b","Type":"ContainerDied","Data":"c8661d4c0d93a309e73bc92eb94616e235b745e23bed1c410170ccd4055bc659"}
Sep 10 19:20:35.022337 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:35.022296    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d477566c-8rcd6"]
Sep 10 19:20:35.025636 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:35.025614    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-8rcd6"
Sep 10 19:20:35.028421 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:35.028401    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 10 19:20:35.028551 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:35.028441    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 10 19:20:35.028785 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:35.028770    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4qpwt\""
Sep 10 19:20:35.029084 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:35.029068    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 10 19:20:35.029348 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:35.029333    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 10 19:20:35.047382 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:35.047342    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d477566c-8rcd6"]
Sep 10 19:20:35.101450 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:35.101409    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c10ecbe-e921-4f4b-b077-0bf836adf8b5-tmp\") pod \"opendatahub-operator-controller-manager-5d477566c-8rcd6\" (UID: \"8c10ecbe-e921-4f4b-b077-0bf836adf8b5\") " pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-8rcd6"
Sep 10 19:20:35.101640 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:35.101470    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8c10ecbe-e921-4f4b-b077-0bf836adf8b5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d477566c-8rcd6\" (UID: \"8c10ecbe-e921-4f4b-b077-0bf836adf8b5\") " pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-8rcd6"
Sep 10 19:20:35.101640 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:35.101540    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8c10ecbe-e921-4f4b-b077-0bf836adf8b5-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d477566c-8rcd6\" (UID: \"8c10ecbe-e921-4f4b-b077-0bf836adf8b5\") " pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-8rcd6"
Sep 10 19:20:35.101640 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:35.101576    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wlzf\" (UniqueName: \"kubernetes.io/projected/8c10ecbe-e921-4f4b-b077-0bf836adf8b5-kube-api-access-5wlzf\") pod \"opendatahub-operator-controller-manager-5d477566c-8rcd6\" (UID: \"8c10ecbe-e921-4f4b-b077-0bf836adf8b5\") " pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-8rcd6"
Sep 10 19:20:35.101640 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:35.101609    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8c10ecbe-e921-4f4b-b077-0bf836adf8b5-manifests\") pod \"opendatahub-operator-controller-manager-5d477566c-8rcd6\" (UID: \"8c10ecbe-e921-4f4b-b077-0bf836adf8b5\") " pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-8rcd6"
Sep 10 19:20:35.203041 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:35.202997    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8c10ecbe-e921-4f4b-b077-0bf836adf8b5-manifests\") pod \"opendatahub-operator-controller-manager-5d477566c-8rcd6\" (UID: \"8c10ecbe-e921-4f4b-b077-0bf836adf8b5\") " pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-8rcd6"
Sep 10 19:20:35.203242 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:35.203065    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c10ecbe-e921-4f4b-b077-0bf836adf8b5-tmp\") pod \"opendatahub-operator-controller-manager-5d477566c-8rcd6\" (UID: \"8c10ecbe-e921-4f4b-b077-0bf836adf8b5\") " pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-8rcd6"
Sep 10 19:20:35.203242 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:35.203111    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8c10ecbe-e921-4f4b-b077-0bf836adf8b5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d477566c-8rcd6\" (UID: \"8c10ecbe-e921-4f4b-b077-0bf836adf8b5\") " pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-8rcd6"
Sep 10 19:20:35.203242 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:35.203144    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8c10ecbe-e921-4f4b-b077-0bf836adf8b5-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d477566c-8rcd6\" (UID: \"8c10ecbe-e921-4f4b-b077-0bf836adf8b5\") " pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-8rcd6"
Sep 10 19:20:35.203413 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:35.203276    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wlzf\" (UniqueName: \"kubernetes.io/projected/8c10ecbe-e921-4f4b-b077-0bf836adf8b5-kube-api-access-5wlzf\") pod \"opendatahub-operator-controller-manager-5d477566c-8rcd6\" (UID: \"8c10ecbe-e921-4f4b-b077-0bf836adf8b5\") " pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-8rcd6"
Sep 10 19:20:35.203469 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:35.203423    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8c10ecbe-e921-4f4b-b077-0bf836adf8b5-manifests\") pod \"opendatahub-operator-controller-manager-5d477566c-8rcd6\" (UID: \"8c10ecbe-e921-4f4b-b077-0bf836adf8b5\") " pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-8rcd6"
Sep 10 19:20:35.205591 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:35.205565    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c10ecbe-e921-4f4b-b077-0bf836adf8b5-tmp\") pod \"opendatahub-operator-controller-manager-5d477566c-8rcd6\" (UID: \"8c10ecbe-e921-4f4b-b077-0bf836adf8b5\") " pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-8rcd6"
Sep 10 19:20:35.205708 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:35.205657    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8c10ecbe-e921-4f4b-b077-0bf836adf8b5-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d477566c-8rcd6\" (UID: \"8c10ecbe-e921-4f4b-b077-0bf836adf8b5\") " pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-8rcd6"
Sep 10 19:20:35.205814 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:35.205766    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8c10ecbe-e921-4f4b-b077-0bf836adf8b5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d477566c-8rcd6\" (UID: \"8c10ecbe-e921-4f4b-b077-0bf836adf8b5\") " pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-8rcd6"
Sep 10 19:20:35.218103 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:35.218079    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wlzf\" (UniqueName: \"kubernetes.io/projected/8c10ecbe-e921-4f4b-b077-0bf836adf8b5-kube-api-access-5wlzf\") pod \"opendatahub-operator-controller-manager-5d477566c-8rcd6\" (UID: \"8c10ecbe-e921-4f4b-b077-0bf836adf8b5\") " pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-8rcd6"
Sep 10 19:20:35.336658 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:35.336575    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-8rcd6"
Sep 10 19:20:35.609164 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:35.609076    2579 generic.go:356] "Generic (PLEG): container finished" podID="daa200da-6daa-49ad-addd-2907c0455a4b" containerID="f234adaa833b2620b81af66ea2a64f2a276edd85c74254cf151deaaad5b2f922" exitCode=0
Sep 10 19:20:35.609164 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:35.609140    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z4s9f" event={"ID":"daa200da-6daa-49ad-addd-2907c0455a4b","Type":"ContainerDied","Data":"f234adaa833b2620b81af66ea2a64f2a276edd85c74254cf151deaaad5b2f922"}
Sep 10 19:20:35.615775 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:35.615682    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d477566c-8rcd6"]
Sep 10 19:20:35.618899 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:20:35.618875    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c10ecbe_e921_4f4b_b077_0bf836adf8b5.slice/crio-7ea40236ba9599d570c31610891ebdd7631fd1a6776541a2138b552ad39d3852 WatchSource:0}: Error finding container 7ea40236ba9599d570c31610891ebdd7631fd1a6776541a2138b552ad39d3852: Status 404 returned error can't find the container with id 7ea40236ba9599d570c31610891ebdd7631fd1a6776541a2138b552ad39d3852
Sep 10 19:20:36.616616 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:36.616570    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-8rcd6" event={"ID":"8c10ecbe-e921-4f4b-b077-0bf836adf8b5","Type":"ContainerStarted","Data":"7ea40236ba9599d570c31610891ebdd7631fd1a6776541a2138b552ad39d3852"}
Sep 10 19:20:36.772918 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:36.772853    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z4s9f"
Sep 10 19:20:36.918665 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:36.918270    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jg65c\" (UniqueName: \"kubernetes.io/projected/daa200da-6daa-49ad-addd-2907c0455a4b-kube-api-access-jg65c\") pod \"daa200da-6daa-49ad-addd-2907c0455a4b\" (UID: \"daa200da-6daa-49ad-addd-2907c0455a4b\") "
Sep 10 19:20:36.918665 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:36.918377    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/daa200da-6daa-49ad-addd-2907c0455a4b-bundle\") pod \"daa200da-6daa-49ad-addd-2907c0455a4b\" (UID: \"daa200da-6daa-49ad-addd-2907c0455a4b\") "
Sep 10 19:20:36.918665 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:36.918431    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/daa200da-6daa-49ad-addd-2907c0455a4b-util\") pod \"daa200da-6daa-49ad-addd-2907c0455a4b\" (UID: \"daa200da-6daa-49ad-addd-2907c0455a4b\") "
Sep 10 19:20:36.919568 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:36.919522    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daa200da-6daa-49ad-addd-2907c0455a4b-bundle" (OuterVolumeSpecName: "bundle") pod "daa200da-6daa-49ad-addd-2907c0455a4b" (UID: "daa200da-6daa-49ad-addd-2907c0455a4b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:20:36.921068 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:36.921024    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daa200da-6daa-49ad-addd-2907c0455a4b-kube-api-access-jg65c" (OuterVolumeSpecName: "kube-api-access-jg65c") pod "daa200da-6daa-49ad-addd-2907c0455a4b" (UID: "daa200da-6daa-49ad-addd-2907c0455a4b"). InnerVolumeSpecName "kube-api-access-jg65c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:20:36.928509 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:36.928463    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daa200da-6daa-49ad-addd-2907c0455a4b-util" (OuterVolumeSpecName: "util") pod "daa200da-6daa-49ad-addd-2907c0455a4b" (UID: "daa200da-6daa-49ad-addd-2907c0455a4b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:20:37.019417 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:37.019381    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/daa200da-6daa-49ad-addd-2907c0455a4b-bundle\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:20:37.019417 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:37.019417    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/daa200da-6daa-49ad-addd-2907c0455a4b-util\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:20:37.019417 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:37.019431    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jg65c\" (UniqueName: \"kubernetes.io/projected/daa200da-6daa-49ad-addd-2907c0455a4b-kube-api-access-jg65c\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:20:37.623495 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:37.623411    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z4s9f" event={"ID":"daa200da-6daa-49ad-addd-2907c0455a4b","Type":"ContainerDied","Data":"08255c6208897f292c6083a228c18d3cb3f4600e07cc2bfb8a45e9fc5c1eb7f0"}
Sep 10 19:20:37.623495 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:37.623468    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="08255c6208897f292c6083a228c18d3cb3f4600e07cc2bfb8a45e9fc5c1eb7f0"
Sep 10 19:20:37.623495 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:37.623473    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z4s9f"
Sep 10 19:20:39.634976 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:39.634941    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-8rcd6" event={"ID":"8c10ecbe-e921-4f4b-b077-0bf836adf8b5","Type":"ContainerStarted","Data":"b722e052d129c0a4846536c1c55721fe3872269db4ea79f299f9e28fc13f2851"}
Sep 10 19:20:40.641368 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:40.641333    2579 generic.go:356] "Generic (PLEG): container finished" podID="8c10ecbe-e921-4f4b-b077-0bf836adf8b5" containerID="b722e052d129c0a4846536c1c55721fe3872269db4ea79f299f9e28fc13f2851" exitCode=0
Sep 10 19:20:40.641743 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:40.641430    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-8rcd6" event={"ID":"8c10ecbe-e921-4f4b-b077-0bf836adf8b5","Type":"ContainerDied","Data":"b722e052d129c0a4846536c1c55721fe3872269db4ea79f299f9e28fc13f2851"}
Sep 10 19:20:41.648474 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:41.648429    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-8rcd6" event={"ID":"8c10ecbe-e921-4f4b-b077-0bf836adf8b5","Type":"ContainerStarted","Data":"315f8add7391a1b6eeefe4c410a7f8419b344e0ba5ec86a43a74b39b84363aac"}
Sep 10 19:20:41.648859 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:41.648591    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-8rcd6"
Sep 10 19:20:41.671326 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:41.671265    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-8rcd6" podStartSLOduration=3.077401616 podStartE2EDuration="7.671244048s" totalImagesPullingTime="3.564721885s" totalInitContainerRuntime="1.029120547s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:20:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:20:41.669110583 +0000 UTC m=+296.888959056" watchObservedRunningTime="2026-09-10 19:20:41.671244048 +0000 UTC m=+296.891092530"
Sep 10 19:20:45.108947 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:45.108913    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bmdg9_9817270c-8629-44c4-ab8f-f3982e1f0e2f/console-operator/1.log"
Sep 10 19:20:45.110351 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:45.110325    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bmdg9_9817270c-8629-44c4-ab8f-f3982e1f0e2f/console-operator/1.log"
Sep 10 19:20:45.114547 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:45.114525    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8r66s_2826f8ec-78f6-4cab-879e-47a21854ed5a/ovn-acl-logging/0.log"
Sep 10 19:20:45.115864 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:45.115847    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8r66s_2826f8ec-78f6-4cab-879e-47a21854ed5a/ovn-acl-logging/0.log"
Sep 10 19:20:45.123303 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:45.123288    2579 kubelet.go:1648] "Image garbage collection succeeded"
Sep 10 19:20:52.655310 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:20:52.655280    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5d477566c-8rcd6"
Sep 10 19:21:05.139858 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.139812    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5dd7bdc797-jljr6"]
Sep 10 19:21:05.140464 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.140427    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="daa200da-6daa-49ad-addd-2907c0455a4b" containerName="util"
Sep 10 19:21:05.140464 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.140445    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="daa200da-6daa-49ad-addd-2907c0455a4b" containerName="util"
Sep 10 19:21:05.140584 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.140493    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="daa200da-6daa-49ad-addd-2907c0455a4b" containerName="pull"
Sep 10 19:21:05.140584 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.140501    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="daa200da-6daa-49ad-addd-2907c0455a4b" containerName="pull"
Sep 10 19:21:05.140584 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.140524    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="daa200da-6daa-49ad-addd-2907c0455a4b" containerName="extract"
Sep 10 19:21:05.140584 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.140532    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="daa200da-6daa-49ad-addd-2907c0455a4b" containerName="extract"
Sep 10 19:21:05.140799 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.140665    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="daa200da-6daa-49ad-addd-2907c0455a4b" containerName="extract"
Sep 10 19:21:05.150622 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.150543    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5dd7bdc797-jljr6"]
Sep 10 19:21:05.150801 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.150652    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5dd7bdc797-jljr6"
Sep 10 19:21:05.152725 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.152696    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 10 19:21:05.153295 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.153266    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 10 19:21:05.153413 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.153321    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 10 19:21:05.153413 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.153342    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-bqx62\""
Sep 10 19:21:05.153532 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.153283    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 10 19:21:05.260987 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.260631    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bl7nn\" (UniqueName: \"kubernetes.io/projected/896dee6e-5770-4d1a-9637-e2573b9ebc16-kube-api-access-bl7nn\") pod \"kube-auth-proxy-5dd7bdc797-jljr6\" (UID: \"896dee6e-5770-4d1a-9637-e2573b9ebc16\") " pod="openshift-ingress/kube-auth-proxy-5dd7bdc797-jljr6"
Sep 10 19:21:05.260987 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.260674    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/896dee6e-5770-4d1a-9637-e2573b9ebc16-tls-certs\") pod \"kube-auth-proxy-5dd7bdc797-jljr6\" (UID: \"896dee6e-5770-4d1a-9637-e2573b9ebc16\") " pod="openshift-ingress/kube-auth-proxy-5dd7bdc797-jljr6"
Sep 10 19:21:05.260987 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.260867    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/896dee6e-5770-4d1a-9637-e2573b9ebc16-tmp\") pod \"kube-auth-proxy-5dd7bdc797-jljr6\" (UID: \"896dee6e-5770-4d1a-9637-e2573b9ebc16\") " pod="openshift-ingress/kube-auth-proxy-5dd7bdc797-jljr6"
Sep 10 19:21:05.263107 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.263081    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2gt8j"]
Sep 10 19:21:05.266365 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.266347    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-2gt8j"
Sep 10 19:21:05.268862 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.268829    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-f9lng\""
Sep 10 19:21:05.268997 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.268971    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 10 19:21:05.275137 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.275113    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2gt8j"]
Sep 10 19:21:05.361929 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.361895    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/896dee6e-5770-4d1a-9637-e2573b9ebc16-tmp\") pod \"kube-auth-proxy-5dd7bdc797-jljr6\" (UID: \"896dee6e-5770-4d1a-9637-e2573b9ebc16\") " pod="openshift-ingress/kube-auth-proxy-5dd7bdc797-jljr6"
Sep 10 19:21:05.362097 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.361947    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bl7nn\" (UniqueName: \"kubernetes.io/projected/896dee6e-5770-4d1a-9637-e2573b9ebc16-kube-api-access-bl7nn\") pod \"kube-auth-proxy-5dd7bdc797-jljr6\" (UID: \"896dee6e-5770-4d1a-9637-e2573b9ebc16\") " pod="openshift-ingress/kube-auth-proxy-5dd7bdc797-jljr6"
Sep 10 19:21:05.362097 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.362065    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/896dee6e-5770-4d1a-9637-e2573b9ebc16-tls-certs\") pod \"kube-auth-proxy-5dd7bdc797-jljr6\" (UID: \"896dee6e-5770-4d1a-9637-e2573b9ebc16\") " pod="openshift-ingress/kube-auth-proxy-5dd7bdc797-jljr6"
Sep 10 19:21:05.362209 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:21:05.362191    2579 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 10 19:21:05.362276 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:21:05.362265    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/896dee6e-5770-4d1a-9637-e2573b9ebc16-tls-certs podName:896dee6e-5770-4d1a-9637-e2573b9ebc16 nodeName:}" failed. No retries permitted until 2026-09-10 19:21:05.862248646 +0000 UTC m=+321.082097104 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/896dee6e-5770-4d1a-9637-e2573b9ebc16-tls-certs") pod "kube-auth-proxy-5dd7bdc797-jljr6" (UID: "896dee6e-5770-4d1a-9637-e2573b9ebc16") : secret "kube-auth-proxy-tls" not found
Sep 10 19:21:05.364242 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.364215    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/896dee6e-5770-4d1a-9637-e2573b9ebc16-tmp\") pod \"kube-auth-proxy-5dd7bdc797-jljr6\" (UID: \"896dee6e-5770-4d1a-9637-e2573b9ebc16\") " pod="openshift-ingress/kube-auth-proxy-5dd7bdc797-jljr6"
Sep 10 19:21:05.373122 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.373105    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl7nn\" (UniqueName: \"kubernetes.io/projected/896dee6e-5770-4d1a-9637-e2573b9ebc16-kube-api-access-bl7nn\") pod \"kube-auth-proxy-5dd7bdc797-jljr6\" (UID: \"896dee6e-5770-4d1a-9637-e2573b9ebc16\") " pod="openshift-ingress/kube-auth-proxy-5dd7bdc797-jljr6"
Sep 10 19:21:05.462922 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.462827    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4cc26bdf-0a6c-42a8-a000-a97fdd40b51d-cert\") pod \"odh-model-controller-7998c67dc4-2gt8j\" (UID: \"4cc26bdf-0a6c-42a8-a000-a97fdd40b51d\") " pod="opendatahub/odh-model-controller-7998c67dc4-2gt8j"
Sep 10 19:21:05.462922 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.462878    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2vtn\" (UniqueName: \"kubernetes.io/projected/4cc26bdf-0a6c-42a8-a000-a97fdd40b51d-kube-api-access-d2vtn\") pod \"odh-model-controller-7998c67dc4-2gt8j\" (UID: \"4cc26bdf-0a6c-42a8-a000-a97fdd40b51d\") " pod="opendatahub/odh-model-controller-7998c67dc4-2gt8j"
Sep 10 19:21:05.564380 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.564342    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4cc26bdf-0a6c-42a8-a000-a97fdd40b51d-cert\") pod \"odh-model-controller-7998c67dc4-2gt8j\" (UID: \"4cc26bdf-0a6c-42a8-a000-a97fdd40b51d\") " pod="opendatahub/odh-model-controller-7998c67dc4-2gt8j"
Sep 10 19:21:05.564548 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.564413    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2vtn\" (UniqueName: \"kubernetes.io/projected/4cc26bdf-0a6c-42a8-a000-a97fdd40b51d-kube-api-access-d2vtn\") pod \"odh-model-controller-7998c67dc4-2gt8j\" (UID: \"4cc26bdf-0a6c-42a8-a000-a97fdd40b51d\") " pod="opendatahub/odh-model-controller-7998c67dc4-2gt8j"
Sep 10 19:21:05.564548 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:21:05.564488    2579 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 10 19:21:05.564618 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:21:05.564570    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4cc26bdf-0a6c-42a8-a000-a97fdd40b51d-cert podName:4cc26bdf-0a6c-42a8-a000-a97fdd40b51d nodeName:}" failed. No retries permitted until 2026-09-10 19:21:06.064552629 +0000 UTC m=+321.284401103 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4cc26bdf-0a6c-42a8-a000-a97fdd40b51d-cert") pod "odh-model-controller-7998c67dc4-2gt8j" (UID: "4cc26bdf-0a6c-42a8-a000-a97fdd40b51d") : secret "odh-model-controller-webhook-cert" not found
Sep 10 19:21:05.572403 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.572381    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2vtn\" (UniqueName: \"kubernetes.io/projected/4cc26bdf-0a6c-42a8-a000-a97fdd40b51d-kube-api-access-d2vtn\") pod \"odh-model-controller-7998c67dc4-2gt8j\" (UID: \"4cc26bdf-0a6c-42a8-a000-a97fdd40b51d\") " pod="opendatahub/odh-model-controller-7998c67dc4-2gt8j"
Sep 10 19:21:05.614904 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.614875    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-w2wms"]
Sep 10 19:21:05.618458 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.618436    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-w2wms"
Sep 10 19:21:05.620432 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.620415    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 10 19:21:05.627951 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.627920    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-w2wms"]
Sep 10 19:21:05.767326 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.767217    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c472bc15-0bbd-43bb-8cd3-277fb3d01ffd-tls-certs\") pod \"model-serving-api-54d89bd5fc-w2wms\" (UID: \"c472bc15-0bbd-43bb-8cd3-277fb3d01ffd\") " pod="opendatahub/model-serving-api-54d89bd5fc-w2wms"
Sep 10 19:21:05.767326 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.767272    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ln86l\" (UniqueName: \"kubernetes.io/projected/c472bc15-0bbd-43bb-8cd3-277fb3d01ffd-kube-api-access-ln86l\") pod \"model-serving-api-54d89bd5fc-w2wms\" (UID: \"c472bc15-0bbd-43bb-8cd3-277fb3d01ffd\") " pod="opendatahub/model-serving-api-54d89bd5fc-w2wms"
Sep 10 19:21:05.868372 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.868342    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ln86l\" (UniqueName: \"kubernetes.io/projected/c472bc15-0bbd-43bb-8cd3-277fb3d01ffd-kube-api-access-ln86l\") pod \"model-serving-api-54d89bd5fc-w2wms\" (UID: \"c472bc15-0bbd-43bb-8cd3-277fb3d01ffd\") " pod="opendatahub/model-serving-api-54d89bd5fc-w2wms"
Sep 10 19:21:05.868575 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.868394    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/896dee6e-5770-4d1a-9637-e2573b9ebc16-tls-certs\") pod \"kube-auth-proxy-5dd7bdc797-jljr6\" (UID: \"896dee6e-5770-4d1a-9637-e2573b9ebc16\") " pod="openshift-ingress/kube-auth-proxy-5dd7bdc797-jljr6"
Sep 10 19:21:05.868575 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.868464    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c472bc15-0bbd-43bb-8cd3-277fb3d01ffd-tls-certs\") pod \"model-serving-api-54d89bd5fc-w2wms\" (UID: \"c472bc15-0bbd-43bb-8cd3-277fb3d01ffd\") " pod="opendatahub/model-serving-api-54d89bd5fc-w2wms"
Sep 10 19:21:05.868575 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:21:05.868555    2579 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 10 19:21:05.868742 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:21:05.868598    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c472bc15-0bbd-43bb-8cd3-277fb3d01ffd-tls-certs podName:c472bc15-0bbd-43bb-8cd3-277fb3d01ffd nodeName:}" failed. No retries permitted until 2026-09-10 19:21:06.368585681 +0000 UTC m=+321.588434161 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/c472bc15-0bbd-43bb-8cd3-277fb3d01ffd-tls-certs") pod "model-serving-api-54d89bd5fc-w2wms" (UID: "c472bc15-0bbd-43bb-8cd3-277fb3d01ffd") : secret "model-serving-api-tls" not found
Sep 10 19:21:05.871045 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.871022    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/896dee6e-5770-4d1a-9637-e2573b9ebc16-tls-certs\") pod \"kube-auth-proxy-5dd7bdc797-jljr6\" (UID: \"896dee6e-5770-4d1a-9637-e2573b9ebc16\") " pod="openshift-ingress/kube-auth-proxy-5dd7bdc797-jljr6"
Sep 10 19:21:05.889507 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:05.889479    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ln86l\" (UniqueName: \"kubernetes.io/projected/c472bc15-0bbd-43bb-8cd3-277fb3d01ffd-kube-api-access-ln86l\") pod \"model-serving-api-54d89bd5fc-w2wms\" (UID: \"c472bc15-0bbd-43bb-8cd3-277fb3d01ffd\") " pod="opendatahub/model-serving-api-54d89bd5fc-w2wms"
Sep 10 19:21:06.060860 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.060775    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5dd7bdc797-jljr6"
Sep 10 19:21:06.070205 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.070177    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4cc26bdf-0a6c-42a8-a000-a97fdd40b51d-cert\") pod \"odh-model-controller-7998c67dc4-2gt8j\" (UID: \"4cc26bdf-0a6c-42a8-a000-a97fdd40b51d\") " pod="opendatahub/odh-model-controller-7998c67dc4-2gt8j"
Sep 10 19:21:06.072739 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.072719    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4cc26bdf-0a6c-42a8-a000-a97fdd40b51d-cert\") pod \"odh-model-controller-7998c67dc4-2gt8j\" (UID: \"4cc26bdf-0a6c-42a8-a000-a97fdd40b51d\") " pod="opendatahub/odh-model-controller-7998c67dc4-2gt8j"
Sep 10 19:21:06.179076 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.179039    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-2gt8j"
Sep 10 19:21:06.339366 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.339332    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5dd7bdc797-jljr6"]
Sep 10 19:21:06.340522 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:21:06.340498    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod896dee6e_5770_4d1a_9637_e2573b9ebc16.slice/crio-9607ab1f29b56157a19de5fef0f8f6868b0b013d1e19646b89be60091ddb5224 WatchSource:0}: Error finding container 9607ab1f29b56157a19de5fef0f8f6868b0b013d1e19646b89be60091ddb5224: Status 404 returned error can't find the container with id 9607ab1f29b56157a19de5fef0f8f6868b0b013d1e19646b89be60091ddb5224
Sep 10 19:21:06.343039 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.343022    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 19:21:06.372852 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.372825    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c472bc15-0bbd-43bb-8cd3-277fb3d01ffd-tls-certs\") pod \"model-serving-api-54d89bd5fc-w2wms\" (UID: \"c472bc15-0bbd-43bb-8cd3-277fb3d01ffd\") " pod="opendatahub/model-serving-api-54d89bd5fc-w2wms"
Sep 10 19:21:06.376264 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.376225    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c472bc15-0bbd-43bb-8cd3-277fb3d01ffd-tls-certs\") pod \"model-serving-api-54d89bd5fc-w2wms\" (UID: \"c472bc15-0bbd-43bb-8cd3-277fb3d01ffd\") " pod="opendatahub/model-serving-api-54d89bd5fc-w2wms"
Sep 10 19:21:06.411369 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.410686    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd"]
Sep 10 19:21:06.414966 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.414941    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd"
Sep 10 19:21:06.417202 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.417168    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 10 19:21:06.417349 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.417291    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 10 19:21:06.417349 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.417291    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 10 19:21:06.417476 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.417305    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 19:21:06.417476 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.417306    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-2rv4h\""
Sep 10 19:21:06.427577 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.427553    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd"]
Sep 10 19:21:06.453987 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.453964    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2gt8j"]
Sep 10 19:21:06.456039 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:21:06.456013    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cc26bdf_0a6c_42a8_a000_a97fdd40b51d.slice/crio-52ed01205518dc8fdeb843593c48482a76333ed3f6cd17b1d637c52fe1678fbe WatchSource:0}: Error finding container 52ed01205518dc8fdeb843593c48482a76333ed3f6cd17b1d637c52fe1678fbe: Status 404 returned error can't find the container with id 52ed01205518dc8fdeb843593c48482a76333ed3f6cd17b1d637c52fe1678fbe
Sep 10 19:21:06.473674 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.473646    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84s4f\" (UniqueName: \"kubernetes.io/projected/0c9472e1-2da1-43b1-b384-f033c370ee4b-kube-api-access-84s4f\") pod \"istiod-openshift-gateway-7b7995d5cd-gxwvd\" (UID: \"0c9472e1-2da1-43b1-b384-f033c370ee4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd"
Sep 10 19:21:06.473811 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.473687    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c9472e1-2da1-43b1-b384-f033c370ee4b-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-gxwvd\" (UID: \"0c9472e1-2da1-43b1-b384-f033c370ee4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd"
Sep 10 19:21:06.473811 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.473710    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c9472e1-2da1-43b1-b384-f033c370ee4b-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-gxwvd\" (UID: \"0c9472e1-2da1-43b1-b384-f033c370ee4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd"
Sep 10 19:21:06.473944 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.473811    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0c9472e1-2da1-43b1-b384-f033c370ee4b-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-gxwvd\" (UID: \"0c9472e1-2da1-43b1-b384-f033c370ee4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd"
Sep 10 19:21:06.473944 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.473852    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0c9472e1-2da1-43b1-b384-f033c370ee4b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-gxwvd\" (UID: \"0c9472e1-2da1-43b1-b384-f033c370ee4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd"
Sep 10 19:21:06.473944 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.473884    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0c9472e1-2da1-43b1-b384-f033c370ee4b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-gxwvd\" (UID: \"0c9472e1-2da1-43b1-b384-f033c370ee4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd"
Sep 10 19:21:06.473944 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.473923    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0c9472e1-2da1-43b1-b384-f033c370ee4b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-gxwvd\" (UID: \"0c9472e1-2da1-43b1-b384-f033c370ee4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd"
Sep 10 19:21:06.530201 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.530164    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-w2wms"
Sep 10 19:21:06.574373 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.574341    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c9472e1-2da1-43b1-b384-f033c370ee4b-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-gxwvd\" (UID: \"0c9472e1-2da1-43b1-b384-f033c370ee4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd"
Sep 10 19:21:06.574555 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.574383    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c9472e1-2da1-43b1-b384-f033c370ee4b-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-gxwvd\" (UID: \"0c9472e1-2da1-43b1-b384-f033c370ee4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd"
Sep 10 19:21:06.574555 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.574431    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0c9472e1-2da1-43b1-b384-f033c370ee4b-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-gxwvd\" (UID: \"0c9472e1-2da1-43b1-b384-f033c370ee4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd"
Sep 10 19:21:06.574555 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.574453    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0c9472e1-2da1-43b1-b384-f033c370ee4b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-gxwvd\" (UID: \"0c9472e1-2da1-43b1-b384-f033c370ee4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd"
Sep 10 19:21:06.574555 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.574477    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0c9472e1-2da1-43b1-b384-f033c370ee4b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-gxwvd\" (UID: \"0c9472e1-2da1-43b1-b384-f033c370ee4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd"
Sep 10 19:21:06.574555 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.574515    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0c9472e1-2da1-43b1-b384-f033c370ee4b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-gxwvd\" (UID: \"0c9472e1-2da1-43b1-b384-f033c370ee4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd"
Sep 10 19:21:06.574836 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.574587    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84s4f\" (UniqueName: \"kubernetes.io/projected/0c9472e1-2da1-43b1-b384-f033c370ee4b-kube-api-access-84s4f\") pod \"istiod-openshift-gateway-7b7995d5cd-gxwvd\" (UID: \"0c9472e1-2da1-43b1-b384-f033c370ee4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd"
Sep 10 19:21:06.575284 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.575232    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0c9472e1-2da1-43b1-b384-f033c370ee4b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-gxwvd\" (UID: \"0c9472e1-2da1-43b1-b384-f033c370ee4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd"
Sep 10 19:21:06.576847 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.576819    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c9472e1-2da1-43b1-b384-f033c370ee4b-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-gxwvd\" (UID: \"0c9472e1-2da1-43b1-b384-f033c370ee4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd"
Sep 10 19:21:06.577164 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.577135    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0c9472e1-2da1-43b1-b384-f033c370ee4b-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-gxwvd\" (UID: \"0c9472e1-2da1-43b1-b384-f033c370ee4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd"
Sep 10 19:21:06.577360 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.577339    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0c9472e1-2da1-43b1-b384-f033c370ee4b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-gxwvd\" (UID: \"0c9472e1-2da1-43b1-b384-f033c370ee4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd"
Sep 10 19:21:06.577433 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.577393    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0c9472e1-2da1-43b1-b384-f033c370ee4b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-gxwvd\" (UID: \"0c9472e1-2da1-43b1-b384-f033c370ee4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd"
Sep 10 19:21:06.584293 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.584240    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c9472e1-2da1-43b1-b384-f033c370ee4b-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-gxwvd\" (UID: \"0c9472e1-2da1-43b1-b384-f033c370ee4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd"
Sep 10 19:21:06.584890 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.584769    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84s4f\" (UniqueName: \"kubernetes.io/projected/0c9472e1-2da1-43b1-b384-f033c370ee4b-kube-api-access-84s4f\") pod \"istiod-openshift-gateway-7b7995d5cd-gxwvd\" (UID: \"0c9472e1-2da1-43b1-b384-f033c370ee4b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd"
Sep 10 19:21:06.725180 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.725140    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd"
Sep 10 19:21:06.762954 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.762916    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-2gt8j" event={"ID":"4cc26bdf-0a6c-42a8-a000-a97fdd40b51d","Type":"ContainerStarted","Data":"52ed01205518dc8fdeb843593c48482a76333ed3f6cd17b1d637c52fe1678fbe"}
Sep 10 19:21:06.764357 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.764252    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5dd7bdc797-jljr6" event={"ID":"896dee6e-5770-4d1a-9637-e2573b9ebc16","Type":"ContainerStarted","Data":"9607ab1f29b56157a19de5fef0f8f6868b0b013d1e19646b89be60091ddb5224"}
Sep 10 19:21:06.808840 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:06.808798    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-w2wms"]
Sep 10 19:21:06.810572 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:21:06.810549    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc472bc15_0bbd_43bb_8cd3_277fb3d01ffd.slice/crio-bd063e960a2a56e2d353bbf7a04c6f983fd5a83324dbf72dc1002bf1a7cb36d9 WatchSource:0}: Error finding container bd063e960a2a56e2d353bbf7a04c6f983fd5a83324dbf72dc1002bf1a7cb36d9: Status 404 returned error can't find the container with id bd063e960a2a56e2d353bbf7a04c6f983fd5a83324dbf72dc1002bf1a7cb36d9
Sep 10 19:21:07.000770 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:07.000718    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd"]
Sep 10 19:21:07.003146 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:21:07.003113    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c9472e1_2da1_43b1_b384_f033c370ee4b.slice/crio-ca0889af135a07c537a46d4112afba406b3a91f23d0615028ed69c78ea584b2c WatchSource:0}: Error finding container ca0889af135a07c537a46d4112afba406b3a91f23d0615028ed69c78ea584b2c: Status 404 returned error can't find the container with id ca0889af135a07c537a46d4112afba406b3a91f23d0615028ed69c78ea584b2c
Sep 10 19:21:07.777865 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:07.776936    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-w2wms" event={"ID":"c472bc15-0bbd-43bb-8cd3-277fb3d01ffd","Type":"ContainerStarted","Data":"bd063e960a2a56e2d353bbf7a04c6f983fd5a83324dbf72dc1002bf1a7cb36d9"}
Sep 10 19:21:07.778995 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:07.778963    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd" event={"ID":"0c9472e1-2da1-43b1-b384-f033c370ee4b","Type":"ContainerStarted","Data":"ca0889af135a07c537a46d4112afba406b3a91f23d0615028ed69c78ea584b2c"}
Sep 10 19:21:12.793808 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:12.793775    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:21:12.794182 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:12.793837    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:21:13.819910 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:13.819874    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-w2wms" event={"ID":"c472bc15-0bbd-43bb-8cd3-277fb3d01ffd","Type":"ContainerStarted","Data":"ecfffcce58d6bb139c41ddbdcf8e8f0a6cc0ec6560f4ec84671b68ab57bd5c16"}
Sep 10 19:21:13.820372 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:13.819984    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-w2wms"
Sep 10 19:21:13.821717 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:13.821689    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-2gt8j" event={"ID":"4cc26bdf-0a6c-42a8-a000-a97fdd40b51d","Type":"ContainerStarted","Data":"0eeb75850b43503587601c7298405b2bb498712d7dadc02ba92bec705ca068da"}
Sep 10 19:21:13.821876 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:13.821790    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-2gt8j"
Sep 10 19:21:13.823494 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:13.823471    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5dd7bdc797-jljr6" event={"ID":"896dee6e-5770-4d1a-9637-e2573b9ebc16","Type":"ContainerStarted","Data":"c5952b9112c6d91b054ba5f2e44d35c1b645839b278352073e8ff44179d88f5a"}
Sep 10 19:21:13.825834 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:13.825811    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd" event={"ID":"0c9472e1-2da1-43b1-b384-f033c370ee4b","Type":"ContainerStarted","Data":"3b8cb5633cd5103b6b0c551c8c9cd5d061fe5d70f0a77e6bd00e0ca31a646ce8"}
Sep 10 19:21:13.825993 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:13.825974    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd"
Sep 10 19:21:13.828558 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:13.828530    2579 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-gxwvd container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 19:21:13.828652 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:13.828609    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd" podUID="0c9472e1-2da1-43b1-b384-f033c370ee4b" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 19:21:13.846347 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:13.846293    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-w2wms" podStartSLOduration=2.874867473 podStartE2EDuration="8.846273248s" totalImagesPullingTime="5.971405775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:21:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:21:13.843805686 +0000 UTC m=+329.063654164" watchObservedRunningTime="2026-09-10 19:21:13.846273248 +0000 UTC m=+329.066121760"
Sep 10 19:21:13.854011 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:13.853975    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79"]
Sep 10 19:21:13.860616 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:13.860589    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79"
Sep 10 19:21:13.864643 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:13.864618    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-2gt7n\""
Sep 10 19:21:13.873396 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:13.873370    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79"]
Sep 10 19:21:13.944130 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:13.944072    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5dd7bdc797-jljr6" podStartSLOduration=2.493187932 podStartE2EDuration="8.944052366s" totalImagesPullingTime="6.450864434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:21:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:21:13.941807148 +0000 UTC m=+329.161655631" watchObservedRunningTime="2026-09-10 19:21:13.944052366 +0000 UTC m=+329.163900847"
Sep 10 19:21:13.944340 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:13.944304    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd" podStartSLOduration=2.155994209 podStartE2EDuration="7.944291227s" totalImagesPullingTime="5.788297018s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:21:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:21:13.900832674 +0000 UTC m=+329.120681155" watchObservedRunningTime="2026-09-10 19:21:13.944291227 +0000 UTC m=+329.164139708"
Sep 10 19:21:13.951381 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:13.951355    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcnsz79\" (UID: \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79"
Sep 10 19:21:13.951532 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:13.951390    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcnsz79\" (UID: \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79"
Sep 10 19:21:13.951532 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:13.951406    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcnsz79\" (UID: \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79"
Sep 10 19:21:13.951532 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:13.951509    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcnsz79\" (UID: \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79"
Sep 10 19:21:13.951672 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:13.951541    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcnsz79\" (UID: \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79"
Sep 10 19:21:13.951672 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:13.951589    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcnsz79\" (UID: \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79"
Sep 10 19:21:13.951672 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:13.951642    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcnsz79\" (UID: \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79"
Sep 10 19:21:13.951795 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:13.951682    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msg58\" (UniqueName: \"kubernetes.io/projected/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-kube-api-access-msg58\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcnsz79\" (UID: \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79"
Sep 10 19:21:13.951795 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:13.951704    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcnsz79\" (UID: \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79"
Sep 10 19:21:13.971060 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:13.971023    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k"]
Sep 10 19:21:13.973671 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:13.973625    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-2gt8j" podStartSLOduration=2.65158889 podStartE2EDuration="8.973609523s" totalImagesPullingTime="6.322020633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:21:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:21:13.97219861 +0000 UTC m=+329.192047084" watchObservedRunningTime="2026-09-10 19:21:13.973609523 +0000 UTC m=+329.193458028"
Sep 10 19:21:13.975215 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:13.975198    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k"
Sep 10 19:21:14.001870 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.001842    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k"]
Sep 10 19:21:14.052417 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.052184    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcnsz79\" (UID: \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79"
Sep 10 19:21:14.052660 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.052445    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msg58\" (UniqueName: \"kubernetes.io/projected/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-kube-api-access-msg58\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcnsz79\" (UID: \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79"
Sep 10 19:21:14.052660 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.052491    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcnsz79\" (UID: \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79"
Sep 10 19:21:14.052660 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.052547    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e52d0ec4-e194-4be2-9690-c2c541a41f7f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k\" (UID: \"e52d0ec4-e194-4be2-9690-c2c541a41f7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k"
Sep 10 19:21:14.052972 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.052828    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcnsz79\" (UID: \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79"
Sep 10 19:21:14.052972 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.052836    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e52d0ec4-e194-4be2-9690-c2c541a41f7f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k\" (UID: \"e52d0ec4-e194-4be2-9690-c2c541a41f7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k"
Sep 10 19:21:14.052972 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.052914    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e52d0ec4-e194-4be2-9690-c2c541a41f7f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k\" (UID: \"e52d0ec4-e194-4be2-9690-c2c541a41f7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k"
Sep 10 19:21:14.053130 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.052992    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcnsz79\" (UID: \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79"
Sep 10 19:21:14.053130 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.053037    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e52d0ec4-e194-4be2-9690-c2c541a41f7f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k\" (UID: \"e52d0ec4-e194-4be2-9690-c2c541a41f7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k"
Sep 10 19:21:14.053130 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.053070    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94pdg\" (UniqueName: \"kubernetes.io/projected/e52d0ec4-e194-4be2-9690-c2c541a41f7f-kube-api-access-94pdg\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k\" (UID: \"e52d0ec4-e194-4be2-9690-c2c541a41f7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k"
Sep 10 19:21:14.053130 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.053095    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e52d0ec4-e194-4be2-9690-c2c541a41f7f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k\" (UID: \"e52d0ec4-e194-4be2-9690-c2c541a41f7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k"
Sep 10 19:21:14.053130 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.053124    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcnsz79\" (UID: \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79"
Sep 10 19:21:14.053364 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.053150    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcnsz79\" (UID: \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79"
Sep 10 19:21:14.053364 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.053188    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e52d0ec4-e194-4be2-9690-c2c541a41f7f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k\" (UID: \"e52d0ec4-e194-4be2-9690-c2c541a41f7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k"
Sep 10 19:21:14.053364 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.053219    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e52d0ec4-e194-4be2-9690-c2c541a41f7f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k\" (UID: \"e52d0ec4-e194-4be2-9690-c2c541a41f7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k"
Sep 10 19:21:14.053364 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.053283    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcnsz79\" (UID: \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79"
Sep 10 19:21:14.053364 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.053310    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcnsz79\" (UID: \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79"
Sep 10 19:21:14.053600 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.053428    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcnsz79\" (UID: \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79"
Sep 10 19:21:14.053600 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.053560    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e52d0ec4-e194-4be2-9690-c2c541a41f7f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k\" (UID: \"e52d0ec4-e194-4be2-9690-c2c541a41f7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k"
Sep 10 19:21:14.053701 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.053618    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcnsz79\" (UID: \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79"
Sep 10 19:21:14.053701 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.053624    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcnsz79\" (UID: \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79"
Sep 10 19:21:14.053701 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.053670    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcnsz79\" (UID: \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79"
Sep 10 19:21:14.054009 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.053988    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcnsz79\" (UID: \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79"
Sep 10 19:21:14.055801 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.055776    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcnsz79\" (UID: \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79"
Sep 10 19:21:14.055912 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.055897    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcnsz79\" (UID: \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79"
Sep 10 19:21:14.074652 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.074579    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcnsz79\" (UID: \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79"
Sep 10 19:21:14.074783 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.074687    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msg58\" (UniqueName: \"kubernetes.io/projected/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-kube-api-access-msg58\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcnsz79\" (UID: \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79"
Sep 10 19:21:14.154352 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.154308    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e52d0ec4-e194-4be2-9690-c2c541a41f7f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k\" (UID: \"e52d0ec4-e194-4be2-9690-c2c541a41f7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k"
Sep 10 19:21:14.154352 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.154353    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e52d0ec4-e194-4be2-9690-c2c541a41f7f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k\" (UID: \"e52d0ec4-e194-4be2-9690-c2c541a41f7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k"
Sep 10 19:21:14.154600 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.154379    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e52d0ec4-e194-4be2-9690-c2c541a41f7f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k\" (UID: \"e52d0ec4-e194-4be2-9690-c2c541a41f7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k"
Sep 10 19:21:14.154600 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.154413    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e52d0ec4-e194-4be2-9690-c2c541a41f7f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k\" (UID: \"e52d0ec4-e194-4be2-9690-c2c541a41f7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k"
Sep 10 19:21:14.154600 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.154440    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94pdg\" (UniqueName: \"kubernetes.io/projected/e52d0ec4-e194-4be2-9690-c2c541a41f7f-kube-api-access-94pdg\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k\" (UID: \"e52d0ec4-e194-4be2-9690-c2c541a41f7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k"
Sep 10 19:21:14.154600 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.154463    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e52d0ec4-e194-4be2-9690-c2c541a41f7f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k\" (UID: \"e52d0ec4-e194-4be2-9690-c2c541a41f7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k"
Sep 10 19:21:14.154600 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.154502    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e52d0ec4-e194-4be2-9690-c2c541a41f7f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k\" (UID: \"e52d0ec4-e194-4be2-9690-c2c541a41f7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k"
Sep 10 19:21:14.154600 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.154529    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e52d0ec4-e194-4be2-9690-c2c541a41f7f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k\" (UID: \"e52d0ec4-e194-4be2-9690-c2c541a41f7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k"
Sep 10 19:21:14.154600 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.154588    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e52d0ec4-e194-4be2-9690-c2c541a41f7f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k\" (UID: \"e52d0ec4-e194-4be2-9690-c2c541a41f7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k"
Sep 10 19:21:14.154964 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.154920    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e52d0ec4-e194-4be2-9690-c2c541a41f7f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k\" (UID: \"e52d0ec4-e194-4be2-9690-c2c541a41f7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k"
Sep 10 19:21:14.154964 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.154943    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e52d0ec4-e194-4be2-9690-c2c541a41f7f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k\" (UID: \"e52d0ec4-e194-4be2-9690-c2c541a41f7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k"
Sep 10 19:21:14.155072 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.155040    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e52d0ec4-e194-4be2-9690-c2c541a41f7f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k\" (UID: \"e52d0ec4-e194-4be2-9690-c2c541a41f7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k"
Sep 10 19:21:14.155160 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.155143    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e52d0ec4-e194-4be2-9690-c2c541a41f7f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k\" (UID: \"e52d0ec4-e194-4be2-9690-c2c541a41f7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k"
Sep 10 19:21:14.155734 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.155711    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e52d0ec4-e194-4be2-9690-c2c541a41f7f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k\" (UID: \"e52d0ec4-e194-4be2-9690-c2c541a41f7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k"
Sep 10 19:21:14.156921 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.156899    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e52d0ec4-e194-4be2-9690-c2c541a41f7f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k\" (UID: \"e52d0ec4-e194-4be2-9690-c2c541a41f7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k"
Sep 10 19:21:14.157175 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.157158    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e52d0ec4-e194-4be2-9690-c2c541a41f7f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k\" (UID: \"e52d0ec4-e194-4be2-9690-c2c541a41f7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k"
Sep 10 19:21:14.162448 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.162424    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e52d0ec4-e194-4be2-9690-c2c541a41f7f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k\" (UID: \"e52d0ec4-e194-4be2-9690-c2c541a41f7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k"
Sep 10 19:21:14.162592 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.162574    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94pdg\" (UniqueName: \"kubernetes.io/projected/e52d0ec4-e194-4be2-9690-c2c541a41f7f-kube-api-access-94pdg\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k\" (UID: \"e52d0ec4-e194-4be2-9690-c2c541a41f7f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k"
Sep 10 19:21:14.171464 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.171443    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79"
Sep 10 19:21:14.286044 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.286002    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k"
Sep 10 19:21:14.476636 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.476605    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79"]
Sep 10 19:21:14.480147 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:21:14.480117    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fe60f3d_468d_42d9_8d2e_ddfd7e4877b2.slice/crio-676f3f3d6dc29aba79a77d7efcd7be0217b55da5fd49d1ba48152ca6fee311bd WatchSource:0}: Error finding container 676f3f3d6dc29aba79a77d7efcd7be0217b55da5fd49d1ba48152ca6fee311bd: Status 404 returned error can't find the container with id 676f3f3d6dc29aba79a77d7efcd7be0217b55da5fd49d1ba48152ca6fee311bd
Sep 10 19:21:14.576777 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.576715    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k"]
Sep 10 19:21:14.579656 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:21:14.579625    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode52d0ec4_e194_4be2_9690_c2c541a41f7f.slice/crio-a054de809058b94005fe49140aafcaa98073511b52576f0d66898fc3c9b429ef WatchSource:0}: Error finding container a054de809058b94005fe49140aafcaa98073511b52576f0d66898fc3c9b429ef: Status 404 returned error can't find the container with id a054de809058b94005fe49140aafcaa98073511b52576f0d66898fc3c9b429ef
Sep 10 19:21:14.832178 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.832138    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79" event={"ID":"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2","Type":"ContainerStarted","Data":"676f3f3d6dc29aba79a77d7efcd7be0217b55da5fd49d1ba48152ca6fee311bd"}
Sep 10 19:21:14.833793 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.833554    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k" event={"ID":"e52d0ec4-e194-4be2-9690-c2c541a41f7f","Type":"ContainerStarted","Data":"a054de809058b94005fe49140aafcaa98073511b52576f0d66898fc3c9b429ef"}
Sep 10 19:21:14.834297 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:14.834279    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd"
Sep 10 19:21:15.372339 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:15.372303    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-h2rcd"]
Sep 10 19:21:15.376409 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:15.376384    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h2rcd"
Sep 10 19:21:15.379625 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:15.379588    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 10 19:21:15.379835 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:15.379590    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-z9475\""
Sep 10 19:21:15.387215 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:15.387186    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-h2rcd"]
Sep 10 19:21:15.467259 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:15.467227    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87c2c3c1-8abb-447e-a1c2-dc779a3cb86a-cert\") pod \"kserve-controller-manager-d6f4b65b8-h2rcd\" (UID: \"87c2c3c1-8abb-447e-a1c2-dc779a3cb86a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h2rcd"
Sep 10 19:21:15.467259 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:15.467270    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ch7lf\" (UniqueName: \"kubernetes.io/projected/87c2c3c1-8abb-447e-a1c2-dc779a3cb86a-kube-api-access-ch7lf\") pod \"kserve-controller-manager-d6f4b65b8-h2rcd\" (UID: \"87c2c3c1-8abb-447e-a1c2-dc779a3cb86a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h2rcd"
Sep 10 19:21:15.568323 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:15.568279    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87c2c3c1-8abb-447e-a1c2-dc779a3cb86a-cert\") pod \"kserve-controller-manager-d6f4b65b8-h2rcd\" (UID: \"87c2c3c1-8abb-447e-a1c2-dc779a3cb86a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h2rcd"
Sep 10 19:21:15.568515 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:15.568337    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ch7lf\" (UniqueName: \"kubernetes.io/projected/87c2c3c1-8abb-447e-a1c2-dc779a3cb86a-kube-api-access-ch7lf\") pod \"kserve-controller-manager-d6f4b65b8-h2rcd\" (UID: \"87c2c3c1-8abb-447e-a1c2-dc779a3cb86a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h2rcd"
Sep 10 19:21:15.568515 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:21:15.568496    2579 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Sep 10 19:21:15.568679 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:21:15.568585    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87c2c3c1-8abb-447e-a1c2-dc779a3cb86a-cert podName:87c2c3c1-8abb-447e-a1c2-dc779a3cb86a nodeName:}" failed. No retries permitted until 2026-09-10 19:21:16.068553923 +0000 UTC m=+331.288402396 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/87c2c3c1-8abb-447e-a1c2-dc779a3cb86a-cert") pod "kserve-controller-manager-d6f4b65b8-h2rcd" (UID: "87c2c3c1-8abb-447e-a1c2-dc779a3cb86a") : secret "kserve-webhook-server-cert" not found
Sep 10 19:21:15.586239 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:15.586211    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ch7lf\" (UniqueName: \"kubernetes.io/projected/87c2c3c1-8abb-447e-a1c2-dc779a3cb86a-kube-api-access-ch7lf\") pod \"kserve-controller-manager-d6f4b65b8-h2rcd\" (UID: \"87c2c3c1-8abb-447e-a1c2-dc779a3cb86a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h2rcd"
Sep 10 19:21:16.072789 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:16.072722    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87c2c3c1-8abb-447e-a1c2-dc779a3cb86a-cert\") pod \"kserve-controller-manager-d6f4b65b8-h2rcd\" (UID: \"87c2c3c1-8abb-447e-a1c2-dc779a3cb86a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h2rcd"
Sep 10 19:21:16.075825 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:16.075800    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87c2c3c1-8abb-447e-a1c2-dc779a3cb86a-cert\") pod \"kserve-controller-manager-d6f4b65b8-h2rcd\" (UID: \"87c2c3c1-8abb-447e-a1c2-dc779a3cb86a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-h2rcd"
Sep 10 19:21:16.293910 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:16.293877    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h2rcd"
Sep 10 19:21:16.720147 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:16.720112    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd"]
Sep 10 19:21:16.846644 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:16.846600    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd" podUID="0c9472e1-2da1-43b1-b384-f033c370ee4b" containerName="discovery" containerID="cri-o://3b8cb5633cd5103b6b0c551c8c9cd5d061fe5d70f0a77e6bd00e0ca31a646ce8" gracePeriod=30
Sep 10 19:21:17.342404 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.342360    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:21:17.342808 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.342452    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:21:17.342808 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.342498    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:21:17.343219 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.343194    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:21:17.343274 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.343262    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:21:17.343317 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.343293    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:21:17.410825 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.410649    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd"
Sep 10 19:21:17.486940 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.486908    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c9472e1-2da1-43b1-b384-f033c370ee4b-istio-token\") pod \"0c9472e1-2da1-43b1-b384-f033c370ee4b\" (UID: \"0c9472e1-2da1-43b1-b384-f033c370ee4b\") "
Sep 10 19:21:17.487104 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.486952    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0c9472e1-2da1-43b1-b384-f033c370ee4b-cacerts\") pod \"0c9472e1-2da1-43b1-b384-f033c370ee4b\" (UID: \"0c9472e1-2da1-43b1-b384-f033c370ee4b\") "
Sep 10 19:21:17.487104 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.486981    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0c9472e1-2da1-43b1-b384-f033c370ee4b-istio-kubeconfig\") pod \"0c9472e1-2da1-43b1-b384-f033c370ee4b\" (UID: \"0c9472e1-2da1-43b1-b384-f033c370ee4b\") "
Sep 10 19:21:17.487104 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.487008    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0c9472e1-2da1-43b1-b384-f033c370ee4b-istio-csr-dns-cert\") pod \"0c9472e1-2da1-43b1-b384-f033c370ee4b\" (UID: \"0c9472e1-2da1-43b1-b384-f033c370ee4b\") "
Sep 10 19:21:17.487104 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.487082    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0c9472e1-2da1-43b1-b384-f033c370ee4b-istio-csr-ca-configmap\") pod \"0c9472e1-2da1-43b1-b384-f033c370ee4b\" (UID: \"0c9472e1-2da1-43b1-b384-f033c370ee4b\") "
Sep 10 19:21:17.487323 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.487113    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84s4f\" (UniqueName: \"kubernetes.io/projected/0c9472e1-2da1-43b1-b384-f033c370ee4b-kube-api-access-84s4f\") pod \"0c9472e1-2da1-43b1-b384-f033c370ee4b\" (UID: \"0c9472e1-2da1-43b1-b384-f033c370ee4b\") "
Sep 10 19:21:17.487323 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.487185    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c9472e1-2da1-43b1-b384-f033c370ee4b-local-certs\") pod \"0c9472e1-2da1-43b1-b384-f033c370ee4b\" (UID: \"0c9472e1-2da1-43b1-b384-f033c370ee4b\") "
Sep 10 19:21:17.487871 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.487814    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c9472e1-2da1-43b1-b384-f033c370ee4b-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "0c9472e1-2da1-43b1-b384-f033c370ee4b" (UID: "0c9472e1-2da1-43b1-b384-f033c370ee4b"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:21:17.491005 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.490958    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c9472e1-2da1-43b1-b384-f033c370ee4b-istio-token" (OuterVolumeSpecName: "istio-token") pod "0c9472e1-2da1-43b1-b384-f033c370ee4b" (UID: "0c9472e1-2da1-43b1-b384-f033c370ee4b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:21:17.491005 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.490963    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c9472e1-2da1-43b1-b384-f033c370ee4b-local-certs" (OuterVolumeSpecName: "local-certs") pod "0c9472e1-2da1-43b1-b384-f033c370ee4b" (UID: "0c9472e1-2da1-43b1-b384-f033c370ee4b"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:21:17.491206 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.491134    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c9472e1-2da1-43b1-b384-f033c370ee4b-kube-api-access-84s4f" (OuterVolumeSpecName: "kube-api-access-84s4f") pod "0c9472e1-2da1-43b1-b384-f033c370ee4b" (UID: "0c9472e1-2da1-43b1-b384-f033c370ee4b"). InnerVolumeSpecName "kube-api-access-84s4f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:21:17.491267 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.491215    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c9472e1-2da1-43b1-b384-f033c370ee4b-cacerts" (OuterVolumeSpecName: "cacerts") pod "0c9472e1-2da1-43b1-b384-f033c370ee4b" (UID: "0c9472e1-2da1-43b1-b384-f033c370ee4b"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:21:17.491267 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.491255    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c9472e1-2da1-43b1-b384-f033c370ee4b-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "0c9472e1-2da1-43b1-b384-f033c370ee4b" (UID: "0c9472e1-2da1-43b1-b384-f033c370ee4b"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:21:17.492639 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.492609    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c9472e1-2da1-43b1-b384-f033c370ee4b-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "0c9472e1-2da1-43b1-b384-f033c370ee4b" (UID: "0c9472e1-2da1-43b1-b384-f033c370ee4b"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:21:17.531534 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.531366    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-h2rcd"]
Sep 10 19:21:17.534373 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:21:17.534339    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87c2c3c1_8abb_447e_a1c2_dc779a3cb86a.slice/crio-c7b9533f6e2cb0fcb7d2513d19254dfa3d29dd9736eb4131bad939059d53e870 WatchSource:0}: Error finding container c7b9533f6e2cb0fcb7d2513d19254dfa3d29dd9736eb4131bad939059d53e870: Status 404 returned error can't find the container with id c7b9533f6e2cb0fcb7d2513d19254dfa3d29dd9736eb4131bad939059d53e870
Sep 10 19:21:17.588541 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.588510    2579 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c9472e1-2da1-43b1-b384-f033c370ee4b-local-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:21:17.588541 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.588536    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c9472e1-2da1-43b1-b384-f033c370ee4b-istio-token\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:21:17.588541 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.588545    2579 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0c9472e1-2da1-43b1-b384-f033c370ee4b-cacerts\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:21:17.588781 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.588553    2579 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0c9472e1-2da1-43b1-b384-f033c370ee4b-istio-kubeconfig\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:21:17.588781 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.588561    2579 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0c9472e1-2da1-43b1-b384-f033c370ee4b-istio-csr-dns-cert\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:21:17.588781 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.588570    2579 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0c9472e1-2da1-43b1-b384-f033c370ee4b-istio-csr-ca-configmap\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:21:17.588781 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.588580    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84s4f\" (UniqueName: \"kubernetes.io/projected/0c9472e1-2da1-43b1-b384-f033c370ee4b-kube-api-access-84s4f\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:21:17.854140 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.854043    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79" event={"ID":"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2","Type":"ContainerStarted","Data":"41ce1056053df75a996f7cbb7ce183c4af402765ab3a0f7df80eacfee8ad2bb5"}
Sep 10 19:21:17.855349 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.855324    2579 generic.go:356] "Generic (PLEG): container finished" podID="0c9472e1-2da1-43b1-b384-f033c370ee4b" containerID="3b8cb5633cd5103b6b0c551c8c9cd5d061fe5d70f0a77e6bd00e0ca31a646ce8" exitCode=0
Sep 10 19:21:17.855496 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.855364    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd"
Sep 10 19:21:17.855496 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.855401    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd" event={"ID":"0c9472e1-2da1-43b1-b384-f033c370ee4b","Type":"ContainerDied","Data":"3b8cb5633cd5103b6b0c551c8c9cd5d061fe5d70f0a77e6bd00e0ca31a646ce8"}
Sep 10 19:21:17.855496 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.855439    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd" event={"ID":"0c9472e1-2da1-43b1-b384-f033c370ee4b","Type":"ContainerDied","Data":"ca0889af135a07c537a46d4112afba406b3a91f23d0615028ed69c78ea584b2c"}
Sep 10 19:21:17.855496 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.855463    2579 scope.go:117] "RemoveContainer" containerID="3b8cb5633cd5103b6b0c551c8c9cd5d061fe5d70f0a77e6bd00e0ca31a646ce8"
Sep 10 19:21:17.857231 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.857196    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k" event={"ID":"e52d0ec4-e194-4be2-9690-c2c541a41f7f","Type":"ContainerStarted","Data":"419cdc221e458016a95688a61cdae527cc7ba03aa9f345e4b631fdd10fa36394"}
Sep 10 19:21:17.858708 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.858688    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h2rcd" event={"ID":"87c2c3c1-8abb-447e-a1c2-dc779a3cb86a","Type":"ContainerStarted","Data":"c7b9533f6e2cb0fcb7d2513d19254dfa3d29dd9736eb4131bad939059d53e870"}
Sep 10 19:21:17.864612 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.864594    2579 scope.go:117] "RemoveContainer" containerID="3b8cb5633cd5103b6b0c551c8c9cd5d061fe5d70f0a77e6bd00e0ca31a646ce8"
Sep 10 19:21:17.864909 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:21:17.864888    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b8cb5633cd5103b6b0c551c8c9cd5d061fe5d70f0a77e6bd00e0ca31a646ce8\": container with ID starting with 3b8cb5633cd5103b6b0c551c8c9cd5d061fe5d70f0a77e6bd00e0ca31a646ce8 not found: ID does not exist" containerID="3b8cb5633cd5103b6b0c551c8c9cd5d061fe5d70f0a77e6bd00e0ca31a646ce8"
Sep 10 19:21:17.864969 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.864921    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b8cb5633cd5103b6b0c551c8c9cd5d061fe5d70f0a77e6bd00e0ca31a646ce8"} err="failed to get container status \"3b8cb5633cd5103b6b0c551c8c9cd5d061fe5d70f0a77e6bd00e0ca31a646ce8\": rpc error: code = NotFound desc = could not find container \"3b8cb5633cd5103b6b0c551c8c9cd5d061fe5d70f0a77e6bd00e0ca31a646ce8\": container with ID starting with 3b8cb5633cd5103b6b0c551c8c9cd5d061fe5d70f0a77e6bd00e0ca31a646ce8 not found: ID does not exist"
Sep 10 19:21:17.877305 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.877263    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79" podStartSLOduration=2.016417286 podStartE2EDuration="4.877249919s" totalImagesPullingTime="2.860832633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:21:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:21:17.874638355 +0000 UTC m=+333.094486837" watchObservedRunningTime="2026-09-10 19:21:17.877249919 +0000 UTC m=+333.097098400"
Sep 10 19:21:17.893791 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.893744    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd"]
Sep 10 19:21:17.895449 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.895431    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd"]
Sep 10 19:21:17.916897 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:17.916853    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k" podStartSLOduration=2.156410955 podStartE2EDuration="4.916838153s" totalImagesPullingTime="2.760427198s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:21:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:21:17.915130035 +0000 UTC m=+333.134978515" watchObservedRunningTime="2026-09-10 19:21:17.916838153 +0000 UTC m=+333.136686634"
Sep 10 19:21:18.172536 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:18.172505    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79"
Sep 10 19:21:18.173906 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:18.173887    2579 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Sep 10 19:21:18.173977 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:18.173933    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79" podUID="9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Sep 10 19:21:18.286285 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:18.286251    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k"
Sep 10 19:21:18.291071 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:18.291045    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k"
Sep 10 19:21:18.865432 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:18.865399    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k"
Sep 10 19:21:18.866534 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:18.866511    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k"
Sep 10 19:21:18.919891 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:18.919857    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79"]
Sep 10 19:21:19.171829 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:19.171796    2579 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Sep 10 19:21:19.172002 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:19.171852    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79" podUID="9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Sep 10 19:21:19.249983 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:19.249936    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c9472e1-2da1-43b1-b384-f033c370ee4b" path="/var/lib/kubelet/pods/0c9472e1-2da1-43b1-b384-f033c370ee4b/volumes"
Sep 10 19:21:20.172175 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:20.172137    2579 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Sep 10 19:21:20.172631 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:20.172205    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79" podUID="9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Sep 10 19:21:20.878608 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:20.878510    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h2rcd" event={"ID":"87c2c3c1-8abb-447e-a1c2-dc779a3cb86a","Type":"ContainerStarted","Data":"11a2957898f4e8b80e8c4819c80642c529bdc8351a41f36c4c8befd83684d3d2"}
Sep 10 19:21:20.878806 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:20.878628    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79" podUID="9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2" containerName="istio-proxy" containerID="cri-o://41ce1056053df75a996f7cbb7ce183c4af402765ab3a0f7df80eacfee8ad2bb5" gracePeriod=30
Sep 10 19:21:20.878806 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:20.878669    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h2rcd"
Sep 10 19:21:20.929860 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:20.929809    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h2rcd" podStartSLOduration=3.051191747 podStartE2EDuration="5.929793637s" totalImagesPullingTime="2.87860189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:21:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:21:20.928169237 +0000 UTC m=+336.148017717" watchObservedRunningTime="2026-09-10 19:21:20.929793637 +0000 UTC m=+336.149642124"
Sep 10 19:21:24.836178 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:24.836145    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-2gt8j"
Sep 10 19:21:24.839583 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:24.839556    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-w2wms"
Sep 10 19:21:26.133173 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:26.133151    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79"
Sep 10 19:21:26.270508 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:26.270424    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-credential-socket\") pod \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\" (UID: \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\") "
Sep 10 19:21:26.270508 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:26.270489    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-istiod-ca-cert\") pod \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\" (UID: \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\") "
Sep 10 19:21:26.270731 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:26.270519    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-istio-envoy\") pod \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\" (UID: \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\") "
Sep 10 19:21:26.270731 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:26.270543    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-istio-token\") pod \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\" (UID: \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\") "
Sep 10 19:21:26.270731 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:26.270559    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-istio-podinfo\") pod \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\" (UID: \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\") "
Sep 10 19:21:26.270731 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:26.270577    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-msg58\" (UniqueName: \"kubernetes.io/projected/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-kube-api-access-msg58\") pod \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\" (UID: \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\") "
Sep 10 19:21:26.270731 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:26.270600    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-istio-data\") pod \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\" (UID: \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\") "
Sep 10 19:21:26.270731 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:26.270663    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-workload-certs\") pod \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\" (UID: \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\") "
Sep 10 19:21:26.270731 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:26.270684    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-workload-socket\") pod \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\" (UID: \"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2\") "
Sep 10 19:21:26.271091 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:26.270801    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2" (UID: "9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:21:26.271091 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:26.270991    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2" (UID: "9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:21:26.271091 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:26.271003    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-credential-socket\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:21:26.271231 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:26.271169    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2" (UID: "9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:21:26.271231 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:26.271161    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-istio-data" (OuterVolumeSpecName: "istio-data") pod "9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2" (UID: "9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:21:26.271415 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:26.271331    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2" (UID: "9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:21:26.273065 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:26.273034    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2" (UID: "9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:21:26.273178 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:26.273065    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-istio-token" (OuterVolumeSpecName: "istio-token") pod "9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2" (UID: "9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:21:26.273243 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:26.273222    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2" (UID: "9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:21:26.273289 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:26.273251    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-kube-api-access-msg58" (OuterVolumeSpecName: "kube-api-access-msg58") pod "9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2" (UID: "9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2"). InnerVolumeSpecName "kube-api-access-msg58". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:21:26.372415 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:26.372383    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-istio-token\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:21:26.372415 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:26.372410    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-istio-podinfo\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:21:26.372415 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:26.372419    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-msg58\" (UniqueName: \"kubernetes.io/projected/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-kube-api-access-msg58\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:21:26.372653 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:26.372432    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-istio-data\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:21:26.372653 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:26.372442    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-workload-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:21:26.372653 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:26.372449    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-workload-socket\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:21:26.372653 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:26.372463    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-istiod-ca-cert\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:21:26.372653 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:26.372472    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2-istio-envoy\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:21:26.910130 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:26.910086    2579 generic.go:356] "Generic (PLEG): container finished" podID="9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2" containerID="41ce1056053df75a996f7cbb7ce183c4af402765ab3a0f7df80eacfee8ad2bb5" exitCode=0
Sep 10 19:21:26.910130 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:26.910135    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79"
Sep 10 19:21:26.910347 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:26.910152    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79" event={"ID":"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2","Type":"ContainerDied","Data":"41ce1056053df75a996f7cbb7ce183c4af402765ab3a0f7df80eacfee8ad2bb5"}
Sep 10 19:21:26.910347 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:26.910207    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79" event={"ID":"9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2","Type":"ContainerDied","Data":"676f3f3d6dc29aba79a77d7efcd7be0217b55da5fd49d1ba48152ca6fee311bd"}
Sep 10 19:21:26.910347 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:26.910237    2579 scope.go:117] "RemoveContainer" containerID="41ce1056053df75a996f7cbb7ce183c4af402765ab3a0f7df80eacfee8ad2bb5"
Sep 10 19:21:26.919982 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:26.919961    2579 scope.go:117] "RemoveContainer" containerID="41ce1056053df75a996f7cbb7ce183c4af402765ab3a0f7df80eacfee8ad2bb5"
Sep 10 19:21:26.920232 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:21:26.920210    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41ce1056053df75a996f7cbb7ce183c4af402765ab3a0f7df80eacfee8ad2bb5\": container with ID starting with 41ce1056053df75a996f7cbb7ce183c4af402765ab3a0f7df80eacfee8ad2bb5 not found: ID does not exist" containerID="41ce1056053df75a996f7cbb7ce183c4af402765ab3a0f7df80eacfee8ad2bb5"
Sep 10 19:21:26.920290 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:26.920239    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41ce1056053df75a996f7cbb7ce183c4af402765ab3a0f7df80eacfee8ad2bb5"} err="failed to get container status \"41ce1056053df75a996f7cbb7ce183c4af402765ab3a0f7df80eacfee8ad2bb5\": rpc error: code = NotFound desc = could not find container \"41ce1056053df75a996f7cbb7ce183c4af402765ab3a0f7df80eacfee8ad2bb5\": container with ID starting with 41ce1056053df75a996f7cbb7ce183c4af402765ab3a0f7df80eacfee8ad2bb5 not found: ID does not exist"
Sep 10 19:21:26.932652 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:26.932629    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79"]
Sep 10 19:21:26.936695 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:26.936672    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79"]
Sep 10 19:21:27.239127 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:27.239045    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2" path="/var/lib/kubelet/pods/9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2/volumes"
Sep 10 19:21:51.889416 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:21:51.889387    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-h2rcd"
Sep 10 19:22:10.313257 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.313222    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sw8t7"]
Sep 10 19:22:10.313696 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.313568    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c9472e1-2da1-43b1-b384-f033c370ee4b" containerName="discovery"
Sep 10 19:22:10.313696 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.313579    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c9472e1-2da1-43b1-b384-f033c370ee4b" containerName="discovery"
Sep 10 19:22:10.313696 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.313596    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2" containerName="istio-proxy"
Sep 10 19:22:10.313696 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.313603    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2" containerName="istio-proxy"
Sep 10 19:22:10.313696 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.313663    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c9472e1-2da1-43b1-b384-f033c370ee4b" containerName="discovery"
Sep 10 19:22:10.313696 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.313671    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fe60f3d-468d-42d9-8d2e-ddfd7e4877b2" containerName="istio-proxy"
Sep 10 19:22:10.316863 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.316846    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sw8t7"
Sep 10 19:22:10.318865 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.318842    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ccnqb\""
Sep 10 19:22:10.319006 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.318876    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 19:22:10.319383 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.319368    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 19:22:10.324700 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.324680    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sw8t7"]
Sep 10 19:22:10.409248 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.409214    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmf48s"]
Sep 10 19:22:10.412700 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.412679    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmf48s"
Sep 10 19:22:10.421617 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.421595    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmf48s"]
Sep 10 19:22:10.450766 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.450730    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48bh4\" (UniqueName: \"kubernetes.io/projected/0cb12c1c-488e-4826-a87d-e08ab3eff96d-kube-api-access-48bh4\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sw8t7\" (UID: \"0cb12c1c-488e-4826-a87d-e08ab3eff96d\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sw8t7"
Sep 10 19:22:10.450883 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.450815    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0cb12c1c-488e-4826-a87d-e08ab3eff96d-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sw8t7\" (UID: \"0cb12c1c-488e-4826-a87d-e08ab3eff96d\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sw8t7"
Sep 10 19:22:10.450883 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.450847    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0cb12c1c-488e-4826-a87d-e08ab3eff96d-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sw8t7\" (UID: \"0cb12c1c-488e-4826-a87d-e08ab3eff96d\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sw8t7"
Sep 10 19:22:10.511032 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.511002    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apvpqv"]
Sep 10 19:22:10.514403 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.514386    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apvpqv"
Sep 10 19:22:10.522820 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.522799    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apvpqv"]
Sep 10 19:22:10.551529 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.551500    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0581c2a-2a71-41a0-8805-6ba636380ca5-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmf48s\" (UID: \"e0581c2a-2a71-41a0-8805-6ba636380ca5\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmf48s"
Sep 10 19:22:10.551672 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.551556    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48bh4\" (UniqueName: \"kubernetes.io/projected/0cb12c1c-488e-4826-a87d-e08ab3eff96d-kube-api-access-48bh4\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sw8t7\" (UID: \"0cb12c1c-488e-4826-a87d-e08ab3eff96d\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sw8t7"
Sep 10 19:22:10.551672 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.551580    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0cb12c1c-488e-4826-a87d-e08ab3eff96d-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sw8t7\" (UID: \"0cb12c1c-488e-4826-a87d-e08ab3eff96d\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sw8t7"
Sep 10 19:22:10.551672 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.551597    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5jpr\" (UniqueName: \"kubernetes.io/projected/e0581c2a-2a71-41a0-8805-6ba636380ca5-kube-api-access-x5jpr\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmf48s\" (UID: \"e0581c2a-2a71-41a0-8805-6ba636380ca5\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmf48s"
Sep 10 19:22:10.551672 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.551621    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0cb12c1c-488e-4826-a87d-e08ab3eff96d-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sw8t7\" (UID: \"0cb12c1c-488e-4826-a87d-e08ab3eff96d\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sw8t7"
Sep 10 19:22:10.551672 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.551651    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0581c2a-2a71-41a0-8805-6ba636380ca5-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmf48s\" (UID: \"e0581c2a-2a71-41a0-8805-6ba636380ca5\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmf48s"
Sep 10 19:22:10.551980 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.551963    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0cb12c1c-488e-4826-a87d-e08ab3eff96d-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sw8t7\" (UID: \"0cb12c1c-488e-4826-a87d-e08ab3eff96d\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sw8t7"
Sep 10 19:22:10.552019 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.551991    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0cb12c1c-488e-4826-a87d-e08ab3eff96d-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sw8t7\" (UID: \"0cb12c1c-488e-4826-a87d-e08ab3eff96d\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sw8t7"
Sep 10 19:22:10.561505 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.561482    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48bh4\" (UniqueName: \"kubernetes.io/projected/0cb12c1c-488e-4826-a87d-e08ab3eff96d-kube-api-access-48bh4\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sw8t7\" (UID: \"0cb12c1c-488e-4826-a87d-e08ab3eff96d\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sw8t7"
Sep 10 19:22:10.609297 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.609220    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zvpr2"]
Sep 10 19:22:10.612956 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.612936    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zvpr2"
Sep 10 19:22:10.620637 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.620612    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zvpr2"]
Sep 10 19:22:10.626556 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.626532    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sw8t7"
Sep 10 19:22:10.652125 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.652103    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/196b3dd9-3349-4687-8d92-9eb83c1a4f5b-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zvpr2\" (UID: \"196b3dd9-3349-4687-8d92-9eb83c1a4f5b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zvpr2"
Sep 10 19:22:10.652246 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.652137    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p66bv\" (UniqueName: \"kubernetes.io/projected/196b3dd9-3349-4687-8d92-9eb83c1a4f5b-kube-api-access-p66bv\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zvpr2\" (UID: \"196b3dd9-3349-4687-8d92-9eb83c1a4f5b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zvpr2"
Sep 10 19:22:10.652246 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.652170    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0581c2a-2a71-41a0-8805-6ba636380ca5-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmf48s\" (UID: \"e0581c2a-2a71-41a0-8805-6ba636380ca5\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmf48s"
Sep 10 19:22:10.652246 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.652234    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0581c2a-2a71-41a0-8805-6ba636380ca5-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmf48s\" (UID: \"e0581c2a-2a71-41a0-8805-6ba636380ca5\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmf48s"
Sep 10 19:22:10.652364 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.652266    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a517fbb-495a-452c-9f8a-f3886baf3f15-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apvpqv\" (UID: \"9a517fbb-495a-452c-9f8a-f3886baf3f15\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apvpqv"
Sep 10 19:22:10.652364 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.652321    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/196b3dd9-3349-4687-8d92-9eb83c1a4f5b-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zvpr2\" (UID: \"196b3dd9-3349-4687-8d92-9eb83c1a4f5b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zvpr2"
Sep 10 19:22:10.652364 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.652356    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a517fbb-495a-452c-9f8a-f3886baf3f15-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apvpqv\" (UID: \"9a517fbb-495a-452c-9f8a-f3886baf3f15\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apvpqv"
Sep 10 19:22:10.652520 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.652374    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx2l5\" (UniqueName: \"kubernetes.io/projected/9a517fbb-495a-452c-9f8a-f3886baf3f15-kube-api-access-cx2l5\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apvpqv\" (UID: \"9a517fbb-495a-452c-9f8a-f3886baf3f15\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apvpqv"
Sep 10 19:22:10.652520 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.652469    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0581c2a-2a71-41a0-8805-6ba636380ca5-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmf48s\" (UID: \"e0581c2a-2a71-41a0-8805-6ba636380ca5\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmf48s"
Sep 10 19:22:10.652520 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.652478    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5jpr\" (UniqueName: \"kubernetes.io/projected/e0581c2a-2a71-41a0-8805-6ba636380ca5-kube-api-access-x5jpr\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmf48s\" (UID: \"e0581c2a-2a71-41a0-8805-6ba636380ca5\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmf48s"
Sep 10 19:22:10.652643 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.652521    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0581c2a-2a71-41a0-8805-6ba636380ca5-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmf48s\" (UID: \"e0581c2a-2a71-41a0-8805-6ba636380ca5\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmf48s"
Sep 10 19:22:10.660471 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.660442    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5jpr\" (UniqueName: \"kubernetes.io/projected/e0581c2a-2a71-41a0-8805-6ba636380ca5-kube-api-access-x5jpr\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmf48s\" (UID: \"e0581c2a-2a71-41a0-8805-6ba636380ca5\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmf48s"
Sep 10 19:22:10.722194 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.722159    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmf48s"
Sep 10 19:22:10.753114 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.753084    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p66bv\" (UniqueName: \"kubernetes.io/projected/196b3dd9-3349-4687-8d92-9eb83c1a4f5b-kube-api-access-p66bv\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zvpr2\" (UID: \"196b3dd9-3349-4687-8d92-9eb83c1a4f5b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zvpr2"
Sep 10 19:22:10.753254 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.753149    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a517fbb-495a-452c-9f8a-f3886baf3f15-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apvpqv\" (UID: \"9a517fbb-495a-452c-9f8a-f3886baf3f15\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apvpqv"
Sep 10 19:22:10.753254 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.753195    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/196b3dd9-3349-4687-8d92-9eb83c1a4f5b-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zvpr2\" (UID: \"196b3dd9-3349-4687-8d92-9eb83c1a4f5b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zvpr2"
Sep 10 19:22:10.753254 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.753242    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a517fbb-495a-452c-9f8a-f3886baf3f15-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apvpqv\" (UID: \"9a517fbb-495a-452c-9f8a-f3886baf3f15\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apvpqv"
Sep 10 19:22:10.753405 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.753267    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx2l5\" (UniqueName: \"kubernetes.io/projected/9a517fbb-495a-452c-9f8a-f3886baf3f15-kube-api-access-cx2l5\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apvpqv\" (UID: \"9a517fbb-495a-452c-9f8a-f3886baf3f15\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apvpqv"
Sep 10 19:22:10.753405 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.753324    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/196b3dd9-3349-4687-8d92-9eb83c1a4f5b-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zvpr2\" (UID: \"196b3dd9-3349-4687-8d92-9eb83c1a4f5b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zvpr2"
Sep 10 19:22:10.753551 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.753528    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a517fbb-495a-452c-9f8a-f3886baf3f15-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apvpqv\" (UID: \"9a517fbb-495a-452c-9f8a-f3886baf3f15\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apvpqv"
Sep 10 19:22:10.753639 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.753616    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a517fbb-495a-452c-9f8a-f3886baf3f15-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apvpqv\" (UID: \"9a517fbb-495a-452c-9f8a-f3886baf3f15\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apvpqv"
Sep 10 19:22:10.753699 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.753659    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/196b3dd9-3349-4687-8d92-9eb83c1a4f5b-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zvpr2\" (UID: \"196b3dd9-3349-4687-8d92-9eb83c1a4f5b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zvpr2"
Sep 10 19:22:10.753699 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.753667    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/196b3dd9-3349-4687-8d92-9eb83c1a4f5b-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zvpr2\" (UID: \"196b3dd9-3349-4687-8d92-9eb83c1a4f5b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zvpr2"
Sep 10 19:22:10.762548 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.762523    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx2l5\" (UniqueName: \"kubernetes.io/projected/9a517fbb-495a-452c-9f8a-f3886baf3f15-kube-api-access-cx2l5\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apvpqv\" (UID: \"9a517fbb-495a-452c-9f8a-f3886baf3f15\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apvpqv"
Sep 10 19:22:10.762982 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.762965    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p66bv\" (UniqueName: \"kubernetes.io/projected/196b3dd9-3349-4687-8d92-9eb83c1a4f5b-kube-api-access-p66bv\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zvpr2\" (UID: \"196b3dd9-3349-4687-8d92-9eb83c1a4f5b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zvpr2"
Sep 10 19:22:10.823913 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.823877    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apvpqv"
Sep 10 19:22:10.901828 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.901803    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sw8t7"]
Sep 10 19:22:10.903979 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:22:10.903947    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cb12c1c_488e_4826_a87d_e08ab3eff96d.slice/crio-aa1c2e78d676876fb0be7c6ce6f5efede3e7f1e97192271e56a42d6684dd376f WatchSource:0}: Error finding container aa1c2e78d676876fb0be7c6ce6f5efede3e7f1e97192271e56a42d6684dd376f: Status 404 returned error can't find the container with id aa1c2e78d676876fb0be7c6ce6f5efede3e7f1e97192271e56a42d6684dd376f
Sep 10 19:22:10.923102 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:10.923083    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zvpr2"
Sep 10 19:22:11.000535 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:11.000500    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmf48s"]
Sep 10 19:22:11.094383 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:11.094360    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apvpqv"]
Sep 10 19:22:11.096439 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:22:11.096414    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a517fbb_495a_452c_9f8a_f3886baf3f15.slice/crio-c1155f553a39c6a26785a32347532b32676949cdf8dd0b33355517546a1dcd3a WatchSource:0}: Error finding container c1155f553a39c6a26785a32347532b32676949cdf8dd0b33355517546a1dcd3a: Status 404 returned error can't find the container with id c1155f553a39c6a26785a32347532b32676949cdf8dd0b33355517546a1dcd3a
Sep 10 19:22:11.129166 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:11.129099    2579 generic.go:356] "Generic (PLEG): container finished" podID="0cb12c1c-488e-4826-a87d-e08ab3eff96d" containerID="90b070f885401089c01f2a2df5db1870b16a9af4153dadd996ae1c77088041b0" exitCode=0
Sep 10 19:22:11.129300 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:11.129183    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sw8t7" event={"ID":"0cb12c1c-488e-4826-a87d-e08ab3eff96d","Type":"ContainerDied","Data":"90b070f885401089c01f2a2df5db1870b16a9af4153dadd996ae1c77088041b0"}
Sep 10 19:22:11.129300 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:11.129238    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sw8t7" event={"ID":"0cb12c1c-488e-4826-a87d-e08ab3eff96d","Type":"ContainerStarted","Data":"aa1c2e78d676876fb0be7c6ce6f5efede3e7f1e97192271e56a42d6684dd376f"}
Sep 10 19:22:11.130581 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:11.130555    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apvpqv" event={"ID":"9a517fbb-495a-452c-9f8a-f3886baf3f15","Type":"ContainerStarted","Data":"c1155f553a39c6a26785a32347532b32676949cdf8dd0b33355517546a1dcd3a"}
Sep 10 19:22:11.132703 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:11.132655    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmf48s" event={"ID":"e0581c2a-2a71-41a0-8805-6ba636380ca5","Type":"ContainerStarted","Data":"1fafe77f8767b5543a7c73ac78806c1a3012eee36bb86df1c0e8bc24bd114805"}
Sep 10 19:22:11.132819 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:11.132704    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmf48s" event={"ID":"e0581c2a-2a71-41a0-8805-6ba636380ca5","Type":"ContainerStarted","Data":"c0d74a77409c32ba24498c37b8e02f5bdd4997d88209b0e6c637b625118bba3c"}
Sep 10 19:22:11.197739 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:11.197711    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zvpr2"]
Sep 10 19:22:11.231599 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:22:11.231565    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod196b3dd9_3349_4687_8d92_9eb83c1a4f5b.slice/crio-e3b2d0ba67caef878963add47109039051b8db71c61cb6254552abd616fcd237 WatchSource:0}: Error finding container e3b2d0ba67caef878963add47109039051b8db71c61cb6254552abd616fcd237: Status 404 returned error can't find the container with id e3b2d0ba67caef878963add47109039051b8db71c61cb6254552abd616fcd237
Sep 10 19:22:12.140103 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:12.140073    2579 generic.go:356] "Generic (PLEG): container finished" podID="196b3dd9-3349-4687-8d92-9eb83c1a4f5b" containerID="8c6a14754afe61ed7884b2d9c4a8e5e6b9cb4bdab32ed879bb01293f79abb4a6" exitCode=0
Sep 10 19:22:12.140415 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:12.140154    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zvpr2" event={"ID":"196b3dd9-3349-4687-8d92-9eb83c1a4f5b","Type":"ContainerDied","Data":"8c6a14754afe61ed7884b2d9c4a8e5e6b9cb4bdab32ed879bb01293f79abb4a6"}
Sep 10 19:22:12.140415 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:12.140207    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zvpr2" event={"ID":"196b3dd9-3349-4687-8d92-9eb83c1a4f5b","Type":"ContainerStarted","Data":"e3b2d0ba67caef878963add47109039051b8db71c61cb6254552abd616fcd237"}
Sep 10 19:22:12.142411 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:12.142388    2579 generic.go:356] "Generic (PLEG): container finished" podID="0cb12c1c-488e-4826-a87d-e08ab3eff96d" containerID="cfbef89dbf299f1920e8921e8fdb39f18638ff7b5fe71d08a6100fd776467932" exitCode=0
Sep 10 19:22:12.142513 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:12.142468    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sw8t7" event={"ID":"0cb12c1c-488e-4826-a87d-e08ab3eff96d","Type":"ContainerDied","Data":"cfbef89dbf299f1920e8921e8fdb39f18638ff7b5fe71d08a6100fd776467932"}
Sep 10 19:22:12.144373 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:12.144353    2579 generic.go:356] "Generic (PLEG): container finished" podID="9a517fbb-495a-452c-9f8a-f3886baf3f15" containerID="21ef4df5ac24a426d98bd571663ecb460494b1b6a67543957f0723d845c03410" exitCode=0
Sep 10 19:22:12.144483 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:12.144373    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apvpqv" event={"ID":"9a517fbb-495a-452c-9f8a-f3886baf3f15","Type":"ContainerDied","Data":"21ef4df5ac24a426d98bd571663ecb460494b1b6a67543957f0723d845c03410"}
Sep 10 19:22:12.146200 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:12.146179    2579 generic.go:356] "Generic (PLEG): container finished" podID="e0581c2a-2a71-41a0-8805-6ba636380ca5" containerID="1fafe77f8767b5543a7c73ac78806c1a3012eee36bb86df1c0e8bc24bd114805" exitCode=0
Sep 10 19:22:12.146278 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:12.146226    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmf48s" event={"ID":"e0581c2a-2a71-41a0-8805-6ba636380ca5","Type":"ContainerDied","Data":"1fafe77f8767b5543a7c73ac78806c1a3012eee36bb86df1c0e8bc24bd114805"}
Sep 10 19:22:13.156263 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:13.156236    2579 generic.go:356] "Generic (PLEG): container finished" podID="0cb12c1c-488e-4826-a87d-e08ab3eff96d" containerID="94de9193e6dc998f3f5d925b12011f05ae3321c6cd8116a927a85c33c5b71a6b" exitCode=0
Sep 10 19:22:13.156693 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:13.156317    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sw8t7" event={"ID":"0cb12c1c-488e-4826-a87d-e08ab3eff96d","Type":"ContainerDied","Data":"94de9193e6dc998f3f5d925b12011f05ae3321c6cd8116a927a85c33c5b71a6b"}
Sep 10 19:22:14.169053 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:14.169018    2579 generic.go:356] "Generic (PLEG): container finished" podID="9a517fbb-495a-452c-9f8a-f3886baf3f15" containerID="325eca014caddb035fff800715d341c5a4ff0d0e0d73d34218fc709fdc25435c" exitCode=0
Sep 10 19:22:14.169456 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:14.169098    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apvpqv" event={"ID":"9a517fbb-495a-452c-9f8a-f3886baf3f15","Type":"ContainerDied","Data":"325eca014caddb035fff800715d341c5a4ff0d0e0d73d34218fc709fdc25435c"}
Sep 10 19:22:14.171209 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:14.171189    2579 generic.go:356] "Generic (PLEG): container finished" podID="e0581c2a-2a71-41a0-8805-6ba636380ca5" containerID="b29f7546d816477a86d204c8e255f27a12672b394b149e0a28a9d843dbb95c60" exitCode=0
Sep 10 19:22:14.171306 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:14.171268    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmf48s" event={"ID":"e0581c2a-2a71-41a0-8805-6ba636380ca5","Type":"ContainerDied","Data":"b29f7546d816477a86d204c8e255f27a12672b394b149e0a28a9d843dbb95c60"}
Sep 10 19:22:14.319614 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:14.319586    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sw8t7"
Sep 10 19:22:14.385085 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:14.385062    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0cb12c1c-488e-4826-a87d-e08ab3eff96d-util\") pod \"0cb12c1c-488e-4826-a87d-e08ab3eff96d\" (UID: \"0cb12c1c-488e-4826-a87d-e08ab3eff96d\") "
Sep 10 19:22:14.385185 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:14.385122    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0cb12c1c-488e-4826-a87d-e08ab3eff96d-bundle\") pod \"0cb12c1c-488e-4826-a87d-e08ab3eff96d\" (UID: \"0cb12c1c-488e-4826-a87d-e08ab3eff96d\") "
Sep 10 19:22:14.385185 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:14.385170    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-48bh4\" (UniqueName: \"kubernetes.io/projected/0cb12c1c-488e-4826-a87d-e08ab3eff96d-kube-api-access-48bh4\") pod \"0cb12c1c-488e-4826-a87d-e08ab3eff96d\" (UID: \"0cb12c1c-488e-4826-a87d-e08ab3eff96d\") "
Sep 10 19:22:14.385681 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:14.385631    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cb12c1c-488e-4826-a87d-e08ab3eff96d-bundle" (OuterVolumeSpecName: "bundle") pod "0cb12c1c-488e-4826-a87d-e08ab3eff96d" (UID: "0cb12c1c-488e-4826-a87d-e08ab3eff96d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:22:14.387684 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:14.387665    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cb12c1c-488e-4826-a87d-e08ab3eff96d-kube-api-access-48bh4" (OuterVolumeSpecName: "kube-api-access-48bh4") pod "0cb12c1c-488e-4826-a87d-e08ab3eff96d" (UID: "0cb12c1c-488e-4826-a87d-e08ab3eff96d"). InnerVolumeSpecName "kube-api-access-48bh4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:22:14.391274 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:14.391249    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cb12c1c-488e-4826-a87d-e08ab3eff96d-util" (OuterVolumeSpecName: "util") pod "0cb12c1c-488e-4826-a87d-e08ab3eff96d" (UID: "0cb12c1c-488e-4826-a87d-e08ab3eff96d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:22:14.486670 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:14.486593    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0cb12c1c-488e-4826-a87d-e08ab3eff96d-util\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:22:14.486670 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:14.486616    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0cb12c1c-488e-4826-a87d-e08ab3eff96d-bundle\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:22:14.486670 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:14.486626    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48bh4\" (UniqueName: \"kubernetes.io/projected/0cb12c1c-488e-4826-a87d-e08ab3eff96d-kube-api-access-48bh4\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:22:15.177888 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:15.177858    2579 generic.go:356] "Generic (PLEG): container finished" podID="196b3dd9-3349-4687-8d92-9eb83c1a4f5b" containerID="dfcb8d63ec8bfc811ee019dd466464b805ce88324f2a0db69c77e1ed8fdb03e6" exitCode=0
Sep 10 19:22:15.178296 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:15.177949    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zvpr2" event={"ID":"196b3dd9-3349-4687-8d92-9eb83c1a4f5b","Type":"ContainerDied","Data":"dfcb8d63ec8bfc811ee019dd466464b805ce88324f2a0db69c77e1ed8fdb03e6"}
Sep 10 19:22:15.180098 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:15.180076    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sw8t7"
Sep 10 19:22:15.180199 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:15.180110    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6sw8t7" event={"ID":"0cb12c1c-488e-4826-a87d-e08ab3eff96d","Type":"ContainerDied","Data":"aa1c2e78d676876fb0be7c6ce6f5efede3e7f1e97192271e56a42d6684dd376f"}
Sep 10 19:22:15.180199 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:15.180154    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aa1c2e78d676876fb0be7c6ce6f5efede3e7f1e97192271e56a42d6684dd376f"
Sep 10 19:22:15.182443 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:15.182420    2579 generic.go:356] "Generic (PLEG): container finished" podID="9a517fbb-495a-452c-9f8a-f3886baf3f15" containerID="168417dbbf511793be7e8732ef7defed6c96b4aa954e1ea182ee17192ddd4160" exitCode=0
Sep 10 19:22:15.182546 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:15.182494    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apvpqv" event={"ID":"9a517fbb-495a-452c-9f8a-f3886baf3f15","Type":"ContainerDied","Data":"168417dbbf511793be7e8732ef7defed6c96b4aa954e1ea182ee17192ddd4160"}
Sep 10 19:22:15.184796 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:15.184773    2579 generic.go:356] "Generic (PLEG): container finished" podID="e0581c2a-2a71-41a0-8805-6ba636380ca5" containerID="697b6ed83f4fddc24548fb092dd753add904d8887ade61789b95679f6404ddd5" exitCode=0
Sep 10 19:22:15.184889 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:15.184869    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmf48s" event={"ID":"e0581c2a-2a71-41a0-8805-6ba636380ca5","Type":"ContainerDied","Data":"697b6ed83f4fddc24548fb092dd753add904d8887ade61789b95679f6404ddd5"}
Sep 10 19:22:16.191817 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:16.191785    2579 generic.go:356] "Generic (PLEG): container finished" podID="196b3dd9-3349-4687-8d92-9eb83c1a4f5b" containerID="3c5ec4995c206ceb530cea4ee123f4f61a839feb77eff4169f0f61fc02c3132b" exitCode=0
Sep 10 19:22:16.192301 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:16.191865    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zvpr2" event={"ID":"196b3dd9-3349-4687-8d92-9eb83c1a4f5b","Type":"ContainerDied","Data":"3c5ec4995c206ceb530cea4ee123f4f61a839feb77eff4169f0f61fc02c3132b"}
Sep 10 19:22:16.347055 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:16.347028    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmf48s"
Sep 10 19:22:16.374654 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:16.374635    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apvpqv"
Sep 10 19:22:16.424027 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:16.423994    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x5jpr\" (UniqueName: \"kubernetes.io/projected/e0581c2a-2a71-41a0-8805-6ba636380ca5-kube-api-access-x5jpr\") pod \"e0581c2a-2a71-41a0-8805-6ba636380ca5\" (UID: \"e0581c2a-2a71-41a0-8805-6ba636380ca5\") "
Sep 10 19:22:16.424196 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:16.424112    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0581c2a-2a71-41a0-8805-6ba636380ca5-util\") pod \"e0581c2a-2a71-41a0-8805-6ba636380ca5\" (UID: \"e0581c2a-2a71-41a0-8805-6ba636380ca5\") "
Sep 10 19:22:16.424196 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:16.424151    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0581c2a-2a71-41a0-8805-6ba636380ca5-bundle\") pod \"e0581c2a-2a71-41a0-8805-6ba636380ca5\" (UID: \"e0581c2a-2a71-41a0-8805-6ba636380ca5\") "
Sep 10 19:22:16.424684 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:16.424649    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0581c2a-2a71-41a0-8805-6ba636380ca5-bundle" (OuterVolumeSpecName: "bundle") pod "e0581c2a-2a71-41a0-8805-6ba636380ca5" (UID: "e0581c2a-2a71-41a0-8805-6ba636380ca5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:22:16.426250 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:16.426228    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0581c2a-2a71-41a0-8805-6ba636380ca5-kube-api-access-x5jpr" (OuterVolumeSpecName: "kube-api-access-x5jpr") pod "e0581c2a-2a71-41a0-8805-6ba636380ca5" (UID: "e0581c2a-2a71-41a0-8805-6ba636380ca5"). InnerVolumeSpecName "kube-api-access-x5jpr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:22:16.432247 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:16.432212    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0581c2a-2a71-41a0-8805-6ba636380ca5-util" (OuterVolumeSpecName: "util") pod "e0581c2a-2a71-41a0-8805-6ba636380ca5" (UID: "e0581c2a-2a71-41a0-8805-6ba636380ca5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:22:16.525088 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:16.525021    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a517fbb-495a-452c-9f8a-f3886baf3f15-util\") pod \"9a517fbb-495a-452c-9f8a-f3886baf3f15\" (UID: \"9a517fbb-495a-452c-9f8a-f3886baf3f15\") "
Sep 10 19:22:16.525088 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:16.525065    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a517fbb-495a-452c-9f8a-f3886baf3f15-bundle\") pod \"9a517fbb-495a-452c-9f8a-f3886baf3f15\" (UID: \"9a517fbb-495a-452c-9f8a-f3886baf3f15\") "
Sep 10 19:22:16.525247 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:16.525163    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cx2l5\" (UniqueName: \"kubernetes.io/projected/9a517fbb-495a-452c-9f8a-f3886baf3f15-kube-api-access-cx2l5\") pod \"9a517fbb-495a-452c-9f8a-f3886baf3f15\" (UID: \"9a517fbb-495a-452c-9f8a-f3886baf3f15\") "
Sep 10 19:22:16.525308 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:16.525297    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x5jpr\" (UniqueName: \"kubernetes.io/projected/e0581c2a-2a71-41a0-8805-6ba636380ca5-kube-api-access-x5jpr\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:22:16.525346 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:16.525309    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0581c2a-2a71-41a0-8805-6ba636380ca5-util\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:22:16.525346 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:16.525318    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0581c2a-2a71-41a0-8805-6ba636380ca5-bundle\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:22:16.525659 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:16.525629    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a517fbb-495a-452c-9f8a-f3886baf3f15-bundle" (OuterVolumeSpecName: "bundle") pod "9a517fbb-495a-452c-9f8a-f3886baf3f15" (UID: "9a517fbb-495a-452c-9f8a-f3886baf3f15"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:22:16.527306 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:16.527279    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a517fbb-495a-452c-9f8a-f3886baf3f15-kube-api-access-cx2l5" (OuterVolumeSpecName: "kube-api-access-cx2l5") pod "9a517fbb-495a-452c-9f8a-f3886baf3f15" (UID: "9a517fbb-495a-452c-9f8a-f3886baf3f15"). InnerVolumeSpecName "kube-api-access-cx2l5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:22:16.625735 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:16.625704    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a517fbb-495a-452c-9f8a-f3886baf3f15-bundle\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:22:16.625735 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:16.625730    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cx2l5\" (UniqueName: \"kubernetes.io/projected/9a517fbb-495a-452c-9f8a-f3886baf3f15-kube-api-access-cx2l5\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:22:16.642338 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:16.642298    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a517fbb-495a-452c-9f8a-f3886baf3f15-util" (OuterVolumeSpecName: "util") pod "9a517fbb-495a-452c-9f8a-f3886baf3f15" (UID: "9a517fbb-495a-452c-9f8a-f3886baf3f15"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:22:16.726818 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:16.726786    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a517fbb-495a-452c-9f8a-f3886baf3f15-util\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:22:17.198423 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:17.198382    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apvpqv" event={"ID":"9a517fbb-495a-452c-9f8a-f3886baf3f15","Type":"ContainerDied","Data":"c1155f553a39c6a26785a32347532b32676949cdf8dd0b33355517546a1dcd3a"}
Sep 10 19:22:17.198871 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:17.198463    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c1155f553a39c6a26785a32347532b32676949cdf8dd0b33355517546a1dcd3a"
Sep 10 19:22:17.198871 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:17.198396    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apvpqv"
Sep 10 19:22:17.200532 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:17.200513    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmf48s"
Sep 10 19:22:17.200671 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:17.200549    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmf48s" event={"ID":"e0581c2a-2a71-41a0-8805-6ba636380ca5","Type":"ContainerDied","Data":"c0d74a77409c32ba24498c37b8e02f5bdd4997d88209b0e6c637b625118bba3c"}
Sep 10 19:22:17.200671 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:17.200588    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c0d74a77409c32ba24498c37b8e02f5bdd4997d88209b0e6c637b625118bba3c"
Sep 10 19:22:17.319988 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:17.319967    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zvpr2"
Sep 10 19:22:17.432280 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:17.432255    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/196b3dd9-3349-4687-8d92-9eb83c1a4f5b-bundle\") pod \"196b3dd9-3349-4687-8d92-9eb83c1a4f5b\" (UID: \"196b3dd9-3349-4687-8d92-9eb83c1a4f5b\") "
Sep 10 19:22:17.432496 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:17.432335    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/196b3dd9-3349-4687-8d92-9eb83c1a4f5b-util\") pod \"196b3dd9-3349-4687-8d92-9eb83c1a4f5b\" (UID: \"196b3dd9-3349-4687-8d92-9eb83c1a4f5b\") "
Sep 10 19:22:17.432496 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:17.432362    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p66bv\" (UniqueName: \"kubernetes.io/projected/196b3dd9-3349-4687-8d92-9eb83c1a4f5b-kube-api-access-p66bv\") pod \"196b3dd9-3349-4687-8d92-9eb83c1a4f5b\" (UID: \"196b3dd9-3349-4687-8d92-9eb83c1a4f5b\") "
Sep 10 19:22:17.432935 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:17.432891    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/196b3dd9-3349-4687-8d92-9eb83c1a4f5b-bundle" (OuterVolumeSpecName: "bundle") pod "196b3dd9-3349-4687-8d92-9eb83c1a4f5b" (UID: "196b3dd9-3349-4687-8d92-9eb83c1a4f5b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:22:17.434666 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:17.434638    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/196b3dd9-3349-4687-8d92-9eb83c1a4f5b-kube-api-access-p66bv" (OuterVolumeSpecName: "kube-api-access-p66bv") pod "196b3dd9-3349-4687-8d92-9eb83c1a4f5b" (UID: "196b3dd9-3349-4687-8d92-9eb83c1a4f5b"). InnerVolumeSpecName "kube-api-access-p66bv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:22:17.436713 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:17.436687    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/196b3dd9-3349-4687-8d92-9eb83c1a4f5b-util" (OuterVolumeSpecName: "util") pod "196b3dd9-3349-4687-8d92-9eb83c1a4f5b" (UID: "196b3dd9-3349-4687-8d92-9eb83c1a4f5b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:22:17.533706 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:17.533673    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/196b3dd9-3349-4687-8d92-9eb83c1a4f5b-util\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:22:17.533706 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:17.533703    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p66bv\" (UniqueName: \"kubernetes.io/projected/196b3dd9-3349-4687-8d92-9eb83c1a4f5b-kube-api-access-p66bv\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:22:17.533891 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:17.533718    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/196b3dd9-3349-4687-8d92-9eb83c1a4f5b-bundle\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:22:18.207667 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:18.207631    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zvpr2" event={"ID":"196b3dd9-3349-4687-8d92-9eb83c1a4f5b","Type":"ContainerDied","Data":"e3b2d0ba67caef878963add47109039051b8db71c61cb6254552abd616fcd237"}
Sep 10 19:22:18.208044 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:18.207685    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e3b2d0ba67caef878963add47109039051b8db71c61cb6254552abd616fcd237"
Sep 10 19:22:18.208044 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:18.207644    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zvpr2"
Sep 10 19:22:19.469870 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.469839    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-f87fb7bb8-rzrd4"]
Sep 10 19:22:19.470229 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.470156    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cb12c1c-488e-4826-a87d-e08ab3eff96d" containerName="extract"
Sep 10 19:22:19.470229 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.470166    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cb12c1c-488e-4826-a87d-e08ab3eff96d" containerName="extract"
Sep 10 19:22:19.470229 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.470175    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a517fbb-495a-452c-9f8a-f3886baf3f15" containerName="util"
Sep 10 19:22:19.470229 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.470181    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a517fbb-495a-452c-9f8a-f3886baf3f15" containerName="util"
Sep 10 19:22:19.470229 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.470189    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a517fbb-495a-452c-9f8a-f3886baf3f15" containerName="extract"
Sep 10 19:22:19.470229 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.470195    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a517fbb-495a-452c-9f8a-f3886baf3f15" containerName="extract"
Sep 10 19:22:19.470229 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.470201    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a517fbb-495a-452c-9f8a-f3886baf3f15" containerName="pull"
Sep 10 19:22:19.470229 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.470206    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a517fbb-495a-452c-9f8a-f3886baf3f15" containerName="pull"
Sep 10 19:22:19.470229 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.470214    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cb12c1c-488e-4826-a87d-e08ab3eff96d" containerName="pull"
Sep 10 19:22:19.470229 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.470218    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cb12c1c-488e-4826-a87d-e08ab3eff96d" containerName="pull"
Sep 10 19:22:19.470229 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.470225    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0581c2a-2a71-41a0-8805-6ba636380ca5" containerName="extract"
Sep 10 19:22:19.470229 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.470230    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0581c2a-2a71-41a0-8805-6ba636380ca5" containerName="extract"
Sep 10 19:22:19.470596 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.470239    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="196b3dd9-3349-4687-8d92-9eb83c1a4f5b" containerName="util"
Sep 10 19:22:19.470596 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.470244    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="196b3dd9-3349-4687-8d92-9eb83c1a4f5b" containerName="util"
Sep 10 19:22:19.470596 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.470250    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="196b3dd9-3349-4687-8d92-9eb83c1a4f5b" containerName="pull"
Sep 10 19:22:19.470596 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.470255    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="196b3dd9-3349-4687-8d92-9eb83c1a4f5b" containerName="pull"
Sep 10 19:22:19.470596 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.470263    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0581c2a-2a71-41a0-8805-6ba636380ca5" containerName="pull"
Sep 10 19:22:19.470596 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.470268    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0581c2a-2a71-41a0-8805-6ba636380ca5" containerName="pull"
Sep 10 19:22:19.470596 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.470275    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cb12c1c-488e-4826-a87d-e08ab3eff96d" containerName="util"
Sep 10 19:22:19.470596 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.470280    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cb12c1c-488e-4826-a87d-e08ab3eff96d" containerName="util"
Sep 10 19:22:19.470596 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.470285    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0581c2a-2a71-41a0-8805-6ba636380ca5" containerName="util"
Sep 10 19:22:19.470596 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.470290    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0581c2a-2a71-41a0-8805-6ba636380ca5" containerName="util"
Sep 10 19:22:19.470596 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.470295    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="196b3dd9-3349-4687-8d92-9eb83c1a4f5b" containerName="extract"
Sep 10 19:22:19.470596 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.470300    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="196b3dd9-3349-4687-8d92-9eb83c1a4f5b" containerName="extract"
Sep 10 19:22:19.470596 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.470350    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="196b3dd9-3349-4687-8d92-9eb83c1a4f5b" containerName="extract"
Sep 10 19:22:19.470596 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.470357    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cb12c1c-488e-4826-a87d-e08ab3eff96d" containerName="extract"
Sep 10 19:22:19.470596 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.470361    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a517fbb-495a-452c-9f8a-f3886baf3f15" containerName="extract"
Sep 10 19:22:19.470596 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.470369    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0581c2a-2a71-41a0-8805-6ba636380ca5" containerName="extract"
Sep 10 19:22:19.474921 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.474900    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f87fb7bb8-rzrd4"
Sep 10 19:22:19.486964 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.486937    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f87fb7bb8-rzrd4"]
Sep 10 19:22:19.550314 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.550283    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lw67k\" (UniqueName: \"kubernetes.io/projected/303f0788-2826-4563-9f31-b5f843038645-kube-api-access-lw67k\") pod \"console-f87fb7bb8-rzrd4\" (UID: \"303f0788-2826-4563-9f31-b5f843038645\") " pod="openshift-console/console-f87fb7bb8-rzrd4"
Sep 10 19:22:19.550470 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.550334    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/303f0788-2826-4563-9f31-b5f843038645-console-config\") pod \"console-f87fb7bb8-rzrd4\" (UID: \"303f0788-2826-4563-9f31-b5f843038645\") " pod="openshift-console/console-f87fb7bb8-rzrd4"
Sep 10 19:22:19.550470 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.550361    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/303f0788-2826-4563-9f31-b5f843038645-service-ca\") pod \"console-f87fb7bb8-rzrd4\" (UID: \"303f0788-2826-4563-9f31-b5f843038645\") " pod="openshift-console/console-f87fb7bb8-rzrd4"
Sep 10 19:22:19.550470 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.550374    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/303f0788-2826-4563-9f31-b5f843038645-trusted-ca-bundle\") pod \"console-f87fb7bb8-rzrd4\" (UID: \"303f0788-2826-4563-9f31-b5f843038645\") " pod="openshift-console/console-f87fb7bb8-rzrd4"
Sep 10 19:22:19.550470 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.550413    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/303f0788-2826-4563-9f31-b5f843038645-oauth-serving-cert\") pod \"console-f87fb7bb8-rzrd4\" (UID: \"303f0788-2826-4563-9f31-b5f843038645\") " pod="openshift-console/console-f87fb7bb8-rzrd4"
Sep 10 19:22:19.550470 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.550451    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/303f0788-2826-4563-9f31-b5f843038645-console-oauth-config\") pod \"console-f87fb7bb8-rzrd4\" (UID: \"303f0788-2826-4563-9f31-b5f843038645\") " pod="openshift-console/console-f87fb7bb8-rzrd4"
Sep 10 19:22:19.550633 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.550531    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/303f0788-2826-4563-9f31-b5f843038645-console-serving-cert\") pod \"console-f87fb7bb8-rzrd4\" (UID: \"303f0788-2826-4563-9f31-b5f843038645\") " pod="openshift-console/console-f87fb7bb8-rzrd4"
Sep 10 19:22:19.651106 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.651075    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/303f0788-2826-4563-9f31-b5f843038645-service-ca\") pod \"console-f87fb7bb8-rzrd4\" (UID: \"303f0788-2826-4563-9f31-b5f843038645\") " pod="openshift-console/console-f87fb7bb8-rzrd4"
Sep 10 19:22:19.651106 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.651109    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/303f0788-2826-4563-9f31-b5f843038645-trusted-ca-bundle\") pod \"console-f87fb7bb8-rzrd4\" (UID: \"303f0788-2826-4563-9f31-b5f843038645\") " pod="openshift-console/console-f87fb7bb8-rzrd4"
Sep 10 19:22:19.651317 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.651129    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/303f0788-2826-4563-9f31-b5f843038645-oauth-serving-cert\") pod \"console-f87fb7bb8-rzrd4\" (UID: \"303f0788-2826-4563-9f31-b5f843038645\") " pod="openshift-console/console-f87fb7bb8-rzrd4"
Sep 10 19:22:19.651317 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.651162    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/303f0788-2826-4563-9f31-b5f843038645-console-oauth-config\") pod \"console-f87fb7bb8-rzrd4\" (UID: \"303f0788-2826-4563-9f31-b5f843038645\") " pod="openshift-console/console-f87fb7bb8-rzrd4"
Sep 10 19:22:19.651317 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.651206    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/303f0788-2826-4563-9f31-b5f843038645-console-serving-cert\") pod \"console-f87fb7bb8-rzrd4\" (UID: \"303f0788-2826-4563-9f31-b5f843038645\") " pod="openshift-console/console-f87fb7bb8-rzrd4"
Sep 10 19:22:19.651317 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.651226    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lw67k\" (UniqueName: \"kubernetes.io/projected/303f0788-2826-4563-9f31-b5f843038645-kube-api-access-lw67k\") pod \"console-f87fb7bb8-rzrd4\" (UID: \"303f0788-2826-4563-9f31-b5f843038645\") " pod="openshift-console/console-f87fb7bb8-rzrd4"
Sep 10 19:22:19.651317 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.651279    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/303f0788-2826-4563-9f31-b5f843038645-console-config\") pod \"console-f87fb7bb8-rzrd4\" (UID: \"303f0788-2826-4563-9f31-b5f843038645\") " pod="openshift-console/console-f87fb7bb8-rzrd4"
Sep 10 19:22:19.651986 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.651943    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/303f0788-2826-4563-9f31-b5f843038645-oauth-serving-cert\") pod \"console-f87fb7bb8-rzrd4\" (UID: \"303f0788-2826-4563-9f31-b5f843038645\") " pod="openshift-console/console-f87fb7bb8-rzrd4"
Sep 10 19:22:19.652313 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.652004    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/303f0788-2826-4563-9f31-b5f843038645-service-ca\") pod \"console-f87fb7bb8-rzrd4\" (UID: \"303f0788-2826-4563-9f31-b5f843038645\") " pod="openshift-console/console-f87fb7bb8-rzrd4"
Sep 10 19:22:19.652313 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.652060    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/303f0788-2826-4563-9f31-b5f843038645-trusted-ca-bundle\") pod \"console-f87fb7bb8-rzrd4\" (UID: \"303f0788-2826-4563-9f31-b5f843038645\") " pod="openshift-console/console-f87fb7bb8-rzrd4"
Sep 10 19:22:19.652485 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.652055    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/303f0788-2826-4563-9f31-b5f843038645-console-config\") pod \"console-f87fb7bb8-rzrd4\" (UID: \"303f0788-2826-4563-9f31-b5f843038645\") " pod="openshift-console/console-f87fb7bb8-rzrd4"
Sep 10 19:22:19.653822 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.653798    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/303f0788-2826-4563-9f31-b5f843038645-console-serving-cert\") pod \"console-f87fb7bb8-rzrd4\" (UID: \"303f0788-2826-4563-9f31-b5f843038645\") " pod="openshift-console/console-f87fb7bb8-rzrd4"
Sep 10 19:22:19.653953 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.653932    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/303f0788-2826-4563-9f31-b5f843038645-console-oauth-config\") pod \"console-f87fb7bb8-rzrd4\" (UID: \"303f0788-2826-4563-9f31-b5f843038645\") " pod="openshift-console/console-f87fb7bb8-rzrd4"
Sep 10 19:22:19.658952 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.658930    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lw67k\" (UniqueName: \"kubernetes.io/projected/303f0788-2826-4563-9f31-b5f843038645-kube-api-access-lw67k\") pod \"console-f87fb7bb8-rzrd4\" (UID: \"303f0788-2826-4563-9f31-b5f843038645\") " pod="openshift-console/console-f87fb7bb8-rzrd4"
Sep 10 19:22:19.786469 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:19.786383    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f87fb7bb8-rzrd4"
Sep 10 19:22:20.072031 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:20.072003    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f87fb7bb8-rzrd4"]
Sep 10 19:22:20.074623 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:22:20.074600    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod303f0788_2826_4563_9f31_b5f843038645.slice/crio-217702cade699655f494259d14a1576498d4a32a0f352c2ff1fd75355da62436 WatchSource:0}: Error finding container 217702cade699655f494259d14a1576498d4a32a0f352c2ff1fd75355da62436: Status 404 returned error can't find the container with id 217702cade699655f494259d14a1576498d4a32a0f352c2ff1fd75355da62436
Sep 10 19:22:20.220715 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:20.220680    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f87fb7bb8-rzrd4" event={"ID":"303f0788-2826-4563-9f31-b5f843038645","Type":"ContainerStarted","Data":"588145d4532ebabfd9ce3d7eebcf8c90ad18a7e730be405caa3c529923a6e9f4"}
Sep 10 19:22:20.220929 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:20.220732    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f87fb7bb8-rzrd4" event={"ID":"303f0788-2826-4563-9f31-b5f843038645","Type":"ContainerStarted","Data":"217702cade699655f494259d14a1576498d4a32a0f352c2ff1fd75355da62436"}
Sep 10 19:22:20.242067 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:20.242018    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-f87fb7bb8-rzrd4" podStartSLOduration=1.242002478 podStartE2EDuration="1.242002478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:22:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:22:20.240022904 +0000 UTC m=+395.459871386" watchObservedRunningTime="2026-09-10 19:22:20.242002478 +0000 UTC m=+395.461851015"
Sep 10 19:22:28.925000 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:28.924957    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-r6nh4"]
Sep 10 19:22:28.929051 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:28.929029    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-r6nh4"
Sep 10 19:22:28.931088 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:28.931070    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-2r8zh\""
Sep 10 19:22:28.931088 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:28.931079    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 19:22:28.931396 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:28.931382    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 19:22:28.941006 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:28.940982    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-r6nh4"]
Sep 10 19:22:29.035832 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:29.035793    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2a3d7cbe-988c-4e0c-a73a-bc84272fada5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-r6nh4\" (UID: \"2a3d7cbe-988c-4e0c-a73a-bc84272fada5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-r6nh4"
Sep 10 19:22:29.035832 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:29.035840    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trjzr\" (UniqueName: \"kubernetes.io/projected/2a3d7cbe-988c-4e0c-a73a-bc84272fada5-kube-api-access-trjzr\") pod \"kuadrant-operator-controller-manager-76f8f686bf-r6nh4\" (UID: \"2a3d7cbe-988c-4e0c-a73a-bc84272fada5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-r6nh4"
Sep 10 19:22:29.137168 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:29.137127    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2a3d7cbe-988c-4e0c-a73a-bc84272fada5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-r6nh4\" (UID: \"2a3d7cbe-988c-4e0c-a73a-bc84272fada5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-r6nh4"
Sep 10 19:22:29.137168 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:29.137174    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trjzr\" (UniqueName: \"kubernetes.io/projected/2a3d7cbe-988c-4e0c-a73a-bc84272fada5-kube-api-access-trjzr\") pod \"kuadrant-operator-controller-manager-76f8f686bf-r6nh4\" (UID: \"2a3d7cbe-988c-4e0c-a73a-bc84272fada5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-r6nh4"
Sep 10 19:22:29.137580 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:29.137554    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2a3d7cbe-988c-4e0c-a73a-bc84272fada5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-r6nh4\" (UID: \"2a3d7cbe-988c-4e0c-a73a-bc84272fada5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-r6nh4"
Sep 10 19:22:29.146803 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:29.146773    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trjzr\" (UniqueName: \"kubernetes.io/projected/2a3d7cbe-988c-4e0c-a73a-bc84272fada5-kube-api-access-trjzr\") pod \"kuadrant-operator-controller-manager-76f8f686bf-r6nh4\" (UID: \"2a3d7cbe-988c-4e0c-a73a-bc84272fada5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-r6nh4"
Sep 10 19:22:29.240764 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:29.240674    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-r6nh4"
Sep 10 19:22:29.542836 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:22:29.542801    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a3d7cbe_988c_4e0c_a73a_bc84272fada5.slice/crio-b921cde6a157e14c6b8d958ba94da39dabe8d9f3ad78dd8c3c6e38322ee0f7fa WatchSource:0}: Error finding container b921cde6a157e14c6b8d958ba94da39dabe8d9f3ad78dd8c3c6e38322ee0f7fa: Status 404 returned error can't find the container with id b921cde6a157e14c6b8d958ba94da39dabe8d9f3ad78dd8c3c6e38322ee0f7fa
Sep 10 19:22:29.547651 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:29.547628    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-r6nh4"]
Sep 10 19:22:29.787030 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:29.786990    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-f87fb7bb8-rzrd4"
Sep 10 19:22:29.787203 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:29.787055    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-f87fb7bb8-rzrd4"
Sep 10 19:22:29.791854 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:29.791785    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-f87fb7bb8-rzrd4"
Sep 10 19:22:30.273113 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:30.273075    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-r6nh4" event={"ID":"2a3d7cbe-988c-4e0c-a73a-bc84272fada5","Type":"ContainerStarted","Data":"b921cde6a157e14c6b8d958ba94da39dabe8d9f3ad78dd8c3c6e38322ee0f7fa"}
Sep 10 19:22:30.277581 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:30.277554    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-f87fb7bb8-rzrd4"
Sep 10 19:22:30.337741 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:30.337699    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f8ddf84ff-x9544"]
Sep 10 19:22:30.940802 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:30.940706    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zw56j"]
Sep 10 19:22:30.944690 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:30.944662    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zw56j"
Sep 10 19:22:30.953973 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:30.953690    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-h9sct\""
Sep 10 19:22:30.954959 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:30.954939    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 10 19:22:30.957124 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:30.957096    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vt5xf\" (UniqueName: \"kubernetes.io/projected/0c31700a-7f6f-464d-940a-77e756cb8f9b-kube-api-access-vt5xf\") pod \"dns-operator-controller-manager-7b84cd7c9c-zw56j\" (UID: \"0c31700a-7f6f-464d-940a-77e756cb8f9b\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zw56j"
Sep 10 19:22:30.972070 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:30.972038    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zw56j"]
Sep 10 19:22:31.057953 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:31.057917    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt5xf\" (UniqueName: \"kubernetes.io/projected/0c31700a-7f6f-464d-940a-77e756cb8f9b-kube-api-access-vt5xf\") pod \"dns-operator-controller-manager-7b84cd7c9c-zw56j\" (UID: \"0c31700a-7f6f-464d-940a-77e756cb8f9b\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zw56j"
Sep 10 19:22:31.072840 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:31.072807    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vt5xf\" (UniqueName: \"kubernetes.io/projected/0c31700a-7f6f-464d-940a-77e756cb8f9b-kube-api-access-vt5xf\") pod \"dns-operator-controller-manager-7b84cd7c9c-zw56j\" (UID: \"0c31700a-7f6f-464d-940a-77e756cb8f9b\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zw56j"
Sep 10 19:22:31.259777 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:31.259621    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zw56j"
Sep 10 19:22:32.266527 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:32.266500    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zw56j"]
Sep 10 19:22:32.268249 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:22:32.268222    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c31700a_7f6f_464d_940a_77e756cb8f9b.slice/crio-15ce55621e9c3db8af5390838982cf9e1f4b5d2921d378a6b87d59059a766cb5 WatchSource:0}: Error finding container 15ce55621e9c3db8af5390838982cf9e1f4b5d2921d378a6b87d59059a766cb5: Status 404 returned error can't find the container with id 15ce55621e9c3db8af5390838982cf9e1f4b5d2921d378a6b87d59059a766cb5
Sep 10 19:22:32.288642 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:32.288592    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zw56j" event={"ID":"0c31700a-7f6f-464d-940a-77e756cb8f9b","Type":"ContainerStarted","Data":"15ce55621e9c3db8af5390838982cf9e1f4b5d2921d378a6b87d59059a766cb5"}
Sep 10 19:22:33.562917 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:33.562876    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-87q4t"]
Sep 10 19:22:33.567687 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:33.567660    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-87q4t"
Sep 10 19:22:33.570077 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:33.570056    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-hwln2\""
Sep 10 19:22:33.576914 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:33.576887    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-87q4t"]
Sep 10 19:22:33.582381 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:33.582349    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lh9wp\" (UniqueName: \"kubernetes.io/projected/93192a08-7a1c-4d1d-9f9c-943e6078228b-kube-api-access-lh9wp\") pod \"limitador-operator-controller-manager-74495c69f8-87q4t\" (UID: \"93192a08-7a1c-4d1d-9f9c-943e6078228b\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-87q4t"
Sep 10 19:22:33.683230 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:33.683173    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lh9wp\" (UniqueName: \"kubernetes.io/projected/93192a08-7a1c-4d1d-9f9c-943e6078228b-kube-api-access-lh9wp\") pod \"limitador-operator-controller-manager-74495c69f8-87q4t\" (UID: \"93192a08-7a1c-4d1d-9f9c-943e6078228b\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-87q4t"
Sep 10 19:22:33.695616 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:33.695582    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lh9wp\" (UniqueName: \"kubernetes.io/projected/93192a08-7a1c-4d1d-9f9c-943e6078228b-kube-api-access-lh9wp\") pod \"limitador-operator-controller-manager-74495c69f8-87q4t\" (UID: \"93192a08-7a1c-4d1d-9f9c-943e6078228b\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-87q4t"
Sep 10 19:22:33.882094 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:33.882059    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-87q4t"
Sep 10 19:22:34.173372 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:34.173346    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-87q4t"]
Sep 10 19:22:34.175302 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:22:34.175272    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93192a08_7a1c_4d1d_9f9c_943e6078228b.slice/crio-c1f6806506a0b4e38c5ab76d6beb7979f41fc0d9ff9cbc7c7cf82159f62d35a6 WatchSource:0}: Error finding container c1f6806506a0b4e38c5ab76d6beb7979f41fc0d9ff9cbc7c7cf82159f62d35a6: Status 404 returned error can't find the container with id c1f6806506a0b4e38c5ab76d6beb7979f41fc0d9ff9cbc7c7cf82159f62d35a6
Sep 10 19:22:34.305770 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:34.305708    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-87q4t" event={"ID":"93192a08-7a1c-4d1d-9f9c-943e6078228b","Type":"ContainerStarted","Data":"c1f6806506a0b4e38c5ab76d6beb7979f41fc0d9ff9cbc7c7cf82159f62d35a6"}
Sep 10 19:22:34.307844 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:34.307804    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-r6nh4" event={"ID":"2a3d7cbe-988c-4e0c-a73a-bc84272fada5","Type":"ContainerStarted","Data":"cc52d73e0d88ff47eb21e223eec880395afec30d2801db6c7691dd346ae07c34"}
Sep 10 19:22:34.307989 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:34.307930    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-r6nh4"
Sep 10 19:22:34.339569 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:34.339515    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-r6nh4" podStartSLOduration=2.139967193 podStartE2EDuration="6.339499146s" totalImagesPullingTime="4.199531953s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:22:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:22:34.336600335 +0000 UTC m=+409.556448815" watchObservedRunningTime="2026-09-10 19:22:34.339499146 +0000 UTC m=+409.559347627"
Sep 10 19:22:35.316065 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:35.315970    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zw56j" event={"ID":"0c31700a-7f6f-464d-940a-77e756cb8f9b","Type":"ContainerStarted","Data":"fb08c0536ce7a546415910574e991180bf617957670277515068de3942fd03ff"}
Sep 10 19:22:35.316435 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:35.316239    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zw56j"
Sep 10 19:22:35.339225 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:35.339172    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zw56j" podStartSLOduration=2.83605149 podStartE2EDuration="5.339156202s" totalImagesPullingTime="2.503104712s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:22:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:22:35.335306713 +0000 UTC m=+410.555155194" watchObservedRunningTime="2026-09-10 19:22:35.339156202 +0000 UTC m=+410.559004683"
Sep 10 19:22:36.067253 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:36.067142    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-hjhj4"]
Sep 10 19:22:36.071188 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:36.071163    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-hjhj4"
Sep 10 19:22:36.076507 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:36.076484    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-x7wjn\""
Sep 10 19:22:36.090956 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:36.090926    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-hjhj4"]
Sep 10 19:22:36.112919 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:36.112868    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdv6q\" (UniqueName: \"kubernetes.io/projected/3a2b3262-dec1-4ede-9891-b202c0ee6574-kube-api-access-qdv6q\") pod \"authorino-operator-76b478bd78-hjhj4\" (UID: \"3a2b3262-dec1-4ede-9891-b202c0ee6574\") " pod="kuadrant-system/authorino-operator-76b478bd78-hjhj4"
Sep 10 19:22:36.213629 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:36.213590    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdv6q\" (UniqueName: \"kubernetes.io/projected/3a2b3262-dec1-4ede-9891-b202c0ee6574-kube-api-access-qdv6q\") pod \"authorino-operator-76b478bd78-hjhj4\" (UID: \"3a2b3262-dec1-4ede-9891-b202c0ee6574\") " pod="kuadrant-system/authorino-operator-76b478bd78-hjhj4"
Sep 10 19:22:36.222534 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:36.222506    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdv6q\" (UniqueName: \"kubernetes.io/projected/3a2b3262-dec1-4ede-9891-b202c0ee6574-kube-api-access-qdv6q\") pod \"authorino-operator-76b478bd78-hjhj4\" (UID: \"3a2b3262-dec1-4ede-9891-b202c0ee6574\") " pod="kuadrant-system/authorino-operator-76b478bd78-hjhj4"
Sep 10 19:22:36.325655 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:36.325565    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-87q4t" event={"ID":"93192a08-7a1c-4d1d-9f9c-943e6078228b","Type":"ContainerStarted","Data":"2d8af6c0bf033344e61c81a09dbb8e63bf826ef9cbb2aa60da4c6e0e4a506072"}
Sep 10 19:22:36.343664 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:36.343604    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-87q4t" podStartSLOduration=1.85139232 podStartE2EDuration="3.343582833s" totalImagesPullingTime="1.492190513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:22:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:22:36.342695831 +0000 UTC m=+411.562544326" watchObservedRunningTime="2026-09-10 19:22:36.343582833 +0000 UTC m=+411.563431314"
Sep 10 19:22:36.380987 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:36.380950    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-hjhj4"
Sep 10 19:22:36.661736 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:36.661707    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-hjhj4"]
Sep 10 19:22:36.663859 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:22:36.663823    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a2b3262_dec1_4ede_9891_b202c0ee6574.slice/crio-8c98c99e3dee606b1c7a7e04be4e9fb11d61b3a8e84d463401ef3eb837ddc18b WatchSource:0}: Error finding container 8c98c99e3dee606b1c7a7e04be4e9fb11d61b3a8e84d463401ef3eb837ddc18b: Status 404 returned error can't find the container with id 8c98c99e3dee606b1c7a7e04be4e9fb11d61b3a8e84d463401ef3eb837ddc18b
Sep 10 19:22:37.096971 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:37.096890    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-97xw4"]
Sep 10 19:22:37.100298 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:37.100276    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-97xw4"
Sep 10 19:22:37.104318 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:37.104283    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-6txr8\""
Sep 10 19:22:37.104600 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:37.104585    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 10 19:22:37.104736 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:37.104716    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 10 19:22:37.115612 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:37.115585    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-97xw4"]
Sep 10 19:22:37.223393 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:37.223353    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6151b239-2930-4ba6-b5c0-1d12b366a700-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-97xw4\" (UID: \"6151b239-2930-4ba6-b5c0-1d12b366a700\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-97xw4"
Sep 10 19:22:37.223582 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:37.223401    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2xjr\" (UniqueName: \"kubernetes.io/projected/6151b239-2930-4ba6-b5c0-1d12b366a700-kube-api-access-n2xjr\") pod \"kuadrant-console-plugin-fbdd4f445-97xw4\" (UID: \"6151b239-2930-4ba6-b5c0-1d12b366a700\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-97xw4"
Sep 10 19:22:37.223582 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:37.223437    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6151b239-2930-4ba6-b5c0-1d12b366a700-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-97xw4\" (UID: \"6151b239-2930-4ba6-b5c0-1d12b366a700\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-97xw4"
Sep 10 19:22:37.324806 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:37.324767    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6151b239-2930-4ba6-b5c0-1d12b366a700-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-97xw4\" (UID: \"6151b239-2930-4ba6-b5c0-1d12b366a700\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-97xw4"
Sep 10 19:22:37.325003 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:37.324823    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2xjr\" (UniqueName: \"kubernetes.io/projected/6151b239-2930-4ba6-b5c0-1d12b366a700-kube-api-access-n2xjr\") pod \"kuadrant-console-plugin-fbdd4f445-97xw4\" (UID: \"6151b239-2930-4ba6-b5c0-1d12b366a700\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-97xw4"
Sep 10 19:22:37.325003 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:37.324866    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6151b239-2930-4ba6-b5c0-1d12b366a700-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-97xw4\" (UID: \"6151b239-2930-4ba6-b5c0-1d12b366a700\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-97xw4"
Sep 10 19:22:37.325472 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:37.325449    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6151b239-2930-4ba6-b5c0-1d12b366a700-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-97xw4\" (UID: \"6151b239-2930-4ba6-b5c0-1d12b366a700\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-97xw4"
Sep 10 19:22:37.327481 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:37.327462    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6151b239-2930-4ba6-b5c0-1d12b366a700-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-97xw4\" (UID: \"6151b239-2930-4ba6-b5c0-1d12b366a700\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-97xw4"
Sep 10 19:22:37.332709 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:37.332679    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-hjhj4" event={"ID":"3a2b3262-dec1-4ede-9891-b202c0ee6574","Type":"ContainerStarted","Data":"8c98c99e3dee606b1c7a7e04be4e9fb11d61b3a8e84d463401ef3eb837ddc18b"}
Sep 10 19:22:37.332832 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:37.332786    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-87q4t"
Sep 10 19:22:37.333410 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:37.333393    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2xjr\" (UniqueName: \"kubernetes.io/projected/6151b239-2930-4ba6-b5c0-1d12b366a700-kube-api-access-n2xjr\") pod \"kuadrant-console-plugin-fbdd4f445-97xw4\" (UID: \"6151b239-2930-4ba6-b5c0-1d12b366a700\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-97xw4"
Sep 10 19:22:37.410674 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:37.410641    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-97xw4"
Sep 10 19:22:37.691195 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:37.691168    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-97xw4"]
Sep 10 19:22:37.693239 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:22:37.693204    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6151b239_2930_4ba6_b5c0_1d12b366a700.slice/crio-347be1b753819e34021006e78a73a397387311825eb81acc868ec1f6b38fa67e WatchSource:0}: Error finding container 347be1b753819e34021006e78a73a397387311825eb81acc868ec1f6b38fa67e: Status 404 returned error can't find the container with id 347be1b753819e34021006e78a73a397387311825eb81acc868ec1f6b38fa67e
Sep 10 19:22:38.340385 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:38.340352    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-hjhj4" event={"ID":"3a2b3262-dec1-4ede-9891-b202c0ee6574","Type":"ContainerStarted","Data":"7c1987b33270d00a3a3bfca1145731c9be409ad2b150a2fa80162c568f3829ec"}
Sep 10 19:22:38.340817 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:38.340455    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-hjhj4"
Sep 10 19:22:38.344393 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:38.344354    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-97xw4" event={"ID":"6151b239-2930-4ba6-b5c0-1d12b366a700","Type":"ContainerStarted","Data":"347be1b753819e34021006e78a73a397387311825eb81acc868ec1f6b38fa67e"}
Sep 10 19:22:38.371833 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:38.371772    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-hjhj4" podStartSLOduration=0.766692008 podStartE2EDuration="2.371729758s" totalImagesPullingTime="1.60503775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:22:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:22:38.368651498 +0000 UTC m=+413.588499970" watchObservedRunningTime="2026-09-10 19:22:38.371729758 +0000 UTC m=+413.591578238"
Sep 10 19:22:45.318587 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:45.318556    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-r6nh4"
Sep 10 19:22:46.328195 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:46.328168    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zw56j"
Sep 10 19:22:47.573854 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:47.573816    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-r6nh4"]
Sep 10 19:22:47.574278 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:47.573992    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-r6nh4" podUID="2a3d7cbe-988c-4e0c-a73a-bc84272fada5" containerName="manager" containerID="cri-o://cc52d73e0d88ff47eb21e223eec880395afec30d2801db6c7691dd346ae07c34" gracePeriod=2
Sep 10 19:22:47.586241 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:47.586213    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-r6nh4"]
Sep 10 19:22:47.597449 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:22:47.597405    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"2a3d7cbe-988c-4e0c-a73a-bc84272fada5\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-76f8f686bf-r6nh4\": pods \"kuadrant-operator-controller-manager-76f8f686bf-r6nh4\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-r6nh4"
Sep 10 19:22:47.603239 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:47.603211    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lxkl4"]
Sep 10 19:22:47.603596 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:47.603581    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a3d7cbe-988c-4e0c-a73a-bc84272fada5" containerName="manager"
Sep 10 19:22:47.603596 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:47.603597    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a3d7cbe-988c-4e0c-a73a-bc84272fada5" containerName="manager"
Sep 10 19:22:47.603725 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:47.603666    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a3d7cbe-988c-4e0c-a73a-bc84272fada5" containerName="manager"
Sep 10 19:22:47.606859 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:47.606840    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-87q4t"]
Sep 10 19:22:47.606971 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:47.606959    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lxkl4"
Sep 10 19:22:47.607037 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:47.606997    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-87q4t" podUID="93192a08-7a1c-4d1d-9f9c-943e6078228b" containerName="manager" containerID="cri-o://2d8af6c0bf033344e61c81a09dbb8e63bf826ef9cbb2aa60da4c6e0e4a506072" gracePeriod=2
Sep 10 19:22:47.609016 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:22:47.608975    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-r6nh4\" is forbidden: User \"system:node:ip-10-0-142-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-20.ec2.internal' and this object" podUID="2a3d7cbe-988c-4e0c-a73a-bc84272fada5" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-r6nh4"
Sep 10 19:22:47.609178 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:47.609161    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-87q4t"
Sep 10 19:22:47.620638 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:47.620610    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-87q4t"]
Sep 10 19:22:47.623649 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:22:47.623569    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"93192a08-7a1c-4d1d-9f9c-943e6078228b\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-74495c69f8-87q4t\": pods \"limitador-operator-controller-manager-74495c69f8-87q4t\" not found" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-87q4t"
Sep 10 19:22:47.625868 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:47.625848    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lxkl4"]
Sep 10 19:22:47.637605 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:47.637582    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-5svd7"]
Sep 10 19:22:47.637994 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:47.637981    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93192a08-7a1c-4d1d-9f9c-943e6078228b" containerName="manager"
Sep 10 19:22:47.638043 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:47.637996    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="93192a08-7a1c-4d1d-9f9c-943e6078228b" containerName="manager"
Sep 10 19:22:47.638079 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:47.638054    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="93192a08-7a1c-4d1d-9f9c-943e6078228b" containerName="manager"
Sep 10 19:22:47.641071 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:47.641043    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5svd7"
Sep 10 19:22:47.664832 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:47.664786    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-5svd7"]
Sep 10 19:22:47.684100 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:22:47.684058    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-r6nh4\" is forbidden: User \"system:node:ip-10-0-142-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-20.ec2.internal' and this object" podUID="2a3d7cbe-988c-4e0c-a73a-bc84272fada5" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-r6nh4"
Sep 10 19:22:47.685558 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:22:47.685528    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-87q4t\" is forbidden: User \"system:node:ip-10-0-142-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-20.ec2.internal' and this object" podUID="93192a08-7a1c-4d1d-9f9c-943e6078228b" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-87q4t"
Sep 10 19:22:47.720416 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:47.720383    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9tdd\" (UniqueName: \"kubernetes.io/projected/37dfe835-023f-4221-aa07-f8025479ee52-kube-api-access-m9tdd\") pod \"limitador-operator-controller-manager-74495c69f8-5svd7\" (UID: \"37dfe835-023f-4221-aa07-f8025479ee52\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5svd7"
Sep 10 19:22:47.720527 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:47.720457    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9604dc6b-d6fd-475f-8016-a5b65615378d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-lxkl4\" (UID: \"9604dc6b-d6fd-475f-8016-a5b65615378d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lxkl4"
Sep 10 19:22:47.720527 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:47.720489    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgpcg\" (UniqueName: \"kubernetes.io/projected/9604dc6b-d6fd-475f-8016-a5b65615378d-kube-api-access-dgpcg\") pod \"kuadrant-operator-controller-manager-76f8f686bf-lxkl4\" (UID: \"9604dc6b-d6fd-475f-8016-a5b65615378d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lxkl4"
Sep 10 19:22:47.821369 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:47.821330    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9tdd\" (UniqueName: \"kubernetes.io/projected/37dfe835-023f-4221-aa07-f8025479ee52-kube-api-access-m9tdd\") pod \"limitador-operator-controller-manager-74495c69f8-5svd7\" (UID: \"37dfe835-023f-4221-aa07-f8025479ee52\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5svd7"
Sep 10 19:22:47.821555 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:47.821380    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9604dc6b-d6fd-475f-8016-a5b65615378d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-lxkl4\" (UID: \"9604dc6b-d6fd-475f-8016-a5b65615378d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lxkl4"
Sep 10 19:22:47.821555 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:47.821400    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgpcg\" (UniqueName: \"kubernetes.io/projected/9604dc6b-d6fd-475f-8016-a5b65615378d-kube-api-access-dgpcg\") pod \"kuadrant-operator-controller-manager-76f8f686bf-lxkl4\" (UID: \"9604dc6b-d6fd-475f-8016-a5b65615378d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lxkl4"
Sep 10 19:22:47.821848 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:47.821823    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9604dc6b-d6fd-475f-8016-a5b65615378d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-lxkl4\" (UID: \"9604dc6b-d6fd-475f-8016-a5b65615378d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lxkl4"
Sep 10 19:22:47.833570 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:47.833529    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9tdd\" (UniqueName: \"kubernetes.io/projected/37dfe835-023f-4221-aa07-f8025479ee52-kube-api-access-m9tdd\") pod \"limitador-operator-controller-manager-74495c69f8-5svd7\" (UID: \"37dfe835-023f-4221-aa07-f8025479ee52\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5svd7"
Sep 10 19:22:47.833980 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:47.833947    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgpcg\" (UniqueName: \"kubernetes.io/projected/9604dc6b-d6fd-475f-8016-a5b65615378d-kube-api-access-dgpcg\") pod \"kuadrant-operator-controller-manager-76f8f686bf-lxkl4\" (UID: \"9604dc6b-d6fd-475f-8016-a5b65615378d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lxkl4"
Sep 10 19:22:47.860269 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:47.860247    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-r6nh4"
Sep 10 19:22:47.862335 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:47.862318    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-87q4t"
Sep 10 19:22:47.864414 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:22:47.864393    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-r6nh4\" is forbidden: User \"system:node:ip-10-0-142-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-20.ec2.internal' and this object" podUID="2a3d7cbe-988c-4e0c-a73a-bc84272fada5" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-r6nh4"
Sep 10 19:22:47.876581 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:22:47.876555    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-87q4t\" is forbidden: User \"system:node:ip-10-0-142-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-20.ec2.internal' and this object" podUID="93192a08-7a1c-4d1d-9f9c-943e6078228b" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-87q4t"
Sep 10 19:22:47.880917 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:22:47.880898    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-r6nh4\" is forbidden: User \"system:node:ip-10-0-142-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-20.ec2.internal' and this object" podUID="2a3d7cbe-988c-4e0c-a73a-bc84272fada5" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-r6nh4"
Sep 10 19:22:47.886181 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:22:47.886161    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-87q4t\" is forbidden: User \"system:node:ip-10-0-142-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-20.ec2.internal' and this object" podUID="93192a08-7a1c-4d1d-9f9c-943e6078228b" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-87q4t"
Sep 10 19:22:47.922486 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:47.922461    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2a3d7cbe-988c-4e0c-a73a-bc84272fada5-extensions-socket-volume\") pod \"2a3d7cbe-988c-4e0c-a73a-bc84272fada5\" (UID: \"2a3d7cbe-988c-4e0c-a73a-bc84272fada5\") "
Sep 10 19:22:47.922614 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:47.922515    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-trjzr\" (UniqueName: \"kubernetes.io/projected/2a3d7cbe-988c-4e0c-a73a-bc84272fada5-kube-api-access-trjzr\") pod \"2a3d7cbe-988c-4e0c-a73a-bc84272fada5\" (UID: \"2a3d7cbe-988c-4e0c-a73a-bc84272fada5\") "
Sep 10 19:22:47.922614 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:47.922548    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lh9wp\" (UniqueName: \"kubernetes.io/projected/93192a08-7a1c-4d1d-9f9c-943e6078228b-kube-api-access-lh9wp\") pod \"93192a08-7a1c-4d1d-9f9c-943e6078228b\" (UID: \"93192a08-7a1c-4d1d-9f9c-943e6078228b\") "
Sep 10 19:22:47.922847 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:47.922824    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a3d7cbe-988c-4e0c-a73a-bc84272fada5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2a3d7cbe-988c-4e0c-a73a-bc84272fada5" (UID: "2a3d7cbe-988c-4e0c-a73a-bc84272fada5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:22:47.924868 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:47.924836    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93192a08-7a1c-4d1d-9f9c-943e6078228b-kube-api-access-lh9wp" (OuterVolumeSpecName: "kube-api-access-lh9wp") pod "93192a08-7a1c-4d1d-9f9c-943e6078228b" (UID: "93192a08-7a1c-4d1d-9f9c-943e6078228b"). InnerVolumeSpecName "kube-api-access-lh9wp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:22:47.924943 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:47.924862    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a3d7cbe-988c-4e0c-a73a-bc84272fada5-kube-api-access-trjzr" (OuterVolumeSpecName: "kube-api-access-trjzr") pod "2a3d7cbe-988c-4e0c-a73a-bc84272fada5" (UID: "2a3d7cbe-988c-4e0c-a73a-bc84272fada5"). InnerVolumeSpecName "kube-api-access-trjzr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:22:47.993651 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:47.993613    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lxkl4"
Sep 10 19:22:47.998877 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:47.998849    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5svd7"
Sep 10 19:22:48.024171 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:48.024142    2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2a3d7cbe-988c-4e0c-a73a-bc84272fada5-extensions-socket-volume\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:22:48.024171 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:48.024168    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trjzr\" (UniqueName: \"kubernetes.io/projected/2a3d7cbe-988c-4e0c-a73a-bc84272fada5-kube-api-access-trjzr\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:22:48.024358 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:48.024180    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lh9wp\" (UniqueName: \"kubernetes.io/projected/93192a08-7a1c-4d1d-9f9c-943e6078228b-kube-api-access-lh9wp\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:22:48.301240 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:48.301141    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lxkl4"]
Sep 10 19:22:48.304818 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:48.304791    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-5svd7"]
Sep 10 19:22:48.305506 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:22:48.305480    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37dfe835_023f_4221_aa07_f8025479ee52.slice/crio-aafc87ae6abe3cf996bde61cbecfb9bdb6e87db86f93c5116af077ffbd08fb4f WatchSource:0}: Error finding container aafc87ae6abe3cf996bde61cbecfb9bdb6e87db86f93c5116af077ffbd08fb4f: Status 404 returned error can't find the container with id aafc87ae6abe3cf996bde61cbecfb9bdb6e87db86f93c5116af077ffbd08fb4f
Sep 10 19:22:48.312372 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:22:48.311862    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9604dc6b_d6fd_475f_8016_a5b65615378d.slice/crio-e5ce3f3d9c215819f8b897f7c381dbbe4829afc54eb26d2c4e547670db7b3a60 WatchSource:0}: Error finding container e5ce3f3d9c215819f8b897f7c381dbbe4829afc54eb26d2c4e547670db7b3a60: Status 404 returned error can't find the container with id e5ce3f3d9c215819f8b897f7c381dbbe4829afc54eb26d2c4e547670db7b3a60
Sep 10 19:22:48.407262 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:48.407128    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lxkl4" event={"ID":"9604dc6b-d6fd-475f-8016-a5b65615378d","Type":"ContainerStarted","Data":"e5ce3f3d9c215819f8b897f7c381dbbe4829afc54eb26d2c4e547670db7b3a60"}
Sep 10 19:22:48.414646 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:48.410127    2579 generic.go:356] "Generic (PLEG): container finished" podID="2a3d7cbe-988c-4e0c-a73a-bc84272fada5" containerID="cc52d73e0d88ff47eb21e223eec880395afec30d2801db6c7691dd346ae07c34" exitCode=0
Sep 10 19:22:48.414646 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:48.410208    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-r6nh4"
Sep 10 19:22:48.414646 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:48.410272    2579 scope.go:117] "RemoveContainer" containerID="cc52d73e0d88ff47eb21e223eec880395afec30d2801db6c7691dd346ae07c34"
Sep 10 19:22:48.419925 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:22:48.419557    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-r6nh4\" is forbidden: User \"system:node:ip-10-0-142-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-20.ec2.internal' and this object" podUID="2a3d7cbe-988c-4e0c-a73a-bc84272fada5" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-r6nh4"
Sep 10 19:22:48.423346 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:48.423316    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5svd7" event={"ID":"37dfe835-023f-4221-aa07-f8025479ee52","Type":"ContainerStarted","Data":"aafc87ae6abe3cf996bde61cbecfb9bdb6e87db86f93c5116af077ffbd08fb4f"}
Sep 10 19:22:48.425064 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:22:48.425030    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-87q4t\" is forbidden: User \"system:node:ip-10-0-142-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-20.ec2.internal' and this object" podUID="93192a08-7a1c-4d1d-9f9c-943e6078228b" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-87q4t"
Sep 10 19:22:48.426824 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:48.426800    2579 generic.go:356] "Generic (PLEG): container finished" podID="93192a08-7a1c-4d1d-9f9c-943e6078228b" containerID="2d8af6c0bf033344e61c81a09dbb8e63bf826ef9cbb2aa60da4c6e0e4a506072" exitCode=0
Sep 10 19:22:48.426940 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:48.426926    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-87q4t"
Sep 10 19:22:48.427566 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:22:48.427506    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-r6nh4\" is forbidden: User \"system:node:ip-10-0-142-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-20.ec2.internal' and this object" podUID="2a3d7cbe-988c-4e0c-a73a-bc84272fada5" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-r6nh4"
Sep 10 19:22:48.430198 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:22:48.430165    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-87q4t\" is forbidden: User \"system:node:ip-10-0-142-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-20.ec2.internal' and this object" podUID="93192a08-7a1c-4d1d-9f9c-943e6078228b" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-87q4t"
Sep 10 19:22:48.432941 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:22:48.432836    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-r6nh4\" is forbidden: User \"system:node:ip-10-0-142-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-20.ec2.internal' and this object" podUID="2a3d7cbe-988c-4e0c-a73a-bc84272fada5" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-r6nh4"
Sep 10 19:22:48.435277 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:22:48.435129    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-87q4t\" is forbidden: User \"system:node:ip-10-0-142-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-20.ec2.internal' and this object" podUID="93192a08-7a1c-4d1d-9f9c-943e6078228b" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-87q4t"
Sep 10 19:22:48.435277 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:48.435207    2579 scope.go:117] "RemoveContainer" containerID="cc52d73e0d88ff47eb21e223eec880395afec30d2801db6c7691dd346ae07c34"
Sep 10 19:22:48.435623 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:22:48.435503    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc52d73e0d88ff47eb21e223eec880395afec30d2801db6c7691dd346ae07c34\": container with ID starting with cc52d73e0d88ff47eb21e223eec880395afec30d2801db6c7691dd346ae07c34 not found: ID does not exist" containerID="cc52d73e0d88ff47eb21e223eec880395afec30d2801db6c7691dd346ae07c34"
Sep 10 19:22:48.435623 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:48.435534    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc52d73e0d88ff47eb21e223eec880395afec30d2801db6c7691dd346ae07c34"} err="failed to get container status \"cc52d73e0d88ff47eb21e223eec880395afec30d2801db6c7691dd346ae07c34\": rpc error: code = NotFound desc = could not find container \"cc52d73e0d88ff47eb21e223eec880395afec30d2801db6c7691dd346ae07c34\": container with ID starting with cc52d73e0d88ff47eb21e223eec880395afec30d2801db6c7691dd346ae07c34 not found: ID does not exist"
Sep 10 19:22:48.435623 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:48.435555    2579 scope.go:117] "RemoveContainer" containerID="2d8af6c0bf033344e61c81a09dbb8e63bf826ef9cbb2aa60da4c6e0e4a506072"
Sep 10 19:22:48.442920 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:22:48.442786    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-87q4t\" is forbidden: User \"system:node:ip-10-0-142-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-20.ec2.internal' and this object" podUID="93192a08-7a1c-4d1d-9f9c-943e6078228b" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-87q4t"
Sep 10 19:22:48.444465 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:22:48.444431    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-r6nh4\" is forbidden: User \"system:node:ip-10-0-142-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-20.ec2.internal' and this object" podUID="2a3d7cbe-988c-4e0c-a73a-bc84272fada5" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-r6nh4"
Sep 10 19:22:48.450397 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:48.450381    2579 scope.go:117] "RemoveContainer" containerID="2d8af6c0bf033344e61c81a09dbb8e63bf826ef9cbb2aa60da4c6e0e4a506072"
Sep 10 19:22:48.450804 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:22:48.450779    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d8af6c0bf033344e61c81a09dbb8e63bf826ef9cbb2aa60da4c6e0e4a506072\": container with ID starting with 2d8af6c0bf033344e61c81a09dbb8e63bf826ef9cbb2aa60da4c6e0e4a506072 not found: ID does not exist" containerID="2d8af6c0bf033344e61c81a09dbb8e63bf826ef9cbb2aa60da4c6e0e4a506072"
Sep 10 19:22:48.450887 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:48.450812    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d8af6c0bf033344e61c81a09dbb8e63bf826ef9cbb2aa60da4c6e0e4a506072"} err="failed to get container status \"2d8af6c0bf033344e61c81a09dbb8e63bf826ef9cbb2aa60da4c6e0e4a506072\": rpc error: code = NotFound desc = could not find container \"2d8af6c0bf033344e61c81a09dbb8e63bf826ef9cbb2aa60da4c6e0e4a506072\": container with ID starting with 2d8af6c0bf033344e61c81a09dbb8e63bf826ef9cbb2aa60da4c6e0e4a506072 not found: ID does not exist"
Sep 10 19:22:49.243081 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:49.243050    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a3d7cbe-988c-4e0c-a73a-bc84272fada5" path="/var/lib/kubelet/pods/2a3d7cbe-988c-4e0c-a73a-bc84272fada5/volumes"
Sep 10 19:22:49.243605 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:49.243446    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93192a08-7a1c-4d1d-9f9c-943e6078228b" path="/var/lib/kubelet/pods/93192a08-7a1c-4d1d-9f9c-943e6078228b/volumes"
Sep 10 19:22:49.351851 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:49.351589    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-hjhj4"
Sep 10 19:22:49.437044 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:49.437002    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5svd7" event={"ID":"37dfe835-023f-4221-aa07-f8025479ee52","Type":"ContainerStarted","Data":"5c2cac7d05413e3f39d7d5b1c1d4b8bebc099ef34cb44eb56207bfc2f5d151ae"}
Sep 10 19:22:49.437279 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:49.437086    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5svd7"
Sep 10 19:22:49.440825 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:49.440796    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lxkl4" event={"ID":"9604dc6b-d6fd-475f-8016-a5b65615378d","Type":"ContainerStarted","Data":"e396d392adef6a305143f351a65ac18e278ac5ac390071dfab6ad9f763f44621"}
Sep 10 19:22:49.440994 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:49.440870    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lxkl4"
Sep 10 19:22:49.462052 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:49.461991    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5svd7" podStartSLOduration=2.46197081 podStartE2EDuration="2.46197081s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:22:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:22:49.460188238 +0000 UTC m=+424.680036731" watchObservedRunningTime="2026-09-10 19:22:49.46197081 +0000 UTC m=+424.681819299"
Sep 10 19:22:49.483134 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:49.483071    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lxkl4" podStartSLOduration=2.483050939 podStartE2EDuration="2.483050939s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:22:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:22:49.480121294 +0000 UTC m=+424.699969781" watchObservedRunningTime="2026-09-10 19:22:49.483050939 +0000 UTC m=+424.702899421"
Sep 10 19:22:52.462933 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:52.462898    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-97xw4" event={"ID":"6151b239-2930-4ba6-b5c0-1d12b366a700","Type":"ContainerStarted","Data":"3c63b04c0fea4e37ef8fb8ead3518ec4e81baca1bc4d533abdd00106b3269ce1"}
Sep 10 19:22:52.498157 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:52.498107    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-97xw4" podStartSLOduration=1.109713733 podStartE2EDuration="15.498090475s" totalImagesPullingTime="14.388376742s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:22:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:22:52.496796766 +0000 UTC m=+427.716645247" watchObservedRunningTime="2026-09-10 19:22:52.498090475 +0000 UTC m=+427.717938955"
Sep 10 19:22:55.369541 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:55.369495    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7f8ddf84ff-x9544" podUID="ba2017b0-d836-41df-abd0-eef93715adc6" containerName="console" containerID="cri-o://fd5f625d6c67a315c2a44dbedb2fd0fab78b62f804dca17e3659724f21b7eb82" gracePeriod=15
Sep 10 19:22:55.622570 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:55.622509    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f8ddf84ff-x9544"
Sep 10 19:22:55.702445 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:55.702417    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ba2017b0-d836-41df-abd0-eef93715adc6-console-serving-cert\") pod \"ba2017b0-d836-41df-abd0-eef93715adc6\" (UID: \"ba2017b0-d836-41df-abd0-eef93715adc6\") "
Sep 10 19:22:55.702598 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:55.702499    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ba2017b0-d836-41df-abd0-eef93715adc6-service-ca\") pod \"ba2017b0-d836-41df-abd0-eef93715adc6\" (UID: \"ba2017b0-d836-41df-abd0-eef93715adc6\") "
Sep 10 19:22:55.702598 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:55.702528    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ba2017b0-d836-41df-abd0-eef93715adc6-oauth-serving-cert\") pod \"ba2017b0-d836-41df-abd0-eef93715adc6\" (UID: \"ba2017b0-d836-41df-abd0-eef93715adc6\") "
Sep 10 19:22:55.702598 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:55.702580    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ms4rj\" (UniqueName: \"kubernetes.io/projected/ba2017b0-d836-41df-abd0-eef93715adc6-kube-api-access-ms4rj\") pod \"ba2017b0-d836-41df-abd0-eef93715adc6\" (UID: \"ba2017b0-d836-41df-abd0-eef93715adc6\") "
Sep 10 19:22:55.702849 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:55.702603    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ba2017b0-d836-41df-abd0-eef93715adc6-console-config\") pod \"ba2017b0-d836-41df-abd0-eef93715adc6\" (UID: \"ba2017b0-d836-41df-abd0-eef93715adc6\") "
Sep 10 19:22:55.702849 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:55.702635    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ba2017b0-d836-41df-abd0-eef93715adc6-console-oauth-config\") pod \"ba2017b0-d836-41df-abd0-eef93715adc6\" (UID: \"ba2017b0-d836-41df-abd0-eef93715adc6\") "
Sep 10 19:22:55.702849 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:55.702674    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba2017b0-d836-41df-abd0-eef93715adc6-trusted-ca-bundle\") pod \"ba2017b0-d836-41df-abd0-eef93715adc6\" (UID: \"ba2017b0-d836-41df-abd0-eef93715adc6\") "
Sep 10 19:22:55.703140 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:55.703011    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba2017b0-d836-41df-abd0-eef93715adc6-service-ca" (OuterVolumeSpecName: "service-ca") pod "ba2017b0-d836-41df-abd0-eef93715adc6" (UID: "ba2017b0-d836-41df-abd0-eef93715adc6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:22:55.703321 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:55.703293    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba2017b0-d836-41df-abd0-eef93715adc6-console-config" (OuterVolumeSpecName: "console-config") pod "ba2017b0-d836-41df-abd0-eef93715adc6" (UID: "ba2017b0-d836-41df-abd0-eef93715adc6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:22:55.703420 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:55.703393    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba2017b0-d836-41df-abd0-eef93715adc6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ba2017b0-d836-41df-abd0-eef93715adc6" (UID: "ba2017b0-d836-41df-abd0-eef93715adc6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:22:55.703420 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:55.703402    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba2017b0-d836-41df-abd0-eef93715adc6-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ba2017b0-d836-41df-abd0-eef93715adc6" (UID: "ba2017b0-d836-41df-abd0-eef93715adc6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:22:55.704863 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:55.704824    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba2017b0-d836-41df-abd0-eef93715adc6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ba2017b0-d836-41df-abd0-eef93715adc6" (UID: "ba2017b0-d836-41df-abd0-eef93715adc6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:22:55.704977 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:55.704937    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba2017b0-d836-41df-abd0-eef93715adc6-kube-api-access-ms4rj" (OuterVolumeSpecName: "kube-api-access-ms4rj") pod "ba2017b0-d836-41df-abd0-eef93715adc6" (UID: "ba2017b0-d836-41df-abd0-eef93715adc6"). InnerVolumeSpecName "kube-api-access-ms4rj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:22:55.705213 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:55.705191    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba2017b0-d836-41df-abd0-eef93715adc6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ba2017b0-d836-41df-abd0-eef93715adc6" (UID: "ba2017b0-d836-41df-abd0-eef93715adc6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:22:55.803838 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:55.803811    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ba2017b0-d836-41df-abd0-eef93715adc6-service-ca\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:22:55.803838 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:55.803834    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ba2017b0-d836-41df-abd0-eef93715adc6-oauth-serving-cert\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:22:55.803838 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:55.803844    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ms4rj\" (UniqueName: \"kubernetes.io/projected/ba2017b0-d836-41df-abd0-eef93715adc6-kube-api-access-ms4rj\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:22:55.804031 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:55.803853    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ba2017b0-d836-41df-abd0-eef93715adc6-console-config\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:22:55.804031 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:55.803862    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ba2017b0-d836-41df-abd0-eef93715adc6-console-oauth-config\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:22:55.804031 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:55.803869    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba2017b0-d836-41df-abd0-eef93715adc6-trusted-ca-bundle\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:22:55.804031 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:55.803877    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ba2017b0-d836-41df-abd0-eef93715adc6-console-serving-cert\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:22:56.487431 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:56.487395    2579 generic.go:356] "Generic (PLEG): container finished" podID="ba2017b0-d836-41df-abd0-eef93715adc6" containerID="fd5f625d6c67a315c2a44dbedb2fd0fab78b62f804dca17e3659724f21b7eb82" exitCode=0
Sep 10 19:22:56.487915 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:56.487459    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f8ddf84ff-x9544"
Sep 10 19:22:56.487915 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:56.487479    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f8ddf84ff-x9544" event={"ID":"ba2017b0-d836-41df-abd0-eef93715adc6","Type":"ContainerDied","Data":"fd5f625d6c67a315c2a44dbedb2fd0fab78b62f804dca17e3659724f21b7eb82"}
Sep 10 19:22:56.487915 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:56.487533    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f8ddf84ff-x9544" event={"ID":"ba2017b0-d836-41df-abd0-eef93715adc6","Type":"ContainerDied","Data":"bee010a5d9fbcd8b0d8e9de93ac6dedef1cbeb9362a317e80d38c7628c547f9c"}
Sep 10 19:22:56.487915 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:56.487553    2579 scope.go:117] "RemoveContainer" containerID="fd5f625d6c67a315c2a44dbedb2fd0fab78b62f804dca17e3659724f21b7eb82"
Sep 10 19:22:56.497593 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:56.497578    2579 scope.go:117] "RemoveContainer" containerID="fd5f625d6c67a315c2a44dbedb2fd0fab78b62f804dca17e3659724f21b7eb82"
Sep 10 19:22:56.497879 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:22:56.497862    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd5f625d6c67a315c2a44dbedb2fd0fab78b62f804dca17e3659724f21b7eb82\": container with ID starting with fd5f625d6c67a315c2a44dbedb2fd0fab78b62f804dca17e3659724f21b7eb82 not found: ID does not exist" containerID="fd5f625d6c67a315c2a44dbedb2fd0fab78b62f804dca17e3659724f21b7eb82"
Sep 10 19:22:56.497926 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:56.497889    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd5f625d6c67a315c2a44dbedb2fd0fab78b62f804dca17e3659724f21b7eb82"} err="failed to get container status \"fd5f625d6c67a315c2a44dbedb2fd0fab78b62f804dca17e3659724f21b7eb82\": rpc error: code = NotFound desc = could not find container \"fd5f625d6c67a315c2a44dbedb2fd0fab78b62f804dca17e3659724f21b7eb82\": container with ID starting with fd5f625d6c67a315c2a44dbedb2fd0fab78b62f804dca17e3659724f21b7eb82 not found: ID does not exist"
Sep 10 19:22:56.509262 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:56.509237    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f8ddf84ff-x9544"]
Sep 10 19:22:56.512614 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:56.512595    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7f8ddf84ff-x9544"]
Sep 10 19:22:57.240009 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:22:57.239975    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba2017b0-d836-41df-abd0-eef93715adc6" path="/var/lib/kubelet/pods/ba2017b0-d836-41df-abd0-eef93715adc6/volumes"
Sep 10 19:23:00.451257 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:00.451228    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5svd7"
Sep 10 19:23:00.451649 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:00.451301    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lxkl4"
Sep 10 19:23:04.169165 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:04.169135    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lxkl4"]
Sep 10 19:23:04.169700 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:04.169317    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lxkl4" podUID="9604dc6b-d6fd-475f-8016-a5b65615378d" containerName="manager" containerID="cri-o://e396d392adef6a305143f351a65ac18e278ac5ac390071dfab6ad9f763f44621" gracePeriod=10
Sep 10 19:23:04.420636 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:04.420573    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lxkl4"
Sep 10 19:23:04.483925 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:04.483891    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgpcg\" (UniqueName: \"kubernetes.io/projected/9604dc6b-d6fd-475f-8016-a5b65615378d-kube-api-access-dgpcg\") pod \"9604dc6b-d6fd-475f-8016-a5b65615378d\" (UID: \"9604dc6b-d6fd-475f-8016-a5b65615378d\") "
Sep 10 19:23:04.484090 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:04.483948    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9604dc6b-d6fd-475f-8016-a5b65615378d-extensions-socket-volume\") pod \"9604dc6b-d6fd-475f-8016-a5b65615378d\" (UID: \"9604dc6b-d6fd-475f-8016-a5b65615378d\") "
Sep 10 19:23:04.484345 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:04.484324    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9604dc6b-d6fd-475f-8016-a5b65615378d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9604dc6b-d6fd-475f-8016-a5b65615378d" (UID: "9604dc6b-d6fd-475f-8016-a5b65615378d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:23:04.486106 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:04.486079    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9604dc6b-d6fd-475f-8016-a5b65615378d-kube-api-access-dgpcg" (OuterVolumeSpecName: "kube-api-access-dgpcg") pod "9604dc6b-d6fd-475f-8016-a5b65615378d" (UID: "9604dc6b-d6fd-475f-8016-a5b65615378d"). InnerVolumeSpecName "kube-api-access-dgpcg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:23:04.535357 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:04.535322    2579 generic.go:356] "Generic (PLEG): container finished" podID="9604dc6b-d6fd-475f-8016-a5b65615378d" containerID="e396d392adef6a305143f351a65ac18e278ac5ac390071dfab6ad9f763f44621" exitCode=0
Sep 10 19:23:04.535530 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:04.535359    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lxkl4" event={"ID":"9604dc6b-d6fd-475f-8016-a5b65615378d","Type":"ContainerDied","Data":"e396d392adef6a305143f351a65ac18e278ac5ac390071dfab6ad9f763f44621"}
Sep 10 19:23:04.535530 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:04.535383    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lxkl4"
Sep 10 19:23:04.535530 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:04.535460    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lxkl4" event={"ID":"9604dc6b-d6fd-475f-8016-a5b65615378d","Type":"ContainerDied","Data":"e5ce3f3d9c215819f8b897f7c381dbbe4829afc54eb26d2c4e547670db7b3a60"}
Sep 10 19:23:04.535530 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:04.535494    2579 scope.go:117] "RemoveContainer" containerID="e396d392adef6a305143f351a65ac18e278ac5ac390071dfab6ad9f763f44621"
Sep 10 19:23:04.545233 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:04.545218    2579 scope.go:117] "RemoveContainer" containerID="e396d392adef6a305143f351a65ac18e278ac5ac390071dfab6ad9f763f44621"
Sep 10 19:23:04.545505 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:23:04.545488    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e396d392adef6a305143f351a65ac18e278ac5ac390071dfab6ad9f763f44621\": container with ID starting with e396d392adef6a305143f351a65ac18e278ac5ac390071dfab6ad9f763f44621 not found: ID does not exist" containerID="e396d392adef6a305143f351a65ac18e278ac5ac390071dfab6ad9f763f44621"
Sep 10 19:23:04.545562 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:04.545513    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e396d392adef6a305143f351a65ac18e278ac5ac390071dfab6ad9f763f44621"} err="failed to get container status \"e396d392adef6a305143f351a65ac18e278ac5ac390071dfab6ad9f763f44621\": rpc error: code = NotFound desc = could not find container \"e396d392adef6a305143f351a65ac18e278ac5ac390071dfab6ad9f763f44621\": container with ID starting with e396d392adef6a305143f351a65ac18e278ac5ac390071dfab6ad9f763f44621 not found: ID does not exist"
Sep 10 19:23:04.559422 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:04.559397    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lxkl4"]
Sep 10 19:23:04.565868 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:04.565838    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lxkl4"]
Sep 10 19:23:04.584497 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:04.584466    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgpcg\" (UniqueName: \"kubernetes.io/projected/9604dc6b-d6fd-475f-8016-a5b65615378d-kube-api-access-dgpcg\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:23:04.584497 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:04.584491    2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9604dc6b-d6fd-475f-8016-a5b65615378d-extensions-socket-volume\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:23:05.239828 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:05.239795    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9604dc6b-d6fd-475f-8016-a5b65615378d" path="/var/lib/kubelet/pods/9604dc6b-d6fd-475f-8016-a5b65615378d/volumes"
Sep 10 19:23:20.520043 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:20.519947    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"]
Sep 10 19:23:20.520482 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:20.520303    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba2017b0-d836-41df-abd0-eef93715adc6" containerName="console"
Sep 10 19:23:20.520482 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:20.520313    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba2017b0-d836-41df-abd0-eef93715adc6" containerName="console"
Sep 10 19:23:20.520482 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:20.520323    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9604dc6b-d6fd-475f-8016-a5b65615378d" containerName="manager"
Sep 10 19:23:20.520482 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:20.520328    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9604dc6b-d6fd-475f-8016-a5b65615378d" containerName="manager"
Sep 10 19:23:20.520482 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:20.520409    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba2017b0-d836-41df-abd0-eef93715adc6" containerName="console"
Sep 10 19:23:20.520482 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:20.520419    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9604dc6b-d6fd-475f-8016-a5b65615378d" containerName="manager"
Sep 10 19:23:20.526905 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:20.526884    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"
Sep 10 19:23:20.529175 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:20.529152    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-plrwc\""
Sep 10 19:23:20.539665 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:20.539641    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"]
Sep 10 19:23:20.598081 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:20.598043    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fd6f983-387d-48bd-9942-0157de3ce6de-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltgk2\" (UID: \"4fd6f983-387d-48bd-9942-0157de3ce6de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"
Sep 10 19:23:20.598081 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:20.598084    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4fd6f983-387d-48bd-9942-0157de3ce6de-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltgk2\" (UID: \"4fd6f983-387d-48bd-9942-0157de3ce6de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"
Sep 10 19:23:20.598300 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:20.598110    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4fd6f983-387d-48bd-9942-0157de3ce6de-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltgk2\" (UID: \"4fd6f983-387d-48bd-9942-0157de3ce6de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"
Sep 10 19:23:20.598300 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:20.598183    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrnpt\" (UniqueName: \"kubernetes.io/projected/4fd6f983-387d-48bd-9942-0157de3ce6de-kube-api-access-hrnpt\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltgk2\" (UID: \"4fd6f983-387d-48bd-9942-0157de3ce6de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"
Sep 10 19:23:20.598300 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:20.598241    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4fd6f983-387d-48bd-9942-0157de3ce6de-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltgk2\" (UID: \"4fd6f983-387d-48bd-9942-0157de3ce6de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"
Sep 10 19:23:20.598300 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:20.598269    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fd6f983-387d-48bd-9942-0157de3ce6de-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltgk2\" (UID: \"4fd6f983-387d-48bd-9942-0157de3ce6de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"
Sep 10 19:23:20.598300 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:20.598294    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4fd6f983-387d-48bd-9942-0157de3ce6de-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltgk2\" (UID: \"4fd6f983-387d-48bd-9942-0157de3ce6de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"
Sep 10 19:23:20.598511 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:20.598386    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4fd6f983-387d-48bd-9942-0157de3ce6de-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltgk2\" (UID: \"4fd6f983-387d-48bd-9942-0157de3ce6de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"
Sep 10 19:23:20.598511 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:20.598414    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4fd6f983-387d-48bd-9942-0157de3ce6de-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltgk2\" (UID: \"4fd6f983-387d-48bd-9942-0157de3ce6de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"
Sep 10 19:23:20.699738 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:20.699694    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4fd6f983-387d-48bd-9942-0157de3ce6de-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltgk2\" (UID: \"4fd6f983-387d-48bd-9942-0157de3ce6de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"
Sep 10 19:23:20.699968 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:20.699790    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fd6f983-387d-48bd-9942-0157de3ce6de-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltgk2\" (UID: \"4fd6f983-387d-48bd-9942-0157de3ce6de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"
Sep 10 19:23:20.699968 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:20.699892    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4fd6f983-387d-48bd-9942-0157de3ce6de-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltgk2\" (UID: \"4fd6f983-387d-48bd-9942-0157de3ce6de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"
Sep 10 19:23:20.699968 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:20.699924    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4fd6f983-387d-48bd-9942-0157de3ce6de-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltgk2\" (UID: \"4fd6f983-387d-48bd-9942-0157de3ce6de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"
Sep 10 19:23:20.699968 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:20.699963    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrnpt\" (UniqueName: \"kubernetes.io/projected/4fd6f983-387d-48bd-9942-0157de3ce6de-kube-api-access-hrnpt\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltgk2\" (UID: \"4fd6f983-387d-48bd-9942-0157de3ce6de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"
Sep 10 19:23:20.700205 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:20.700015    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4fd6f983-387d-48bd-9942-0157de3ce6de-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltgk2\" (UID: \"4fd6f983-387d-48bd-9942-0157de3ce6de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"
Sep 10 19:23:20.700205 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:20.700051    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fd6f983-387d-48bd-9942-0157de3ce6de-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltgk2\" (UID: \"4fd6f983-387d-48bd-9942-0157de3ce6de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"
Sep 10 19:23:20.700205 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:20.700097    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4fd6f983-387d-48bd-9942-0157de3ce6de-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltgk2\" (UID: \"4fd6f983-387d-48bd-9942-0157de3ce6de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"
Sep 10 19:23:20.700205 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:20.700116    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fd6f983-387d-48bd-9942-0157de3ce6de-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltgk2\" (UID: \"4fd6f983-387d-48bd-9942-0157de3ce6de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"
Sep 10 19:23:20.700205 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:20.700187    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4fd6f983-387d-48bd-9942-0157de3ce6de-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltgk2\" (UID: \"4fd6f983-387d-48bd-9942-0157de3ce6de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"
Sep 10 19:23:20.700449 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:20.700229    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4fd6f983-387d-48bd-9942-0157de3ce6de-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltgk2\" (UID: \"4fd6f983-387d-48bd-9942-0157de3ce6de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"
Sep 10 19:23:20.700449 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:20.700329    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4fd6f983-387d-48bd-9942-0157de3ce6de-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltgk2\" (UID: \"4fd6f983-387d-48bd-9942-0157de3ce6de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"
Sep 10 19:23:20.700449 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:20.700428    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fd6f983-387d-48bd-9942-0157de3ce6de-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltgk2\" (UID: \"4fd6f983-387d-48bd-9942-0157de3ce6de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"
Sep 10 19:23:20.700650 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:20.700627    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4fd6f983-387d-48bd-9942-0157de3ce6de-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltgk2\" (UID: \"4fd6f983-387d-48bd-9942-0157de3ce6de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"
Sep 10 19:23:20.702557 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:20.702535    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4fd6f983-387d-48bd-9942-0157de3ce6de-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltgk2\" (UID: \"4fd6f983-387d-48bd-9942-0157de3ce6de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"
Sep 10 19:23:20.702655 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:20.702617    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4fd6f983-387d-48bd-9942-0157de3ce6de-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltgk2\" (UID: \"4fd6f983-387d-48bd-9942-0157de3ce6de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"
Sep 10 19:23:20.707935 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:20.707913    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4fd6f983-387d-48bd-9942-0157de3ce6de-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltgk2\" (UID: \"4fd6f983-387d-48bd-9942-0157de3ce6de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"
Sep 10 19:23:20.708116 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:20.708098    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrnpt\" (UniqueName: \"kubernetes.io/projected/4fd6f983-387d-48bd-9942-0157de3ce6de-kube-api-access-hrnpt\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltgk2\" (UID: \"4fd6f983-387d-48bd-9942-0157de3ce6de\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"
Sep 10 19:23:20.837463 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:20.837371    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"
Sep 10 19:23:21.127469 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:21.127432    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"]
Sep 10 19:23:21.128616 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:23:21.128589    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fd6f983_387d_48bd_9942_0157de3ce6de.slice/crio-6d62ce6cc874f99ee13001fb98c7e09bf3052d1eaa5e57daed717300a1d57756 WatchSource:0}: Error finding container 6d62ce6cc874f99ee13001fb98c7e09bf3052d1eaa5e57daed717300a1d57756: Status 404 returned error can't find the container with id 6d62ce6cc874f99ee13001fb98c7e09bf3052d1eaa5e57daed717300a1d57756
Sep 10 19:23:21.130732 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:21.130698    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:23:21.130825 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:21.130804    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:23:21.130871 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:21.130857    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:23:21.631132 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:21.631093    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2" event={"ID":"4fd6f983-387d-48bd-9942-0157de3ce6de","Type":"ContainerStarted","Data":"709e9612c2ed39872932b9f9e30ba6f8caedcf81e828943916270d31fa24c029"}
Sep 10 19:23:21.631599 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:21.631147    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2" event={"ID":"4fd6f983-387d-48bd-9942-0157de3ce6de","Type":"ContainerStarted","Data":"6d62ce6cc874f99ee13001fb98c7e09bf3052d1eaa5e57daed717300a1d57756"}
Sep 10 19:23:21.654714 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:21.654666    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2" podStartSLOduration=1.654650567 podStartE2EDuration="1.654650567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:23:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:23:21.652958536 +0000 UTC m=+456.872807024" watchObservedRunningTime="2026-09-10 19:23:21.654650567 +0000 UTC m=+456.874499049"
Sep 10 19:23:21.837898 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:21.837865    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"
Sep 10 19:23:22.643271 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:22.643231    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-vqc2d"]
Sep 10 19:23:22.646290 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:22.646270    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-vqc2d"
Sep 10 19:23:22.648528 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:22.648504    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 10 19:23:22.660057 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:22.660033    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-vqc2d"]
Sep 10 19:23:22.718228 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:22.718196    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c7b7234a-083b-426a-a7c6-4d81aecf4c76-config-file\") pod \"limitador-limitador-8588749977-vqc2d\" (UID: \"c7b7234a-083b-426a-a7c6-4d81aecf4c76\") " pod="kuadrant-system/limitador-limitador-8588749977-vqc2d"
Sep 10 19:23:22.718393 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:22.718347    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mjr9\" (UniqueName: \"kubernetes.io/projected/c7b7234a-083b-426a-a7c6-4d81aecf4c76-kube-api-access-6mjr9\") pod \"limitador-limitador-8588749977-vqc2d\" (UID: \"c7b7234a-083b-426a-a7c6-4d81aecf4c76\") " pod="kuadrant-system/limitador-limitador-8588749977-vqc2d"
Sep 10 19:23:22.746930 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:22.746894    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-vqc2d"]
Sep 10 19:23:22.819585 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:22.819549    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c7b7234a-083b-426a-a7c6-4d81aecf4c76-config-file\") pod \"limitador-limitador-8588749977-vqc2d\" (UID: \"c7b7234a-083b-426a-a7c6-4d81aecf4c76\") " pod="kuadrant-system/limitador-limitador-8588749977-vqc2d"
Sep 10 19:23:22.819738 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:22.819599    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mjr9\" (UniqueName: \"kubernetes.io/projected/c7b7234a-083b-426a-a7c6-4d81aecf4c76-kube-api-access-6mjr9\") pod \"limitador-limitador-8588749977-vqc2d\" (UID: \"c7b7234a-083b-426a-a7c6-4d81aecf4c76\") " pod="kuadrant-system/limitador-limitador-8588749977-vqc2d"
Sep 10 19:23:22.820185 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:22.820165    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c7b7234a-083b-426a-a7c6-4d81aecf4c76-config-file\") pod \"limitador-limitador-8588749977-vqc2d\" (UID: \"c7b7234a-083b-426a-a7c6-4d81aecf4c76\") " pod="kuadrant-system/limitador-limitador-8588749977-vqc2d"
Sep 10 19:23:22.828658 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:22.828634    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mjr9\" (UniqueName: \"kubernetes.io/projected/c7b7234a-083b-426a-a7c6-4d81aecf4c76-kube-api-access-6mjr9\") pod \"limitador-limitador-8588749977-vqc2d\" (UID: \"c7b7234a-083b-426a-a7c6-4d81aecf4c76\") " pod="kuadrant-system/limitador-limitador-8588749977-vqc2d"
Sep 10 19:23:22.843034 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:22.843011    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"
Sep 10 19:23:22.956996 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:22.956904    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-vqc2d"
Sep 10 19:23:23.237232 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:23:23.237194    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7b7234a_083b_426a_a7c6_4d81aecf4c76.slice/crio-ea89a460f911df5c6d28d37dda60861dc338f299df20505a697f891978877627 WatchSource:0}: Error finding container ea89a460f911df5c6d28d37dda60861dc338f299df20505a697f891978877627: Status 404 returned error can't find the container with id ea89a460f911df5c6d28d37dda60861dc338f299df20505a697f891978877627
Sep 10 19:23:23.240799 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:23.240774    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-vqc2d"]
Sep 10 19:23:23.509202 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:23.509123    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-72svm"]
Sep 10 19:23:23.512758 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:23.512728    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-72svm"
Sep 10 19:23:23.515814 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:23.515794    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-vj929\""
Sep 10 19:23:23.521834 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:23.521810    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-72svm"]
Sep 10 19:23:23.526635 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:23.526581    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hj4cb\" (UniqueName: \"kubernetes.io/projected/7a867b26-16ab-47e1-b3d9-3373fc9863d5-kube-api-access-hj4cb\") pod \"authorino-7bcfd8f9cd-72svm\" (UID: \"7a867b26-16ab-47e1-b3d9-3373fc9863d5\") " pod="kuadrant-system/authorino-7bcfd8f9cd-72svm"
Sep 10 19:23:23.627860 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:23.627824    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hj4cb\" (UniqueName: \"kubernetes.io/projected/7a867b26-16ab-47e1-b3d9-3373fc9863d5-kube-api-access-hj4cb\") pod \"authorino-7bcfd8f9cd-72svm\" (UID: \"7a867b26-16ab-47e1-b3d9-3373fc9863d5\") " pod="kuadrant-system/authorino-7bcfd8f9cd-72svm"
Sep 10 19:23:23.642511 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:23.642476    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hj4cb\" (UniqueName: \"kubernetes.io/projected/7a867b26-16ab-47e1-b3d9-3373fc9863d5-kube-api-access-hj4cb\") pod \"authorino-7bcfd8f9cd-72svm\" (UID: \"7a867b26-16ab-47e1-b3d9-3373fc9863d5\") " pod="kuadrant-system/authorino-7bcfd8f9cd-72svm"
Sep 10 19:23:23.644807 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:23.644781    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-vqc2d" event={"ID":"c7b7234a-083b-426a-a7c6-4d81aecf4c76","Type":"ContainerStarted","Data":"ea89a460f911df5c6d28d37dda60861dc338f299df20505a697f891978877627"}
Sep 10 19:23:23.645230 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:23.645069    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"
Sep 10 19:23:23.646113 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:23.646093    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"
Sep 10 19:23:23.824680 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:23.824592    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-72svm"
Sep 10 19:23:24.096815 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:24.096792    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-72svm"]
Sep 10 19:23:24.652422 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:24.652383    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-vqc2d" event={"ID":"c7b7234a-083b-426a-a7c6-4d81aecf4c76","Type":"ContainerStarted","Data":"4331944511e77d9f44f7b14252b79e592f224db5eda9b04ce72f8277b0542823"}
Sep 10 19:23:24.652892 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:24.652527    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-vqc2d"
Sep 10 19:23:24.653641 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:24.653621    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-72svm" event={"ID":"7a867b26-16ab-47e1-b3d9-3373fc9863d5","Type":"ContainerStarted","Data":"99e1f8acee05f61c62a0bb8ea1667954a7a58e82c5741a8a403c767a7cc05017"}
Sep 10 19:23:24.670183 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:24.670127    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-vqc2d" podStartSLOduration=1.371501986 podStartE2EDuration="2.670111444s" totalImagesPullingTime="1.298609458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:23:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:23:24.668388238 +0000 UTC m=+459.888236720" watchObservedRunningTime="2026-09-10 19:23:24.670111444 +0000 UTC m=+459.889959925"
Sep 10 19:23:26.669987 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:26.669953    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-72svm" event={"ID":"7a867b26-16ab-47e1-b3d9-3373fc9863d5","Type":"ContainerStarted","Data":"f62736d37170dc094d779523975fa471883ed3d869f42a3d9a34949f7f7b491c"}
Sep 10 19:23:26.689607 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:26.689550    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-72svm" podStartSLOduration=1.708809773 podStartE2EDuration="3.68952997s" totalImagesPullingTime="1.980720197s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:23:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:23:26.686702226 +0000 UTC m=+461.906550705" watchObservedRunningTime="2026-09-10 19:23:26.68952997 +0000 UTC m=+461.909378453"
Sep 10 19:23:27.534899 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:27.534869    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-72svm"]
Sep 10 19:23:28.689399 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:28.689357    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-72svm" podUID="7a867b26-16ab-47e1-b3d9-3373fc9863d5" containerName="authorino" containerID="cri-o://f62736d37170dc094d779523975fa471883ed3d869f42a3d9a34949f7f7b491c" gracePeriod=30
Sep 10 19:23:28.938903 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:28.938876    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-72svm"
Sep 10 19:23:28.977673 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:28.977588    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hj4cb\" (UniqueName: \"kubernetes.io/projected/7a867b26-16ab-47e1-b3d9-3373fc9863d5-kube-api-access-hj4cb\") pod \"7a867b26-16ab-47e1-b3d9-3373fc9863d5\" (UID: \"7a867b26-16ab-47e1-b3d9-3373fc9863d5\") "
Sep 10 19:23:28.979882 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:28.979852    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a867b26-16ab-47e1-b3d9-3373fc9863d5-kube-api-access-hj4cb" (OuterVolumeSpecName: "kube-api-access-hj4cb") pod "7a867b26-16ab-47e1-b3d9-3373fc9863d5" (UID: "7a867b26-16ab-47e1-b3d9-3373fc9863d5"). InnerVolumeSpecName "kube-api-access-hj4cb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:23:29.078897 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:29.078857    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hj4cb\" (UniqueName: \"kubernetes.io/projected/7a867b26-16ab-47e1-b3d9-3373fc9863d5-kube-api-access-hj4cb\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:23:29.696479 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:29.696447    2579 generic.go:356] "Generic (PLEG): container finished" podID="7a867b26-16ab-47e1-b3d9-3373fc9863d5" containerID="f62736d37170dc094d779523975fa471883ed3d869f42a3d9a34949f7f7b491c" exitCode=0
Sep 10 19:23:29.696947 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:29.696490    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-72svm"
Sep 10 19:23:29.696947 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:29.696502    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-72svm" event={"ID":"7a867b26-16ab-47e1-b3d9-3373fc9863d5","Type":"ContainerDied","Data":"f62736d37170dc094d779523975fa471883ed3d869f42a3d9a34949f7f7b491c"}
Sep 10 19:23:29.696947 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:29.696545    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-72svm" event={"ID":"7a867b26-16ab-47e1-b3d9-3373fc9863d5","Type":"ContainerDied","Data":"99e1f8acee05f61c62a0bb8ea1667954a7a58e82c5741a8a403c767a7cc05017"}
Sep 10 19:23:29.696947 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:29.696561    2579 scope.go:117] "RemoveContainer" containerID="f62736d37170dc094d779523975fa471883ed3d869f42a3d9a34949f7f7b491c"
Sep 10 19:23:29.705219 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:29.705202    2579 scope.go:117] "RemoveContainer" containerID="f62736d37170dc094d779523975fa471883ed3d869f42a3d9a34949f7f7b491c"
Sep 10 19:23:29.705471 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:23:29.705451    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f62736d37170dc094d779523975fa471883ed3d869f42a3d9a34949f7f7b491c\": container with ID starting with f62736d37170dc094d779523975fa471883ed3d869f42a3d9a34949f7f7b491c not found: ID does not exist" containerID="f62736d37170dc094d779523975fa471883ed3d869f42a3d9a34949f7f7b491c"
Sep 10 19:23:29.705534 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:29.705479    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f62736d37170dc094d779523975fa471883ed3d869f42a3d9a34949f7f7b491c"} err="failed to get container status \"f62736d37170dc094d779523975fa471883ed3d869f42a3d9a34949f7f7b491c\": rpc error: code = NotFound desc = could not find container \"f62736d37170dc094d779523975fa471883ed3d869f42a3d9a34949f7f7b491c\": container with ID starting with f62736d37170dc094d779523975fa471883ed3d869f42a3d9a34949f7f7b491c not found: ID does not exist"
Sep 10 19:23:29.712115 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:29.712091    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-72svm"]
Sep 10 19:23:29.719054 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:29.719033    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-72svm"]
Sep 10 19:23:31.238842 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:31.238813    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a867b26-16ab-47e1-b3d9-3373fc9863d5" path="/var/lib/kubelet/pods/7a867b26-16ab-47e1-b3d9-3373fc9863d5/volumes"
Sep 10 19:23:35.662408 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:35.662376    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-vqc2d"
Sep 10 19:23:37.543857 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:37.543821    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-vqc2d"]
Sep 10 19:23:37.544330 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:37.544000    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-vqc2d" podUID="c7b7234a-083b-426a-a7c6-4d81aecf4c76" containerName="limitador" containerID="cri-o://4331944511e77d9f44f7b14252b79e592f224db5eda9b04ce72f8277b0542823" gracePeriod=30
Sep 10 19:23:38.099760 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:38.099717    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-vqc2d"
Sep 10 19:23:38.163553 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:38.163519    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c7b7234a-083b-426a-a7c6-4d81aecf4c76-config-file\") pod \"c7b7234a-083b-426a-a7c6-4d81aecf4c76\" (UID: \"c7b7234a-083b-426a-a7c6-4d81aecf4c76\") "
Sep 10 19:23:38.163730 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:38.163631    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6mjr9\" (UniqueName: \"kubernetes.io/projected/c7b7234a-083b-426a-a7c6-4d81aecf4c76-kube-api-access-6mjr9\") pod \"c7b7234a-083b-426a-a7c6-4d81aecf4c76\" (UID: \"c7b7234a-083b-426a-a7c6-4d81aecf4c76\") "
Sep 10 19:23:38.163930 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:38.163907    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7b7234a-083b-426a-a7c6-4d81aecf4c76-config-file" (OuterVolumeSpecName: "config-file") pod "c7b7234a-083b-426a-a7c6-4d81aecf4c76" (UID: "c7b7234a-083b-426a-a7c6-4d81aecf4c76"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:23:38.165743 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:38.165719    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7b7234a-083b-426a-a7c6-4d81aecf4c76-kube-api-access-6mjr9" (OuterVolumeSpecName: "kube-api-access-6mjr9") pod "c7b7234a-083b-426a-a7c6-4d81aecf4c76" (UID: "c7b7234a-083b-426a-a7c6-4d81aecf4c76"). InnerVolumeSpecName "kube-api-access-6mjr9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:23:38.264857 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:38.264821    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mjr9\" (UniqueName: \"kubernetes.io/projected/c7b7234a-083b-426a-a7c6-4d81aecf4c76-kube-api-access-6mjr9\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:23:38.264857 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:38.264850    2579 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c7b7234a-083b-426a-a7c6-4d81aecf4c76-config-file\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:23:38.748497 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:38.748465    2579 generic.go:356] "Generic (PLEG): container finished" podID="c7b7234a-083b-426a-a7c6-4d81aecf4c76" containerID="4331944511e77d9f44f7b14252b79e592f224db5eda9b04ce72f8277b0542823" exitCode=0
Sep 10 19:23:38.748968 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:38.748513    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-vqc2d"
Sep 10 19:23:38.748968 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:38.748530    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-vqc2d" event={"ID":"c7b7234a-083b-426a-a7c6-4d81aecf4c76","Type":"ContainerDied","Data":"4331944511e77d9f44f7b14252b79e592f224db5eda9b04ce72f8277b0542823"}
Sep 10 19:23:38.748968 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:38.748572    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-vqc2d" event={"ID":"c7b7234a-083b-426a-a7c6-4d81aecf4c76","Type":"ContainerDied","Data":"ea89a460f911df5c6d28d37dda60861dc338f299df20505a697f891978877627"}
Sep 10 19:23:38.748968 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:38.748594    2579 scope.go:117] "RemoveContainer" containerID="4331944511e77d9f44f7b14252b79e592f224db5eda9b04ce72f8277b0542823"
Sep 10 19:23:38.757684 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:38.757665    2579 scope.go:117] "RemoveContainer" containerID="4331944511e77d9f44f7b14252b79e592f224db5eda9b04ce72f8277b0542823"
Sep 10 19:23:38.757977 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:23:38.757958    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4331944511e77d9f44f7b14252b79e592f224db5eda9b04ce72f8277b0542823\": container with ID starting with 4331944511e77d9f44f7b14252b79e592f224db5eda9b04ce72f8277b0542823 not found: ID does not exist" containerID="4331944511e77d9f44f7b14252b79e592f224db5eda9b04ce72f8277b0542823"
Sep 10 19:23:38.758049 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:38.757991    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4331944511e77d9f44f7b14252b79e592f224db5eda9b04ce72f8277b0542823"} err="failed to get container status \"4331944511e77d9f44f7b14252b79e592f224db5eda9b04ce72f8277b0542823\": rpc error: code = NotFound desc = could not find container \"4331944511e77d9f44f7b14252b79e592f224db5eda9b04ce72f8277b0542823\": container with ID starting with 4331944511e77d9f44f7b14252b79e592f224db5eda9b04ce72f8277b0542823 not found: ID does not exist"
Sep 10 19:23:38.769344 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:38.769319    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-vqc2d"]
Sep 10 19:23:38.777692 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:38.777665    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-vqc2d"]
Sep 10 19:23:39.152689 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:39.152658    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-2vq2x"]
Sep 10 19:23:39.153053 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:39.153040    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a867b26-16ab-47e1-b3d9-3373fc9863d5" containerName="authorino"
Sep 10 19:23:39.153111 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:39.153056    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a867b26-16ab-47e1-b3d9-3373fc9863d5" containerName="authorino"
Sep 10 19:23:39.153111 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:39.153082    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7b7234a-083b-426a-a7c6-4d81aecf4c76" containerName="limitador"
Sep 10 19:23:39.153111 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:39.153088    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7b7234a-083b-426a-a7c6-4d81aecf4c76" containerName="limitador"
Sep 10 19:23:39.153209 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:39.153144    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a867b26-16ab-47e1-b3d9-3373fc9863d5" containerName="authorino"
Sep 10 19:23:39.153209 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:39.153152    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7b7234a-083b-426a-a7c6-4d81aecf4c76" containerName="limitador"
Sep 10 19:23:39.157511 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:39.157496    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2vq2x"
Sep 10 19:23:39.159649 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:39.159625    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:23:39.159801 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:39.159658    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 10 19:23:39.160182 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:39.160162    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 19:23:39.160294 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:39.160165    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-dn2q5\""
Sep 10 19:23:39.169560 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:39.169459    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-2vq2x"]
Sep 10 19:23:39.239612 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:39.239581    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7b7234a-083b-426a-a7c6-4d81aecf4c76" path="/var/lib/kubelet/pods/c7b7234a-083b-426a-a7c6-4d81aecf4c76/volumes"
Sep 10 19:23:39.273657 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:39.273625    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/79c2ba5f-2254-4d6e-8f90-697212b36ff1-data\") pod \"postgres-69fb4594cf-2vq2x\" (UID: \"79c2ba5f-2254-4d6e-8f90-697212b36ff1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-2vq2x"
Sep 10 19:23:39.273830 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:39.273794    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7b7tw\" (UniqueName: \"kubernetes.io/projected/79c2ba5f-2254-4d6e-8f90-697212b36ff1-kube-api-access-7b7tw\") pod \"postgres-69fb4594cf-2vq2x\" (UID: \"79c2ba5f-2254-4d6e-8f90-697212b36ff1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-2vq2x"
Sep 10 19:23:39.375097 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:39.375058    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7b7tw\" (UniqueName: \"kubernetes.io/projected/79c2ba5f-2254-4d6e-8f90-697212b36ff1-kube-api-access-7b7tw\") pod \"postgres-69fb4594cf-2vq2x\" (UID: \"79c2ba5f-2254-4d6e-8f90-697212b36ff1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-2vq2x"
Sep 10 19:23:39.375289 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:39.375108    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/79c2ba5f-2254-4d6e-8f90-697212b36ff1-data\") pod \"postgres-69fb4594cf-2vq2x\" (UID: \"79c2ba5f-2254-4d6e-8f90-697212b36ff1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-2vq2x"
Sep 10 19:23:39.375511 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:39.375488    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/79c2ba5f-2254-4d6e-8f90-697212b36ff1-data\") pod \"postgres-69fb4594cf-2vq2x\" (UID: \"79c2ba5f-2254-4d6e-8f90-697212b36ff1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-2vq2x"
Sep 10 19:23:39.383653 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:39.383624    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7b7tw\" (UniqueName: \"kubernetes.io/projected/79c2ba5f-2254-4d6e-8f90-697212b36ff1-kube-api-access-7b7tw\") pod \"postgres-69fb4594cf-2vq2x\" (UID: \"79c2ba5f-2254-4d6e-8f90-697212b36ff1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-2vq2x"
Sep 10 19:23:39.474383 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:39.474288    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2vq2x"
Sep 10 19:23:39.745980 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:39.745949    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-2vq2x"]
Sep 10 19:23:39.747651 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:23:39.747630    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79c2ba5f_2254_4d6e_8f90_697212b36ff1.slice/crio-e73f7185468586586b987ed4ffb2dbdde212b997f881a3c40637fd11ee3f29be WatchSource:0}: Error finding container e73f7185468586586b987ed4ffb2dbdde212b997f881a3c40637fd11ee3f29be: Status 404 returned error can't find the container with id e73f7185468586586b987ed4ffb2dbdde212b997f881a3c40637fd11ee3f29be
Sep 10 19:23:39.755619 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:39.755594    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2vq2x" event={"ID":"79c2ba5f-2254-4d6e-8f90-697212b36ff1","Type":"ContainerStarted","Data":"e73f7185468586586b987ed4ffb2dbdde212b997f881a3c40637fd11ee3f29be"}
Sep 10 19:23:42.776847 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:42.776809    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2vq2x" event={"ID":"79c2ba5f-2254-4d6e-8f90-697212b36ff1","Type":"ContainerStarted","Data":"b0587e4525664f1eca225621b5debbec38d98bb80550d6d4bb4f7051bf275061"}
Sep 10 19:23:42.777325 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:42.776977    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2vq2x"
Sep 10 19:23:48.812525 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:48.812485    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2vq2x"
Sep 10 19:23:48.830137 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:48.830092    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2vq2x" podStartSLOduration=7.549502416 podStartE2EDuration="9.830077903s" totalImagesPullingTime="2.280575487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:23:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:23:42.791936631 +0000 UTC m=+478.011785136" watchObservedRunningTime="2026-09-10 19:23:48.830077903 +0000 UTC m=+484.049926381"
Sep 10 19:23:49.345967 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:49.345930    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-jklh7"]
Sep 10 19:23:49.352513 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:49.352477    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-jklh7"
Sep 10 19:23:49.354765 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:49.354730    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-vj929\""
Sep 10 19:23:49.359106 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:49.359037    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-jklh7"]
Sep 10 19:23:49.469138 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:49.469099    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22xw6\" (UniqueName: \"kubernetes.io/projected/e13ca353-9ca8-44c5-a619-1c047904898e-kube-api-access-22xw6\") pod \"authorino-557f876f69-jklh7\" (UID: \"e13ca353-9ca8-44c5-a619-1c047904898e\") " pod="kuadrant-system/authorino-557f876f69-jklh7"
Sep 10 19:23:49.570674 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:49.570639    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22xw6\" (UniqueName: \"kubernetes.io/projected/e13ca353-9ca8-44c5-a619-1c047904898e-kube-api-access-22xw6\") pod \"authorino-557f876f69-jklh7\" (UID: \"e13ca353-9ca8-44c5-a619-1c047904898e\") " pod="kuadrant-system/authorino-557f876f69-jklh7"
Sep 10 19:23:49.582109 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:49.582076    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-jklh7"]
Sep 10 19:23:49.582397 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:23:49.582376    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-22xw6], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-557f876f69-jklh7" podUID="e13ca353-9ca8-44c5-a619-1c047904898e"
Sep 10 19:23:49.585334 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:49.585309    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22xw6\" (UniqueName: \"kubernetes.io/projected/e13ca353-9ca8-44c5-a619-1c047904898e-kube-api-access-22xw6\") pod \"authorino-557f876f69-jklh7\" (UID: \"e13ca353-9ca8-44c5-a619-1c047904898e\") " pod="kuadrant-system/authorino-557f876f69-jklh7"
Sep 10 19:23:49.622799 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:49.622708    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-687cb9f6b4-zt952"]
Sep 10 19:23:49.627222 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:49.627205    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687cb9f6b4-zt952"
Sep 10 19:23:49.637189 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:49.637160    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-687cb9f6b4-zt952"]
Sep 10 19:23:49.659590 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:49.659558    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-687cb9f6b4-zt952"]
Sep 10 19:23:49.659902 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:23:49.659879    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-fwdvt], unattached volumes=[], failed to process volumes=[kube-api-access-fwdvt]: context canceled" pod="kuadrant-system/authorino-687cb9f6b4-zt952" podUID="8f78e8a6-c9dc-40f5-9e87-b14c04e29eda"
Sep 10 19:23:49.694097 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:49.694062    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c5bbb57d-swn54"]
Sep 10 19:23:49.698467 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:49.698448    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c5bbb57d-swn54"
Sep 10 19:23:49.700598 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:49.700572    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 10 19:23:49.703700 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:49.703673    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c5bbb57d-swn54"]
Sep 10 19:23:49.772973 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:49.772923    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwdvt\" (UniqueName: \"kubernetes.io/projected/8f78e8a6-c9dc-40f5-9e87-b14c04e29eda-kube-api-access-fwdvt\") pod \"authorino-687cb9f6b4-zt952\" (UID: \"8f78e8a6-c9dc-40f5-9e87-b14c04e29eda\") " pod="kuadrant-system/authorino-687cb9f6b4-zt952"
Sep 10 19:23:49.772973 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:49.772975    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d200af66-98aa-4db1-8ff7-54ecd1884852-tls-cert\") pod \"authorino-c5bbb57d-swn54\" (UID: \"d200af66-98aa-4db1-8ff7-54ecd1884852\") " pod="kuadrant-system/authorino-c5bbb57d-swn54"
Sep 10 19:23:49.773182 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:49.773060    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdr9p\" (UniqueName: \"kubernetes.io/projected/d200af66-98aa-4db1-8ff7-54ecd1884852-kube-api-access-bdr9p\") pod \"authorino-c5bbb57d-swn54\" (UID: \"d200af66-98aa-4db1-8ff7-54ecd1884852\") " pod="kuadrant-system/authorino-c5bbb57d-swn54"
Sep 10 19:23:49.817184 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:49.817155    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687cb9f6b4-zt952"
Sep 10 19:23:49.817571 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:49.817158    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-jklh7"
Sep 10 19:23:49.824938 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:49.824905    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687cb9f6b4-zt952"
Sep 10 19:23:49.828112 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:49.828097    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-jklh7"
Sep 10 19:23:49.874049 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:49.873958    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwdvt\" (UniqueName: \"kubernetes.io/projected/8f78e8a6-c9dc-40f5-9e87-b14c04e29eda-kube-api-access-fwdvt\") pod \"authorino-687cb9f6b4-zt952\" (UID: \"8f78e8a6-c9dc-40f5-9e87-b14c04e29eda\") " pod="kuadrant-system/authorino-687cb9f6b4-zt952"
Sep 10 19:23:49.874049 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:49.873992    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d200af66-98aa-4db1-8ff7-54ecd1884852-tls-cert\") pod \"authorino-c5bbb57d-swn54\" (UID: \"d200af66-98aa-4db1-8ff7-54ecd1884852\") " pod="kuadrant-system/authorino-c5bbb57d-swn54"
Sep 10 19:23:49.874049 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:49.874034    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdr9p\" (UniqueName: \"kubernetes.io/projected/d200af66-98aa-4db1-8ff7-54ecd1884852-kube-api-access-bdr9p\") pod \"authorino-c5bbb57d-swn54\" (UID: \"d200af66-98aa-4db1-8ff7-54ecd1884852\") " pod="kuadrant-system/authorino-c5bbb57d-swn54"
Sep 10 19:23:49.876705 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:49.876684    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d200af66-98aa-4db1-8ff7-54ecd1884852-tls-cert\") pod \"authorino-c5bbb57d-swn54\" (UID: \"d200af66-98aa-4db1-8ff7-54ecd1884852\") " pod="kuadrant-system/authorino-c5bbb57d-swn54"
Sep 10 19:23:49.881891 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:49.881865    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwdvt\" (UniqueName: \"kubernetes.io/projected/8f78e8a6-c9dc-40f5-9e87-b14c04e29eda-kube-api-access-fwdvt\") pod \"authorino-687cb9f6b4-zt952\" (UID: \"8f78e8a6-c9dc-40f5-9e87-b14c04e29eda\") " pod="kuadrant-system/authorino-687cb9f6b4-zt952"
Sep 10 19:23:49.881891 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:49.881882    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdr9p\" (UniqueName: \"kubernetes.io/projected/d200af66-98aa-4db1-8ff7-54ecd1884852-kube-api-access-bdr9p\") pod \"authorino-c5bbb57d-swn54\" (UID: \"d200af66-98aa-4db1-8ff7-54ecd1884852\") " pod="kuadrant-system/authorino-c5bbb57d-swn54"
Sep 10 19:23:49.974611 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:49.974571    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fwdvt\" (UniqueName: \"kubernetes.io/projected/8f78e8a6-c9dc-40f5-9e87-b14c04e29eda-kube-api-access-fwdvt\") pod \"8f78e8a6-c9dc-40f5-9e87-b14c04e29eda\" (UID: \"8f78e8a6-c9dc-40f5-9e87-b14c04e29eda\") "
Sep 10 19:23:49.974838 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:49.974652    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-22xw6\" (UniqueName: \"kubernetes.io/projected/e13ca353-9ca8-44c5-a619-1c047904898e-kube-api-access-22xw6\") pod \"e13ca353-9ca8-44c5-a619-1c047904898e\" (UID: \"e13ca353-9ca8-44c5-a619-1c047904898e\") "
Sep 10 19:23:49.976937 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:49.976902    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f78e8a6-c9dc-40f5-9e87-b14c04e29eda-kube-api-access-fwdvt" (OuterVolumeSpecName: "kube-api-access-fwdvt") pod "8f78e8a6-c9dc-40f5-9e87-b14c04e29eda" (UID: "8f78e8a6-c9dc-40f5-9e87-b14c04e29eda"). InnerVolumeSpecName "kube-api-access-fwdvt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:23:49.977047 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:49.976909    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e13ca353-9ca8-44c5-a619-1c047904898e-kube-api-access-22xw6" (OuterVolumeSpecName: "kube-api-access-22xw6") pod "e13ca353-9ca8-44c5-a619-1c047904898e" (UID: "e13ca353-9ca8-44c5-a619-1c047904898e"). InnerVolumeSpecName "kube-api-access-22xw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:23:50.008661 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:50.008624    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c5bbb57d-swn54"
Sep 10 19:23:50.075991 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:50.075960    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22xw6\" (UniqueName: \"kubernetes.io/projected/e13ca353-9ca8-44c5-a619-1c047904898e-kube-api-access-22xw6\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:23:50.075991 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:50.075987    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwdvt\" (UniqueName: \"kubernetes.io/projected/8f78e8a6-c9dc-40f5-9e87-b14c04e29eda-kube-api-access-fwdvt\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:23:50.284132 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:50.284110    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c5bbb57d-swn54"]
Sep 10 19:23:50.285090 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:23:50.285068    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd200af66_98aa_4db1_8ff7_54ecd1884852.slice/crio-87b92cb8810e4bd31af46372193906dc7380d00fb01a9e7e101e57258a5e5e3b WatchSource:0}: Error finding container 87b92cb8810e4bd31af46372193906dc7380d00fb01a9e7e101e57258a5e5e3b: Status 404 returned error can't find the container with id 87b92cb8810e4bd31af46372193906dc7380d00fb01a9e7e101e57258a5e5e3b
Sep 10 19:23:50.824209 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:50.824174    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c5bbb57d-swn54" event={"ID":"d200af66-98aa-4db1-8ff7-54ecd1884852","Type":"ContainerStarted","Data":"87b92cb8810e4bd31af46372193906dc7380d00fb01a9e7e101e57258a5e5e3b"}
Sep 10 19:23:50.824607 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:50.824216    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-jklh7"
Sep 10 19:23:50.824607 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:50.824220    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687cb9f6b4-zt952"
Sep 10 19:23:50.858391 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:50.858354    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-jklh7"]
Sep 10 19:23:50.863527 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:50.863502    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-jklh7"]
Sep 10 19:23:50.883094 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:50.883064    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-687cb9f6b4-zt952"]
Sep 10 19:23:50.887042 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:50.887016    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-687cb9f6b4-zt952"]
Sep 10 19:23:51.240198 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:51.240166    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f78e8a6-c9dc-40f5-9e87-b14c04e29eda" path="/var/lib/kubelet/pods/8f78e8a6-c9dc-40f5-9e87-b14c04e29eda/volumes"
Sep 10 19:23:51.240473 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:51.240433    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e13ca353-9ca8-44c5-a619-1c047904898e" path="/var/lib/kubelet/pods/e13ca353-9ca8-44c5-a619-1c047904898e/volumes"
Sep 10 19:23:51.837824 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:51.837789    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c5bbb57d-swn54" event={"ID":"d200af66-98aa-4db1-8ff7-54ecd1884852","Type":"ContainerStarted","Data":"200127a86974bfef5cc439f88b83c11cd5c08a541080af9096c29991427aa4a5"}
Sep 10 19:23:51.855636 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:51.855583    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c5bbb57d-swn54" podStartSLOduration=2.117816939 podStartE2EDuration="2.855567668s" totalImagesPullingTime="737.750729ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:23:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:23:51.853681056 +0000 UTC m=+487.073529536" watchObservedRunningTime="2026-09-10 19:23:51.855567668 +0000 UTC m=+487.075416152"
Sep 10 19:23:56.590606 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:56.590569    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-56887f8cd7-9zrl2"]
Sep 10 19:23:56.600421 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:56.600396    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56887f8cd7-9zrl2"
Sep 10 19:23:56.606809 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:56.606778    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 19:23:56.607307 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:56.607139    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-d7nmk\""
Sep 10 19:23:56.607517 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:56.607500    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 19:23:56.608589 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:56.608567    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 19:23:56.613538 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:56.612903    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56887f8cd7-9zrl2"]
Sep 10 19:23:56.732699 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:56.732663    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eba62e6f-1fbb-42bb-aa66-c5460e6e7753-webhook-cert\") pod \"maas-controller-56887f8cd7-9zrl2\" (UID: \"eba62e6f-1fbb-42bb-aa66-c5460e6e7753\") " pod="opendatahub/maas-controller-56887f8cd7-9zrl2"
Sep 10 19:23:56.732699 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:56.732705    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zm9m7\" (UniqueName: \"kubernetes.io/projected/eba62e6f-1fbb-42bb-aa66-c5460e6e7753-kube-api-access-zm9m7\") pod \"maas-controller-56887f8cd7-9zrl2\" (UID: \"eba62e6f-1fbb-42bb-aa66-c5460e6e7753\") " pod="opendatahub/maas-controller-56887f8cd7-9zrl2"
Sep 10 19:23:56.732937 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:56.732731    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eba62e6f-1fbb-42bb-aa66-c5460e6e7753-metrics-certs\") pod \"maas-controller-56887f8cd7-9zrl2\" (UID: \"eba62e6f-1fbb-42bb-aa66-c5460e6e7753\") " pod="opendatahub/maas-controller-56887f8cd7-9zrl2"
Sep 10 19:23:56.762411 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:56.762380    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-77c474c76b-h9hhf"]
Sep 10 19:23:56.766106 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:56.766084    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77c474c76b-h9hhf"
Sep 10 19:23:56.777605 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:56.777585    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77c474c76b-h9hhf"]
Sep 10 19:23:56.833901 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:56.833869    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eba62e6f-1fbb-42bb-aa66-c5460e6e7753-webhook-cert\") pod \"maas-controller-56887f8cd7-9zrl2\" (UID: \"eba62e6f-1fbb-42bb-aa66-c5460e6e7753\") " pod="opendatahub/maas-controller-56887f8cd7-9zrl2"
Sep 10 19:23:56.834106 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:56.833912    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zm9m7\" (UniqueName: \"kubernetes.io/projected/eba62e6f-1fbb-42bb-aa66-c5460e6e7753-kube-api-access-zm9m7\") pod \"maas-controller-56887f8cd7-9zrl2\" (UID: \"eba62e6f-1fbb-42bb-aa66-c5460e6e7753\") " pod="opendatahub/maas-controller-56887f8cd7-9zrl2"
Sep 10 19:23:56.834106 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:56.834046    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eba62e6f-1fbb-42bb-aa66-c5460e6e7753-metrics-certs\") pod \"maas-controller-56887f8cd7-9zrl2\" (UID: \"eba62e6f-1fbb-42bb-aa66-c5460e6e7753\") " pod="opendatahub/maas-controller-56887f8cd7-9zrl2"
Sep 10 19:23:56.834231 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:23:56.834165    2579 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 10 19:23:56.834275 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:23:56.834239    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eba62e6f-1fbb-42bb-aa66-c5460e6e7753-metrics-certs podName:eba62e6f-1fbb-42bb-aa66-c5460e6e7753 nodeName:}" failed. No retries permitted until 2026-09-10 19:23:57.33422214 +0000 UTC m=+492.554070603 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eba62e6f-1fbb-42bb-aa66-c5460e6e7753-metrics-certs") pod "maas-controller-56887f8cd7-9zrl2" (UID: "eba62e6f-1fbb-42bb-aa66-c5460e6e7753") : secret "maas-controller-metrics-tls" not found
Sep 10 19:23:56.836583 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:56.836558    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eba62e6f-1fbb-42bb-aa66-c5460e6e7753-webhook-cert\") pod \"maas-controller-56887f8cd7-9zrl2\" (UID: \"eba62e6f-1fbb-42bb-aa66-c5460e6e7753\") " pod="opendatahub/maas-controller-56887f8cd7-9zrl2"
Sep 10 19:23:56.843559 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:56.843485    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zm9m7\" (UniqueName: \"kubernetes.io/projected/eba62e6f-1fbb-42bb-aa66-c5460e6e7753-kube-api-access-zm9m7\") pod \"maas-controller-56887f8cd7-9zrl2\" (UID: \"eba62e6f-1fbb-42bb-aa66-c5460e6e7753\") " pod="opendatahub/maas-controller-56887f8cd7-9zrl2"
Sep 10 19:23:56.935146 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:56.935109    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6738a3a0-7730-4f3a-8251-f28b713bd83a-metrics-certs\") pod \"maas-controller-77c474c76b-h9hhf\" (UID: \"6738a3a0-7730-4f3a-8251-f28b713bd83a\") " pod="opendatahub/maas-controller-77c474c76b-h9hhf"
Sep 10 19:23:56.935312 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:56.935187    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ffj9\" (UniqueName: \"kubernetes.io/projected/6738a3a0-7730-4f3a-8251-f28b713bd83a-kube-api-access-7ffj9\") pod \"maas-controller-77c474c76b-h9hhf\" (UID: \"6738a3a0-7730-4f3a-8251-f28b713bd83a\") " pod="opendatahub/maas-controller-77c474c76b-h9hhf"
Sep 10 19:23:56.935312 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:56.935232    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6738a3a0-7730-4f3a-8251-f28b713bd83a-webhook-cert\") pod \"maas-controller-77c474c76b-h9hhf\" (UID: \"6738a3a0-7730-4f3a-8251-f28b713bd83a\") " pod="opendatahub/maas-controller-77c474c76b-h9hhf"
Sep 10 19:23:57.036052 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:57.036013    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6738a3a0-7730-4f3a-8251-f28b713bd83a-metrics-certs\") pod \"maas-controller-77c474c76b-h9hhf\" (UID: \"6738a3a0-7730-4f3a-8251-f28b713bd83a\") " pod="opendatahub/maas-controller-77c474c76b-h9hhf"
Sep 10 19:23:57.036195 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:57.036086    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ffj9\" (UniqueName: \"kubernetes.io/projected/6738a3a0-7730-4f3a-8251-f28b713bd83a-kube-api-access-7ffj9\") pod \"maas-controller-77c474c76b-h9hhf\" (UID: \"6738a3a0-7730-4f3a-8251-f28b713bd83a\") " pod="opendatahub/maas-controller-77c474c76b-h9hhf"
Sep 10 19:23:57.036195 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:57.036134    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6738a3a0-7730-4f3a-8251-f28b713bd83a-webhook-cert\") pod \"maas-controller-77c474c76b-h9hhf\" (UID: \"6738a3a0-7730-4f3a-8251-f28b713bd83a\") " pod="opendatahub/maas-controller-77c474c76b-h9hhf"
Sep 10 19:23:57.036195 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:23:57.036170    2579 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 10 19:23:57.036319 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:23:57.036234    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6738a3a0-7730-4f3a-8251-f28b713bd83a-metrics-certs podName:6738a3a0-7730-4f3a-8251-f28b713bd83a nodeName:}" failed. No retries permitted until 2026-09-10 19:23:57.536219108 +0000 UTC m=+492.756067568 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6738a3a0-7730-4f3a-8251-f28b713bd83a-metrics-certs") pod "maas-controller-77c474c76b-h9hhf" (UID: "6738a3a0-7730-4f3a-8251-f28b713bd83a") : secret "maas-controller-metrics-tls" not found
Sep 10 19:23:57.038687 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:57.038662    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6738a3a0-7730-4f3a-8251-f28b713bd83a-webhook-cert\") pod \"maas-controller-77c474c76b-h9hhf\" (UID: \"6738a3a0-7730-4f3a-8251-f28b713bd83a\") " pod="opendatahub/maas-controller-77c474c76b-h9hhf"
Sep 10 19:23:57.047039 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:57.047018    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ffj9\" (UniqueName: \"kubernetes.io/projected/6738a3a0-7730-4f3a-8251-f28b713bd83a-kube-api-access-7ffj9\") pod \"maas-controller-77c474c76b-h9hhf\" (UID: \"6738a3a0-7730-4f3a-8251-f28b713bd83a\") " pod="opendatahub/maas-controller-77c474c76b-h9hhf"
Sep 10 19:23:57.338226 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:57.338188    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eba62e6f-1fbb-42bb-aa66-c5460e6e7753-metrics-certs\") pod \"maas-controller-56887f8cd7-9zrl2\" (UID: \"eba62e6f-1fbb-42bb-aa66-c5460e6e7753\") " pod="opendatahub/maas-controller-56887f8cd7-9zrl2"
Sep 10 19:23:57.340785 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:57.340733    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eba62e6f-1fbb-42bb-aa66-c5460e6e7753-metrics-certs\") pod \"maas-controller-56887f8cd7-9zrl2\" (UID: \"eba62e6f-1fbb-42bb-aa66-c5460e6e7753\") " pod="opendatahub/maas-controller-56887f8cd7-9zrl2"
Sep 10 19:23:57.512557 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:57.512510    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56887f8cd7-9zrl2"
Sep 10 19:23:57.540165 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:57.540132    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6738a3a0-7730-4f3a-8251-f28b713bd83a-metrics-certs\") pod \"maas-controller-77c474c76b-h9hhf\" (UID: \"6738a3a0-7730-4f3a-8251-f28b713bd83a\") " pod="opendatahub/maas-controller-77c474c76b-h9hhf"
Sep 10 19:23:57.542703 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:57.542679    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6738a3a0-7730-4f3a-8251-f28b713bd83a-metrics-certs\") pod \"maas-controller-77c474c76b-h9hhf\" (UID: \"6738a3a0-7730-4f3a-8251-f28b713bd83a\") " pod="opendatahub/maas-controller-77c474c76b-h9hhf"
Sep 10 19:23:57.677514 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:57.677475    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77c474c76b-h9hhf"
Sep 10 19:23:57.806370 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:57.806344    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56887f8cd7-9zrl2"]
Sep 10 19:23:57.806866 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:23:57.806842    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeba62e6f_1fbb_42bb_aa66_c5460e6e7753.slice/crio-ca9aa883d1f407a638e97a2df6e20a39b3f1282ee94e3eac5e5996626af8c5a1 WatchSource:0}: Error finding container ca9aa883d1f407a638e97a2df6e20a39b3f1282ee94e3eac5e5996626af8c5a1: Status 404 returned error can't find the container with id ca9aa883d1f407a638e97a2df6e20a39b3f1282ee94e3eac5e5996626af8c5a1
Sep 10 19:23:57.872908 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:57.872820    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56887f8cd7-9zrl2" event={"ID":"eba62e6f-1fbb-42bb-aa66-c5460e6e7753","Type":"ContainerStarted","Data":"ca9aa883d1f407a638e97a2df6e20a39b3f1282ee94e3eac5e5996626af8c5a1"}
Sep 10 19:23:57.952984 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:57.952955    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77c474c76b-h9hhf"]
Sep 10 19:23:57.954461 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:23:57.954429    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6738a3a0_7730_4f3a_8251_f28b713bd83a.slice/crio-8bae8b1845e4a4d31a56ce5334995b593a113bfb0b51cc382d0f2b673511e74d WatchSource:0}: Error finding container 8bae8b1845e4a4d31a56ce5334995b593a113bfb0b51cc382d0f2b673511e74d: Status 404 returned error can't find the container with id 8bae8b1845e4a4d31a56ce5334995b593a113bfb0b51cc382d0f2b673511e74d
Sep 10 19:23:58.886921 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:23:58.886883    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77c474c76b-h9hhf" event={"ID":"6738a3a0-7730-4f3a-8251-f28b713bd83a","Type":"ContainerStarted","Data":"8bae8b1845e4a4d31a56ce5334995b593a113bfb0b51cc382d0f2b673511e74d"}
Sep 10 19:24:00.902896 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:00.902855    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56887f8cd7-9zrl2" event={"ID":"eba62e6f-1fbb-42bb-aa66-c5460e6e7753","Type":"ContainerStarted","Data":"2abb5c83977f5bb1822530cb9a7bc495b3990354fef03462247c2e1260fa30a6"}
Sep 10 19:24:00.903409 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:00.903001    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-56887f8cd7-9zrl2"
Sep 10 19:24:00.905055 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:00.904979    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77c474c76b-h9hhf" event={"ID":"6738a3a0-7730-4f3a-8251-f28b713bd83a","Type":"ContainerStarted","Data":"def77155daa1e6517ffe74a0564a53796a807585c2723bdf7bc6e954df7ec71a"}
Sep 10 19:24:00.905226 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:00.905081    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-77c474c76b-h9hhf"
Sep 10 19:24:00.932519 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:00.932460    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-56887f8cd7-9zrl2" podStartSLOduration=2.331271729 podStartE2EDuration="4.932442883s" totalImagesPullingTime="2.601171154s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:23:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:24:00.930100559 +0000 UTC m=+496.149949052" watchObservedRunningTime="2026-09-10 19:24:00.932442883 +0000 UTC m=+496.152291363"
Sep 10 19:24:00.951402 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:00.951341    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-77c474c76b-h9hhf" podStartSLOduration=2.490998276 podStartE2EDuration="4.95132476s" totalImagesPullingTime="2.460326484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:23:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:24:00.950832093 +0000 UTC m=+496.170680574" watchObservedRunningTime="2026-09-10 19:24:00.95132476 +0000 UTC m=+496.171173243"
Sep 10 19:24:11.913928 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:11.913894    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-56887f8cd7-9zrl2"
Sep 10 19:24:11.914501 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:11.913970    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-77c474c76b-h9hhf"
Sep 10 19:24:11.984780 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:11.984716    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-56887f8cd7-9zrl2"]
Sep 10 19:24:11.985009 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:11.984956    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-56887f8cd7-9zrl2" podUID="eba62e6f-1fbb-42bb-aa66-c5460e6e7753" containerName="manager" containerID="cri-o://2abb5c83977f5bb1822530cb9a7bc495b3990354fef03462247c2e1260fa30a6" gracePeriod=30
Sep 10 19:24:12.232459 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:12.232428    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56887f8cd7-9zrl2"
Sep 10 19:24:12.277706 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:12.277675    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eba62e6f-1fbb-42bb-aa66-c5460e6e7753-metrics-certs\") pod \"eba62e6f-1fbb-42bb-aa66-c5460e6e7753\" (UID: \"eba62e6f-1fbb-42bb-aa66-c5460e6e7753\") "
Sep 10 19:24:12.277909 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:12.277721    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eba62e6f-1fbb-42bb-aa66-c5460e6e7753-webhook-cert\") pod \"eba62e6f-1fbb-42bb-aa66-c5460e6e7753\" (UID: \"eba62e6f-1fbb-42bb-aa66-c5460e6e7753\") "
Sep 10 19:24:12.277909 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:12.277775    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zm9m7\" (UniqueName: \"kubernetes.io/projected/eba62e6f-1fbb-42bb-aa66-c5460e6e7753-kube-api-access-zm9m7\") pod \"eba62e6f-1fbb-42bb-aa66-c5460e6e7753\" (UID: \"eba62e6f-1fbb-42bb-aa66-c5460e6e7753\") "
Sep 10 19:24:12.280280 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:12.280249    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eba62e6f-1fbb-42bb-aa66-c5460e6e7753-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "eba62e6f-1fbb-42bb-aa66-c5460e6e7753" (UID: "eba62e6f-1fbb-42bb-aa66-c5460e6e7753"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:24:12.280464 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:12.280440    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eba62e6f-1fbb-42bb-aa66-c5460e6e7753-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "eba62e6f-1fbb-42bb-aa66-c5460e6e7753" (UID: "eba62e6f-1fbb-42bb-aa66-c5460e6e7753"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:24:12.280518 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:12.280457    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eba62e6f-1fbb-42bb-aa66-c5460e6e7753-kube-api-access-zm9m7" (OuterVolumeSpecName: "kube-api-access-zm9m7") pod "eba62e6f-1fbb-42bb-aa66-c5460e6e7753" (UID: "eba62e6f-1fbb-42bb-aa66-c5460e6e7753"). InnerVolumeSpecName "kube-api-access-zm9m7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:24:12.378874 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:12.378840    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eba62e6f-1fbb-42bb-aa66-c5460e6e7753-metrics-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:24:12.378874 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:12.378867    2579 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eba62e6f-1fbb-42bb-aa66-c5460e6e7753-webhook-cert\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:24:12.378874 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:12.378876    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zm9m7\" (UniqueName: \"kubernetes.io/projected/eba62e6f-1fbb-42bb-aa66-c5460e6e7753-kube-api-access-zm9m7\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:24:12.976559 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:12.976514    2579 generic.go:356] "Generic (PLEG): container finished" podID="eba62e6f-1fbb-42bb-aa66-c5460e6e7753" containerID="2abb5c83977f5bb1822530cb9a7bc495b3990354fef03462247c2e1260fa30a6" exitCode=0
Sep 10 19:24:12.977020 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:12.976568    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56887f8cd7-9zrl2"
Sep 10 19:24:12.977020 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:12.976568    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56887f8cd7-9zrl2" event={"ID":"eba62e6f-1fbb-42bb-aa66-c5460e6e7753","Type":"ContainerDied","Data":"2abb5c83977f5bb1822530cb9a7bc495b3990354fef03462247c2e1260fa30a6"}
Sep 10 19:24:12.977020 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:12.976624    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56887f8cd7-9zrl2" event={"ID":"eba62e6f-1fbb-42bb-aa66-c5460e6e7753","Type":"ContainerDied","Data":"ca9aa883d1f407a638e97a2df6e20a39b3f1282ee94e3eac5e5996626af8c5a1"}
Sep 10 19:24:12.977020 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:12.976654    2579 scope.go:117] "RemoveContainer" containerID="2abb5c83977f5bb1822530cb9a7bc495b3990354fef03462247c2e1260fa30a6"
Sep 10 19:24:12.986774 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:12.986556    2579 scope.go:117] "RemoveContainer" containerID="2abb5c83977f5bb1822530cb9a7bc495b3990354fef03462247c2e1260fa30a6"
Sep 10 19:24:12.986864 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:24:12.986845    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2abb5c83977f5bb1822530cb9a7bc495b3990354fef03462247c2e1260fa30a6\": container with ID starting with 2abb5c83977f5bb1822530cb9a7bc495b3990354fef03462247c2e1260fa30a6 not found: ID does not exist" containerID="2abb5c83977f5bb1822530cb9a7bc495b3990354fef03462247c2e1260fa30a6"
Sep 10 19:24:12.986906 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:12.986873    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2abb5c83977f5bb1822530cb9a7bc495b3990354fef03462247c2e1260fa30a6"} err="failed to get container status \"2abb5c83977f5bb1822530cb9a7bc495b3990354fef03462247c2e1260fa30a6\": rpc error: code = NotFound desc = could not find container \"2abb5c83977f5bb1822530cb9a7bc495b3990354fef03462247c2e1260fa30a6\": container with ID starting with 2abb5c83977f5bb1822530cb9a7bc495b3990354fef03462247c2e1260fa30a6 not found: ID does not exist"
Sep 10 19:24:12.998705 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:12.998681    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-56887f8cd7-9zrl2"]
Sep 10 19:24:13.000908 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:13.000890    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-56887f8cd7-9zrl2"]
Sep 10 19:24:13.239374 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:13.239297    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eba62e6f-1fbb-42bb-aa66-c5460e6e7753" path="/var/lib/kubelet/pods/eba62e6f-1fbb-42bb-aa66-c5460e6e7753/volumes"
Sep 10 19:24:34.095833 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:34.095798    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7d4dc54f4c-wvlr7"]
Sep 10 19:24:34.096314 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:34.096297    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eba62e6f-1fbb-42bb-aa66-c5460e6e7753" containerName="manager"
Sep 10 19:24:34.096369 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:34.096317    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="eba62e6f-1fbb-42bb-aa66-c5460e6e7753" containerName="manager"
Sep 10 19:24:34.096446 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:34.096426    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="eba62e6f-1fbb-42bb-aa66-c5460e6e7753" containerName="manager"
Sep 10 19:24:34.103400 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:34.103381    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-wvlr7"
Sep 10 19:24:34.105618 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:34.105597    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 19:24:34.106044 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:34.106025    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-m8g2m\""
Sep 10 19:24:34.106120 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:34.106025    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 10 19:24:34.111556 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:34.111516    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7d4dc54f4c-wvlr7"]
Sep 10 19:24:34.179850 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:34.179810    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d2ff726-2d16-4771-b26e-4a11e90db6f5-openshift-service-ca\") pod \"maas-api-7d4dc54f4c-wvlr7\" (UID: \"3d2ff726-2d16-4771-b26e-4a11e90db6f5\") " pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-wvlr7"
Sep 10 19:24:34.179850 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:34.179855    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d2ff726-2d16-4771-b26e-4a11e90db6f5-metrics-certs\") pod \"maas-api-7d4dc54f4c-wvlr7\" (UID: \"3d2ff726-2d16-4771-b26e-4a11e90db6f5\") " pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-wvlr7"
Sep 10 19:24:34.180062 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:34.179995    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2xzm\" (UniqueName: \"kubernetes.io/projected/3d2ff726-2d16-4771-b26e-4a11e90db6f5-kube-api-access-g2xzm\") pod \"maas-api-7d4dc54f4c-wvlr7\" (UID: \"3d2ff726-2d16-4771-b26e-4a11e90db6f5\") " pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-wvlr7"
Sep 10 19:24:34.180132 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:34.180114    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d2ff726-2d16-4771-b26e-4a11e90db6f5-maas-api-tls\") pod \"maas-api-7d4dc54f4c-wvlr7\" (UID: \"3d2ff726-2d16-4771-b26e-4a11e90db6f5\") " pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-wvlr7"
Sep 10 19:24:34.281118 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:34.281085    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d2ff726-2d16-4771-b26e-4a11e90db6f5-maas-api-tls\") pod \"maas-api-7d4dc54f4c-wvlr7\" (UID: \"3d2ff726-2d16-4771-b26e-4a11e90db6f5\") " pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-wvlr7"
Sep 10 19:24:34.281286 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:34.281130    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d2ff726-2d16-4771-b26e-4a11e90db6f5-openshift-service-ca\") pod \"maas-api-7d4dc54f4c-wvlr7\" (UID: \"3d2ff726-2d16-4771-b26e-4a11e90db6f5\") " pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-wvlr7"
Sep 10 19:24:34.281286 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:34.281148    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d2ff726-2d16-4771-b26e-4a11e90db6f5-metrics-certs\") pod \"maas-api-7d4dc54f4c-wvlr7\" (UID: \"3d2ff726-2d16-4771-b26e-4a11e90db6f5\") " pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-wvlr7"
Sep 10 19:24:34.281286 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:34.281215    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2xzm\" (UniqueName: \"kubernetes.io/projected/3d2ff726-2d16-4771-b26e-4a11e90db6f5-kube-api-access-g2xzm\") pod \"maas-api-7d4dc54f4c-wvlr7\" (UID: \"3d2ff726-2d16-4771-b26e-4a11e90db6f5\") " pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-wvlr7"
Sep 10 19:24:34.281396 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:24:34.281299    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found
Sep 10 19:24:34.281396 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:24:34.281360    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d2ff726-2d16-4771-b26e-4a11e90db6f5-metrics-certs podName:3d2ff726-2d16-4771-b26e-4a11e90db6f5 nodeName:}" failed. No retries permitted until 2026-09-10 19:24:34.781345677 +0000 UTC m=+530.001194135 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d2ff726-2d16-4771-b26e-4a11e90db6f5-metrics-certs") pod "maas-api-7d4dc54f4c-wvlr7" (UID: "3d2ff726-2d16-4771-b26e-4a11e90db6f5") : secret "maas-api-metrics-tls" not found
Sep 10 19:24:34.281828 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:34.281801    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d2ff726-2d16-4771-b26e-4a11e90db6f5-openshift-service-ca\") pod \"maas-api-7d4dc54f4c-wvlr7\" (UID: \"3d2ff726-2d16-4771-b26e-4a11e90db6f5\") " pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-wvlr7"
Sep 10 19:24:34.283821 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:34.283799    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d2ff726-2d16-4771-b26e-4a11e90db6f5-maas-api-tls\") pod \"maas-api-7d4dc54f4c-wvlr7\" (UID: \"3d2ff726-2d16-4771-b26e-4a11e90db6f5\") " pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-wvlr7"
Sep 10 19:24:34.292801 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:34.292774    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2xzm\" (UniqueName: \"kubernetes.io/projected/3d2ff726-2d16-4771-b26e-4a11e90db6f5-kube-api-access-g2xzm\") pod \"maas-api-7d4dc54f4c-wvlr7\" (UID: \"3d2ff726-2d16-4771-b26e-4a11e90db6f5\") " pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-wvlr7"
Sep 10 19:24:34.786211 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:34.786171    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d2ff726-2d16-4771-b26e-4a11e90db6f5-metrics-certs\") pod \"maas-api-7d4dc54f4c-wvlr7\" (UID: \"3d2ff726-2d16-4771-b26e-4a11e90db6f5\") " pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-wvlr7"
Sep 10 19:24:34.789239 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:34.788958    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d2ff726-2d16-4771-b26e-4a11e90db6f5-metrics-certs\") pod \"maas-api-7d4dc54f4c-wvlr7\" (UID: \"3d2ff726-2d16-4771-b26e-4a11e90db6f5\") " pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-wvlr7"
Sep 10 19:24:35.014897 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:35.014862    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-wvlr7"
Sep 10 19:24:35.305157 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:35.305128    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7d4dc54f4c-wvlr7"]
Sep 10 19:24:36.115176 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:36.115141    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-wvlr7" event={"ID":"3d2ff726-2d16-4771-b26e-4a11e90db6f5","Type":"ContainerStarted","Data":"6079a9e0086d14da8447b9b5035955b100b36a35556478f9a3d7bc7da2a6b9b8"}
Sep 10 19:24:38.128997 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:38.128956    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-wvlr7" event={"ID":"3d2ff726-2d16-4771-b26e-4a11e90db6f5","Type":"ContainerStarted","Data":"48512e9bb64b1eec18611391efe314d8110b1c273b6592fc17fda3bb02201d55"}
Sep 10 19:24:38.129433 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:38.129105    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-wvlr7"
Sep 10 19:24:38.147724 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:38.147668    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-wvlr7" podStartSLOduration=2.253460817 podStartE2EDuration="4.147654258s" totalImagesPullingTime="1.894193441s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:24:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:24:38.145158925 +0000 UTC m=+533.365007405" watchObservedRunningTime="2026-09-10 19:24:38.147654258 +0000 UTC m=+533.367502738"
Sep 10 19:24:44.140533 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:44.140500    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-wvlr7"
Sep 10 19:24:45.172181 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:45.172150    2579 generic.go:356] "Generic (PLEG): container finished" podID="4fd6f983-387d-48bd-9942-0157de3ce6de" containerID="709e9612c2ed39872932b9f9e30ba6f8caedcf81e828943916270d31fa24c029" exitCode=0
Sep 10 19:24:45.172600 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:45.172225    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2" event={"ID":"4fd6f983-387d-48bd-9942-0157de3ce6de","Type":"ContainerDied","Data":"709e9612c2ed39872932b9f9e30ba6f8caedcf81e828943916270d31fa24c029"}
Sep 10 19:24:45.172652 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:45.172614    2579 scope.go:117] "RemoveContainer" containerID="709e9612c2ed39872932b9f9e30ba6f8caedcf81e828943916270d31fa24c029"
Sep 10 19:24:45.173567 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:45.173545    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:24:45.173630 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:45.173609    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:24:45.173671 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:45.173640    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:24:46.181528 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:46.181494    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2" event={"ID":"4fd6f983-387d-48bd-9942-0157de3ce6de","Type":"ContainerStarted","Data":"ef06804e8239c6f493a445772aae0b7c879dc781ebe45cf01f9599291748417f"}
Sep 10 19:24:46.837798 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:46.837739    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"
Sep 10 19:24:46.838024 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:46.838003    2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-ltgk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 10 19:24:46.838081 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:46.838061    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2" podUID="4fd6f983-387d-48bd-9942-0157de3ce6de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 10 19:24:47.838718 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:47.838675    2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-ltgk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 10 19:24:47.839136 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:47.838731    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2" podUID="4fd6f983-387d-48bd-9942-0157de3ce6de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 10 19:24:48.838488 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:48.838452    2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-ltgk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 10 19:24:48.838682 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:48.838506    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2" podUID="4fd6f983-387d-48bd-9942-0157de3ce6de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 10 19:24:49.838180 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:49.838145    2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-ltgk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 10 19:24:49.838554 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:49.838212    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2" podUID="4fd6f983-387d-48bd-9942-0157de3ce6de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 10 19:24:50.838108 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:50.838067    2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-ltgk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 10 19:24:50.838295 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:50.838125    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2" podUID="4fd6f983-387d-48bd-9942-0157de3ce6de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 10 19:24:51.841913 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:51.841885    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"
Sep 10 19:24:51.842351 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:51.842330    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"
Sep 10 19:24:51.843124 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:24:51.843103    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2"
Sep 10 19:25:02.893663 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:02.893624    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk"]
Sep 10 19:25:02.898341 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:02.898322    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk"
Sep 10 19:25:02.901547 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:02.901521    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 10 19:25:02.901680 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:02.901547    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 10 19:25:02.901680 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:02.901521    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6nnj8\""
Sep 10 19:25:02.901680 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:02.901582    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 10 19:25:02.907902 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:02.907874    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk"]
Sep 10 19:25:02.949172 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:02.949141    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9eb974ab-1b17-4f69-9829-397b61fd5aae-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk\" (UID: \"9eb974ab-1b17-4f69-9829-397b61fd5aae\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk"
Sep 10 19:25:02.949368 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:02.949187    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8lk8\" (UniqueName: \"kubernetes.io/projected/9eb974ab-1b17-4f69-9829-397b61fd5aae-kube-api-access-b8lk8\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk\" (UID: \"9eb974ab-1b17-4f69-9829-397b61fd5aae\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk"
Sep 10 19:25:02.949368 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:02.949223    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9eb974ab-1b17-4f69-9829-397b61fd5aae-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk\" (UID: \"9eb974ab-1b17-4f69-9829-397b61fd5aae\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk"
Sep 10 19:25:02.949368 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:02.949287    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9eb974ab-1b17-4f69-9829-397b61fd5aae-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk\" (UID: \"9eb974ab-1b17-4f69-9829-397b61fd5aae\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk"
Sep 10 19:25:02.949368 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:02.949329    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9eb974ab-1b17-4f69-9829-397b61fd5aae-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk\" (UID: \"9eb974ab-1b17-4f69-9829-397b61fd5aae\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk"
Sep 10 19:25:02.949519 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:02.949377    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9eb974ab-1b17-4f69-9829-397b61fd5aae-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk\" (UID: \"9eb974ab-1b17-4f69-9829-397b61fd5aae\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk"
Sep 10 19:25:03.050627 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:03.050581    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9eb974ab-1b17-4f69-9829-397b61fd5aae-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk\" (UID: \"9eb974ab-1b17-4f69-9829-397b61fd5aae\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk"
Sep 10 19:25:03.050867 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:03.050656    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8lk8\" (UniqueName: \"kubernetes.io/projected/9eb974ab-1b17-4f69-9829-397b61fd5aae-kube-api-access-b8lk8\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk\" (UID: \"9eb974ab-1b17-4f69-9829-397b61fd5aae\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk"
Sep 10 19:25:03.050867 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:03.050717    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9eb974ab-1b17-4f69-9829-397b61fd5aae-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk\" (UID: \"9eb974ab-1b17-4f69-9829-397b61fd5aae\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk"
Sep 10 19:25:03.050867 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:03.050803    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9eb974ab-1b17-4f69-9829-397b61fd5aae-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk\" (UID: \"9eb974ab-1b17-4f69-9829-397b61fd5aae\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk"
Sep 10 19:25:03.050867 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:03.050827    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9eb974ab-1b17-4f69-9829-397b61fd5aae-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk\" (UID: \"9eb974ab-1b17-4f69-9829-397b61fd5aae\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk"
Sep 10 19:25:03.051110 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:03.050916    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9eb974ab-1b17-4f69-9829-397b61fd5aae-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk\" (UID: \"9eb974ab-1b17-4f69-9829-397b61fd5aae\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk"
Sep 10 19:25:03.051333 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:03.051310    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9eb974ab-1b17-4f69-9829-397b61fd5aae-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk\" (UID: \"9eb974ab-1b17-4f69-9829-397b61fd5aae\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk"
Sep 10 19:25:03.051465 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:03.051357    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9eb974ab-1b17-4f69-9829-397b61fd5aae-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk\" (UID: \"9eb974ab-1b17-4f69-9829-397b61fd5aae\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk"
Sep 10 19:25:03.051586 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:03.051557    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9eb974ab-1b17-4f69-9829-397b61fd5aae-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk\" (UID: \"9eb974ab-1b17-4f69-9829-397b61fd5aae\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk"
Sep 10 19:25:03.053873 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:03.053847    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9eb974ab-1b17-4f69-9829-397b61fd5aae-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk\" (UID: \"9eb974ab-1b17-4f69-9829-397b61fd5aae\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk"
Sep 10 19:25:03.054087 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:03.054070    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9eb974ab-1b17-4f69-9829-397b61fd5aae-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk\" (UID: \"9eb974ab-1b17-4f69-9829-397b61fd5aae\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk"
Sep 10 19:25:03.062918 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:03.062893    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8lk8\" (UniqueName: \"kubernetes.io/projected/9eb974ab-1b17-4f69-9829-397b61fd5aae-kube-api-access-b8lk8\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk\" (UID: \"9eb974ab-1b17-4f69-9829-397b61fd5aae\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk"
Sep 10 19:25:03.211675 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:03.211589    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk"
Sep 10 19:25:03.491700 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:03.491668    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk"]
Sep 10 19:25:03.491935 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:25:03.491911    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eb974ab_1b17_4f69_9829_397b61fd5aae.slice/crio-dcadf4cbdba9b270a116aa4509fa20e91242660d8045531f6264d4b912f68af9 WatchSource:0}: Error finding container dcadf4cbdba9b270a116aa4509fa20e91242660d8045531f6264d4b912f68af9: Status 404 returned error can't find the container with id dcadf4cbdba9b270a116aa4509fa20e91242660d8045531f6264d4b912f68af9
Sep 10 19:25:04.289915 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:04.289875    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk" event={"ID":"9eb974ab-1b17-4f69-9829-397b61fd5aae","Type":"ContainerStarted","Data":"dcadf4cbdba9b270a116aa4509fa20e91242660d8045531f6264d4b912f68af9"}
Sep 10 19:25:05.296511 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:05.296474    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk" event={"ID":"9eb974ab-1b17-4f69-9829-397b61fd5aae","Type":"ContainerStarted","Data":"9514b35919ddd68d0d8f85ebaeaa4bbd758c777552c06bcdf7d0c5262fa0393e"}
Sep 10 19:25:05.319958 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:05.319899    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk" podStartSLOduration=2.108059005 podStartE2EDuration="3.319882884s" totalImagesPullingTime="1.211823879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:25:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:25:05.316235552 +0000 UTC m=+560.536084037" watchObservedRunningTime="2026-09-10 19:25:05.319882884 +0000 UTC m=+560.539731365"
Sep 10 19:25:13.211907 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:13.211864    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk"
Sep 10 19:25:13.212310 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:13.211958    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk"
Sep 10 19:25:13.224879 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:13.224852    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk"
Sep 10 19:25:13.352709 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:13.352678    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk"
Sep 10 19:25:15.974583 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:15.974544    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-z8q2v"]
Sep 10 19:25:15.978222 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:15.978203    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z8q2v"
Sep 10 19:25:15.980375 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:15.980354    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 10 19:25:15.991550 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:15.991523    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-z8q2v"]
Sep 10 19:25:16.079650 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:16.079612    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmh87\" (UniqueName: \"kubernetes.io/projected/8a97f317-2ff5-4001-bb6c-f3a09d29ca52-kube-api-access-pmh87\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z8q2v\" (UID: \"8a97f317-2ff5-4001-bb6c-f3a09d29ca52\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z8q2v"
Sep 10 19:25:16.079863 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:16.079656    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a97f317-2ff5-4001-bb6c-f3a09d29ca52-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z8q2v\" (UID: \"8a97f317-2ff5-4001-bb6c-f3a09d29ca52\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z8q2v"
Sep 10 19:25:16.079863 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:16.079675    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a97f317-2ff5-4001-bb6c-f3a09d29ca52-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z8q2v\" (UID: \"8a97f317-2ff5-4001-bb6c-f3a09d29ca52\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z8q2v"
Sep 10 19:25:16.079863 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:16.079834    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a97f317-2ff5-4001-bb6c-f3a09d29ca52-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z8q2v\" (UID: \"8a97f317-2ff5-4001-bb6c-f3a09d29ca52\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z8q2v"
Sep 10 19:25:16.079863 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:16.079860    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a97f317-2ff5-4001-bb6c-f3a09d29ca52-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z8q2v\" (UID: \"8a97f317-2ff5-4001-bb6c-f3a09d29ca52\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z8q2v"
Sep 10 19:25:16.080029 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:16.079881    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a97f317-2ff5-4001-bb6c-f3a09d29ca52-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z8q2v\" (UID: \"8a97f317-2ff5-4001-bb6c-f3a09d29ca52\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z8q2v"
Sep 10 19:25:16.180714 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:16.180667    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmh87\" (UniqueName: \"kubernetes.io/projected/8a97f317-2ff5-4001-bb6c-f3a09d29ca52-kube-api-access-pmh87\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z8q2v\" (UID: \"8a97f317-2ff5-4001-bb6c-f3a09d29ca52\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z8q2v"
Sep 10 19:25:16.180714 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:16.180717    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a97f317-2ff5-4001-bb6c-f3a09d29ca52-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z8q2v\" (UID: \"8a97f317-2ff5-4001-bb6c-f3a09d29ca52\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z8q2v"
Sep 10 19:25:16.181014 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:16.180734    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a97f317-2ff5-4001-bb6c-f3a09d29ca52-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z8q2v\" (UID: \"8a97f317-2ff5-4001-bb6c-f3a09d29ca52\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z8q2v"
Sep 10 19:25:16.181014 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:16.180819    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a97f317-2ff5-4001-bb6c-f3a09d29ca52-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z8q2v\" (UID: \"8a97f317-2ff5-4001-bb6c-f3a09d29ca52\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z8q2v"
Sep 10 19:25:16.181014 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:16.180837    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a97f317-2ff5-4001-bb6c-f3a09d29ca52-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z8q2v\" (UID: \"8a97f317-2ff5-4001-bb6c-f3a09d29ca52\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z8q2v"
Sep 10 19:25:16.181014 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:16.180853    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a97f317-2ff5-4001-bb6c-f3a09d29ca52-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z8q2v\" (UID: \"8a97f317-2ff5-4001-bb6c-f3a09d29ca52\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z8q2v"
Sep 10 19:25:16.181231 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:16.181205    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a97f317-2ff5-4001-bb6c-f3a09d29ca52-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z8q2v\" (UID: \"8a97f317-2ff5-4001-bb6c-f3a09d29ca52\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z8q2v"
Sep 10 19:25:16.181280 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:16.181250    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a97f317-2ff5-4001-bb6c-f3a09d29ca52-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z8q2v\" (UID: \"8a97f317-2ff5-4001-bb6c-f3a09d29ca52\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z8q2v"
Sep 10 19:25:16.181313 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:16.181277    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a97f317-2ff5-4001-bb6c-f3a09d29ca52-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z8q2v\" (UID: \"8a97f317-2ff5-4001-bb6c-f3a09d29ca52\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z8q2v"
Sep 10 19:25:16.183150 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:16.183130    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a97f317-2ff5-4001-bb6c-f3a09d29ca52-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z8q2v\" (UID: \"8a97f317-2ff5-4001-bb6c-f3a09d29ca52\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z8q2v"
Sep 10 19:25:16.183402 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:16.183384    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a97f317-2ff5-4001-bb6c-f3a09d29ca52-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z8q2v\" (UID: \"8a97f317-2ff5-4001-bb6c-f3a09d29ca52\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z8q2v"
Sep 10 19:25:16.189457 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:16.189435    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmh87\" (UniqueName: \"kubernetes.io/projected/8a97f317-2ff5-4001-bb6c-f3a09d29ca52-kube-api-access-pmh87\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z8q2v\" (UID: \"8a97f317-2ff5-4001-bb6c-f3a09d29ca52\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z8q2v"
Sep 10 19:25:16.289103 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:16.289013    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z8q2v"
Sep 10 19:25:16.563944 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:16.563911    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-z8q2v"]
Sep 10 19:25:16.565420 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:25:16.565394    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a97f317_2ff5_4001_bb6c_f3a09d29ca52.slice/crio-a504c5ccd8f696a79a369b4dfaf1b34d037b8dabda7385dbc4e434378a118fdc WatchSource:0}: Error finding container a504c5ccd8f696a79a369b4dfaf1b34d037b8dabda7385dbc4e434378a118fdc: Status 404 returned error can't find the container with id a504c5ccd8f696a79a369b4dfaf1b34d037b8dabda7385dbc4e434378a118fdc
Sep 10 19:25:17.368078 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:17.368039    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z8q2v" event={"ID":"8a97f317-2ff5-4001-bb6c-f3a09d29ca52","Type":"ContainerStarted","Data":"e85d6f9ec34ea68af3766717a97705e5e01c35f7643010443ad6bbd0f6f08e8b"}
Sep 10 19:25:17.368469 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:17.368088    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z8q2v" event={"ID":"8a97f317-2ff5-4001-bb6c-f3a09d29ca52","Type":"ContainerStarted","Data":"a504c5ccd8f696a79a369b4dfaf1b34d037b8dabda7385dbc4e434378a118fdc"}
Sep 10 19:25:17.387848 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:17.387794    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z8q2v" podStartSLOduration=2.195294564 podStartE2EDuration="2.387778437s" totalImagesPullingTime="192.483873ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:25:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:25:17.384839069 +0000 UTC m=+572.604687554" watchObservedRunningTime="2026-09-10 19:25:17.387778437 +0000 UTC m=+572.607626912"
Sep 10 19:25:26.289210 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:26.289169    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z8q2v"
Sep 10 19:25:26.289609 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:26.289250    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z8q2v"
Sep 10 19:25:26.302064 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:26.302034    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z8q2v"
Sep 10 19:25:26.432615 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:26.432587    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z8q2v"
Sep 10 19:25:45.153349 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:45.153325    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bmdg9_9817270c-8629-44c4-ab8f-f3982e1f0e2f/console-operator/1.log"
Sep 10 19:25:45.154433 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:45.154412    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bmdg9_9817270c-8629-44c4-ab8f-f3982e1f0e2f/console-operator/1.log"
Sep 10 19:25:45.158435 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:45.158416    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8r66s_2826f8ec-78f6-4cab-879e-47a21854ed5a/ovn-acl-logging/0.log"
Sep 10 19:25:45.159232 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:25:45.159213    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8r66s_2826f8ec-78f6-4cab-879e-47a21854ed5a/ovn-acl-logging/0.log"
Sep 10 19:27:23.608600 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:23.608559    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c5bbb57d-swn54"]
Sep 10 19:27:23.609165 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:23.608855    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-c5bbb57d-swn54" podUID="d200af66-98aa-4db1-8ff7-54ecd1884852" containerName="authorino" containerID="cri-o://200127a86974bfef5cc439f88b83c11cd5c08a541080af9096c29991427aa4a5" gracePeriod=30
Sep 10 19:27:23.869608 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:23.869543    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c5bbb57d-swn54"
Sep 10 19:27:24.002605 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:24.002575    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdr9p\" (UniqueName: \"kubernetes.io/projected/d200af66-98aa-4db1-8ff7-54ecd1884852-kube-api-access-bdr9p\") pod \"d200af66-98aa-4db1-8ff7-54ecd1884852\" (UID: \"d200af66-98aa-4db1-8ff7-54ecd1884852\") "
Sep 10 19:27:24.002825 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:24.002733    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d200af66-98aa-4db1-8ff7-54ecd1884852-tls-cert\") pod \"d200af66-98aa-4db1-8ff7-54ecd1884852\" (UID: \"d200af66-98aa-4db1-8ff7-54ecd1884852\") "
Sep 10 19:27:24.004874 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:24.004847    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d200af66-98aa-4db1-8ff7-54ecd1884852-kube-api-access-bdr9p" (OuterVolumeSpecName: "kube-api-access-bdr9p") pod "d200af66-98aa-4db1-8ff7-54ecd1884852" (UID: "d200af66-98aa-4db1-8ff7-54ecd1884852"). InnerVolumeSpecName "kube-api-access-bdr9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:27:24.015590 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:24.015566    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d200af66-98aa-4db1-8ff7-54ecd1884852-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "d200af66-98aa-4db1-8ff7-54ecd1884852" (UID: "d200af66-98aa-4db1-8ff7-54ecd1884852"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:27:24.103529 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:24.103494    2579 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d200af66-98aa-4db1-8ff7-54ecd1884852-tls-cert\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:27:24.103529 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:24.103527    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdr9p\" (UniqueName: \"kubernetes.io/projected/d200af66-98aa-4db1-8ff7-54ecd1884852-kube-api-access-bdr9p\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:27:24.104669 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:24.104644    2579 generic.go:356] "Generic (PLEG): container finished" podID="d200af66-98aa-4db1-8ff7-54ecd1884852" containerID="200127a86974bfef5cc439f88b83c11cd5c08a541080af9096c29991427aa4a5" exitCode=0
Sep 10 19:27:24.104792 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:24.104688    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c5bbb57d-swn54"
Sep 10 19:27:24.104792 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:24.104733    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c5bbb57d-swn54" event={"ID":"d200af66-98aa-4db1-8ff7-54ecd1884852","Type":"ContainerDied","Data":"200127a86974bfef5cc439f88b83c11cd5c08a541080af9096c29991427aa4a5"}
Sep 10 19:27:24.104792 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:24.104788    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c5bbb57d-swn54" event={"ID":"d200af66-98aa-4db1-8ff7-54ecd1884852","Type":"ContainerDied","Data":"87b92cb8810e4bd31af46372193906dc7380d00fb01a9e7e101e57258a5e5e3b"}
Sep 10 19:27:24.104908 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:24.104805    2579 scope.go:117] "RemoveContainer" containerID="200127a86974bfef5cc439f88b83c11cd5c08a541080af9096c29991427aa4a5"
Sep 10 19:27:24.114866 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:24.114849    2579 scope.go:117] "RemoveContainer" containerID="200127a86974bfef5cc439f88b83c11cd5c08a541080af9096c29991427aa4a5"
Sep 10 19:27:24.115184 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:27:24.115163    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"200127a86974bfef5cc439f88b83c11cd5c08a541080af9096c29991427aa4a5\": container with ID starting with 200127a86974bfef5cc439f88b83c11cd5c08a541080af9096c29991427aa4a5 not found: ID does not exist" containerID="200127a86974bfef5cc439f88b83c11cd5c08a541080af9096c29991427aa4a5"
Sep 10 19:27:24.115240 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:24.115193    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"200127a86974bfef5cc439f88b83c11cd5c08a541080af9096c29991427aa4a5"} err="failed to get container status \"200127a86974bfef5cc439f88b83c11cd5c08a541080af9096c29991427aa4a5\": rpc error: code = NotFound desc = could not find container \"200127a86974bfef5cc439f88b83c11cd5c08a541080af9096c29991427aa4a5\": container with ID starting with 200127a86974bfef5cc439f88b83c11cd5c08a541080af9096c29991427aa4a5 not found: ID does not exist"
Sep 10 19:27:24.133592 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:24.133561    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c5bbb57d-swn54"]
Sep 10 19:27:24.142222 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:24.142193    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c5bbb57d-swn54"]
Sep 10 19:27:25.243181 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:25.243150    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d200af66-98aa-4db1-8ff7-54ecd1884852" path="/var/lib/kubelet/pods/d200af66-98aa-4db1-8ff7-54ecd1884852/volumes"
Sep 10 19:27:42.864112 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:42.864030    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"]
Sep 10 19:27:42.864680 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:42.864659    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d200af66-98aa-4db1-8ff7-54ecd1884852" containerName="authorino"
Sep 10 19:27:42.864733 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:42.864683    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d200af66-98aa-4db1-8ff7-54ecd1884852" containerName="authorino"
Sep 10 19:27:42.864826 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:42.864815    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d200af66-98aa-4db1-8ff7-54ecd1884852" containerName="authorino"
Sep 10 19:27:42.868298 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:42.868276    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:27:42.870460 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:42.870440    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-8300b9-openshift-default-dockercfg-wfzz6\""
Sep 10 19:27:42.875212 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:42.875164    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e405b89-cc1d-4169-a326-070959a02f66-credential-socket\") pod \"e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc\" (UID: \"6e405b89-cc1d-4169-a326-070959a02f66\") " pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:27:42.875212 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:42.875207    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6zss\" (UniqueName: \"kubernetes.io/projected/6e405b89-cc1d-4169-a326-070959a02f66-kube-api-access-r6zss\") pod \"e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc\" (UID: \"6e405b89-cc1d-4169-a326-070959a02f66\") " pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:27:42.875455 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:42.875247    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e405b89-cc1d-4169-a326-070959a02f66-workload-certs\") pod \"e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc\" (UID: \"6e405b89-cc1d-4169-a326-070959a02f66\") " pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:27:42.875455 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:42.875271    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e405b89-cc1d-4169-a326-070959a02f66-istiod-ca-cert\") pod \"e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc\" (UID: \"6e405b89-cc1d-4169-a326-070959a02f66\") " pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:27:42.875455 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:42.875304    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e405b89-cc1d-4169-a326-070959a02f66-istio-data\") pod \"e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc\" (UID: \"6e405b89-cc1d-4169-a326-070959a02f66\") " pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:27:42.875455 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:42.875332    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e405b89-cc1d-4169-a326-070959a02f66-istio-envoy\") pod \"e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc\" (UID: \"6e405b89-cc1d-4169-a326-070959a02f66\") " pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:27:42.875455 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:42.875412    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e405b89-cc1d-4169-a326-070959a02f66-workload-socket\") pod \"e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc\" (UID: \"6e405b89-cc1d-4169-a326-070959a02f66\") " pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:27:42.875455 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:42.875440    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e405b89-cc1d-4169-a326-070959a02f66-istio-token\") pod \"e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc\" (UID: \"6e405b89-cc1d-4169-a326-070959a02f66\") " pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:27:42.875870 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:42.875472    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e405b89-cc1d-4169-a326-070959a02f66-istio-podinfo\") pod \"e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc\" (UID: \"6e405b89-cc1d-4169-a326-070959a02f66\") " pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:27:42.878821 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:42.878799    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"]
Sep 10 19:27:42.976291 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:42.976217    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e405b89-cc1d-4169-a326-070959a02f66-credential-socket\") pod \"e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc\" (UID: \"6e405b89-cc1d-4169-a326-070959a02f66\") " pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:27:42.976481 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:42.976318    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6zss\" (UniqueName: \"kubernetes.io/projected/6e405b89-cc1d-4169-a326-070959a02f66-kube-api-access-r6zss\") pod \"e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc\" (UID: \"6e405b89-cc1d-4169-a326-070959a02f66\") " pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:27:42.976481 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:42.976366    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e405b89-cc1d-4169-a326-070959a02f66-workload-certs\") pod \"e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc\" (UID: \"6e405b89-cc1d-4169-a326-070959a02f66\") " pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:27:42.976704 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:42.976668    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e405b89-cc1d-4169-a326-070959a02f66-istiod-ca-cert\") pod \"e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc\" (UID: \"6e405b89-cc1d-4169-a326-070959a02f66\") " pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:27:42.976871 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:42.976674    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e405b89-cc1d-4169-a326-070959a02f66-credential-socket\") pod \"e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc\" (UID: \"6e405b89-cc1d-4169-a326-070959a02f66\") " pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:27:42.976871 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:42.976719    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e405b89-cc1d-4169-a326-070959a02f66-istio-data\") pod \"e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc\" (UID: \"6e405b89-cc1d-4169-a326-070959a02f66\") " pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:27:42.976871 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:42.976815    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e405b89-cc1d-4169-a326-070959a02f66-istio-envoy\") pod \"e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc\" (UID: \"6e405b89-cc1d-4169-a326-070959a02f66\") " pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:27:42.976871 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:42.976777    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e405b89-cc1d-4169-a326-070959a02f66-workload-certs\") pod \"e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc\" (UID: \"6e405b89-cc1d-4169-a326-070959a02f66\") " pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:27:42.977110 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:42.976941    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e405b89-cc1d-4169-a326-070959a02f66-workload-socket\") pod \"e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc\" (UID: \"6e405b89-cc1d-4169-a326-070959a02f66\") " pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:27:42.977110 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:42.976983    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e405b89-cc1d-4169-a326-070959a02f66-istio-token\") pod \"e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc\" (UID: \"6e405b89-cc1d-4169-a326-070959a02f66\") " pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:27:42.977110 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:42.977027    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e405b89-cc1d-4169-a326-070959a02f66-istio-data\") pod \"e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc\" (UID: \"6e405b89-cc1d-4169-a326-070959a02f66\") " pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:27:42.977110 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:42.977072    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e405b89-cc1d-4169-a326-070959a02f66-istio-podinfo\") pod \"e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc\" (UID: \"6e405b89-cc1d-4169-a326-070959a02f66\") " pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:27:42.977323 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:42.977244    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e405b89-cc1d-4169-a326-070959a02f66-workload-socket\") pod \"e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc\" (UID: \"6e405b89-cc1d-4169-a326-070959a02f66\") " pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:27:42.977362 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:42.977345    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e405b89-cc1d-4169-a326-070959a02f66-istiod-ca-cert\") pod \"e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc\" (UID: \"6e405b89-cc1d-4169-a326-070959a02f66\") " pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:27:42.979230 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:42.979187    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e405b89-cc1d-4169-a326-070959a02f66-istio-envoy\") pod \"e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc\" (UID: \"6e405b89-cc1d-4169-a326-070959a02f66\") " pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:27:42.979620 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:42.979593    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e405b89-cc1d-4169-a326-070959a02f66-istio-podinfo\") pod \"e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc\" (UID: \"6e405b89-cc1d-4169-a326-070959a02f66\") " pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:27:42.986059 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:42.986032    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e405b89-cc1d-4169-a326-070959a02f66-istio-token\") pod \"e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc\" (UID: \"6e405b89-cc1d-4169-a326-070959a02f66\") " pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:27:42.986167 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:42.986119    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6zss\" (UniqueName: \"kubernetes.io/projected/6e405b89-cc1d-4169-a326-070959a02f66-kube-api-access-r6zss\") pod \"e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc\" (UID: \"6e405b89-cc1d-4169-a326-070959a02f66\") " pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:27:43.181830 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:43.181798    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:27:43.460340 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:43.460309    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"]
Sep 10 19:27:43.461562 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:27:43.461535    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e405b89_cc1d_4169_a326_070959a02f66.slice/crio-0a95c08c672ab34b75a67681e280449ad62551f43fceb4e817848ee325480eec WatchSource:0}: Error finding container 0a95c08c672ab34b75a67681e280449ad62551f43fceb4e817848ee325480eec: Status 404 returned error can't find the container with id 0a95c08c672ab34b75a67681e280449ad62551f43fceb4e817848ee325480eec
Sep 10 19:27:43.464413 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:43.464368    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:27:43.464531 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:43.464464    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:27:43.464531 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:43.464509    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:27:44.225682 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:44.225643    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc" event={"ID":"6e405b89-cc1d-4169-a326-070959a02f66","Type":"ContainerStarted","Data":"eb9fef4c1d2b1b124c17c29676f718401ccf0636caae07cb4e47bdcc9995285e"}
Sep 10 19:27:44.226090 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:44.225705    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc" event={"ID":"6e405b89-cc1d-4169-a326-070959a02f66","Type":"ContainerStarted","Data":"0a95c08c672ab34b75a67681e280449ad62551f43fceb4e817848ee325480eec"}
Sep 10 19:27:44.268143 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:44.268085    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc" podStartSLOduration=2.268070176 podStartE2EDuration="2.268070176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:27:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:27:44.264940198 +0000 UTC m=+719.484788678" watchObservedRunningTime="2026-09-10 19:27:44.268070176 +0000 UTC m=+719.487918657"
Sep 10 19:27:44.786941 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:44.786907    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7467-88fc1e"]
Sep 10 19:27:44.790282 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:44.790267    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7467-88fc1e"
Sep 10 19:27:44.794236 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:44.794215    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-8vfd9\""
Sep 10 19:27:44.801637 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:44.801616    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7467-88fc1e"]
Sep 10 19:27:44.893807 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:44.893740    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lb4n\" (UniqueName: \"kubernetes.io/projected/9749fd37-6afd-487b-ab71-6c8c8b4de6ad-kube-api-access-2lb4n\") pod \"test-curl-7467-88fc1e\" (UID: \"9749fd37-6afd-487b-ab71-6c8c8b4de6ad\") " pod="opendatahub/test-curl-7467-88fc1e"
Sep 10 19:27:44.995127 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:44.995093    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lb4n\" (UniqueName: \"kubernetes.io/projected/9749fd37-6afd-487b-ab71-6c8c8b4de6ad-kube-api-access-2lb4n\") pod \"test-curl-7467-88fc1e\" (UID: \"9749fd37-6afd-487b-ab71-6c8c8b4de6ad\") " pod="opendatahub/test-curl-7467-88fc1e"
Sep 10 19:27:45.003842 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:45.003810    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lb4n\" (UniqueName: \"kubernetes.io/projected/9749fd37-6afd-487b-ab71-6c8c8b4de6ad-kube-api-access-2lb4n\") pod \"test-curl-7467-88fc1e\" (UID: \"9749fd37-6afd-487b-ab71-6c8c8b4de6ad\") " pod="opendatahub/test-curl-7467-88fc1e"
Sep 10 19:27:45.102281 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:45.102210    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-8vfd9\""
Sep 10 19:27:45.110942 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:45.110914    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7467-88fc1e"
Sep 10 19:27:45.182488 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:45.182446    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:27:45.187564 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:45.187535    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:27:45.232079 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:45.232030    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:27:45.243080 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:45.243055    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:27:45.385431 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:45.385405    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7467-88fc1e"]
Sep 10 19:27:45.387216 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:27:45.387197    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9749fd37_6afd_487b_ab71_6c8c8b4de6ad.slice/crio-a14477a3a81e02152d68caccbc7b794ece44fce089be56edfef594fed0d9c3aa WatchSource:0}: Error finding container a14477a3a81e02152d68caccbc7b794ece44fce089be56edfef594fed0d9c3aa: Status 404 returned error can't find the container with id a14477a3a81e02152d68caccbc7b794ece44fce089be56edfef594fed0d9c3aa
Sep 10 19:27:45.389474 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:45.389447    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 19:27:46.240187 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:46.240063    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7467-88fc1e" event={"ID":"9749fd37-6afd-487b-ab71-6c8c8b4de6ad","Type":"ContainerStarted","Data":"965e045b13c782cd0a7e919a3dc0346c27a303aee28226b4e4c52e809d614305"}
Sep 10 19:27:46.240187 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:46.240136    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7467-88fc1e" event={"ID":"9749fd37-6afd-487b-ab71-6c8c8b4de6ad","Type":"ContainerStarted","Data":"a14477a3a81e02152d68caccbc7b794ece44fce089be56edfef594fed0d9c3aa"}
Sep 10 19:27:46.259159 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:46.259108    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7467-88fc1e" podStartSLOduration=1.7365278480000002 podStartE2EDuration="2.259092226s" totalImagesPullingTime="522.564378ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:27:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:27:46.257116486 +0000 UTC m=+721.476964967" watchObservedRunningTime="2026-09-10 19:27:46.259092226 +0000 UTC m=+721.478940701"
Sep 10 19:27:46.896118 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:46.896057    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7467-88fc1e"]
Sep 10 19:27:46.899023 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:46.898991    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7467-88fc1e"]
Sep 10 19:27:47.045096 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:47.045013    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7467-053750"]
Sep 10 19:27:47.045459 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:47.045445    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9749fd37-6afd-487b-ab71-6c8c8b4de6ad" containerName="test-curl-7467-88fc1e"
Sep 10 19:27:47.045459 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:47.045459    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9749fd37-6afd-487b-ab71-6c8c8b4de6ad" containerName="test-curl-7467-88fc1e"
Sep 10 19:27:47.045555 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:47.045519    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9749fd37-6afd-487b-ab71-6c8c8b4de6ad" containerName="test-curl-7467-88fc1e"
Sep 10 19:27:47.048807 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:47.048790    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7467-053750"
Sep 10 19:27:47.056643 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:47.056617    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7467-053750"]
Sep 10 19:27:47.127018 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:47.126987    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mwsz\" (UniqueName: \"kubernetes.io/projected/f83cd7c5-e321-4ee8-b9c5-6826d5cb324c-kube-api-access-8mwsz\") pod \"test-curl-7467-053750\" (UID: \"f83cd7c5-e321-4ee8-b9c5-6826d5cb324c\") " pod="opendatahub/test-curl-7467-053750"
Sep 10 19:27:47.227476 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:47.227444    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mwsz\" (UniqueName: \"kubernetes.io/projected/f83cd7c5-e321-4ee8-b9c5-6826d5cb324c-kube-api-access-8mwsz\") pod \"test-curl-7467-053750\" (UID: \"f83cd7c5-e321-4ee8-b9c5-6826d5cb324c\") " pod="opendatahub/test-curl-7467-053750"
Sep 10 19:27:47.237176 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:47.237143    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mwsz\" (UniqueName: \"kubernetes.io/projected/f83cd7c5-e321-4ee8-b9c5-6826d5cb324c-kube-api-access-8mwsz\") pod \"test-curl-7467-053750\" (UID: \"f83cd7c5-e321-4ee8-b9c5-6826d5cb324c\") " pod="opendatahub/test-curl-7467-053750"
Sep 10 19:27:47.246061 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:27:47.246038    2579 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7467-88fc1e" container="test-curl-7467-88fc1e"
Sep 10 19:27:47.359279 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:47.359190    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7467-053750"
Sep 10 19:27:47.537680 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:47.537644    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29"]
Sep 10 19:27:47.542322 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:47.542300    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29"
Sep 10 19:27:47.544709 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:47.544685    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-8300b9-serving-cert\""
Sep 10 19:27:47.553201 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:47.553166    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29"]
Sep 10 19:27:47.631839 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:47.631810    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5pqv\" (UniqueName: \"kubernetes.io/projected/fb483fb7-6eb9-4b81-ad36-b7e75ed333f0-kube-api-access-s5pqv\") pod \"maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29\" (UID: \"fb483fb7-6eb9-4b81-ad36-b7e75ed333f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29"
Sep 10 19:27:47.631994 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:47.631869    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb483fb7-6eb9-4b81-ad36-b7e75ed333f0-maas-api-tls\") pod \"maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29\" (UID: \"fb483fb7-6eb9-4b81-ad36-b7e75ed333f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29"
Sep 10 19:27:47.631994 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:47.631946    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb483fb7-6eb9-4b81-ad36-b7e75ed333f0-metrics-certs\") pod \"maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29\" (UID: \"fb483fb7-6eb9-4b81-ad36-b7e75ed333f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29"
Sep 10 19:27:47.631994 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:47.631981    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb483fb7-6eb9-4b81-ad36-b7e75ed333f0-openshift-service-ca\") pod \"maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29\" (UID: \"fb483fb7-6eb9-4b81-ad36-b7e75ed333f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29"
Sep 10 19:27:47.648226 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:47.648203    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7467-053750"]
Sep 10 19:27:47.650783 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:27:47.650736    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf83cd7c5_e321_4ee8_b9c5_6826d5cb324c.slice/crio-40b9e0c70ed2f3652c7a0a4a9d0aa8f43297ffd70f4f4b01d7aa38f03ab606a6 WatchSource:0}: Error finding container 40b9e0c70ed2f3652c7a0a4a9d0aa8f43297ffd70f4f4b01d7aa38f03ab606a6: Status 404 returned error can't find the container with id 40b9e0c70ed2f3652c7a0a4a9d0aa8f43297ffd70f4f4b01d7aa38f03ab606a6
Sep 10 19:27:47.733399 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:47.733365    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5pqv\" (UniqueName: \"kubernetes.io/projected/fb483fb7-6eb9-4b81-ad36-b7e75ed333f0-kube-api-access-s5pqv\") pod \"maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29\" (UID: \"fb483fb7-6eb9-4b81-ad36-b7e75ed333f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29"
Sep 10 19:27:47.733562 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:47.733430    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb483fb7-6eb9-4b81-ad36-b7e75ed333f0-maas-api-tls\") pod \"maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29\" (UID: \"fb483fb7-6eb9-4b81-ad36-b7e75ed333f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29"
Sep 10 19:27:47.733562 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:47.733465    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb483fb7-6eb9-4b81-ad36-b7e75ed333f0-metrics-certs\") pod \"maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29\" (UID: \"fb483fb7-6eb9-4b81-ad36-b7e75ed333f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29"
Sep 10 19:27:47.733562 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:47.733500    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb483fb7-6eb9-4b81-ad36-b7e75ed333f0-openshift-service-ca\") pod \"maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29\" (UID: \"fb483fb7-6eb9-4b81-ad36-b7e75ed333f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29"
Sep 10 19:27:47.733686 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:27:47.733576    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-8300b9-serving-cert: secret "maas-api-e2e-inf-a-8300b9-serving-cert" not found
Sep 10 19:27:47.733686 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:27:47.733629    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb483fb7-6eb9-4b81-ad36-b7e75ed333f0-maas-api-tls podName:fb483fb7-6eb9-4b81-ad36-b7e75ed333f0 nodeName:}" failed. No retries permitted until 2026-09-10 19:27:48.233613637 +0000 UTC m=+723.453462096 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fb483fb7-6eb9-4b81-ad36-b7e75ed333f0-maas-api-tls") pod "maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29" (UID: "fb483fb7-6eb9-4b81-ad36-b7e75ed333f0") : secret "maas-api-e2e-inf-a-8300b9-serving-cert" not found
Sep 10 19:27:47.734273 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:47.734214    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb483fb7-6eb9-4b81-ad36-b7e75ed333f0-openshift-service-ca\") pod \"maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29\" (UID: \"fb483fb7-6eb9-4b81-ad36-b7e75ed333f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29"
Sep 10 19:27:47.736010 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:47.735982    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb483fb7-6eb9-4b81-ad36-b7e75ed333f0-metrics-certs\") pod \"maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29\" (UID: \"fb483fb7-6eb9-4b81-ad36-b7e75ed333f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29"
Sep 10 19:27:47.755033 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:47.755003    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5pqv\" (UniqueName: \"kubernetes.io/projected/fb483fb7-6eb9-4b81-ad36-b7e75ed333f0-kube-api-access-s5pqv\") pod \"maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29\" (UID: \"fb483fb7-6eb9-4b81-ad36-b7e75ed333f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29"
Sep 10 19:27:48.239495 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:48.239462    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb483fb7-6eb9-4b81-ad36-b7e75ed333f0-maas-api-tls\") pod \"maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29\" (UID: \"fb483fb7-6eb9-4b81-ad36-b7e75ed333f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29"
Sep 10 19:27:48.242219 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:48.242185    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb483fb7-6eb9-4b81-ad36-b7e75ed333f0-maas-api-tls\") pod \"maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29\" (UID: \"fb483fb7-6eb9-4b81-ad36-b7e75ed333f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29"
Sep 10 19:27:48.253310 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:48.253269    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7467-053750" event={"ID":"f83cd7c5-e321-4ee8-b9c5-6826d5cb324c","Type":"ContainerStarted","Data":"ea1a6f6eea020b7ec819c6ece38da09ab842b0d64d2f0696379bb8156665520f"}
Sep 10 19:27:48.253731 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:48.253320    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7467-053750" event={"ID":"f83cd7c5-e321-4ee8-b9c5-6826d5cb324c","Type":"ContainerStarted","Data":"40b9e0c70ed2f3652c7a0a4a9d0aa8f43297ffd70f4f4b01d7aa38f03ab606a6"}
Sep 10 19:27:48.253731 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:48.253599    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7467-88fc1e" podUID="9749fd37-6afd-487b-ab71-6c8c8b4de6ad" containerName="test-curl-7467-88fc1e" containerID="cri-o://965e045b13c782cd0a7e919a3dc0346c27a303aee28226b4e4c52e809d614305" gracePeriod=2
Sep 10 19:27:48.267976 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:48.267918    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7467-053750" podStartSLOduration=1.267899528 podStartE2EDuration="1.267899528s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:27:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:27:48.267227509 +0000 UTC m=+723.487075990" watchObservedRunningTime="2026-09-10 19:27:48.267899528 +0000 UTC m=+723.487748009"
Sep 10 19:27:48.455679 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:48.455643    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29"
Sep 10 19:27:48.740459 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:48.740422    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29"]
Sep 10 19:27:48.741934 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:27:48.741905    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb483fb7_6eb9_4b81_ad36_b7e75ed333f0.slice/crio-17e93b740d2c18c5760a22b30e9264d848a241dcd6cfd91236c38b1091947eff WatchSource:0}: Error finding container 17e93b740d2c18c5760a22b30e9264d848a241dcd6cfd91236c38b1091947eff: Status 404 returned error can't find the container with id 17e93b740d2c18c5760a22b30e9264d848a241dcd6cfd91236c38b1091947eff
Sep 10 19:27:48.910830 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:48.910794    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7467-053750"]
Sep 10 19:27:48.917282 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:48.917215    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7467-053750"]
Sep 10 19:27:49.263449 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:49.263406    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29" event={"ID":"fb483fb7-6eb9-4b81-ad36-b7e75ed333f0","Type":"ContainerStarted","Data":"3c23150cd6410b1a113229c833e653487af550e0288ece18f3a9741f7dd1bd87"}
Sep 10 19:27:49.263933 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:49.263475    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29" event={"ID":"fb483fb7-6eb9-4b81-ad36-b7e75ed333f0","Type":"ContainerStarted","Data":"17e93b740d2c18c5760a22b30e9264d848a241dcd6cfd91236c38b1091947eff"}
Sep 10 19:27:49.263933 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:27:49.263596    2579 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7467-053750" container="test-curl-7467-053750"
Sep 10 19:27:49.263933 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:49.263633    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29"
Sep 10 19:27:49.285929 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:49.285867    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29" podStartSLOduration=1.8502022120000001 podStartE2EDuration="2.285850579s" totalImagesPullingTime="435.648367ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:27:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:27:49.285065443 +0000 UTC m=+724.504913925" watchObservedRunningTime="2026-09-10 19:27:49.285850579 +0000 UTC m=+724.505699060"
Sep 10 19:27:49.884926 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:49.884894    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"]
Sep 10 19:27:49.885617 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:49.885597    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f83cd7c5-e321-4ee8-b9c5-6826d5cb324c" containerName="test-curl-7467-053750"
Sep 10 19:27:49.885662 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:49.885622    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f83cd7c5-e321-4ee8-b9c5-6826d5cb324c" containerName="test-curl-7467-053750"
Sep 10 19:27:49.885790 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:49.885777    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f83cd7c5-e321-4ee8-b9c5-6826d5cb324c" containerName="test-curl-7467-053750"
Sep 10 19:27:49.889476 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:49.889459    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:27:49.898187 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:49.898164    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-b3108b-openshift-default-dockercfg-2kk4k\""
Sep 10 19:27:49.912653 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:49.912625    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"]
Sep 10 19:27:49.954830 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:49.954798    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-credential-socket\") pod \"e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d\" (UID: \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\") " pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:27:49.955002 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:49.954842    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-istio-podinfo\") pod \"e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d\" (UID: \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\") " pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:27:49.955002 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:49.954920    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-workload-socket\") pod \"e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d\" (UID: \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\") " pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:27:49.955002 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:49.954950    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-istio-envoy\") pod \"e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d\" (UID: \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\") " pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:27:49.955002 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:49.954974    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-workload-certs\") pod \"e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d\" (UID: \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\") " pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:27:49.955002 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:49.954999    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-istio-data\") pod \"e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d\" (UID: \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\") " pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:27:49.955173 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:49.955048    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zfvm\" (UniqueName: \"kubernetes.io/projected/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-kube-api-access-5zfvm\") pod \"e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d\" (UID: \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\") " pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:27:49.955173 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:49.955093    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-istio-token\") pod \"e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d\" (UID: \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\") " pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:27:49.955173 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:49.955130    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-istiod-ca-cert\") pod \"e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d\" (UID: \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\") " pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:27:50.056401 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:50.056362    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zfvm\" (UniqueName: \"kubernetes.io/projected/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-kube-api-access-5zfvm\") pod \"e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d\" (UID: \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\") " pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:27:50.056401 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:50.056409    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-istio-token\") pod \"e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d\" (UID: \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\") " pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:27:50.056644 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:50.056442    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-istiod-ca-cert\") pod \"e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d\" (UID: \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\") " pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:27:50.056644 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:50.056503    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-credential-socket\") pod \"e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d\" (UID: \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\") " pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:27:50.056644 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:50.056537    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-istio-podinfo\") pod \"e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d\" (UID: \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\") " pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:27:50.056644 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:50.056597    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-workload-socket\") pod \"e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d\" (UID: \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\") " pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:27:50.056644 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:50.056631    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-istio-envoy\") pod \"e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d\" (UID: \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\") " pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:27:50.056923 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:50.056661    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-workload-certs\") pod \"e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d\" (UID: \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\") " pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:27:50.056923 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:50.056696    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-istio-data\") pod \"e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d\" (UID: \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\") " pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:27:50.057069 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:50.057044    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-credential-socket\") pod \"e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d\" (UID: \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\") " pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:27:50.057146 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:50.057084    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-workload-certs\") pod \"e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d\" (UID: \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\") " pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:27:50.057146 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:50.057132    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-workload-socket\") pod \"e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d\" (UID: \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\") " pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:27:50.057390 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:50.057363    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-istio-data\") pod \"e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d\" (UID: \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\") " pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:27:50.057390 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:50.057384    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-istiod-ca-cert\") pod \"e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d\" (UID: \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\") " pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:27:50.059232 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:50.059205    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-istio-envoy\") pod \"e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d\" (UID: \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\") " pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:27:50.059321 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:50.059308    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-istio-podinfo\") pod \"e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d\" (UID: \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\") " pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:27:50.066642 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:50.066615    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-istio-token\") pod \"e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d\" (UID: \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\") " pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:27:50.067536 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:50.067513    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zfvm\" (UniqueName: \"kubernetes.io/projected/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-kube-api-access-5zfvm\") pod \"e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d\" (UID: \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\") " pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:27:50.201716 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:50.201629    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:27:50.273526 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:50.273173    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7467-053750" podUID="f83cd7c5-e321-4ee8-b9c5-6826d5cb324c" containerName="test-curl-7467-053750" containerID="cri-o://ea1a6f6eea020b7ec819c6ece38da09ab842b0d64d2f0696379bb8156665520f" gracePeriod=2
Sep 10 19:27:50.412095 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:50.412071    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7467-88fc1e"
Sep 10 19:27:50.460102 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:50.460030    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2lb4n\" (UniqueName: \"kubernetes.io/projected/9749fd37-6afd-487b-ab71-6c8c8b4de6ad-kube-api-access-2lb4n\") pod \"9749fd37-6afd-487b-ab71-6c8c8b4de6ad\" (UID: \"9749fd37-6afd-487b-ab71-6c8c8b4de6ad\") "
Sep 10 19:27:50.462221 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:50.462197    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9749fd37-6afd-487b-ab71-6c8c8b4de6ad-kube-api-access-2lb4n" (OuterVolumeSpecName: "kube-api-access-2lb4n") pod "9749fd37-6afd-487b-ab71-6c8c8b4de6ad" (UID: "9749fd37-6afd-487b-ab71-6c8c8b4de6ad"). InnerVolumeSpecName "kube-api-access-2lb4n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:27:50.490010 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:50.489984    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"]
Sep 10 19:27:50.492241 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:27:50.492214    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4b7c1d3_1982_4ff4_9552_19cb2043c4fe.slice/crio-41aa73ce791e65f8a3f57a8f8b7cdbe29366ce9f415af193746868b814b189a3 WatchSource:0}: Error finding container 41aa73ce791e65f8a3f57a8f8b7cdbe29366ce9f415af193746868b814b189a3: Status 404 returned error can't find the container with id 41aa73ce791e65f8a3f57a8f8b7cdbe29366ce9f415af193746868b814b189a3
Sep 10 19:27:50.494595 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:50.494555    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:27:50.494675 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:50.494651    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:27:50.494714 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:50.494700    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:27:50.561210 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:50.561190    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2lb4n\" (UniqueName: \"kubernetes.io/projected/9749fd37-6afd-487b-ab71-6c8c8b4de6ad-kube-api-access-2lb4n\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:27:51.239648 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:51.239612    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9749fd37-6afd-487b-ab71-6c8c8b4de6ad" path="/var/lib/kubelet/pods/9749fd37-6afd-487b-ab71-6c8c8b4de6ad/volumes"
Sep 10 19:27:51.279889 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:51.279857    2579 generic.go:356] "Generic (PLEG): container finished" podID="9749fd37-6afd-487b-ab71-6c8c8b4de6ad" containerID="965e045b13c782cd0a7e919a3dc0346c27a303aee28226b4e4c52e809d614305" exitCode=137
Sep 10 19:27:51.280275 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:51.279895    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7467-88fc1e"
Sep 10 19:27:51.280275 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:51.279948    2579 scope.go:117] "RemoveContainer" containerID="965e045b13c782cd0a7e919a3dc0346c27a303aee28226b4e4c52e809d614305"
Sep 10 19:27:51.281977 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:51.281946    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d" event={"ID":"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe","Type":"ContainerStarted","Data":"0114dde241e410a36bf0c73d37d33aa20ed73faa71f00caa2fef7ed2c6b77f8d"}
Sep 10 19:27:51.282105 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:51.282007    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d" event={"ID":"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe","Type":"ContainerStarted","Data":"41aa73ce791e65f8a3f57a8f8b7cdbe29366ce9f415af193746868b814b189a3"}
Sep 10 19:27:51.289145 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:51.289131    2579 scope.go:117] "RemoveContainer" containerID="965e045b13c782cd0a7e919a3dc0346c27a303aee28226b4e4c52e809d614305"
Sep 10 19:27:51.289419 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:27:51.289401    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"965e045b13c782cd0a7e919a3dc0346c27a303aee28226b4e4c52e809d614305\": container with ID starting with 965e045b13c782cd0a7e919a3dc0346c27a303aee28226b4e4c52e809d614305 not found: ID does not exist" containerID="965e045b13c782cd0a7e919a3dc0346c27a303aee28226b4e4c52e809d614305"
Sep 10 19:27:51.289481 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:51.289432    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"965e045b13c782cd0a7e919a3dc0346c27a303aee28226b4e4c52e809d614305"} err="failed to get container status \"965e045b13c782cd0a7e919a3dc0346c27a303aee28226b4e4c52e809d614305\": rpc error: code = NotFound desc = could not find container \"965e045b13c782cd0a7e919a3dc0346c27a303aee28226b4e4c52e809d614305\": container with ID starting with 965e045b13c782cd0a7e919a3dc0346c27a303aee28226b4e4c52e809d614305 not found: ID does not exist"
Sep 10 19:27:51.313299 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:51.313256    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d" podStartSLOduration=2.313241091 podStartE2EDuration="2.313241091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:27:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:27:51.310458317 +0000 UTC m=+726.530306797" watchObservedRunningTime="2026-09-10 19:27:51.313241091 +0000 UTC m=+726.533089562"
Sep 10 19:27:52.202241 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:52.202205    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:27:52.207278 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:52.207254    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:27:52.290938 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:52.290917    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:27:52.291965 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:52.291944    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:27:52.451225 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:52.451203    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7467-053750"
Sep 10 19:27:52.483657 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:52.483577    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mwsz\" (UniqueName: \"kubernetes.io/projected/f83cd7c5-e321-4ee8-b9c5-6826d5cb324c-kube-api-access-8mwsz\") pod \"f83cd7c5-e321-4ee8-b9c5-6826d5cb324c\" (UID: \"f83cd7c5-e321-4ee8-b9c5-6826d5cb324c\") "
Sep 10 19:27:52.485792 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:52.485739    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f83cd7c5-e321-4ee8-b9c5-6826d5cb324c-kube-api-access-8mwsz" (OuterVolumeSpecName: "kube-api-access-8mwsz") pod "f83cd7c5-e321-4ee8-b9c5-6826d5cb324c" (UID: "f83cd7c5-e321-4ee8-b9c5-6826d5cb324c"). InnerVolumeSpecName "kube-api-access-8mwsz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:27:52.584889 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:52.584852    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mwsz\" (UniqueName: \"kubernetes.io/projected/f83cd7c5-e321-4ee8-b9c5-6826d5cb324c-kube-api-access-8mwsz\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:27:53.239230 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:53.239196    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f83cd7c5-e321-4ee8-b9c5-6826d5cb324c" path="/var/lib/kubelet/pods/f83cd7c5-e321-4ee8-b9c5-6826d5cb324c/volumes"
Sep 10 19:27:53.300087 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:53.299907    2579 generic.go:356] "Generic (PLEG): container finished" podID="f83cd7c5-e321-4ee8-b9c5-6826d5cb324c" containerID="ea1a6f6eea020b7ec819c6ece38da09ab842b0d64d2f0696379bb8156665520f" exitCode=137
Sep 10 19:27:53.300087 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:53.299962    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7467-053750"
Sep 10 19:27:53.300087 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:53.300029    2579 scope.go:117] "RemoveContainer" containerID="ea1a6f6eea020b7ec819c6ece38da09ab842b0d64d2f0696379bb8156665520f"
Sep 10 19:27:53.311869 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:53.311844    2579 scope.go:117] "RemoveContainer" containerID="ea1a6f6eea020b7ec819c6ece38da09ab842b0d64d2f0696379bb8156665520f"
Sep 10 19:27:53.312226 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:27:53.312204    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea1a6f6eea020b7ec819c6ece38da09ab842b0d64d2f0696379bb8156665520f\": container with ID starting with ea1a6f6eea020b7ec819c6ece38da09ab842b0d64d2f0696379bb8156665520f not found: ID does not exist" containerID="ea1a6f6eea020b7ec819c6ece38da09ab842b0d64d2f0696379bb8156665520f"
Sep 10 19:27:53.312286 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:53.312238    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea1a6f6eea020b7ec819c6ece38da09ab842b0d64d2f0696379bb8156665520f"} err="failed to get container status \"ea1a6f6eea020b7ec819c6ece38da09ab842b0d64d2f0696379bb8156665520f\": rpc error: code = NotFound desc = could not find container \"ea1a6f6eea020b7ec819c6ece38da09ab842b0d64d2f0696379bb8156665520f\": container with ID starting with ea1a6f6eea020b7ec819c6ece38da09ab842b0d64d2f0696379bb8156665520f not found: ID does not exist"
Sep 10 19:27:54.734471 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:54.734431    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd"]
Sep 10 19:27:54.740274 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:54.740248    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd"
Sep 10 19:27:54.742567 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:54.742542    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-b3108b-serving-cert\""
Sep 10 19:27:54.748573 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:54.748545    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd"]
Sep 10 19:27:54.806925 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:54.806887    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/200d7c73-5704-484c-b3cb-047941982034-metrics-certs\") pod \"maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd\" (UID: \"200d7c73-5704-484c-b3cb-047941982034\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd"
Sep 10 19:27:54.806925 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:54.806930    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/200d7c73-5704-484c-b3cb-047941982034-openshift-service-ca\") pod \"maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd\" (UID: \"200d7c73-5704-484c-b3cb-047941982034\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd"
Sep 10 19:27:54.807171 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:54.806962    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/200d7c73-5704-484c-b3cb-047941982034-maas-api-tls\") pod \"maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd\" (UID: \"200d7c73-5704-484c-b3cb-047941982034\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd"
Sep 10 19:27:54.807171 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:54.807044    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kh7l\" (UniqueName: \"kubernetes.io/projected/200d7c73-5704-484c-b3cb-047941982034-kube-api-access-7kh7l\") pod \"maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd\" (UID: \"200d7c73-5704-484c-b3cb-047941982034\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd"
Sep 10 19:27:54.908377 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:54.908327    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/200d7c73-5704-484c-b3cb-047941982034-metrics-certs\") pod \"maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd\" (UID: \"200d7c73-5704-484c-b3cb-047941982034\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd"
Sep 10 19:27:54.908377 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:54.908380    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/200d7c73-5704-484c-b3cb-047941982034-openshift-service-ca\") pod \"maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd\" (UID: \"200d7c73-5704-484c-b3cb-047941982034\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd"
Sep 10 19:27:54.908645 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:54.908411    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/200d7c73-5704-484c-b3cb-047941982034-maas-api-tls\") pod \"maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd\" (UID: \"200d7c73-5704-484c-b3cb-047941982034\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd"
Sep 10 19:27:54.908645 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:54.908435    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kh7l\" (UniqueName: \"kubernetes.io/projected/200d7c73-5704-484c-b3cb-047941982034-kube-api-access-7kh7l\") pod \"maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd\" (UID: \"200d7c73-5704-484c-b3cb-047941982034\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd"
Sep 10 19:27:54.909139 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:54.909114    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/200d7c73-5704-484c-b3cb-047941982034-openshift-service-ca\") pod \"maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd\" (UID: \"200d7c73-5704-484c-b3cb-047941982034\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd"
Sep 10 19:27:54.910895 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:54.910871    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/200d7c73-5704-484c-b3cb-047941982034-metrics-certs\") pod \"maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd\" (UID: \"200d7c73-5704-484c-b3cb-047941982034\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd"
Sep 10 19:27:54.910991 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:54.910957    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/200d7c73-5704-484c-b3cb-047941982034-maas-api-tls\") pod \"maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd\" (UID: \"200d7c73-5704-484c-b3cb-047941982034\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd"
Sep 10 19:27:54.916732 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:54.916711    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kh7l\" (UniqueName: \"kubernetes.io/projected/200d7c73-5704-484c-b3cb-047941982034-kube-api-access-7kh7l\") pod \"maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd\" (UID: \"200d7c73-5704-484c-b3cb-047941982034\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd"
Sep 10 19:27:55.080683 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:55.080590    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd"
Sep 10 19:27:55.285129 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:55.285100    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29"
Sep 10 19:27:55.386583 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:55.386546    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd"]
Sep 10 19:27:55.388357 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:27:55.388329    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod200d7c73_5704_484c_b3cb_047941982034.slice/crio-9fbe154b131c9982e1201a97430ea83c7ec3f99c814fde2fb148f675d625856f WatchSource:0}: Error finding container 9fbe154b131c9982e1201a97430ea83c7ec3f99c814fde2fb148f675d625856f: Status 404 returned error can't find the container with id 9fbe154b131c9982e1201a97430ea83c7ec3f99c814fde2fb148f675d625856f
Sep 10 19:27:56.323990 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:56.323956    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd" event={"ID":"200d7c73-5704-484c-b3cb-047941982034","Type":"ContainerStarted","Data":"fe17db926da08ff4ebebf9978329f02d17a811e9e3a647c426f1bd63c9f31eb7"}
Sep 10 19:27:56.324380 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:56.324028    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd" event={"ID":"200d7c73-5704-484c-b3cb-047941982034","Type":"ContainerStarted","Data":"9fbe154b131c9982e1201a97430ea83c7ec3f99c814fde2fb148f675d625856f"}
Sep 10 19:27:56.324380 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:56.324064    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd"
Sep 10 19:27:56.342667 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:56.342619    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd" podStartSLOduration=2.018200444 podStartE2EDuration="2.342605162s" totalImagesPullingTime="324.404718ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:27:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:27:56.341302501 +0000 UTC m=+731.561150979" watchObservedRunningTime="2026-09-10 19:27:56.342605162 +0000 UTC m=+731.562453707"
Sep 10 19:27:59.255043 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:59.254673    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv"]
Sep 10 19:27:59.260493 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:59.260449    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv"
Sep 10 19:27:59.264225 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:59.264181    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-85e90bf8-serving-cert\""
Sep 10 19:27:59.267039 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:59.266391    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv"]
Sep 10 19:27:59.358166 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:59.358128    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66230911-e709-4279-9b11-5dc52f248145-metrics-certs\") pod \"maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv\" (UID: \"66230911-e709-4279-9b11-5dc52f248145\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv"
Sep 10 19:27:59.358346 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:59.358295    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66230911-e709-4279-9b11-5dc52f248145-maas-api-tls\") pod \"maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv\" (UID: \"66230911-e709-4279-9b11-5dc52f248145\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv"
Sep 10 19:27:59.358407 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:59.358360    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66230911-e709-4279-9b11-5dc52f248145-openshift-service-ca\") pod \"maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv\" (UID: \"66230911-e709-4279-9b11-5dc52f248145\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv"
Sep 10 19:27:59.358407 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:59.358386    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4pfk\" (UniqueName: \"kubernetes.io/projected/66230911-e709-4279-9b11-5dc52f248145-kube-api-access-k4pfk\") pod \"maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv\" (UID: \"66230911-e709-4279-9b11-5dc52f248145\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv"
Sep 10 19:27:59.459921 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:59.459886    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66230911-e709-4279-9b11-5dc52f248145-maas-api-tls\") pod \"maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv\" (UID: \"66230911-e709-4279-9b11-5dc52f248145\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv"
Sep 10 19:27:59.460123 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:59.459947    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66230911-e709-4279-9b11-5dc52f248145-openshift-service-ca\") pod \"maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv\" (UID: \"66230911-e709-4279-9b11-5dc52f248145\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv"
Sep 10 19:27:59.460123 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:59.459978    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4pfk\" (UniqueName: \"kubernetes.io/projected/66230911-e709-4279-9b11-5dc52f248145-kube-api-access-k4pfk\") pod \"maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv\" (UID: \"66230911-e709-4279-9b11-5dc52f248145\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv"
Sep 10 19:27:59.460123 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:59.460049    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66230911-e709-4279-9b11-5dc52f248145-metrics-certs\") pod \"maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv\" (UID: \"66230911-e709-4279-9b11-5dc52f248145\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv"
Sep 10 19:27:59.460286 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:27:59.460052    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-85e90bf8-serving-cert: secret "maas-api-e2e-ait-85e90bf8-serving-cert" not found
Sep 10 19:27:59.460286 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:27:59.460256    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66230911-e709-4279-9b11-5dc52f248145-maas-api-tls podName:66230911-e709-4279-9b11-5dc52f248145 nodeName:}" failed. No retries permitted until 2026-09-10 19:27:59.960224002 +0000 UTC m=+735.180072476 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/66230911-e709-4279-9b11-5dc52f248145-maas-api-tls") pod "maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv" (UID: "66230911-e709-4279-9b11-5dc52f248145") : secret "maas-api-e2e-ait-85e90bf8-serving-cert" not found
Sep 10 19:27:59.460683 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:59.460659    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66230911-e709-4279-9b11-5dc52f248145-openshift-service-ca\") pod \"maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv\" (UID: \"66230911-e709-4279-9b11-5dc52f248145\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv"
Sep 10 19:27:59.462638 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:59.462617    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66230911-e709-4279-9b11-5dc52f248145-metrics-certs\") pod \"maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv\" (UID: \"66230911-e709-4279-9b11-5dc52f248145\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv"
Sep 10 19:27:59.475342 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:59.475316    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4pfk\" (UniqueName: \"kubernetes.io/projected/66230911-e709-4279-9b11-5dc52f248145-kube-api-access-k4pfk\") pod \"maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv\" (UID: \"66230911-e709-4279-9b11-5dc52f248145\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv"
Sep 10 19:27:59.966338 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:59.966296    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66230911-e709-4279-9b11-5dc52f248145-maas-api-tls\") pod \"maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv\" (UID: \"66230911-e709-4279-9b11-5dc52f248145\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv"
Sep 10 19:27:59.969716 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:27:59.969680    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66230911-e709-4279-9b11-5dc52f248145-maas-api-tls\") pod \"maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv\" (UID: \"66230911-e709-4279-9b11-5dc52f248145\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv"
Sep 10 19:28:00.177055 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:00.176785    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv"
Sep 10 19:28:00.480407 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:00.480346    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv"]
Sep 10 19:28:01.273289 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:01.272899    2579 patch_prober.go:28] interesting pod/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 19:28:01.273289 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:01.272960    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc" podUID="6e405b89-cc1d-4169-a326-070959a02f66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 19:28:01.360690 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:01.360648    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv" event={"ID":"66230911-e709-4279-9b11-5dc52f248145","Type":"ContainerStarted","Data":"a4be7b9d7755cdc5961ff55db54e2f5e7a68e1d48dbaa33da828ea8068ed9e1d"}
Sep 10 19:28:01.360911 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:01.360719    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv" event={"ID":"66230911-e709-4279-9b11-5dc52f248145","Type":"ContainerStarted","Data":"5bfb02c96d5173a52f810a5fc84d7aedcb8cf53a644d74ae028bc33f4e37df7f"}
Sep 10 19:28:01.361637 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:01.361610    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv"
Sep 10 19:28:01.385479 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:01.385423    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv" podStartSLOduration=1.75468099 podStartE2EDuration="2.385407341s" totalImagesPullingTime="630.726351ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:27:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:28:01.381622472 +0000 UTC m=+736.601470953" watchObservedRunningTime="2026-09-10 19:28:01.385407341 +0000 UTC m=+736.605255821"
Sep 10 19:28:02.337335 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:02.337159    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd"
Sep 10 19:28:03.384487 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:03.384454    2579 generic.go:356] "Generic (PLEG): container finished" podID="6e405b89-cc1d-4169-a326-070959a02f66" containerID="eb9fef4c1d2b1b124c17c29676f718401ccf0636caae07cb4e47bdcc9995285e" exitCode=0
Sep 10 19:28:03.384879 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:03.384520    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc" event={"ID":"6e405b89-cc1d-4169-a326-070959a02f66","Type":"ContainerDied","Data":"eb9fef4c1d2b1b124c17c29676f718401ccf0636caae07cb4e47bdcc9995285e"}
Sep 10 19:28:03.384952 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:03.384936    2579 scope.go:117] "RemoveContainer" containerID="eb9fef4c1d2b1b124c17c29676f718401ccf0636caae07cb4e47bdcc9995285e"
Sep 10 19:28:03.385959 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:03.385934    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:28:03.386032 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:03.386018    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:28:03.386080 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:03.386057    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:28:04.398035 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:04.397984    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc" event={"ID":"6e405b89-cc1d-4169-a326-070959a02f66","Type":"ContainerStarted","Data":"eb50015b36c5da678c80829aaa8ee50d31a3e3cde288b7bca5d0fc7b23baa3fc"}
Sep 10 19:28:05.183127 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:05.183090    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:28:05.183876 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:05.183418    2579 patch_prober.go:28] interesting pod/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 10 19:28:05.183876 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:05.183565    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc" podUID="6e405b89-cc1d-4169-a326-070959a02f66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 10 19:28:06.182166 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:06.182136    2579 patch_prober.go:28] interesting pod/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 10 19:28:06.182555 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:06.182193    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc" podUID="6e405b89-cc1d-4169-a326-070959a02f66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 10 19:28:07.183188 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:07.183141    2579 patch_prober.go:28] interesting pod/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 10 19:28:07.183581 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:07.183217    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc" podUID="6e405b89-cc1d-4169-a326-070959a02f66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 10 19:28:08.182696 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:08.182651    2579 patch_prober.go:28] interesting pod/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 10 19:28:08.182891 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:08.182722    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc" podUID="6e405b89-cc1d-4169-a326-070959a02f66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 10 19:28:08.332924 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:08.332884    2579 patch_prober.go:28] interesting pod/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 19:28:08.333324 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:08.332987    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d" podUID="c4b7c1d3-1982-4ff4-9552-19cb2043c4fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 19:28:08.389991 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:08.389958    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv"
Sep 10 19:28:09.183008 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:09.182960    2579 patch_prober.go:28] interesting pod/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 10 19:28:09.183197 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:09.183043    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc" podUID="6e405b89-cc1d-4169-a326-070959a02f66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 10 19:28:10.186215 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:10.186184    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:28:10.186626 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:10.186464    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:28:10.187156 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:10.187137    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:28:10.442052 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:10.441971    2579 generic.go:356] "Generic (PLEG): container finished" podID="c4b7c1d3-1982-4ff4-9552-19cb2043c4fe" containerID="0114dde241e410a36bf0c73d37d33aa20ed73faa71f00caa2fef7ed2c6b77f8d" exitCode=0
Sep 10 19:28:10.442195 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:10.442050    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d" event={"ID":"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe","Type":"ContainerDied","Data":"0114dde241e410a36bf0c73d37d33aa20ed73faa71f00caa2fef7ed2c6b77f8d"}
Sep 10 19:28:10.442521 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:10.442504    2579 scope.go:117] "RemoveContainer" containerID="0114dde241e410a36bf0c73d37d33aa20ed73faa71f00caa2fef7ed2c6b77f8d"
Sep 10 19:28:10.443574 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:10.443543    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:28:10.443720 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:10.443628    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:28:10.443720 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:10.443699    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:28:11.450689 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:11.450655    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d" event={"ID":"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe","Type":"ContainerStarted","Data":"4f4d7396ce0d16e0c86f08af912b68910de3426d836f8f03bfdbf49b9f5dc2ed"}
Sep 10 19:28:12.202424 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:12.202390    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:28:12.202616 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:12.202539    2579 patch_prober.go:28] interesting pod/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 10 19:28:12.202616 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:12.202586    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d" podUID="c4b7c1d3-1982-4ff4-9552-19cb2043c4fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 10 19:28:13.202641 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:13.202605    2579 patch_prober.go:28] interesting pod/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 10 19:28:13.203094 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:13.202677    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d" podUID="c4b7c1d3-1982-4ff4-9552-19cb2043c4fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 10 19:28:14.202639 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:14.202600    2579 patch_prober.go:28] interesting pod/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 10 19:28:14.202855 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:14.202654    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d" podUID="c4b7c1d3-1982-4ff4-9552-19cb2043c4fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 10 19:28:15.202123 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:15.202086    2579 patch_prober.go:28] interesting pod/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 10 19:28:15.202319 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:15.202143    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d" podUID="c4b7c1d3-1982-4ff4-9552-19cb2043c4fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 10 19:28:16.149338 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:16.149305    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb"]
Sep 10 19:28:16.177914 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:16.177886    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb"
Sep 10 19:28:16.179413 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:16.179362    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb"]
Sep 10 19:28:16.184094 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:16.181608    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-4247c7-serving-cert\""
Sep 10 19:28:16.202477 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:16.202449    2579 patch_prober.go:28] interesting pod/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 10 19:28:16.202599 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:16.202496    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d" podUID="c4b7c1d3-1982-4ff4-9552-19cb2043c4fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 10 19:28:16.365734 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:16.365697    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ef42da1-2dd3-473b-9e8b-7cf390d640a3-metrics-certs\") pod \"maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb\" (UID: \"3ef42da1-2dd3-473b-9e8b-7cf390d640a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb"
Sep 10 19:28:16.365734 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:16.365738    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5nzk\" (UniqueName: \"kubernetes.io/projected/3ef42da1-2dd3-473b-9e8b-7cf390d640a3-kube-api-access-t5nzk\") pod \"maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb\" (UID: \"3ef42da1-2dd3-473b-9e8b-7cf390d640a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb"
Sep 10 19:28:16.366000 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:16.365825    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ef42da1-2dd3-473b-9e8b-7cf390d640a3-maas-api-tls\") pod \"maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb\" (UID: \"3ef42da1-2dd3-473b-9e8b-7cf390d640a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb"
Sep 10 19:28:16.366000 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:16.365898    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ef42da1-2dd3-473b-9e8b-7cf390d640a3-openshift-service-ca\") pod \"maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb\" (UID: \"3ef42da1-2dd3-473b-9e8b-7cf390d640a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb"
Sep 10 19:28:16.466845 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:16.466764    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ef42da1-2dd3-473b-9e8b-7cf390d640a3-openshift-service-ca\") pod \"maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb\" (UID: \"3ef42da1-2dd3-473b-9e8b-7cf390d640a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb"
Sep 10 19:28:16.467021 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:16.466868    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ef42da1-2dd3-473b-9e8b-7cf390d640a3-metrics-certs\") pod \"maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb\" (UID: \"3ef42da1-2dd3-473b-9e8b-7cf390d640a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb"
Sep 10 19:28:16.467021 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:16.466887    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5nzk\" (UniqueName: \"kubernetes.io/projected/3ef42da1-2dd3-473b-9e8b-7cf390d640a3-kube-api-access-t5nzk\") pod \"maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb\" (UID: \"3ef42da1-2dd3-473b-9e8b-7cf390d640a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb"
Sep 10 19:28:16.467021 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:16.466908    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ef42da1-2dd3-473b-9e8b-7cf390d640a3-maas-api-tls\") pod \"maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb\" (UID: \"3ef42da1-2dd3-473b-9e8b-7cf390d640a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb"
Sep 10 19:28:16.467382 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:16.467359    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ef42da1-2dd3-473b-9e8b-7cf390d640a3-openshift-service-ca\") pod \"maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb\" (UID: \"3ef42da1-2dd3-473b-9e8b-7cf390d640a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb"
Sep 10 19:28:16.469435 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:16.469414    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ef42da1-2dd3-473b-9e8b-7cf390d640a3-maas-api-tls\") pod \"maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb\" (UID: \"3ef42da1-2dd3-473b-9e8b-7cf390d640a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb"
Sep 10 19:28:16.469520 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:16.469415    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ef42da1-2dd3-473b-9e8b-7cf390d640a3-metrics-certs\") pod \"maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb\" (UID: \"3ef42da1-2dd3-473b-9e8b-7cf390d640a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb"
Sep 10 19:28:16.488660 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:16.488641    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5nzk\" (UniqueName: \"kubernetes.io/projected/3ef42da1-2dd3-473b-9e8b-7cf390d640a3-kube-api-access-t5nzk\") pod \"maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb\" (UID: \"3ef42da1-2dd3-473b-9e8b-7cf390d640a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb"
Sep 10 19:28:16.494231 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:16.494205    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb"
Sep 10 19:28:16.776864 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:16.776685    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb"]
Sep 10 19:28:16.779645 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:28:16.779614    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ef42da1_2dd3_473b_9e8b_7cf390d640a3.slice/crio-e98003093732c3182fa5eff86db4463b0cf3ff01d8d78c209526286114788522 WatchSource:0}: Error finding container e98003093732c3182fa5eff86db4463b0cf3ff01d8d78c209526286114788522: Status 404 returned error can't find the container with id e98003093732c3182fa5eff86db4463b0cf3ff01d8d78c209526286114788522
Sep 10 19:28:17.207019 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:17.206994    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:28:17.207546 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:17.207527    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:28:17.208334 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:17.208315    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:28:17.504407 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:17.504303    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb" event={"ID":"3ef42da1-2dd3-473b-9e8b-7cf390d640a3","Type":"ContainerStarted","Data":"2073afa8099436382b414be72309823891399b99080d9ed0e5d4d7a77e8b27b2"}
Sep 10 19:28:17.504407 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:17.504366    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb" event={"ID":"3ef42da1-2dd3-473b-9e8b-7cf390d640a3","Type":"ContainerStarted","Data":"e98003093732c3182fa5eff86db4463b0cf3ff01d8d78c209526286114788522"}
Sep 10 19:28:17.504635 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:17.504522    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb"
Sep 10 19:28:17.530123 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:17.530073    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb" podStartSLOduration=1.144251914 podStartE2EDuration="1.530054136s" totalImagesPullingTime="385.802222ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:28:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:28:17.527393774 +0000 UTC m=+752.747242280" watchObservedRunningTime="2026-09-10 19:28:17.530054136 +0000 UTC m=+752.749902615"
Sep 10 19:28:23.516925 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:23.516895    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb"
Sep 10 19:28:40.379484 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:40.379451    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv"]
Sep 10 19:28:40.380090 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:40.379735    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv" podUID="66230911-e709-4279-9b11-5dc52f248145" containerName="maas-api" containerID="cri-o://a4be7b9d7755cdc5961ff55db54e2f5e7a68e1d48dbaa33da828ea8068ed9e1d" gracePeriod=30
Sep 10 19:28:40.638841 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:40.638812    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv"
Sep 10 19:28:40.651497 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:40.651464    2579 generic.go:356] "Generic (PLEG): container finished" podID="66230911-e709-4279-9b11-5dc52f248145" containerID="a4be7b9d7755cdc5961ff55db54e2f5e7a68e1d48dbaa33da828ea8068ed9e1d" exitCode=0
Sep 10 19:28:40.651765 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:40.651540    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv"
Sep 10 19:28:40.652131 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:40.651549    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv" event={"ID":"66230911-e709-4279-9b11-5dc52f248145","Type":"ContainerDied","Data":"a4be7b9d7755cdc5961ff55db54e2f5e7a68e1d48dbaa33da828ea8068ed9e1d"}
Sep 10 19:28:40.652131 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:40.651897    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv" event={"ID":"66230911-e709-4279-9b11-5dc52f248145","Type":"ContainerDied","Data":"5bfb02c96d5173a52f810a5fc84d7aedcb8cf53a644d74ae028bc33f4e37df7f"}
Sep 10 19:28:40.652131 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:40.651935    2579 scope.go:117] "RemoveContainer" containerID="a4be7b9d7755cdc5961ff55db54e2f5e7a68e1d48dbaa33da828ea8068ed9e1d"
Sep 10 19:28:40.665653 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:40.665611    2579 scope.go:117] "RemoveContainer" containerID="a4be7b9d7755cdc5961ff55db54e2f5e7a68e1d48dbaa33da828ea8068ed9e1d"
Sep 10 19:28:40.666088 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:28:40.666063    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4be7b9d7755cdc5961ff55db54e2f5e7a68e1d48dbaa33da828ea8068ed9e1d\": container with ID starting with a4be7b9d7755cdc5961ff55db54e2f5e7a68e1d48dbaa33da828ea8068ed9e1d not found: ID does not exist" containerID="a4be7b9d7755cdc5961ff55db54e2f5e7a68e1d48dbaa33da828ea8068ed9e1d"
Sep 10 19:28:40.666201 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:40.666097    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4be7b9d7755cdc5961ff55db54e2f5e7a68e1d48dbaa33da828ea8068ed9e1d"} err="failed to get container status \"a4be7b9d7755cdc5961ff55db54e2f5e7a68e1d48dbaa33da828ea8068ed9e1d\": rpc error: code = NotFound desc = could not find container \"a4be7b9d7755cdc5961ff55db54e2f5e7a68e1d48dbaa33da828ea8068ed9e1d\": container with ID starting with a4be7b9d7755cdc5961ff55db54e2f5e7a68e1d48dbaa33da828ea8068ed9e1d not found: ID does not exist"
Sep 10 19:28:40.714370 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:40.714336    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66230911-e709-4279-9b11-5dc52f248145-metrics-certs\") pod \"66230911-e709-4279-9b11-5dc52f248145\" (UID: \"66230911-e709-4279-9b11-5dc52f248145\") "
Sep 10 19:28:40.714521 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:40.714406    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66230911-e709-4279-9b11-5dc52f248145-openshift-service-ca\") pod \"66230911-e709-4279-9b11-5dc52f248145\" (UID: \"66230911-e709-4279-9b11-5dc52f248145\") "
Sep 10 19:28:40.714521 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:40.714431    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66230911-e709-4279-9b11-5dc52f248145-maas-api-tls\") pod \"66230911-e709-4279-9b11-5dc52f248145\" (UID: \"66230911-e709-4279-9b11-5dc52f248145\") "
Sep 10 19:28:40.714521 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:40.714486    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4pfk\" (UniqueName: \"kubernetes.io/projected/66230911-e709-4279-9b11-5dc52f248145-kube-api-access-k4pfk\") pod \"66230911-e709-4279-9b11-5dc52f248145\" (UID: \"66230911-e709-4279-9b11-5dc52f248145\") "
Sep 10 19:28:40.714829 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:40.714793    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66230911-e709-4279-9b11-5dc52f248145-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "66230911-e709-4279-9b11-5dc52f248145" (UID: "66230911-e709-4279-9b11-5dc52f248145"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:28:40.714945 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:40.714925    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66230911-e709-4279-9b11-5dc52f248145-openshift-service-ca\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:28:40.716934 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:40.716906    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66230911-e709-4279-9b11-5dc52f248145-kube-api-access-k4pfk" (OuterVolumeSpecName: "kube-api-access-k4pfk") pod "66230911-e709-4279-9b11-5dc52f248145" (UID: "66230911-e709-4279-9b11-5dc52f248145"). InnerVolumeSpecName "kube-api-access-k4pfk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:28:40.717069 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:40.717047    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66230911-e709-4279-9b11-5dc52f248145-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "66230911-e709-4279-9b11-5dc52f248145" (UID: "66230911-e709-4279-9b11-5dc52f248145"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:28:40.717069 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:40.717057    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66230911-e709-4279-9b11-5dc52f248145-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "66230911-e709-4279-9b11-5dc52f248145" (UID: "66230911-e709-4279-9b11-5dc52f248145"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:28:40.816491 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:40.816454    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66230911-e709-4279-9b11-5dc52f248145-metrics-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:28:40.816491 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:40.816477    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66230911-e709-4279-9b11-5dc52f248145-maas-api-tls\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:28:40.816491 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:40.816492    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4pfk\" (UniqueName: \"kubernetes.io/projected/66230911-e709-4279-9b11-5dc52f248145-kube-api-access-k4pfk\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:28:40.974301 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:40.974262    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv"]
Sep 10 19:28:40.978153 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:40.978132    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-85e90bf8-58867b69c7-zmfzv"]
Sep 10 19:28:41.240058 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:41.239983    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66230911-e709-4279-9b11-5dc52f248145" path="/var/lib/kubelet/pods/66230911-e709-4279-9b11-5dc52f248145/volumes"
Sep 10 19:28:44.605509 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:44.605472    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-80c77821-768c59459f-llwl7"]
Sep 10 19:28:44.606220 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:44.606200    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66230911-e709-4279-9b11-5dc52f248145" containerName="maas-api"
Sep 10 19:28:44.606220 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:44.606223    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="66230911-e709-4279-9b11-5dc52f248145" containerName="maas-api"
Sep 10 19:28:44.606365 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:44.606337    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="66230911-e709-4279-9b11-5dc52f248145" containerName="maas-api"
Sep 10 19:28:44.609772 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:44.609739    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-80c77821-768c59459f-llwl7"
Sep 10 19:28:44.612084 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:44.612063    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-80c77821-serving-cert\""
Sep 10 19:28:44.622825 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:44.622798    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-80c77821-768c59459f-llwl7"]
Sep 10 19:28:44.755479 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:44.755445    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03fa8105-d8d8-4f98-8a8a-7af48991c121-openshift-service-ca\") pod \"maas-api-e2e-migrate-80c77821-768c59459f-llwl7\" (UID: \"03fa8105-d8d8-4f98-8a8a-7af48991c121\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-80c77821-768c59459f-llwl7"
Sep 10 19:28:44.755690 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:44.755526    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjxrx\" (UniqueName: \"kubernetes.io/projected/03fa8105-d8d8-4f98-8a8a-7af48991c121-kube-api-access-vjxrx\") pod \"maas-api-e2e-migrate-80c77821-768c59459f-llwl7\" (UID: \"03fa8105-d8d8-4f98-8a8a-7af48991c121\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-80c77821-768c59459f-llwl7"
Sep 10 19:28:44.755690 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:44.755583    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03fa8105-d8d8-4f98-8a8a-7af48991c121-maas-api-tls\") pod \"maas-api-e2e-migrate-80c77821-768c59459f-llwl7\" (UID: \"03fa8105-d8d8-4f98-8a8a-7af48991c121\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-80c77821-768c59459f-llwl7"
Sep 10 19:28:44.755690 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:44.755637    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03fa8105-d8d8-4f98-8a8a-7af48991c121-metrics-certs\") pod \"maas-api-e2e-migrate-80c77821-768c59459f-llwl7\" (UID: \"03fa8105-d8d8-4f98-8a8a-7af48991c121\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-80c77821-768c59459f-llwl7"
Sep 10 19:28:44.857076 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:44.856985    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjxrx\" (UniqueName: \"kubernetes.io/projected/03fa8105-d8d8-4f98-8a8a-7af48991c121-kube-api-access-vjxrx\") pod \"maas-api-e2e-migrate-80c77821-768c59459f-llwl7\" (UID: \"03fa8105-d8d8-4f98-8a8a-7af48991c121\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-80c77821-768c59459f-llwl7"
Sep 10 19:28:44.857076 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:44.857060    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03fa8105-d8d8-4f98-8a8a-7af48991c121-maas-api-tls\") pod \"maas-api-e2e-migrate-80c77821-768c59459f-llwl7\" (UID: \"03fa8105-d8d8-4f98-8a8a-7af48991c121\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-80c77821-768c59459f-llwl7"
Sep 10 19:28:44.857318 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:44.857086    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03fa8105-d8d8-4f98-8a8a-7af48991c121-metrics-certs\") pod \"maas-api-e2e-migrate-80c77821-768c59459f-llwl7\" (UID: \"03fa8105-d8d8-4f98-8a8a-7af48991c121\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-80c77821-768c59459f-llwl7"
Sep 10 19:28:44.857318 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:44.857179    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03fa8105-d8d8-4f98-8a8a-7af48991c121-openshift-service-ca\") pod \"maas-api-e2e-migrate-80c77821-768c59459f-llwl7\" (UID: \"03fa8105-d8d8-4f98-8a8a-7af48991c121\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-80c77821-768c59459f-llwl7"
Sep 10 19:28:44.857956 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:44.857882    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03fa8105-d8d8-4f98-8a8a-7af48991c121-openshift-service-ca\") pod \"maas-api-e2e-migrate-80c77821-768c59459f-llwl7\" (UID: \"03fa8105-d8d8-4f98-8a8a-7af48991c121\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-80c77821-768c59459f-llwl7"
Sep 10 19:28:44.859862 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:44.859836    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03fa8105-d8d8-4f98-8a8a-7af48991c121-maas-api-tls\") pod \"maas-api-e2e-migrate-80c77821-768c59459f-llwl7\" (UID: \"03fa8105-d8d8-4f98-8a8a-7af48991c121\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-80c77821-768c59459f-llwl7"
Sep 10 19:28:44.859946 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:44.859842    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03fa8105-d8d8-4f98-8a8a-7af48991c121-metrics-certs\") pod \"maas-api-e2e-migrate-80c77821-768c59459f-llwl7\" (UID: \"03fa8105-d8d8-4f98-8a8a-7af48991c121\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-80c77821-768c59459f-llwl7"
Sep 10 19:28:44.870532 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:44.870512    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjxrx\" (UniqueName: \"kubernetes.io/projected/03fa8105-d8d8-4f98-8a8a-7af48991c121-kube-api-access-vjxrx\") pod \"maas-api-e2e-migrate-80c77821-768c59459f-llwl7\" (UID: \"03fa8105-d8d8-4f98-8a8a-7af48991c121\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-80c77821-768c59459f-llwl7"
Sep 10 19:28:44.923541 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:44.923508    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-80c77821-768c59459f-llwl7"
Sep 10 19:28:45.204335 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:45.204310    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-80c77821-768c59459f-llwl7"]
Sep 10 19:28:45.205521 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:28:45.205498    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03fa8105_d8d8_4f98_8a8a_7af48991c121.slice/crio-52bc3330a6813998f684805a05729336a28918cc06b2d063eedbb7095173b9d0 WatchSource:0}: Error finding container 52bc3330a6813998f684805a05729336a28918cc06b2d063eedbb7095173b9d0: Status 404 returned error can't find the container with id 52bc3330a6813998f684805a05729336a28918cc06b2d063eedbb7095173b9d0
Sep 10 19:28:45.687275 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:45.687231    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-80c77821-768c59459f-llwl7" event={"ID":"03fa8105-d8d8-4f98-8a8a-7af48991c121","Type":"ContainerStarted","Data":"2891977d5016adb88ce78682580d5357fcb6ca220aebdc4caa95cc3fdaccce7b"}
Sep 10 19:28:45.687711 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:45.687294    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-80c77821-768c59459f-llwl7" event={"ID":"03fa8105-d8d8-4f98-8a8a-7af48991c121","Type":"ContainerStarted","Data":"52bc3330a6813998f684805a05729336a28918cc06b2d063eedbb7095173b9d0"}
Sep 10 19:28:45.687711 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:45.687424    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-80c77821-768c59459f-llwl7"
Sep 10 19:28:45.711590 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:45.711539    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-80c77821-768c59459f-llwl7" podStartSLOduration=1.3145734390000001 podStartE2EDuration="1.711523723s" totalImagesPullingTime="396.950284ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:28:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:28:45.709881707 +0000 UTC m=+780.929730189" watchObservedRunningTime="2026-09-10 19:28:45.711523723 +0000 UTC m=+780.931372204"
Sep 10 19:28:51.700487 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:28:51.700452    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-80c77821-768c59459f-llwl7"
Sep 10 19:29:16.042904 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:16.042821    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-80c77821-768c59459f-llwl7"]
Sep 10 19:29:16.043478 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:16.043137    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-80c77821-768c59459f-llwl7" podUID="03fa8105-d8d8-4f98-8a8a-7af48991c121" containerName="maas-api" containerID="cri-o://2891977d5016adb88ce78682580d5357fcb6ca220aebdc4caa95cc3fdaccce7b" gracePeriod=30
Sep 10 19:29:16.303273 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:16.303215    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-80c77821-768c59459f-llwl7"
Sep 10 19:29:16.367803 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:16.367744    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03fa8105-d8d8-4f98-8a8a-7af48991c121-openshift-service-ca\") pod \"03fa8105-d8d8-4f98-8a8a-7af48991c121\" (UID: \"03fa8105-d8d8-4f98-8a8a-7af48991c121\") "
Sep 10 19:29:16.367997 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:16.367842    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03fa8105-d8d8-4f98-8a8a-7af48991c121-maas-api-tls\") pod \"03fa8105-d8d8-4f98-8a8a-7af48991c121\" (UID: \"03fa8105-d8d8-4f98-8a8a-7af48991c121\") "
Sep 10 19:29:16.367997 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:16.367884    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vjxrx\" (UniqueName: \"kubernetes.io/projected/03fa8105-d8d8-4f98-8a8a-7af48991c121-kube-api-access-vjxrx\") pod \"03fa8105-d8d8-4f98-8a8a-7af48991c121\" (UID: \"03fa8105-d8d8-4f98-8a8a-7af48991c121\") "
Sep 10 19:29:16.367997 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:16.367953    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03fa8105-d8d8-4f98-8a8a-7af48991c121-metrics-certs\") pod \"03fa8105-d8d8-4f98-8a8a-7af48991c121\" (UID: \"03fa8105-d8d8-4f98-8a8a-7af48991c121\") "
Sep 10 19:29:16.368167 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:16.368055    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03fa8105-d8d8-4f98-8a8a-7af48991c121-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "03fa8105-d8d8-4f98-8a8a-7af48991c121" (UID: "03fa8105-d8d8-4f98-8a8a-7af48991c121"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:29:16.368316 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:16.368295    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03fa8105-d8d8-4f98-8a8a-7af48991c121-openshift-service-ca\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:29:16.370060 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:16.370027    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03fa8105-d8d8-4f98-8a8a-7af48991c121-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "03fa8105-d8d8-4f98-8a8a-7af48991c121" (UID: "03fa8105-d8d8-4f98-8a8a-7af48991c121"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:29:16.370173 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:16.370122    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03fa8105-d8d8-4f98-8a8a-7af48991c121-kube-api-access-vjxrx" (OuterVolumeSpecName: "kube-api-access-vjxrx") pod "03fa8105-d8d8-4f98-8a8a-7af48991c121" (UID: "03fa8105-d8d8-4f98-8a8a-7af48991c121"). InnerVolumeSpecName "kube-api-access-vjxrx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:29:16.370234 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:16.370189    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03fa8105-d8d8-4f98-8a8a-7af48991c121-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "03fa8105-d8d8-4f98-8a8a-7af48991c121" (UID: "03fa8105-d8d8-4f98-8a8a-7af48991c121"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:29:16.468978 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:16.468948    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03fa8105-d8d8-4f98-8a8a-7af48991c121-maas-api-tls\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:29:16.468978 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:16.468973    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjxrx\" (UniqueName: \"kubernetes.io/projected/03fa8105-d8d8-4f98-8a8a-7af48991c121-kube-api-access-vjxrx\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:29:16.468978 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:16.468983    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03fa8105-d8d8-4f98-8a8a-7af48991c121-metrics-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:29:16.877797 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:16.877765    2579 generic.go:356] "Generic (PLEG): container finished" podID="03fa8105-d8d8-4f98-8a8a-7af48991c121" containerID="2891977d5016adb88ce78682580d5357fcb6ca220aebdc4caa95cc3fdaccce7b" exitCode=0
Sep 10 19:29:16.877976 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:16.877808    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-80c77821-768c59459f-llwl7"
Sep 10 19:29:16.877976 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:16.877845    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-80c77821-768c59459f-llwl7" event={"ID":"03fa8105-d8d8-4f98-8a8a-7af48991c121","Type":"ContainerDied","Data":"2891977d5016adb88ce78682580d5357fcb6ca220aebdc4caa95cc3fdaccce7b"}
Sep 10 19:29:16.877976 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:16.877913    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-80c77821-768c59459f-llwl7" event={"ID":"03fa8105-d8d8-4f98-8a8a-7af48991c121","Type":"ContainerDied","Data":"52bc3330a6813998f684805a05729336a28918cc06b2d063eedbb7095173b9d0"}
Sep 10 19:29:16.877976 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:16.877970    2579 scope.go:117] "RemoveContainer" containerID="2891977d5016adb88ce78682580d5357fcb6ca220aebdc4caa95cc3fdaccce7b"
Sep 10 19:29:16.888487 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:16.888460    2579 scope.go:117] "RemoveContainer" containerID="2891977d5016adb88ce78682580d5357fcb6ca220aebdc4caa95cc3fdaccce7b"
Sep 10 19:29:16.888790 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:29:16.888771    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2891977d5016adb88ce78682580d5357fcb6ca220aebdc4caa95cc3fdaccce7b\": container with ID starting with 2891977d5016adb88ce78682580d5357fcb6ca220aebdc4caa95cc3fdaccce7b not found: ID does not exist" containerID="2891977d5016adb88ce78682580d5357fcb6ca220aebdc4caa95cc3fdaccce7b"
Sep 10 19:29:16.888846 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:16.888797    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2891977d5016adb88ce78682580d5357fcb6ca220aebdc4caa95cc3fdaccce7b"} err="failed to get container status \"2891977d5016adb88ce78682580d5357fcb6ca220aebdc4caa95cc3fdaccce7b\": rpc error: code = NotFound desc = could not find container \"2891977d5016adb88ce78682580d5357fcb6ca220aebdc4caa95cc3fdaccce7b\": container with ID starting with 2891977d5016adb88ce78682580d5357fcb6ca220aebdc4caa95cc3fdaccce7b not found: ID does not exist"
Sep 10 19:29:16.904379 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:16.904358    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-80c77821-768c59459f-llwl7"]
Sep 10 19:29:16.909259 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:16.909240    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-80c77821-768c59459f-llwl7"]
Sep 10 19:29:17.239612 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:17.239583    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03fa8105-d8d8-4f98-8a8a-7af48991c121" path="/var/lib/kubelet/pods/03fa8105-d8d8-4f98-8a8a-7af48991c121/volumes"
Sep 10 19:29:20.654220 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:20.654187    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788"]
Sep 10 19:29:20.654872 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:20.654854    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03fa8105-d8d8-4f98-8a8a-7af48991c121" containerName="maas-api"
Sep 10 19:29:20.654976 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:20.654873    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="03fa8105-d8d8-4f98-8a8a-7af48991c121" containerName="maas-api"
Sep 10 19:29:20.655040 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:20.654980    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="03fa8105-d8d8-4f98-8a8a-7af48991c121" containerName="maas-api"
Sep 10 19:29:20.659566 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:20.659546    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788"
Sep 10 19:29:20.661693 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:20.661673    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-f0b1e2fe-serving-cert\""
Sep 10 19:29:20.669062 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:20.669040    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788"]
Sep 10 19:29:20.821379 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:20.821346    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/310772d4-defd-49a5-9658-a0f86edca77f-metrics-certs\") pod \"maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788\" (UID: \"310772d4-defd-49a5-9658-a0f86edca77f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788"
Sep 10 19:29:20.821541 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:20.821428    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlfw9\" (UniqueName: \"kubernetes.io/projected/310772d4-defd-49a5-9658-a0f86edca77f-kube-api-access-hlfw9\") pod \"maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788\" (UID: \"310772d4-defd-49a5-9658-a0f86edca77f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788"
Sep 10 19:29:20.821541 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:20.821464    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/310772d4-defd-49a5-9658-a0f86edca77f-maas-api-tls\") pod \"maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788\" (UID: \"310772d4-defd-49a5-9658-a0f86edca77f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788"
Sep 10 19:29:20.821541 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:20.821531    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/310772d4-defd-49a5-9658-a0f86edca77f-openshift-service-ca\") pod \"maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788\" (UID: \"310772d4-defd-49a5-9658-a0f86edca77f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788"
Sep 10 19:29:20.922329 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:20.922231    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlfw9\" (UniqueName: \"kubernetes.io/projected/310772d4-defd-49a5-9658-a0f86edca77f-kube-api-access-hlfw9\") pod \"maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788\" (UID: \"310772d4-defd-49a5-9658-a0f86edca77f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788"
Sep 10 19:29:20.922329 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:20.922273    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/310772d4-defd-49a5-9658-a0f86edca77f-maas-api-tls\") pod \"maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788\" (UID: \"310772d4-defd-49a5-9658-a0f86edca77f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788"
Sep 10 19:29:20.922329 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:20.922328    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/310772d4-defd-49a5-9658-a0f86edca77f-openshift-service-ca\") pod \"maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788\" (UID: \"310772d4-defd-49a5-9658-a0f86edca77f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788"
Sep 10 19:29:20.922640 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:20.922365    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/310772d4-defd-49a5-9658-a0f86edca77f-metrics-certs\") pod \"maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788\" (UID: \"310772d4-defd-49a5-9658-a0f86edca77f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788"
Sep 10 19:29:20.922640 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:29:20.922471    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-f0b1e2fe-serving-cert: secret "maas-api-e2e-ait-f0b1e2fe-serving-cert" not found
Sep 10 19:29:20.922640 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:29:20.922550    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/310772d4-defd-49a5-9658-a0f86edca77f-maas-api-tls podName:310772d4-defd-49a5-9658-a0f86edca77f nodeName:}" failed. No retries permitted until 2026-09-10 19:29:21.422530544 +0000 UTC m=+816.642379018 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/310772d4-defd-49a5-9658-a0f86edca77f-maas-api-tls") pod "maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788" (UID: "310772d4-defd-49a5-9658-a0f86edca77f") : secret "maas-api-e2e-ait-f0b1e2fe-serving-cert" not found
Sep 10 19:29:20.923049 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:20.923029    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/310772d4-defd-49a5-9658-a0f86edca77f-openshift-service-ca\") pod \"maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788\" (UID: \"310772d4-defd-49a5-9658-a0f86edca77f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788"
Sep 10 19:29:20.925102 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:20.925080    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/310772d4-defd-49a5-9658-a0f86edca77f-metrics-certs\") pod \"maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788\" (UID: \"310772d4-defd-49a5-9658-a0f86edca77f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788"
Sep 10 19:29:20.930688 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:20.930661    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlfw9\" (UniqueName: \"kubernetes.io/projected/310772d4-defd-49a5-9658-a0f86edca77f-kube-api-access-hlfw9\") pod \"maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788\" (UID: \"310772d4-defd-49a5-9658-a0f86edca77f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788"
Sep 10 19:29:21.427054 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:21.427022    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/310772d4-defd-49a5-9658-a0f86edca77f-maas-api-tls\") pod \"maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788\" (UID: \"310772d4-defd-49a5-9658-a0f86edca77f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788"
Sep 10 19:29:21.429544 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:21.429525    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/310772d4-defd-49a5-9658-a0f86edca77f-maas-api-tls\") pod \"maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788\" (UID: \"310772d4-defd-49a5-9658-a0f86edca77f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788"
Sep 10 19:29:21.574667 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:21.574627    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788"
Sep 10 19:29:21.854884 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:21.854855    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788"]
Sep 10 19:29:21.855432 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:29:21.855409    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod310772d4_defd_49a5_9658_a0f86edca77f.slice/crio-5656e7b0af9c76ef1d0e496c569afc4256344bbb7a803f2f026a95565c386af1 WatchSource:0}: Error finding container 5656e7b0af9c76ef1d0e496c569afc4256344bbb7a803f2f026a95565c386af1: Status 404 returned error can't find the container with id 5656e7b0af9c76ef1d0e496c569afc4256344bbb7a803f2f026a95565c386af1
Sep 10 19:29:21.911065 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:21.911031    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788" event={"ID":"310772d4-defd-49a5-9658-a0f86edca77f","Type":"ContainerStarted","Data":"5656e7b0af9c76ef1d0e496c569afc4256344bbb7a803f2f026a95565c386af1"}
Sep 10 19:29:22.918024 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:22.917992    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788" event={"ID":"310772d4-defd-49a5-9658-a0f86edca77f","Type":"ContainerStarted","Data":"527114da7a489f1a17f7efc5ae5b7c6af6224ee4cf95dbe3ac2205fc4799f206"}
Sep 10 19:29:22.918410 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:22.918124    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788"
Sep 10 19:29:22.936886 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:22.936818    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788" podStartSLOduration=2.540913383 podStartE2EDuration="2.936801582s" totalImagesPullingTime="395.888199ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:29:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:29:22.934994578 +0000 UTC m=+818.154843058" watchObservedRunningTime="2026-09-10 19:29:22.936801582 +0000 UTC m=+818.156650062"
Sep 10 19:29:28.929324 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:28.929291    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788"
Sep 10 19:29:50.978721 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:50.978685    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788"]
Sep 10 19:29:50.979344 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:50.978969    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788" podUID="310772d4-defd-49a5-9658-a0f86edca77f" containerName="maas-api" containerID="cri-o://527114da7a489f1a17f7efc5ae5b7c6af6224ee4cf95dbe3ac2205fc4799f206" gracePeriod=30
Sep 10 19:29:51.257449 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:51.257426    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788"
Sep 10 19:29:51.326388 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:51.326362    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hlfw9\" (UniqueName: \"kubernetes.io/projected/310772d4-defd-49a5-9658-a0f86edca77f-kube-api-access-hlfw9\") pod \"310772d4-defd-49a5-9658-a0f86edca77f\" (UID: \"310772d4-defd-49a5-9658-a0f86edca77f\") "
Sep 10 19:29:51.326553 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:51.326426    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/310772d4-defd-49a5-9658-a0f86edca77f-openshift-service-ca\") pod \"310772d4-defd-49a5-9658-a0f86edca77f\" (UID: \"310772d4-defd-49a5-9658-a0f86edca77f\") "
Sep 10 19:29:51.326553 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:51.326535    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/310772d4-defd-49a5-9658-a0f86edca77f-metrics-certs\") pod \"310772d4-defd-49a5-9658-a0f86edca77f\" (UID: \"310772d4-defd-49a5-9658-a0f86edca77f\") "
Sep 10 19:29:51.326648 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:51.326621    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/310772d4-defd-49a5-9658-a0f86edca77f-maas-api-tls\") pod \"310772d4-defd-49a5-9658-a0f86edca77f\" (UID: \"310772d4-defd-49a5-9658-a0f86edca77f\") "
Sep 10 19:29:51.326834 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:51.326809    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/310772d4-defd-49a5-9658-a0f86edca77f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "310772d4-defd-49a5-9658-a0f86edca77f" (UID: "310772d4-defd-49a5-9658-a0f86edca77f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:29:51.327030 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:51.327013    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/310772d4-defd-49a5-9658-a0f86edca77f-openshift-service-ca\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:29:51.328804 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:51.328772    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/310772d4-defd-49a5-9658-a0f86edca77f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "310772d4-defd-49a5-9658-a0f86edca77f" (UID: "310772d4-defd-49a5-9658-a0f86edca77f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:29:51.328909 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:51.328794    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/310772d4-defd-49a5-9658-a0f86edca77f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "310772d4-defd-49a5-9658-a0f86edca77f" (UID: "310772d4-defd-49a5-9658-a0f86edca77f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:29:51.329100 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:51.329083    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/310772d4-defd-49a5-9658-a0f86edca77f-kube-api-access-hlfw9" (OuterVolumeSpecName: "kube-api-access-hlfw9") pod "310772d4-defd-49a5-9658-a0f86edca77f" (UID: "310772d4-defd-49a5-9658-a0f86edca77f"). InnerVolumeSpecName "kube-api-access-hlfw9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:29:51.427739 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:51.427705    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/310772d4-defd-49a5-9658-a0f86edca77f-maas-api-tls\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:29:51.427739 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:51.427735    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hlfw9\" (UniqueName: \"kubernetes.io/projected/310772d4-defd-49a5-9658-a0f86edca77f-kube-api-access-hlfw9\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:29:51.427739 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:51.427744    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/310772d4-defd-49a5-9658-a0f86edca77f-metrics-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:29:52.100807 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:52.100738    2579 generic.go:356] "Generic (PLEG): container finished" podID="310772d4-defd-49a5-9658-a0f86edca77f" containerID="527114da7a489f1a17f7efc5ae5b7c6af6224ee4cf95dbe3ac2205fc4799f206" exitCode=0
Sep 10 19:29:52.101216 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:52.100818    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788"
Sep 10 19:29:52.101216 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:52.100825    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788" event={"ID":"310772d4-defd-49a5-9658-a0f86edca77f","Type":"ContainerDied","Data":"527114da7a489f1a17f7efc5ae5b7c6af6224ee4cf95dbe3ac2205fc4799f206"}
Sep 10 19:29:52.101216 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:52.100885    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788" event={"ID":"310772d4-defd-49a5-9658-a0f86edca77f","Type":"ContainerDied","Data":"5656e7b0af9c76ef1d0e496c569afc4256344bbb7a803f2f026a95565c386af1"}
Sep 10 19:29:52.101216 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:52.100909    2579 scope.go:117] "RemoveContainer" containerID="527114da7a489f1a17f7efc5ae5b7c6af6224ee4cf95dbe3ac2205fc4799f206"
Sep 10 19:29:52.113568 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:52.113551    2579 scope.go:117] "RemoveContainer" containerID="527114da7a489f1a17f7efc5ae5b7c6af6224ee4cf95dbe3ac2205fc4799f206"
Sep 10 19:29:52.113848 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:29:52.113825    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"527114da7a489f1a17f7efc5ae5b7c6af6224ee4cf95dbe3ac2205fc4799f206\": container with ID starting with 527114da7a489f1a17f7efc5ae5b7c6af6224ee4cf95dbe3ac2205fc4799f206 not found: ID does not exist" containerID="527114da7a489f1a17f7efc5ae5b7c6af6224ee4cf95dbe3ac2205fc4799f206"
Sep 10 19:29:52.113935 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:52.113855    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"527114da7a489f1a17f7efc5ae5b7c6af6224ee4cf95dbe3ac2205fc4799f206"} err="failed to get container status \"527114da7a489f1a17f7efc5ae5b7c6af6224ee4cf95dbe3ac2205fc4799f206\": rpc error: code = NotFound desc = could not find container \"527114da7a489f1a17f7efc5ae5b7c6af6224ee4cf95dbe3ac2205fc4799f206\": container with ID starting with 527114da7a489f1a17f7efc5ae5b7c6af6224ee4cf95dbe3ac2205fc4799f206 not found: ID does not exist"
Sep 10 19:29:52.125793 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:52.125771    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788"]
Sep 10 19:29:52.134505 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:52.134483    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f0b1e2fe-5f967c94f-2w788"]
Sep 10 19:29:53.044355 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.044311    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"]
Sep 10 19:29:53.044862 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.044845    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="310772d4-defd-49a5-9658-a0f86edca77f" containerName="maas-api"
Sep 10 19:29:53.044862 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.044864    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="310772d4-defd-49a5-9658-a0f86edca77f" containerName="maas-api"
Sep 10 19:29:53.045032 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.044956    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="310772d4-defd-49a5-9658-a0f86edca77f" containerName="maas-api"
Sep 10 19:29:53.049604 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.049584    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:29:53.051637 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.051621    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-588a50a1-openshift-default-dockercfg-wm8nv\""
Sep 10 19:29:53.059982 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.059959    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"]
Sep 10 19:29:53.145338 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.145292    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f565038-502b-40f9-b776-b02836bd6094-istio-envoy\") pod \"e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq\" (UID: \"5f565038-502b-40f9-b776-b02836bd6094\") " pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:29:53.145338 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.145352    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f565038-502b-40f9-b776-b02836bd6094-istio-podinfo\") pod \"e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq\" (UID: \"5f565038-502b-40f9-b776-b02836bd6094\") " pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:29:53.145884 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.145395    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f565038-502b-40f9-b776-b02836bd6094-istio-data\") pod \"e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq\" (UID: \"5f565038-502b-40f9-b776-b02836bd6094\") " pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:29:53.145884 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.145464    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f565038-502b-40f9-b776-b02836bd6094-workload-certs\") pod \"e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq\" (UID: \"5f565038-502b-40f9-b776-b02836bd6094\") " pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:29:53.145884 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.145499    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f565038-502b-40f9-b776-b02836bd6094-istiod-ca-cert\") pod \"e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq\" (UID: \"5f565038-502b-40f9-b776-b02836bd6094\") " pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:29:53.145884 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.145524    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f565038-502b-40f9-b776-b02836bd6094-credential-socket\") pod \"e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq\" (UID: \"5f565038-502b-40f9-b776-b02836bd6094\") " pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:29:53.145884 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.145569    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f565038-502b-40f9-b776-b02836bd6094-istio-token\") pod \"e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq\" (UID: \"5f565038-502b-40f9-b776-b02836bd6094\") " pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:29:53.145884 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.145608    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f565038-502b-40f9-b776-b02836bd6094-workload-socket\") pod \"e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq\" (UID: \"5f565038-502b-40f9-b776-b02836bd6094\") " pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:29:53.145884 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.145660    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tphxz\" (UniqueName: \"kubernetes.io/projected/5f565038-502b-40f9-b776-b02836bd6094-kube-api-access-tphxz\") pod \"e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq\" (UID: \"5f565038-502b-40f9-b776-b02836bd6094\") " pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:29:53.239985 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.239952    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="310772d4-defd-49a5-9658-a0f86edca77f" path="/var/lib/kubelet/pods/310772d4-defd-49a5-9658-a0f86edca77f/volumes"
Sep 10 19:29:53.247101 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.247076    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tphxz\" (UniqueName: \"kubernetes.io/projected/5f565038-502b-40f9-b776-b02836bd6094-kube-api-access-tphxz\") pod \"e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq\" (UID: \"5f565038-502b-40f9-b776-b02836bd6094\") " pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:29:53.247231 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.247144    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f565038-502b-40f9-b776-b02836bd6094-istio-envoy\") pod \"e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq\" (UID: \"5f565038-502b-40f9-b776-b02836bd6094\") " pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:29:53.247231 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.247169    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f565038-502b-40f9-b776-b02836bd6094-istio-podinfo\") pod \"e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq\" (UID: \"5f565038-502b-40f9-b776-b02836bd6094\") " pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:29:53.247231 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.247196    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f565038-502b-40f9-b776-b02836bd6094-istio-data\") pod \"e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq\" (UID: \"5f565038-502b-40f9-b776-b02836bd6094\") " pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:29:53.247368 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.247249    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f565038-502b-40f9-b776-b02836bd6094-workload-certs\") pod \"e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq\" (UID: \"5f565038-502b-40f9-b776-b02836bd6094\") " pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:29:53.247368 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.247282    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f565038-502b-40f9-b776-b02836bd6094-istiod-ca-cert\") pod \"e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq\" (UID: \"5f565038-502b-40f9-b776-b02836bd6094\") " pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:29:53.247368 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.247307    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f565038-502b-40f9-b776-b02836bd6094-credential-socket\") pod \"e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq\" (UID: \"5f565038-502b-40f9-b776-b02836bd6094\") " pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:29:53.247368 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.247337    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f565038-502b-40f9-b776-b02836bd6094-istio-token\") pod \"e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq\" (UID: \"5f565038-502b-40f9-b776-b02836bd6094\") " pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:29:53.247544 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.247369    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f565038-502b-40f9-b776-b02836bd6094-workload-socket\") pod \"e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq\" (UID: \"5f565038-502b-40f9-b776-b02836bd6094\") " pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:29:53.247628 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.247599    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f565038-502b-40f9-b776-b02836bd6094-istio-data\") pod \"e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq\" (UID: \"5f565038-502b-40f9-b776-b02836bd6094\") " pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:29:53.247776 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.247690    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f565038-502b-40f9-b776-b02836bd6094-workload-certs\") pod \"e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq\" (UID: \"5f565038-502b-40f9-b776-b02836bd6094\") " pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:29:53.247836 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.247763    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f565038-502b-40f9-b776-b02836bd6094-workload-socket\") pod \"e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq\" (UID: \"5f565038-502b-40f9-b776-b02836bd6094\") " pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:29:53.247885 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.247865    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f565038-502b-40f9-b776-b02836bd6094-credential-socket\") pod \"e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq\" (UID: \"5f565038-502b-40f9-b776-b02836bd6094\") " pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:29:53.248196 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.248178    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f565038-502b-40f9-b776-b02836bd6094-istiod-ca-cert\") pod \"e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq\" (UID: \"5f565038-502b-40f9-b776-b02836bd6094\") " pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:29:53.249630 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.249612    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f565038-502b-40f9-b776-b02836bd6094-istio-envoy\") pod \"e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq\" (UID: \"5f565038-502b-40f9-b776-b02836bd6094\") " pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:29:53.250146 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.250130    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f565038-502b-40f9-b776-b02836bd6094-istio-podinfo\") pod \"e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq\" (UID: \"5f565038-502b-40f9-b776-b02836bd6094\") " pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:29:53.255703 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.255679    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f565038-502b-40f9-b776-b02836bd6094-istio-token\") pod \"e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq\" (UID: \"5f565038-502b-40f9-b776-b02836bd6094\") " pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:29:53.255808 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.255726    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tphxz\" (UniqueName: \"kubernetes.io/projected/5f565038-502b-40f9-b776-b02836bd6094-kube-api-access-tphxz\") pod \"e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq\" (UID: \"5f565038-502b-40f9-b776-b02836bd6094\") " pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:29:53.360004 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.359919    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:29:53.640921 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.640890    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"]
Sep 10 19:29:53.642244 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:29:53.642218    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f565038_502b_40f9_b776_b02836bd6094.slice/crio-fe4341f1700f3c801056b9b0c7c7bfaa219da4cdb149c5f93b56d44cf40b7545 WatchSource:0}: Error finding container fe4341f1700f3c801056b9b0c7c7bfaa219da4cdb149c5f93b56d44cf40b7545: Status 404 returned error can't find the container with id fe4341f1700f3c801056b9b0c7c7bfaa219da4cdb149c5f93b56d44cf40b7545
Sep 10 19:29:53.644824 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.644792    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:29:53.644897 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.644877    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:29:53.644936 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:53.644905    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:29:54.115915 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:54.115832    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq" event={"ID":"5f565038-502b-40f9-b776-b02836bd6094","Type":"ContainerStarted","Data":"4540f4d7273facc8a3c7f4eb11cf74ba2e0f745f501257ddc87c7d9998f08a7a"}
Sep 10 19:29:54.115915 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:54.115879    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq" event={"ID":"5f565038-502b-40f9-b776-b02836bd6094","Type":"ContainerStarted","Data":"fe4341f1700f3c801056b9b0c7c7bfaa219da4cdb149c5f93b56d44cf40b7545"}
Sep 10 19:29:54.158762 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:54.158689    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq" podStartSLOduration=1.158661227 podStartE2EDuration="1.158661227s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:29:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:29:54.154629082 +0000 UTC m=+849.374477563" watchObservedRunningTime="2026-09-10 19:29:54.158661227 +0000 UTC m=+849.378509775"
Sep 10 19:29:54.360666 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:54.360628    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:29:54.365967 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:54.365897    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:29:55.122472 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:55.122443    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:29:55.123383 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:55.123364    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:29:56.641543 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:56.641509    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9"]
Sep 10 19:29:56.645816 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:56.645792    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9"
Sep 10 19:29:56.648281 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:56.648259    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-588a50a1-serving-cert\""
Sep 10 19:29:56.657495 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:56.657470    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9"]
Sep 10 19:29:56.783565 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:56.783521    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rb5hr\" (UniqueName: \"kubernetes.io/projected/fd4ef127-564c-40e8-ac73-fe846716fcda-kube-api-access-rb5hr\") pod \"maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9\" (UID: \"fd4ef127-564c-40e8-ac73-fe846716fcda\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9"
Sep 10 19:29:56.783784 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:56.783609    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd4ef127-564c-40e8-ac73-fe846716fcda-maas-api-tls\") pod \"maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9\" (UID: \"fd4ef127-564c-40e8-ac73-fe846716fcda\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9"
Sep 10 19:29:56.783784 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:56.783670    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd4ef127-564c-40e8-ac73-fe846716fcda-openshift-service-ca\") pod \"maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9\" (UID: \"fd4ef127-564c-40e8-ac73-fe846716fcda\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9"
Sep 10 19:29:56.783784 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:56.783693    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd4ef127-564c-40e8-ac73-fe846716fcda-metrics-certs\") pod \"maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9\" (UID: \"fd4ef127-564c-40e8-ac73-fe846716fcda\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9"
Sep 10 19:29:56.885158 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:56.885123    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb5hr\" (UniqueName: \"kubernetes.io/projected/fd4ef127-564c-40e8-ac73-fe846716fcda-kube-api-access-rb5hr\") pod \"maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9\" (UID: \"fd4ef127-564c-40e8-ac73-fe846716fcda\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9"
Sep 10 19:29:56.885372 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:56.885196    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd4ef127-564c-40e8-ac73-fe846716fcda-maas-api-tls\") pod \"maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9\" (UID: \"fd4ef127-564c-40e8-ac73-fe846716fcda\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9"
Sep 10 19:29:56.885372 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:56.885259    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd4ef127-564c-40e8-ac73-fe846716fcda-openshift-service-ca\") pod \"maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9\" (UID: \"fd4ef127-564c-40e8-ac73-fe846716fcda\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9"
Sep 10 19:29:56.885372 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:56.885283    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd4ef127-564c-40e8-ac73-fe846716fcda-metrics-certs\") pod \"maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9\" (UID: \"fd4ef127-564c-40e8-ac73-fe846716fcda\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9"
Sep 10 19:29:56.886069 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:56.885987    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd4ef127-564c-40e8-ac73-fe846716fcda-openshift-service-ca\") pod \"maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9\" (UID: \"fd4ef127-564c-40e8-ac73-fe846716fcda\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9"
Sep 10 19:29:56.888085 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:56.888052    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd4ef127-564c-40e8-ac73-fe846716fcda-metrics-certs\") pod \"maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9\" (UID: \"fd4ef127-564c-40e8-ac73-fe846716fcda\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9"
Sep 10 19:29:56.888188 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:56.888146    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd4ef127-564c-40e8-ac73-fe846716fcda-maas-api-tls\") pod \"maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9\" (UID: \"fd4ef127-564c-40e8-ac73-fe846716fcda\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9"
Sep 10 19:29:56.893014 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:56.892951    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rb5hr\" (UniqueName: \"kubernetes.io/projected/fd4ef127-564c-40e8-ac73-fe846716fcda-kube-api-access-rb5hr\") pod \"maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9\" (UID: \"fd4ef127-564c-40e8-ac73-fe846716fcda\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9"
Sep 10 19:29:56.960836 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:56.960801    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9"
Sep 10 19:29:57.242634 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:29:57.242514    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd4ef127_564c_40e8_ac73_fe846716fcda.slice/crio-52770939bfb00d5aa6fa25d2effc68514d1b35b8e975547ddd2a4dbc06a754a6 WatchSource:0}: Error finding container 52770939bfb00d5aa6fa25d2effc68514d1b35b8e975547ddd2a4dbc06a754a6: Status 404 returned error can't find the container with id 52770939bfb00d5aa6fa25d2effc68514d1b35b8e975547ddd2a4dbc06a754a6
Sep 10 19:29:57.243421 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:57.243398    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9"]
Sep 10 19:29:58.144372 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:58.144340    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9" event={"ID":"fd4ef127-564c-40e8-ac73-fe846716fcda","Type":"ContainerStarted","Data":"08183f004e1f2b87a504d3c269eb8ef9e0ae66d62425d20208c47248a5db7047"}
Sep 10 19:29:58.144806 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:58.144392    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9" event={"ID":"fd4ef127-564c-40e8-ac73-fe846716fcda","Type":"ContainerStarted","Data":"52770939bfb00d5aa6fa25d2effc68514d1b35b8e975547ddd2a4dbc06a754a6"}
Sep 10 19:29:58.144806 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:58.144546    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9"
Sep 10 19:29:58.162348 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:58.162284    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9" podStartSLOduration=1.734052153 podStartE2EDuration="2.16226912s" totalImagesPullingTime="428.216967ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:29:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:29:58.161075564 +0000 UTC m=+853.380924041" watchObservedRunningTime="2026-09-10 19:29:58.16226912 +0000 UTC m=+853.382117601"
Sep 10 19:29:59.792446 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:59.792410    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd"]
Sep 10 19:29:59.794573 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:29:59.793227    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd" podUID="200d7c73-5704-484c-b3cb-047941982034" containerName="maas-api" containerID="cri-o://fe17db926da08ff4ebebf9978329f02d17a811e9e3a647c426f1bd63c9f31eb7" gracePeriod=30
Sep 10 19:30:00.079966 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:00.079938    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd"
Sep 10 19:30:00.160810 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:00.160735    2579 generic.go:356] "Generic (PLEG): container finished" podID="200d7c73-5704-484c-b3cb-047941982034" containerID="fe17db926da08ff4ebebf9978329f02d17a811e9e3a647c426f1bd63c9f31eb7" exitCode=0
Sep 10 19:30:00.160978 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:00.160816    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd"
Sep 10 19:30:00.160978 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:00.160828    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd" event={"ID":"200d7c73-5704-484c-b3cb-047941982034","Type":"ContainerDied","Data":"fe17db926da08ff4ebebf9978329f02d17a811e9e3a647c426f1bd63c9f31eb7"}
Sep 10 19:30:00.160978 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:00.160899    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd" event={"ID":"200d7c73-5704-484c-b3cb-047941982034","Type":"ContainerDied","Data":"9fbe154b131c9982e1201a97430ea83c7ec3f99c814fde2fb148f675d625856f"}
Sep 10 19:30:00.160978 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:00.160929    2579 scope.go:117] "RemoveContainer" containerID="fe17db926da08ff4ebebf9978329f02d17a811e9e3a647c426f1bd63c9f31eb7"
Sep 10 19:30:00.172733 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:00.172715    2579 scope.go:117] "RemoveContainer" containerID="fe17db926da08ff4ebebf9978329f02d17a811e9e3a647c426f1bd63c9f31eb7"
Sep 10 19:30:00.173126 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:30:00.173101    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe17db926da08ff4ebebf9978329f02d17a811e9e3a647c426f1bd63c9f31eb7\": container with ID starting with fe17db926da08ff4ebebf9978329f02d17a811e9e3a647c426f1bd63c9f31eb7 not found: ID does not exist" containerID="fe17db926da08ff4ebebf9978329f02d17a811e9e3a647c426f1bd63c9f31eb7"
Sep 10 19:30:00.173221 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:00.173134    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe17db926da08ff4ebebf9978329f02d17a811e9e3a647c426f1bd63c9f31eb7"} err="failed to get container status \"fe17db926da08ff4ebebf9978329f02d17a811e9e3a647c426f1bd63c9f31eb7\": rpc error: code = NotFound desc = could not find container \"fe17db926da08ff4ebebf9978329f02d17a811e9e3a647c426f1bd63c9f31eb7\": container with ID starting with fe17db926da08ff4ebebf9978329f02d17a811e9e3a647c426f1bd63c9f31eb7 not found: ID does not exist"
Sep 10 19:30:00.222856 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:00.222826    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/200d7c73-5704-484c-b3cb-047941982034-openshift-service-ca\") pod \"200d7c73-5704-484c-b3cb-047941982034\" (UID: \"200d7c73-5704-484c-b3cb-047941982034\") "
Sep 10 19:30:00.223053 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:00.222871    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/200d7c73-5704-484c-b3cb-047941982034-metrics-certs\") pod \"200d7c73-5704-484c-b3cb-047941982034\" (UID: \"200d7c73-5704-484c-b3cb-047941982034\") "
Sep 10 19:30:00.223053 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:00.222921    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/200d7c73-5704-484c-b3cb-047941982034-maas-api-tls\") pod \"200d7c73-5704-484c-b3cb-047941982034\" (UID: \"200d7c73-5704-484c-b3cb-047941982034\") "
Sep 10 19:30:00.223053 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:00.222938    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7kh7l\" (UniqueName: \"kubernetes.io/projected/200d7c73-5704-484c-b3cb-047941982034-kube-api-access-7kh7l\") pod \"200d7c73-5704-484c-b3cb-047941982034\" (UID: \"200d7c73-5704-484c-b3cb-047941982034\") "
Sep 10 19:30:00.223318 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:00.223268    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/200d7c73-5704-484c-b3cb-047941982034-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "200d7c73-5704-484c-b3cb-047941982034" (UID: "200d7c73-5704-484c-b3cb-047941982034"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:30:00.223455 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:00.223371    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/200d7c73-5704-484c-b3cb-047941982034-openshift-service-ca\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:30:00.225312 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:00.225285    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/200d7c73-5704-484c-b3cb-047941982034-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "200d7c73-5704-484c-b3cb-047941982034" (UID: "200d7c73-5704-484c-b3cb-047941982034"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:30:00.225817 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:00.225795    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/200d7c73-5704-484c-b3cb-047941982034-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "200d7c73-5704-484c-b3cb-047941982034" (UID: "200d7c73-5704-484c-b3cb-047941982034"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:30:00.225887 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:00.225832    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/200d7c73-5704-484c-b3cb-047941982034-kube-api-access-7kh7l" (OuterVolumeSpecName: "kube-api-access-7kh7l") pod "200d7c73-5704-484c-b3cb-047941982034" (UID: "200d7c73-5704-484c-b3cb-047941982034"). InnerVolumeSpecName "kube-api-access-7kh7l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:30:00.324362 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:00.324267    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/200d7c73-5704-484c-b3cb-047941982034-metrics-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:30:00.324362 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:00.324292    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/200d7c73-5704-484c-b3cb-047941982034-maas-api-tls\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:30:00.324362 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:00.324303    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7kh7l\" (UniqueName: \"kubernetes.io/projected/200d7c73-5704-484c-b3cb-047941982034-kube-api-access-7kh7l\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:30:00.490349 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:00.490316    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd"]
Sep 10 19:30:00.493628 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:00.493604    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-b3108b-6fc9ff8b59-z9hbd"]
Sep 10 19:30:01.240217 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:01.240185    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="200d7c73-5704-484c-b3cb-047941982034" path="/var/lib/kubelet/pods/200d7c73-5704-484c-b3cb-047941982034/volumes"
Sep 10 19:30:04.163141 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:04.163104    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9"
Sep 10 19:30:07.067492 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:07.067453    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"]
Sep 10 19:30:07.068321 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:07.068285    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d" podUID="c4b7c1d3-1982-4ff4-9552-19cb2043c4fe" containerName="istio-proxy" containerID="cri-o://4f4d7396ce0d16e0c86f08af912b68910de3426d836f8f03bfdbf49b9f5dc2ed" gracePeriod=30
Sep 10 19:30:11.162924 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:11.162882    2579 patch_prober.go:28] interesting pod/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 19:30:11.163308 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:11.162989    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq" podUID="5f565038-502b-40f9-b776-b02836bd6094" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 19:30:11.738458 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:11.738426    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94"]
Sep 10 19:30:11.739147 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:11.739122    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="200d7c73-5704-484c-b3cb-047941982034" containerName="maas-api"
Sep 10 19:30:11.739328 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:11.739313    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="200d7c73-5704-484c-b3cb-047941982034" containerName="maas-api"
Sep 10 19:30:11.739486 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:11.739471    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="200d7c73-5704-484c-b3cb-047941982034" containerName="maas-api"
Sep 10 19:30:11.742216 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:11.742198    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94"
Sep 10 19:30:11.744374 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:11.744354    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-faed1b-serving-cert\""
Sep 10 19:30:11.753627 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:11.753603    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxtwj\" (UniqueName: \"kubernetes.io/projected/a18f1c5a-8e34-452f-9c8c-a73c087b2c4a-kube-api-access-cxtwj\") pod \"maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94\" (UID: \"a18f1c5a-8e34-452f-9c8c-a73c087b2c4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94"
Sep 10 19:30:11.753869 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:11.753850    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a18f1c5a-8e34-452f-9c8c-a73c087b2c4a-maas-api-tls\") pod \"maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94\" (UID: \"a18f1c5a-8e34-452f-9c8c-a73c087b2c4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94"
Sep 10 19:30:11.754030 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:11.754013    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a18f1c5a-8e34-452f-9c8c-a73c087b2c4a-openshift-service-ca\") pod \"maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94\" (UID: \"a18f1c5a-8e34-452f-9c8c-a73c087b2c4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94"
Sep 10 19:30:11.754132 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:11.754119    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a18f1c5a-8e34-452f-9c8c-a73c087b2c4a-metrics-certs\") pod \"maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94\" (UID: \"a18f1c5a-8e34-452f-9c8c-a73c087b2c4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94"
Sep 10 19:30:11.756699 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:11.756679    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94"]
Sep 10 19:30:11.855461 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:11.855430    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a18f1c5a-8e34-452f-9c8c-a73c087b2c4a-openshift-service-ca\") pod \"maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94\" (UID: \"a18f1c5a-8e34-452f-9c8c-a73c087b2c4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94"
Sep 10 19:30:11.855674 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:11.855475    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a18f1c5a-8e34-452f-9c8c-a73c087b2c4a-metrics-certs\") pod \"maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94\" (UID: \"a18f1c5a-8e34-452f-9c8c-a73c087b2c4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94"
Sep 10 19:30:11.855674 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:11.855591    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxtwj\" (UniqueName: \"kubernetes.io/projected/a18f1c5a-8e34-452f-9c8c-a73c087b2c4a-kube-api-access-cxtwj\") pod \"maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94\" (UID: \"a18f1c5a-8e34-452f-9c8c-a73c087b2c4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94"
Sep 10 19:30:11.855841 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:11.855655    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a18f1c5a-8e34-452f-9c8c-a73c087b2c4a-maas-api-tls\") pod \"maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94\" (UID: \"a18f1c5a-8e34-452f-9c8c-a73c087b2c4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94"
Sep 10 19:30:11.856388 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:11.856356    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a18f1c5a-8e34-452f-9c8c-a73c087b2c4a-openshift-service-ca\") pod \"maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94\" (UID: \"a18f1c5a-8e34-452f-9c8c-a73c087b2c4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94"
Sep 10 19:30:11.859250 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:11.859222    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a18f1c5a-8e34-452f-9c8c-a73c087b2c4a-maas-api-tls\") pod \"maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94\" (UID: \"a18f1c5a-8e34-452f-9c8c-a73c087b2c4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94"
Sep 10 19:30:11.859354 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:11.859273    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a18f1c5a-8e34-452f-9c8c-a73c087b2c4a-metrics-certs\") pod \"maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94\" (UID: \"a18f1c5a-8e34-452f-9c8c-a73c087b2c4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94"
Sep 10 19:30:11.864091 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:11.864068    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxtwj\" (UniqueName: \"kubernetes.io/projected/a18f1c5a-8e34-452f-9c8c-a73c087b2c4a-kube-api-access-cxtwj\") pod \"maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94\" (UID: \"a18f1c5a-8e34-452f-9c8c-a73c087b2c4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94"
Sep 10 19:30:12.060072 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.059982    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94"
Sep 10 19:30:12.253856 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.253816    2579 generic.go:356] "Generic (PLEG): container finished" podID="c4b7c1d3-1982-4ff4-9552-19cb2043c4fe" containerID="4f4d7396ce0d16e0c86f08af912b68910de3426d836f8f03bfdbf49b9f5dc2ed" exitCode=0
Sep 10 19:30:12.254295 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.253894    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d" event={"ID":"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe","Type":"ContainerDied","Data":"4f4d7396ce0d16e0c86f08af912b68910de3426d836f8f03bfdbf49b9f5dc2ed"}
Sep 10 19:30:12.254295 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.253959    2579 scope.go:117] "RemoveContainer" containerID="0114dde241e410a36bf0c73d37d33aa20ed73faa71f00caa2fef7ed2c6b77f8d"
Sep 10 19:30:12.256197 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.256175    2579 generic.go:356] "Generic (PLEG): container finished" podID="5f565038-502b-40f9-b776-b02836bd6094" containerID="4540f4d7273facc8a3c7f4eb11cf74ba2e0f745f501257ddc87c7d9998f08a7a" exitCode=0
Sep 10 19:30:12.256296 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.256245    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq" event={"ID":"5f565038-502b-40f9-b776-b02836bd6094","Type":"ContainerDied","Data":"4540f4d7273facc8a3c7f4eb11cf74ba2e0f745f501257ddc87c7d9998f08a7a"}
Sep 10 19:30:12.257278 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.257124    2579 scope.go:117] "RemoveContainer" containerID="4540f4d7273facc8a3c7f4eb11cf74ba2e0f745f501257ddc87c7d9998f08a7a"
Sep 10 19:30:12.258681 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.258641    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:30:12.258829 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.258732    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:30:12.258829 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.258818    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:30:12.345495 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.345465    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94"]
Sep 10 19:30:12.348969 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:30:12.348943    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda18f1c5a_8e34_452f_9c8c_a73c087b2c4a.slice/crio-285c8d430c18262b3c2fcb36559a921de8d349b36f0a25abf3687ddd7f428bc4 WatchSource:0}: Error finding container 285c8d430c18262b3c2fcb36559a921de8d349b36f0a25abf3687ddd7f428bc4: Status 404 returned error can't find the container with id 285c8d430c18262b3c2fcb36559a921de8d349b36f0a25abf3687ddd7f428bc4
Sep 10 19:30:12.366952 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.366927    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:30:12.461703 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.461669    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-workload-socket\") pod \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\" (UID: \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\") "
Sep 10 19:30:12.461886 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.461718    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5zfvm\" (UniqueName: \"kubernetes.io/projected/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-kube-api-access-5zfvm\") pod \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\" (UID: \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\") "
Sep 10 19:30:12.461886 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.461789    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-credential-socket\") pod \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\" (UID: \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\") "
Sep 10 19:30:12.461886 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.461846    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-istio-data\") pod \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\" (UID: \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\") "
Sep 10 19:30:12.462055 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.461891    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-istio-token\") pod \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\" (UID: \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\") "
Sep 10 19:30:12.462055 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.461917    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-istio-podinfo\") pod \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\" (UID: \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\") "
Sep 10 19:30:12.462055 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.461996    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-workload-certs\") pod \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\" (UID: \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\") "
Sep 10 19:30:12.462055 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.462026    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-istiod-ca-cert\") pod \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\" (UID: \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\") "
Sep 10 19:30:12.462055 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.462029    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c4b7c1d3-1982-4ff4-9552-19cb2043c4fe" (UID: "c4b7c1d3-1982-4ff4-9552-19cb2043c4fe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:30:12.462055 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.462036    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c4b7c1d3-1982-4ff4-9552-19cb2043c4fe" (UID: "c4b7c1d3-1982-4ff4-9552-19cb2043c4fe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:30:12.462375 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.462067    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-istio-envoy\") pod \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\" (UID: \"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe\") "
Sep 10 19:30:12.462375 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.462185    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c4b7c1d3-1982-4ff4-9552-19cb2043c4fe" (UID: "c4b7c1d3-1982-4ff4-9552-19cb2043c4fe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:30:12.462375 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.462308    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c4b7c1d3-1982-4ff4-9552-19cb2043c4fe" (UID: "c4b7c1d3-1982-4ff4-9552-19cb2043c4fe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:30:12.462542 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.462451    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-workload-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:30:12.462542 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.462459    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-istio-data" (OuterVolumeSpecName: "istio-data") pod "c4b7c1d3-1982-4ff4-9552-19cb2043c4fe" (UID: "c4b7c1d3-1982-4ff4-9552-19cb2043c4fe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:30:12.462542 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.462471    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-istiod-ca-cert\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:30:12.462542 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.462512    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-workload-socket\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:30:12.462542 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.462530    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-credential-socket\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:30:12.464519 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.464480    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c4b7c1d3-1982-4ff4-9552-19cb2043c4fe" (UID: "c4b7c1d3-1982-4ff4-9552-19cb2043c4fe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:30:12.465219 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.465037    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c4b7c1d3-1982-4ff4-9552-19cb2043c4fe" (UID: "c4b7c1d3-1982-4ff4-9552-19cb2043c4fe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:30:12.465219 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.465040    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-istio-token" (OuterVolumeSpecName: "istio-token") pod "c4b7c1d3-1982-4ff4-9552-19cb2043c4fe" (UID: "c4b7c1d3-1982-4ff4-9552-19cb2043c4fe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:30:12.465412 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.465386    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-kube-api-access-5zfvm" (OuterVolumeSpecName: "kube-api-access-5zfvm") pod "c4b7c1d3-1982-4ff4-9552-19cb2043c4fe" (UID: "c4b7c1d3-1982-4ff4-9552-19cb2043c4fe"). InnerVolumeSpecName "kube-api-access-5zfvm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:30:12.563824 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.563655    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-istio-envoy\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:30:12.563824 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.563692    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5zfvm\" (UniqueName: \"kubernetes.io/projected/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-kube-api-access-5zfvm\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:30:12.563824 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.563706    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-istio-data\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:30:12.563824 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.563719    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-istio-token\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:30:12.563824 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:12.563731    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe-istio-podinfo\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:30:13.269501 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:13.269457    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq" event={"ID":"5f565038-502b-40f9-b776-b02836bd6094","Type":"ContainerStarted","Data":"042a7570d65c39a771cb8dae83faafb37d3152ea18f72cbe92907ca7af8eb557"}
Sep 10 19:30:13.272877 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:13.272825    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94" event={"ID":"a18f1c5a-8e34-452f-9c8c-a73c087b2c4a","Type":"ContainerStarted","Data":"ffbefdd3b09458a9fe47d9a3fc5b321501bb2af5124549fbe71e74798279b80c"}
Sep 10 19:30:13.273038 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:13.272886    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94" event={"ID":"a18f1c5a-8e34-452f-9c8c-a73c087b2c4a","Type":"ContainerStarted","Data":"285c8d430c18262b3c2fcb36559a921de8d349b36f0a25abf3687ddd7f428bc4"}
Sep 10 19:30:13.273494 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:13.273470    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94"
Sep 10 19:30:13.283132 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:13.283105    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"
Sep 10 19:30:13.283132 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:13.283115    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d" event={"ID":"c4b7c1d3-1982-4ff4-9552-19cb2043c4fe","Type":"ContainerDied","Data":"41aa73ce791e65f8a3f57a8f8b7cdbe29366ce9f415af193746868b814b189a3"}
Sep 10 19:30:13.283364 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:13.283187    2579 scope.go:117] "RemoveContainer" containerID="4f4d7396ce0d16e0c86f08af912b68910de3426d836f8f03bfdbf49b9f5dc2ed"
Sep 10 19:30:13.314305 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:13.314278    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"]
Sep 10 19:30:13.324208 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:13.324183    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d"]
Sep 10 19:30:13.341398 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:13.341355    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94" podStartSLOduration=1.988808406 podStartE2EDuration="2.341340913s" totalImagesPullingTime="352.532507ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:30:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:30:13.340527677 +0000 UTC m=+868.560376157" watchObservedRunningTime="2026-09-10 19:30:13.341340913 +0000 UTC m=+868.561189393"
Sep 10 19:30:13.360710 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:13.360665    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:30:13.361148 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:13.361117    2579 patch_prober.go:28] interesting pod/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 10 19:30:13.361260 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:13.361177    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq" podUID="5f565038-502b-40f9-b776-b02836bd6094" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 10 19:30:14.360966 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:14.360929    2579 patch_prober.go:28] interesting pod/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 10 19:30:14.361557 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:14.361523    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq" podUID="5f565038-502b-40f9-b776-b02836bd6094" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 10 19:30:15.240644 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:15.240607    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4b7c1d3-1982-4ff4-9552-19cb2043c4fe" path="/var/lib/kubelet/pods/c4b7c1d3-1982-4ff4-9552-19cb2043c4fe/volumes"
Sep 10 19:30:15.361087 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:15.361053    2579 patch_prober.go:28] interesting pod/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 10 19:30:15.361654 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:15.361622    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq" podUID="5f565038-502b-40f9-b776-b02836bd6094" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 10 19:30:16.361097 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:16.361049    2579 patch_prober.go:28] interesting pod/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 10 19:30:16.361620 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:16.361120    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq" podUID="5f565038-502b-40f9-b776-b02836bd6094" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 10 19:30:17.360864 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:17.360823    2579 patch_prober.go:28] interesting pod/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 10 19:30:17.361079 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:17.360888    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq" podUID="5f565038-502b-40f9-b776-b02836bd6094" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 10 19:30:18.360446 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:18.360402    2579 patch_prober.go:28] interesting pod/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 10 19:30:18.360944 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:18.360475    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq" podUID="5f565038-502b-40f9-b776-b02836bd6094" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 10 19:30:19.364189 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:19.364152    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:30:19.364564 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:19.364454    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:30:19.365266 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:19.365252    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:30:20.311121 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:20.311084    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94"
Sep 10 19:30:37.624799 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:37.624684    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9"]
Sep 10 19:30:37.625340 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:37.624971    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9" podUID="fd4ef127-564c-40e8-ac73-fe846716fcda" containerName="maas-api" containerID="cri-o://08183f004e1f2b87a504d3c269eb8ef9e0ae66d62425d20208c47248a5db7047" gracePeriod=30
Sep 10 19:30:37.889830 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:37.889805    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9"
Sep 10 19:30:38.037633 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:38.037602    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd4ef127-564c-40e8-ac73-fe846716fcda-maas-api-tls\") pod \"fd4ef127-564c-40e8-ac73-fe846716fcda\" (UID: \"fd4ef127-564c-40e8-ac73-fe846716fcda\") "
Sep 10 19:30:38.037834 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:38.037663    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"]
Sep 10 19:30:38.037834 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:38.037699    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd4ef127-564c-40e8-ac73-fe846716fcda-metrics-certs\") pod \"fd4ef127-564c-40e8-ac73-fe846716fcda\" (UID: \"fd4ef127-564c-40e8-ac73-fe846716fcda\") "
Sep 10 19:30:38.037834 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:38.037771    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rb5hr\" (UniqueName: \"kubernetes.io/projected/fd4ef127-564c-40e8-ac73-fe846716fcda-kube-api-access-rb5hr\") pod \"fd4ef127-564c-40e8-ac73-fe846716fcda\" (UID: \"fd4ef127-564c-40e8-ac73-fe846716fcda\") "
Sep 10 19:30:38.037834 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:38.037817    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd4ef127-564c-40e8-ac73-fe846716fcda-openshift-service-ca\") pod \"fd4ef127-564c-40e8-ac73-fe846716fcda\" (UID: \"fd4ef127-564c-40e8-ac73-fe846716fcda\") "
Sep 10 19:30:38.038084 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:38.037897    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq" podUID="5f565038-502b-40f9-b776-b02836bd6094" containerName="istio-proxy" containerID="cri-o://042a7570d65c39a771cb8dae83faafb37d3152ea18f72cbe92907ca7af8eb557" gracePeriod=30
Sep 10 19:30:38.038673 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:38.038483    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd4ef127-564c-40e8-ac73-fe846716fcda-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fd4ef127-564c-40e8-ac73-fe846716fcda" (UID: "fd4ef127-564c-40e8-ac73-fe846716fcda"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:30:38.041206 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:38.041136    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd4ef127-564c-40e8-ac73-fe846716fcda-kube-api-access-rb5hr" (OuterVolumeSpecName: "kube-api-access-rb5hr") pod "fd4ef127-564c-40e8-ac73-fe846716fcda" (UID: "fd4ef127-564c-40e8-ac73-fe846716fcda"). InnerVolumeSpecName "kube-api-access-rb5hr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:30:38.041206 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:38.041154    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd4ef127-564c-40e8-ac73-fe846716fcda-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fd4ef127-564c-40e8-ac73-fe846716fcda" (UID: "fd4ef127-564c-40e8-ac73-fe846716fcda"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:30:38.041429 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:38.041250    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd4ef127-564c-40e8-ac73-fe846716fcda-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "fd4ef127-564c-40e8-ac73-fe846716fcda" (UID: "fd4ef127-564c-40e8-ac73-fe846716fcda"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:30:38.139710 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:38.139672    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd4ef127-564c-40e8-ac73-fe846716fcda-metrics-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:30:38.139710 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:38.139710    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rb5hr\" (UniqueName: \"kubernetes.io/projected/fd4ef127-564c-40e8-ac73-fe846716fcda-kube-api-access-rb5hr\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:30:38.139997 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:38.139726    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd4ef127-564c-40e8-ac73-fe846716fcda-openshift-service-ca\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:30:38.139997 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:38.139740    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd4ef127-564c-40e8-ac73-fe846716fcda-maas-api-tls\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:30:38.455429 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:38.455341    2579 generic.go:356] "Generic (PLEG): container finished" podID="fd4ef127-564c-40e8-ac73-fe846716fcda" containerID="08183f004e1f2b87a504d3c269eb8ef9e0ae66d62425d20208c47248a5db7047" exitCode=0
Sep 10 19:30:38.455429 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:38.455392    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9"
Sep 10 19:30:38.455674 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:38.455428    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9" event={"ID":"fd4ef127-564c-40e8-ac73-fe846716fcda","Type":"ContainerDied","Data":"08183f004e1f2b87a504d3c269eb8ef9e0ae66d62425d20208c47248a5db7047"}
Sep 10 19:30:38.455674 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:38.455488    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9" event={"ID":"fd4ef127-564c-40e8-ac73-fe846716fcda","Type":"ContainerDied","Data":"52770939bfb00d5aa6fa25d2effc68514d1b35b8e975547ddd2a4dbc06a754a6"}
Sep 10 19:30:38.455674 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:38.455518    2579 scope.go:117] "RemoveContainer" containerID="08183f004e1f2b87a504d3c269eb8ef9e0ae66d62425d20208c47248a5db7047"
Sep 10 19:30:38.465828 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:38.465809    2579 scope.go:117] "RemoveContainer" containerID="08183f004e1f2b87a504d3c269eb8ef9e0ae66d62425d20208c47248a5db7047"
Sep 10 19:30:38.466181 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:30:38.466157    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08183f004e1f2b87a504d3c269eb8ef9e0ae66d62425d20208c47248a5db7047\": container with ID starting with 08183f004e1f2b87a504d3c269eb8ef9e0ae66d62425d20208c47248a5db7047 not found: ID does not exist" containerID="08183f004e1f2b87a504d3c269eb8ef9e0ae66d62425d20208c47248a5db7047"
Sep 10 19:30:38.466273 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:38.466196    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08183f004e1f2b87a504d3c269eb8ef9e0ae66d62425d20208c47248a5db7047"} err="failed to get container status \"08183f004e1f2b87a504d3c269eb8ef9e0ae66d62425d20208c47248a5db7047\": rpc error: code = NotFound desc = could not find container \"08183f004e1f2b87a504d3c269eb8ef9e0ae66d62425d20208c47248a5db7047\": container with ID starting with 08183f004e1f2b87a504d3c269eb8ef9e0ae66d62425d20208c47248a5db7047 not found: ID does not exist"
Sep 10 19:30:38.477805 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:38.477777    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9"]
Sep 10 19:30:38.485211 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:38.485189    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-588a50a1-6d654dfff8-g7tn9"]
Sep 10 19:30:39.242579 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:39.242537    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd4ef127-564c-40e8-ac73-fe846716fcda" path="/var/lib/kubelet/pods/fd4ef127-564c-40e8-ac73-fe846716fcda/volumes"
Sep 10 19:30:43.302420 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.302350    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:30:43.393496 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.393469    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f565038-502b-40f9-b776-b02836bd6094-istio-data\") pod \"5f565038-502b-40f9-b776-b02836bd6094\" (UID: \"5f565038-502b-40f9-b776-b02836bd6094\") "
Sep 10 19:30:43.393656 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.393573    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f565038-502b-40f9-b776-b02836bd6094-istio-envoy\") pod \"5f565038-502b-40f9-b776-b02836bd6094\" (UID: \"5f565038-502b-40f9-b776-b02836bd6094\") "
Sep 10 19:30:43.393656 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.393612    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f565038-502b-40f9-b776-b02836bd6094-workload-certs\") pod \"5f565038-502b-40f9-b776-b02836bd6094\" (UID: \"5f565038-502b-40f9-b776-b02836bd6094\") "
Sep 10 19:30:43.393656 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.393632    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f565038-502b-40f9-b776-b02836bd6094-workload-socket\") pod \"5f565038-502b-40f9-b776-b02836bd6094\" (UID: \"5f565038-502b-40f9-b776-b02836bd6094\") "
Sep 10 19:30:43.393800 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.393703    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f565038-502b-40f9-b776-b02836bd6094-istio-data" (OuterVolumeSpecName: "istio-data") pod "5f565038-502b-40f9-b776-b02836bd6094" (UID: "5f565038-502b-40f9-b776-b02836bd6094"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:30:43.393800 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.393707    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f565038-502b-40f9-b776-b02836bd6094-credential-socket\") pod \"5f565038-502b-40f9-b776-b02836bd6094\" (UID: \"5f565038-502b-40f9-b776-b02836bd6094\") "
Sep 10 19:30:43.393885 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.393797    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f565038-502b-40f9-b776-b02836bd6094-istio-podinfo\") pod \"5f565038-502b-40f9-b776-b02836bd6094\" (UID: \"5f565038-502b-40f9-b776-b02836bd6094\") "
Sep 10 19:30:43.393885 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.393858    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f565038-502b-40f9-b776-b02836bd6094-istiod-ca-cert\") pod \"5f565038-502b-40f9-b776-b02836bd6094\" (UID: \"5f565038-502b-40f9-b776-b02836bd6094\") "
Sep 10 19:30:43.393996 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.393915    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f565038-502b-40f9-b776-b02836bd6094-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5f565038-502b-40f9-b776-b02836bd6094" (UID: "5f565038-502b-40f9-b776-b02836bd6094"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:30:43.393996 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.393933    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tphxz\" (UniqueName: \"kubernetes.io/projected/5f565038-502b-40f9-b776-b02836bd6094-kube-api-access-tphxz\") pod \"5f565038-502b-40f9-b776-b02836bd6094\" (UID: \"5f565038-502b-40f9-b776-b02836bd6094\") "
Sep 10 19:30:43.393996 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.393935    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f565038-502b-40f9-b776-b02836bd6094-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5f565038-502b-40f9-b776-b02836bd6094" (UID: "5f565038-502b-40f9-b776-b02836bd6094"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:30:43.393996 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.393959    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f565038-502b-40f9-b776-b02836bd6094-istio-token\") pod \"5f565038-502b-40f9-b776-b02836bd6094\" (UID: \"5f565038-502b-40f9-b776-b02836bd6094\") "
Sep 10 19:30:43.393996 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.393966    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f565038-502b-40f9-b776-b02836bd6094-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5f565038-502b-40f9-b776-b02836bd6094" (UID: "5f565038-502b-40f9-b776-b02836bd6094"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:30:43.394315 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.394288    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f565038-502b-40f9-b776-b02836bd6094-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5f565038-502b-40f9-b776-b02836bd6094" (UID: "5f565038-502b-40f9-b776-b02836bd6094"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:30:43.394668 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.394499    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f565038-502b-40f9-b776-b02836bd6094-istiod-ca-cert\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:30:43.394668 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.394519    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f565038-502b-40f9-b776-b02836bd6094-istio-data\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:30:43.394668 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.394532    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f565038-502b-40f9-b776-b02836bd6094-workload-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:30:43.394668 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.394544    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f565038-502b-40f9-b776-b02836bd6094-workload-socket\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:30:43.394668 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.394557    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f565038-502b-40f9-b776-b02836bd6094-credential-socket\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:30:43.395987 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.395960    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f565038-502b-40f9-b776-b02836bd6094-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5f565038-502b-40f9-b776-b02836bd6094" (UID: "5f565038-502b-40f9-b776-b02836bd6094"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:30:43.396292 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.396275    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5f565038-502b-40f9-b776-b02836bd6094-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5f565038-502b-40f9-b776-b02836bd6094" (UID: "5f565038-502b-40f9-b776-b02836bd6094"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:30:43.396677 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.396653    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f565038-502b-40f9-b776-b02836bd6094-istio-token" (OuterVolumeSpecName: "istio-token") pod "5f565038-502b-40f9-b776-b02836bd6094" (UID: "5f565038-502b-40f9-b776-b02836bd6094"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:30:43.396743 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.396687    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f565038-502b-40f9-b776-b02836bd6094-kube-api-access-tphxz" (OuterVolumeSpecName: "kube-api-access-tphxz") pod "5f565038-502b-40f9-b776-b02836bd6094" (UID: "5f565038-502b-40f9-b776-b02836bd6094"). InnerVolumeSpecName "kube-api-access-tphxz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:30:43.490774 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.490727    2579 generic.go:356] "Generic (PLEG): container finished" podID="5f565038-502b-40f9-b776-b02836bd6094" containerID="042a7570d65c39a771cb8dae83faafb37d3152ea18f72cbe92907ca7af8eb557" exitCode=0
Sep 10 19:30:43.490935 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.490791    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"
Sep 10 19:30:43.490935 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.490815    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq" event={"ID":"5f565038-502b-40f9-b776-b02836bd6094","Type":"ContainerDied","Data":"042a7570d65c39a771cb8dae83faafb37d3152ea18f72cbe92907ca7af8eb557"}
Sep 10 19:30:43.490935 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.490873    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq" event={"ID":"5f565038-502b-40f9-b776-b02836bd6094","Type":"ContainerDied","Data":"fe4341f1700f3c801056b9b0c7c7bfaa219da4cdb149c5f93b56d44cf40b7545"}
Sep 10 19:30:43.490935 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.490897    2579 scope.go:117] "RemoveContainer" containerID="042a7570d65c39a771cb8dae83faafb37d3152ea18f72cbe92907ca7af8eb557"
Sep 10 19:30:43.496150 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.496126    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f565038-502b-40f9-b776-b02836bd6094-istio-podinfo\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:30:43.496150 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.496150    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tphxz\" (UniqueName: \"kubernetes.io/projected/5f565038-502b-40f9-b776-b02836bd6094-kube-api-access-tphxz\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:30:43.496299 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.496161    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f565038-502b-40f9-b776-b02836bd6094-istio-token\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:30:43.496299 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.496174    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f565038-502b-40f9-b776-b02836bd6094-istio-envoy\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:30:43.500742 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.500727    2579 scope.go:117] "RemoveContainer" containerID="4540f4d7273facc8a3c7f4eb11cf74ba2e0f745f501257ddc87c7d9998f08a7a"
Sep 10 19:30:43.509791 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.509768    2579 scope.go:117] "RemoveContainer" containerID="042a7570d65c39a771cb8dae83faafb37d3152ea18f72cbe92907ca7af8eb557"
Sep 10 19:30:43.510110 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:30:43.510089    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"042a7570d65c39a771cb8dae83faafb37d3152ea18f72cbe92907ca7af8eb557\": container with ID starting with 042a7570d65c39a771cb8dae83faafb37d3152ea18f72cbe92907ca7af8eb557 not found: ID does not exist" containerID="042a7570d65c39a771cb8dae83faafb37d3152ea18f72cbe92907ca7af8eb557"
Sep 10 19:30:43.510207 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.510122    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"042a7570d65c39a771cb8dae83faafb37d3152ea18f72cbe92907ca7af8eb557"} err="failed to get container status \"042a7570d65c39a771cb8dae83faafb37d3152ea18f72cbe92907ca7af8eb557\": rpc error: code = NotFound desc = could not find container \"042a7570d65c39a771cb8dae83faafb37d3152ea18f72cbe92907ca7af8eb557\": container with ID starting with 042a7570d65c39a771cb8dae83faafb37d3152ea18f72cbe92907ca7af8eb557 not found: ID does not exist"
Sep 10 19:30:43.510207 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.510146    2579 scope.go:117] "RemoveContainer" containerID="4540f4d7273facc8a3c7f4eb11cf74ba2e0f745f501257ddc87c7d9998f08a7a"
Sep 10 19:30:43.510421 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:30:43.510399    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4540f4d7273facc8a3c7f4eb11cf74ba2e0f745f501257ddc87c7d9998f08a7a\": container with ID starting with 4540f4d7273facc8a3c7f4eb11cf74ba2e0f745f501257ddc87c7d9998f08a7a not found: ID does not exist" containerID="4540f4d7273facc8a3c7f4eb11cf74ba2e0f745f501257ddc87c7d9998f08a7a"
Sep 10 19:30:43.510467 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.510431    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4540f4d7273facc8a3c7f4eb11cf74ba2e0f745f501257ddc87c7d9998f08a7a"} err="failed to get container status \"4540f4d7273facc8a3c7f4eb11cf74ba2e0f745f501257ddc87c7d9998f08a7a\": rpc error: code = NotFound desc = could not find container \"4540f4d7273facc8a3c7f4eb11cf74ba2e0f745f501257ddc87c7d9998f08a7a\": container with ID starting with 4540f4d7273facc8a3c7f4eb11cf74ba2e0f745f501257ddc87c7d9998f08a7a not found: ID does not exist"
Sep 10 19:30:43.513180 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.513142    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"]
Sep 10 19:30:43.516631 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:43.516607    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq"]
Sep 10 19:30:45.195384 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:45.195357    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bmdg9_9817270c-8629-44c4-ab8f-f3982e1f0e2f/console-operator/1.log"
Sep 10 19:30:45.197188 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:45.197170    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bmdg9_9817270c-8629-44c4-ab8f-f3982e1f0e2f/console-operator/1.log"
Sep 10 19:30:45.200243 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:45.200224    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8r66s_2826f8ec-78f6-4cab-879e-47a21854ed5a/ovn-acl-logging/0.log"
Sep 10 19:30:45.201598 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:45.201581    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8r66s_2826f8ec-78f6-4cab-879e-47a21854ed5a/ovn-acl-logging/0.log"
Sep 10 19:30:45.240534 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:45.240504    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f565038-502b-40f9-b776-b02836bd6094" path="/var/lib/kubelet/pods/5f565038-502b-40f9-b776-b02836bd6094/volumes"
Sep 10 19:30:58.999742 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:58.999702    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-faed1b/explicit-model-faed1b-kserve-5c869768bc-ns4vq"]
Sep 10 19:30:59.000350 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:59.000329    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f565038-502b-40f9-b776-b02836bd6094" containerName="istio-proxy"
Sep 10 19:30:59.000428 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:59.000354    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f565038-502b-40f9-b776-b02836bd6094" containerName="istio-proxy"
Sep 10 19:30:59.000428 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:59.000383    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4b7c1d3-1982-4ff4-9552-19cb2043c4fe" containerName="istio-proxy"
Sep 10 19:30:59.000428 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:59.000392    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4b7c1d3-1982-4ff4-9552-19cb2043c4fe" containerName="istio-proxy"
Sep 10 19:30:59.000428 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:59.000416    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd4ef127-564c-40e8-ac73-fe846716fcda" containerName="maas-api"
Sep 10 19:30:59.000428 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:59.000424    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd4ef127-564c-40e8-ac73-fe846716fcda" containerName="maas-api"
Sep 10 19:30:59.000687 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:59.000534    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd4ef127-564c-40e8-ac73-fe846716fcda" containerName="maas-api"
Sep 10 19:30:59.000687 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:59.000550    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f565038-502b-40f9-b776-b02836bd6094" containerName="istio-proxy"
Sep 10 19:30:59.000687 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:59.000561    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4b7c1d3-1982-4ff4-9552-19cb2043c4fe" containerName="istio-proxy"
Sep 10 19:30:59.000687 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:59.000571    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f565038-502b-40f9-b776-b02836bd6094" containerName="istio-proxy"
Sep 10 19:30:59.000687 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:59.000579    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4b7c1d3-1982-4ff4-9552-19cb2043c4fe" containerName="istio-proxy"
Sep 10 19:30:59.005620 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:59.005596    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-faed1b/explicit-model-faed1b-kserve-5c869768bc-ns4vq"
Sep 10 19:30:59.008071 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:59.008051    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-faed1b\"/\"default-dockercfg-jzl5t\""
Sep 10 19:30:59.008705 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:59.008686    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-faed1b\"/\"kube-root-ca.crt\""
Sep 10 19:30:59.019291 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:59.019274    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-faed1b\"/\"openshift-service-ca.crt\""
Sep 10 19:30:59.019636 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:59.019615    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-faed1b\"/\"explicit-model-faed1b-kserve-self-signed-certs\""
Sep 10 19:30:59.023194 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:59.023173    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-faed1b/explicit-model-faed1b-kserve-5c869768bc-ns4vq"]
Sep 10 19:30:59.148900 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:59.148865    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7cfbd55-31e7-436b-a386-f2d0301e4d82-home\") pod \"explicit-model-faed1b-kserve-5c869768bc-ns4vq\" (UID: \"e7cfbd55-31e7-436b-a386-f2d0301e4d82\") " pod="ai-tenant-e2e-explicit-faed1b/explicit-model-faed1b-kserve-5c869768bc-ns4vq"
Sep 10 19:30:59.149068 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:59.148927    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7cfbd55-31e7-436b-a386-f2d0301e4d82-tmp-dir\") pod \"explicit-model-faed1b-kserve-5c869768bc-ns4vq\" (UID: \"e7cfbd55-31e7-436b-a386-f2d0301e4d82\") " pod="ai-tenant-e2e-explicit-faed1b/explicit-model-faed1b-kserve-5c869768bc-ns4vq"
Sep 10 19:30:59.149068 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:59.148955    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mh7bm\" (UniqueName: \"kubernetes.io/projected/e7cfbd55-31e7-436b-a386-f2d0301e4d82-kube-api-access-mh7bm\") pod \"explicit-model-faed1b-kserve-5c869768bc-ns4vq\" (UID: \"e7cfbd55-31e7-436b-a386-f2d0301e4d82\") " pod="ai-tenant-e2e-explicit-faed1b/explicit-model-faed1b-kserve-5c869768bc-ns4vq"
Sep 10 19:30:59.149068 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:59.148982    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7cfbd55-31e7-436b-a386-f2d0301e4d82-tls-certs\") pod \"explicit-model-faed1b-kserve-5c869768bc-ns4vq\" (UID: \"e7cfbd55-31e7-436b-a386-f2d0301e4d82\") " pod="ai-tenant-e2e-explicit-faed1b/explicit-model-faed1b-kserve-5c869768bc-ns4vq"
Sep 10 19:30:59.149068 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:59.149065    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7cfbd55-31e7-436b-a386-f2d0301e4d82-model-cache\") pod \"explicit-model-faed1b-kserve-5c869768bc-ns4vq\" (UID: \"e7cfbd55-31e7-436b-a386-f2d0301e4d82\") " pod="ai-tenant-e2e-explicit-faed1b/explicit-model-faed1b-kserve-5c869768bc-ns4vq"
Sep 10 19:30:59.149262 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:59.149084    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7cfbd55-31e7-436b-a386-f2d0301e4d82-dshm\") pod \"explicit-model-faed1b-kserve-5c869768bc-ns4vq\" (UID: \"e7cfbd55-31e7-436b-a386-f2d0301e4d82\") " pod="ai-tenant-e2e-explicit-faed1b/explicit-model-faed1b-kserve-5c869768bc-ns4vq"
Sep 10 19:30:59.250890 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:59.250707    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7cfbd55-31e7-436b-a386-f2d0301e4d82-tmp-dir\") pod \"explicit-model-faed1b-kserve-5c869768bc-ns4vq\" (UID: \"e7cfbd55-31e7-436b-a386-f2d0301e4d82\") " pod="ai-tenant-e2e-explicit-faed1b/explicit-model-faed1b-kserve-5c869768bc-ns4vq"
Sep 10 19:30:59.250890 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:59.250779    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mh7bm\" (UniqueName: \"kubernetes.io/projected/e7cfbd55-31e7-436b-a386-f2d0301e4d82-kube-api-access-mh7bm\") pod \"explicit-model-faed1b-kserve-5c869768bc-ns4vq\" (UID: \"e7cfbd55-31e7-436b-a386-f2d0301e4d82\") " pod="ai-tenant-e2e-explicit-faed1b/explicit-model-faed1b-kserve-5c869768bc-ns4vq"
Sep 10 19:30:59.250890 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:59.250822    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7cfbd55-31e7-436b-a386-f2d0301e4d82-tls-certs\") pod \"explicit-model-faed1b-kserve-5c869768bc-ns4vq\" (UID: \"e7cfbd55-31e7-436b-a386-f2d0301e4d82\") " pod="ai-tenant-e2e-explicit-faed1b/explicit-model-faed1b-kserve-5c869768bc-ns4vq"
Sep 10 19:30:59.251548 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:59.250929    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7cfbd55-31e7-436b-a386-f2d0301e4d82-model-cache\") pod \"explicit-model-faed1b-kserve-5c869768bc-ns4vq\" (UID: \"e7cfbd55-31e7-436b-a386-f2d0301e4d82\") " pod="ai-tenant-e2e-explicit-faed1b/explicit-model-faed1b-kserve-5c869768bc-ns4vq"
Sep 10 19:30:59.251548 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:59.250963    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7cfbd55-31e7-436b-a386-f2d0301e4d82-dshm\") pod \"explicit-model-faed1b-kserve-5c869768bc-ns4vq\" (UID: \"e7cfbd55-31e7-436b-a386-f2d0301e4d82\") " pod="ai-tenant-e2e-explicit-faed1b/explicit-model-faed1b-kserve-5c869768bc-ns4vq"
Sep 10 19:30:59.251548 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:59.251021    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7cfbd55-31e7-436b-a386-f2d0301e4d82-home\") pod \"explicit-model-faed1b-kserve-5c869768bc-ns4vq\" (UID: \"e7cfbd55-31e7-436b-a386-f2d0301e4d82\") " pod="ai-tenant-e2e-explicit-faed1b/explicit-model-faed1b-kserve-5c869768bc-ns4vq"
Sep 10 19:30:59.251548 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:59.251466    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7cfbd55-31e7-436b-a386-f2d0301e4d82-home\") pod \"explicit-model-faed1b-kserve-5c869768bc-ns4vq\" (UID: \"e7cfbd55-31e7-436b-a386-f2d0301e4d82\") " pod="ai-tenant-e2e-explicit-faed1b/explicit-model-faed1b-kserve-5c869768bc-ns4vq"
Sep 10 19:30:59.251548 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:59.251521    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7cfbd55-31e7-436b-a386-f2d0301e4d82-model-cache\") pod \"explicit-model-faed1b-kserve-5c869768bc-ns4vq\" (UID: \"e7cfbd55-31e7-436b-a386-f2d0301e4d82\") " pod="ai-tenant-e2e-explicit-faed1b/explicit-model-faed1b-kserve-5c869768bc-ns4vq"
Sep 10 19:30:59.252223 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:59.252119    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7cfbd55-31e7-436b-a386-f2d0301e4d82-tmp-dir\") pod \"explicit-model-faed1b-kserve-5c869768bc-ns4vq\" (UID: \"e7cfbd55-31e7-436b-a386-f2d0301e4d82\") " pod="ai-tenant-e2e-explicit-faed1b/explicit-model-faed1b-kserve-5c869768bc-ns4vq"
Sep 10 19:30:59.254238 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:59.254217    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7cfbd55-31e7-436b-a386-f2d0301e4d82-dshm\") pod \"explicit-model-faed1b-kserve-5c869768bc-ns4vq\" (UID: \"e7cfbd55-31e7-436b-a386-f2d0301e4d82\") " pod="ai-tenant-e2e-explicit-faed1b/explicit-model-faed1b-kserve-5c869768bc-ns4vq"
Sep 10 19:30:59.254323 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:59.254240    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7cfbd55-31e7-436b-a386-f2d0301e4d82-tls-certs\") pod \"explicit-model-faed1b-kserve-5c869768bc-ns4vq\" (UID: \"e7cfbd55-31e7-436b-a386-f2d0301e4d82\") " pod="ai-tenant-e2e-explicit-faed1b/explicit-model-faed1b-kserve-5c869768bc-ns4vq"
Sep 10 19:30:59.258818 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:59.258793    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mh7bm\" (UniqueName: \"kubernetes.io/projected/e7cfbd55-31e7-436b-a386-f2d0301e4d82-kube-api-access-mh7bm\") pod \"explicit-model-faed1b-kserve-5c869768bc-ns4vq\" (UID: \"e7cfbd55-31e7-436b-a386-f2d0301e4d82\") " pod="ai-tenant-e2e-explicit-faed1b/explicit-model-faed1b-kserve-5c869768bc-ns4vq"
Sep 10 19:30:59.314762 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:59.314729    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-faed1b/explicit-model-faed1b-kserve-5c869768bc-ns4vq"
Sep 10 19:30:59.592065 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:30:59.591984    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-faed1b/explicit-model-faed1b-kserve-5c869768bc-ns4vq"]
Sep 10 19:30:59.596066 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:30:59.596032    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode7cfbd55_31e7_436b_a386_f2d0301e4d82.slice/crio-efde3006b4bebdca8b6d4dced4907dc8fe8c0fe504ae636b2db103ffe291582d WatchSource:0}: Error finding container efde3006b4bebdca8b6d4dced4907dc8fe8c0fe504ae636b2db103ffe291582d: Status 404 returned error can't find the container with id efde3006b4bebdca8b6d4dced4907dc8fe8c0fe504ae636b2db103ffe291582d
Sep 10 19:31:00.605107 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:00.605070    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-faed1b/explicit-model-faed1b-kserve-5c869768bc-ns4vq" event={"ID":"e7cfbd55-31e7-436b-a386-f2d0301e4d82","Type":"ContainerStarted","Data":"1e028e1255a2790e1dddc65a7bbb591f853f8e56692242b8862a77a09ca319b9"}
Sep 10 19:31:00.605506 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:00.605115    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-faed1b/explicit-model-faed1b-kserve-5c869768bc-ns4vq" event={"ID":"e7cfbd55-31e7-436b-a386-f2d0301e4d82","Type":"ContainerStarted","Data":"efde3006b4bebdca8b6d4dced4907dc8fe8c0fe504ae636b2db103ffe291582d"}
Sep 10 19:31:00.621223 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:00.621171    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-faed1b/explicit-model-faed1b-kserve-5c869768bc-ns4vq" podStartSLOduration=2.621151781 podStartE2EDuration="2.621151781s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:30:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:31:00.620606416 +0000 UTC m=+915.840454897" watchObservedRunningTime="2026-09-10 19:31:00.621151781 +0000 UTC m=+915.841000262"
Sep 10 19:31:04.861221 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:04.861188    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-faed1b/explicit-model-faed1b-kserve-5c869768bc-ns4vq"]
Sep 10 19:31:09.315861 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:09.315830    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-faed1b/explicit-model-faed1b-kserve-5c869768bc-ns4vq"
Sep 10 19:31:15.509476 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:15.509441    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94"]
Sep 10 19:31:15.510060 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:15.509741    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94" podUID="a18f1c5a-8e34-452f-9c8c-a73c087b2c4a" containerName="maas-api" containerID="cri-o://ffbefdd3b09458a9fe47d9a3fc5b321501bb2af5124549fbe71e74798279b80c" gracePeriod=30
Sep 10 19:31:15.699782 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:15.699722    2579 generic.go:356] "Generic (PLEG): container finished" podID="a18f1c5a-8e34-452f-9c8c-a73c087b2c4a" containerID="ffbefdd3b09458a9fe47d9a3fc5b321501bb2af5124549fbe71e74798279b80c" exitCode=0
Sep 10 19:31:15.699928 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:15.699786    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94" event={"ID":"a18f1c5a-8e34-452f-9c8c-a73c087b2c4a","Type":"ContainerDied","Data":"ffbefdd3b09458a9fe47d9a3fc5b321501bb2af5124549fbe71e74798279b80c"}
Sep 10 19:31:15.770191 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:15.770133    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94"
Sep 10 19:31:15.819251 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:15.819227    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a18f1c5a-8e34-452f-9c8c-a73c087b2c4a-metrics-certs\") pod \"a18f1c5a-8e34-452f-9c8c-a73c087b2c4a\" (UID: \"a18f1c5a-8e34-452f-9c8c-a73c087b2c4a\") "
Sep 10 19:31:15.819420 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:15.819285    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a18f1c5a-8e34-452f-9c8c-a73c087b2c4a-openshift-service-ca\") pod \"a18f1c5a-8e34-452f-9c8c-a73c087b2c4a\" (UID: \"a18f1c5a-8e34-452f-9c8c-a73c087b2c4a\") "
Sep 10 19:31:15.819420 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:15.819322    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a18f1c5a-8e34-452f-9c8c-a73c087b2c4a-maas-api-tls\") pod \"a18f1c5a-8e34-452f-9c8c-a73c087b2c4a\" (UID: \"a18f1c5a-8e34-452f-9c8c-a73c087b2c4a\") "
Sep 10 19:31:15.819420 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:15.819363    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cxtwj\" (UniqueName: \"kubernetes.io/projected/a18f1c5a-8e34-452f-9c8c-a73c087b2c4a-kube-api-access-cxtwj\") pod \"a18f1c5a-8e34-452f-9c8c-a73c087b2c4a\" (UID: \"a18f1c5a-8e34-452f-9c8c-a73c087b2c4a\") "
Sep 10 19:31:15.819802 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:15.819740    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a18f1c5a-8e34-452f-9c8c-a73c087b2c4a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a18f1c5a-8e34-452f-9c8c-a73c087b2c4a" (UID: "a18f1c5a-8e34-452f-9c8c-a73c087b2c4a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:31:15.821560 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:15.821535    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a18f1c5a-8e34-452f-9c8c-a73c087b2c4a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a18f1c5a-8e34-452f-9c8c-a73c087b2c4a" (UID: "a18f1c5a-8e34-452f-9c8c-a73c087b2c4a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:31:15.821671 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:15.821638    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a18f1c5a-8e34-452f-9c8c-a73c087b2c4a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a18f1c5a-8e34-452f-9c8c-a73c087b2c4a" (UID: "a18f1c5a-8e34-452f-9c8c-a73c087b2c4a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:31:15.821713 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:15.821688    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a18f1c5a-8e34-452f-9c8c-a73c087b2c4a-kube-api-access-cxtwj" (OuterVolumeSpecName: "kube-api-access-cxtwj") pod "a18f1c5a-8e34-452f-9c8c-a73c087b2c4a" (UID: "a18f1c5a-8e34-452f-9c8c-a73c087b2c4a"). InnerVolumeSpecName "kube-api-access-cxtwj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:31:15.921060 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:15.921026    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a18f1c5a-8e34-452f-9c8c-a73c087b2c4a-maas-api-tls\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:31:15.921060 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:15.921053    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxtwj\" (UniqueName: \"kubernetes.io/projected/a18f1c5a-8e34-452f-9c8c-a73c087b2c4a-kube-api-access-cxtwj\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:31:15.921060 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:15.921063    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a18f1c5a-8e34-452f-9c8c-a73c087b2c4a-metrics-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:31:15.921060 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:15.921071    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a18f1c5a-8e34-452f-9c8c-a73c087b2c4a-openshift-service-ca\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:31:16.707684 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:16.707645    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94" event={"ID":"a18f1c5a-8e34-452f-9c8c-a73c087b2c4a","Type":"ContainerDied","Data":"285c8d430c18262b3c2fcb36559a921de8d349b36f0a25abf3687ddd7f428bc4"}
Sep 10 19:31:16.707684 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:16.707665    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94"
Sep 10 19:31:16.708209 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:16.707744    2579 scope.go:117] "RemoveContainer" containerID="ffbefdd3b09458a9fe47d9a3fc5b321501bb2af5124549fbe71e74798279b80c"
Sep 10 19:31:16.729173 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:16.729149    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94"]
Sep 10 19:31:16.733564 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:16.733541    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-faed1b-765ccd4dbc-gpf94"]
Sep 10 19:31:17.240148 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:17.240118    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a18f1c5a-8e34-452f-9c8c-a73c087b2c4a" path="/var/lib/kubelet/pods/a18f1c5a-8e34-452f-9c8c-a73c087b2c4a/volumes"
Sep 10 19:31:19.883789 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:19.883724    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-faed1b/explicit-model-faed1b-kserve-5c869768bc-ns4vq" podUID="e7cfbd55-31e7-436b-a386-f2d0301e4d82" containerName="main" containerID="cri-o://1e028e1255a2790e1dddc65a7bbb591f853f8e56692242b8862a77a09ca319b9" gracePeriod=45
Sep 10 19:31:20.134513 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:20.134445    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-faed1b/explicit-model-faed1b-kserve-5c869768bc-ns4vq"
Sep 10 19:31:20.166582 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:20.166557    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7cfbd55-31e7-436b-a386-f2d0301e4d82-tls-certs\") pod \"e7cfbd55-31e7-436b-a386-f2d0301e4d82\" (UID: \"e7cfbd55-31e7-436b-a386-f2d0301e4d82\") "
Sep 10 19:31:20.166781 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:20.166629    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7cfbd55-31e7-436b-a386-f2d0301e4d82-home\") pod \"e7cfbd55-31e7-436b-a386-f2d0301e4d82\" (UID: \"e7cfbd55-31e7-436b-a386-f2d0301e4d82\") "
Sep 10 19:31:20.166781 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:20.166700    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7cfbd55-31e7-436b-a386-f2d0301e4d82-dshm\") pod \"e7cfbd55-31e7-436b-a386-f2d0301e4d82\" (UID: \"e7cfbd55-31e7-436b-a386-f2d0301e4d82\") "
Sep 10 19:31:20.166781 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:20.166770    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mh7bm\" (UniqueName: \"kubernetes.io/projected/e7cfbd55-31e7-436b-a386-f2d0301e4d82-kube-api-access-mh7bm\") pod \"e7cfbd55-31e7-436b-a386-f2d0301e4d82\" (UID: \"e7cfbd55-31e7-436b-a386-f2d0301e4d82\") "
Sep 10 19:31:20.166953 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:20.166817    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7cfbd55-31e7-436b-a386-f2d0301e4d82-model-cache\") pod \"e7cfbd55-31e7-436b-a386-f2d0301e4d82\" (UID: \"e7cfbd55-31e7-436b-a386-f2d0301e4d82\") "
Sep 10 19:31:20.166953 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:20.166846    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7cfbd55-31e7-436b-a386-f2d0301e4d82-tmp-dir\") pod \"e7cfbd55-31e7-436b-a386-f2d0301e4d82\" (UID: \"e7cfbd55-31e7-436b-a386-f2d0301e4d82\") "
Sep 10 19:31:20.166953 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:20.166889    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7cfbd55-31e7-436b-a386-f2d0301e4d82-home" (OuterVolumeSpecName: "home") pod "e7cfbd55-31e7-436b-a386-f2d0301e4d82" (UID: "e7cfbd55-31e7-436b-a386-f2d0301e4d82"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:31:20.167145 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:20.167112    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7cfbd55-31e7-436b-a386-f2d0301e4d82-model-cache" (OuterVolumeSpecName: "model-cache") pod "e7cfbd55-31e7-436b-a386-f2d0301e4d82" (UID: "e7cfbd55-31e7-436b-a386-f2d0301e4d82"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:31:20.167309 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:20.167214    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7cfbd55-31e7-436b-a386-f2d0301e4d82-model-cache\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:31:20.167309 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:20.167231    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7cfbd55-31e7-436b-a386-f2d0301e4d82-home\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:31:20.167309 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:20.167236    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7cfbd55-31e7-436b-a386-f2d0301e4d82-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e7cfbd55-31e7-436b-a386-f2d0301e4d82" (UID: "e7cfbd55-31e7-436b-a386-f2d0301e4d82"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:31:20.169115 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:20.169089    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7cfbd55-31e7-436b-a386-f2d0301e4d82-dshm" (OuterVolumeSpecName: "dshm") pod "e7cfbd55-31e7-436b-a386-f2d0301e4d82" (UID: "e7cfbd55-31e7-436b-a386-f2d0301e4d82"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:31:20.169200 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:20.169120    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7cfbd55-31e7-436b-a386-f2d0301e4d82-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e7cfbd55-31e7-436b-a386-f2d0301e4d82" (UID: "e7cfbd55-31e7-436b-a386-f2d0301e4d82"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:31:20.169418 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:20.169405    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7cfbd55-31e7-436b-a386-f2d0301e4d82-kube-api-access-mh7bm" (OuterVolumeSpecName: "kube-api-access-mh7bm") pod "e7cfbd55-31e7-436b-a386-f2d0301e4d82" (UID: "e7cfbd55-31e7-436b-a386-f2d0301e4d82"). InnerVolumeSpecName "kube-api-access-mh7bm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:31:20.268439 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:20.268398    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mh7bm\" (UniqueName: \"kubernetes.io/projected/e7cfbd55-31e7-436b-a386-f2d0301e4d82-kube-api-access-mh7bm\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:31:20.268439 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:20.268433    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7cfbd55-31e7-436b-a386-f2d0301e4d82-tmp-dir\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:31:20.268439 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:20.268442    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7cfbd55-31e7-436b-a386-f2d0301e4d82-tls-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:31:20.268439 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:20.268450    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7cfbd55-31e7-436b-a386-f2d0301e4d82-dshm\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:31:20.741415 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:20.741375    2579 generic.go:356] "Generic (PLEG): container finished" podID="e7cfbd55-31e7-436b-a386-f2d0301e4d82" containerID="1e028e1255a2790e1dddc65a7bbb591f853f8e56692242b8862a77a09ca319b9" exitCode=0
Sep 10 19:31:20.741599 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:20.741439    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-faed1b/explicit-model-faed1b-kserve-5c869768bc-ns4vq"
Sep 10 19:31:20.741599 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:20.741455    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-faed1b/explicit-model-faed1b-kserve-5c869768bc-ns4vq" event={"ID":"e7cfbd55-31e7-436b-a386-f2d0301e4d82","Type":"ContainerDied","Data":"1e028e1255a2790e1dddc65a7bbb591f853f8e56692242b8862a77a09ca319b9"}
Sep 10 19:31:20.741599 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:20.741506    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-faed1b/explicit-model-faed1b-kserve-5c869768bc-ns4vq" event={"ID":"e7cfbd55-31e7-436b-a386-f2d0301e4d82","Type":"ContainerDied","Data":"efde3006b4bebdca8b6d4dced4907dc8fe8c0fe504ae636b2db103ffe291582d"}
Sep 10 19:31:20.741599 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:20.741523    2579 scope.go:117] "RemoveContainer" containerID="1e028e1255a2790e1dddc65a7bbb591f853f8e56692242b8862a77a09ca319b9"
Sep 10 19:31:20.751468 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:20.751449    2579 scope.go:117] "RemoveContainer" containerID="1e028e1255a2790e1dddc65a7bbb591f853f8e56692242b8862a77a09ca319b9"
Sep 10 19:31:20.751795 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:31:20.751743    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e028e1255a2790e1dddc65a7bbb591f853f8e56692242b8862a77a09ca319b9\": container with ID starting with 1e028e1255a2790e1dddc65a7bbb591f853f8e56692242b8862a77a09ca319b9 not found: ID does not exist" containerID="1e028e1255a2790e1dddc65a7bbb591f853f8e56692242b8862a77a09ca319b9"
Sep 10 19:31:20.751894 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:20.751804    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e028e1255a2790e1dddc65a7bbb591f853f8e56692242b8862a77a09ca319b9"} err="failed to get container status \"1e028e1255a2790e1dddc65a7bbb591f853f8e56692242b8862a77a09ca319b9\": rpc error: code = NotFound desc = could not find container \"1e028e1255a2790e1dddc65a7bbb591f853f8e56692242b8862a77a09ca319b9\": container with ID starting with 1e028e1255a2790e1dddc65a7bbb591f853f8e56692242b8862a77a09ca319b9 not found: ID does not exist"
Sep 10 19:31:20.763032 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:20.763010    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-faed1b/explicit-model-faed1b-kserve-5c869768bc-ns4vq"]
Sep 10 19:31:20.766701 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:20.766681    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-faed1b/explicit-model-faed1b-kserve-5c869768bc-ns4vq"]
Sep 10 19:31:21.240391 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:21.240354    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7cfbd55-31e7-436b-a386-f2d0301e4d82" path="/var/lib/kubelet/pods/e7cfbd55-31e7-436b-a386-f2d0301e4d82/volumes"
Sep 10 19:31:23.304597 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.304562    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"]
Sep 10 19:31:23.305089 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.305075    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a18f1c5a-8e34-452f-9c8c-a73c087b2c4a" containerName="maas-api"
Sep 10 19:31:23.305144 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.305091    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a18f1c5a-8e34-452f-9c8c-a73c087b2c4a" containerName="maas-api"
Sep 10 19:31:23.305144 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.305100    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7cfbd55-31e7-436b-a386-f2d0301e4d82" containerName="main"
Sep 10 19:31:23.305144 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.305106    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7cfbd55-31e7-436b-a386-f2d0301e4d82" containerName="main"
Sep 10 19:31:23.305144 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.305125    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f565038-502b-40f9-b776-b02836bd6094" containerName="istio-proxy"
Sep 10 19:31:23.305144 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.305133    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f565038-502b-40f9-b776-b02836bd6094" containerName="istio-proxy"
Sep 10 19:31:23.305300 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.305149    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4b7c1d3-1982-4ff4-9552-19cb2043c4fe" containerName="istio-proxy"
Sep 10 19:31:23.305300 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.305154    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4b7c1d3-1982-4ff4-9552-19cb2043c4fe" containerName="istio-proxy"
Sep 10 19:31:23.305300 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.305223    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7cfbd55-31e7-436b-a386-f2d0301e4d82" containerName="main"
Sep 10 19:31:23.305300 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.305233    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a18f1c5a-8e34-452f-9c8c-a73c087b2c4a" containerName="maas-api"
Sep 10 19:31:23.309613 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.309590    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"
Sep 10 19:31:23.311720 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.311702    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-8d88a2-openshift-default-dockercfg-cqfgb\""
Sep 10 19:31:23.320850 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.320825    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"]
Sep 10 19:31:23.401491 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.401444    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a301d6a-8127-482e-b2f4-7780cbac60ed-istiod-ca-cert\") pod \"e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx\" (UID: \"1a301d6a-8127-482e-b2f4-7780cbac60ed\") " pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"
Sep 10 19:31:23.401704 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.401522    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a301d6a-8127-482e-b2f4-7780cbac60ed-istio-envoy\") pod \"e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx\" (UID: \"1a301d6a-8127-482e-b2f4-7780cbac60ed\") " pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"
Sep 10 19:31:23.401704 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.401558    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlqnm\" (UniqueName: \"kubernetes.io/projected/1a301d6a-8127-482e-b2f4-7780cbac60ed-kube-api-access-mlqnm\") pod \"e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx\" (UID: \"1a301d6a-8127-482e-b2f4-7780cbac60ed\") " pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"
Sep 10 19:31:23.401704 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.401663    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a301d6a-8127-482e-b2f4-7780cbac60ed-workload-socket\") pod \"e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx\" (UID: \"1a301d6a-8127-482e-b2f4-7780cbac60ed\") " pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"
Sep 10 19:31:23.401915 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.401712    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a301d6a-8127-482e-b2f4-7780cbac60ed-workload-certs\") pod \"e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx\" (UID: \"1a301d6a-8127-482e-b2f4-7780cbac60ed\") " pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"
Sep 10 19:31:23.401915 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.401822    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a301d6a-8127-482e-b2f4-7780cbac60ed-credential-socket\") pod \"e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx\" (UID: \"1a301d6a-8127-482e-b2f4-7780cbac60ed\") " pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"
Sep 10 19:31:23.401915 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.401849    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a301d6a-8127-482e-b2f4-7780cbac60ed-istio-data\") pod \"e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx\" (UID: \"1a301d6a-8127-482e-b2f4-7780cbac60ed\") " pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"
Sep 10 19:31:23.401915 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.401891    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a301d6a-8127-482e-b2f4-7780cbac60ed-istio-podinfo\") pod \"e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx\" (UID: \"1a301d6a-8127-482e-b2f4-7780cbac60ed\") " pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"
Sep 10 19:31:23.402138 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.401944    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a301d6a-8127-482e-b2f4-7780cbac60ed-istio-token\") pod \"e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx\" (UID: \"1a301d6a-8127-482e-b2f4-7780cbac60ed\") " pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"
Sep 10 19:31:23.503510 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.503475    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a301d6a-8127-482e-b2f4-7780cbac60ed-credential-socket\") pod \"e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx\" (UID: \"1a301d6a-8127-482e-b2f4-7780cbac60ed\") " pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"
Sep 10 19:31:23.503510 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.503514    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a301d6a-8127-482e-b2f4-7780cbac60ed-istio-data\") pod \"e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx\" (UID: \"1a301d6a-8127-482e-b2f4-7780cbac60ed\") " pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"
Sep 10 19:31:23.503744 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.503544    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a301d6a-8127-482e-b2f4-7780cbac60ed-istio-podinfo\") pod \"e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx\" (UID: \"1a301d6a-8127-482e-b2f4-7780cbac60ed\") " pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"
Sep 10 19:31:23.503744 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.503569    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a301d6a-8127-482e-b2f4-7780cbac60ed-istio-token\") pod \"e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx\" (UID: \"1a301d6a-8127-482e-b2f4-7780cbac60ed\") " pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"
Sep 10 19:31:23.503744 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.503611    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a301d6a-8127-482e-b2f4-7780cbac60ed-istiod-ca-cert\") pod \"e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx\" (UID: \"1a301d6a-8127-482e-b2f4-7780cbac60ed\") " pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"
Sep 10 19:31:23.503744 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.503639    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a301d6a-8127-482e-b2f4-7780cbac60ed-istio-envoy\") pod \"e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx\" (UID: \"1a301d6a-8127-482e-b2f4-7780cbac60ed\") " pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"
Sep 10 19:31:23.503744 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.503684    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlqnm\" (UniqueName: \"kubernetes.io/projected/1a301d6a-8127-482e-b2f4-7780cbac60ed-kube-api-access-mlqnm\") pod \"e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx\" (UID: \"1a301d6a-8127-482e-b2f4-7780cbac60ed\") " pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"
Sep 10 19:31:23.504040 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.503788    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a301d6a-8127-482e-b2f4-7780cbac60ed-workload-socket\") pod \"e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx\" (UID: \"1a301d6a-8127-482e-b2f4-7780cbac60ed\") " pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"
Sep 10 19:31:23.504040 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.503825    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a301d6a-8127-482e-b2f4-7780cbac60ed-workload-certs\") pod \"e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx\" (UID: \"1a301d6a-8127-482e-b2f4-7780cbac60ed\") " pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"
Sep 10 19:31:23.504040 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.503958    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a301d6a-8127-482e-b2f4-7780cbac60ed-istio-data\") pod \"e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx\" (UID: \"1a301d6a-8127-482e-b2f4-7780cbac60ed\") " pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"
Sep 10 19:31:23.504040 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.503972    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a301d6a-8127-482e-b2f4-7780cbac60ed-credential-socket\") pod \"e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx\" (UID: \"1a301d6a-8127-482e-b2f4-7780cbac60ed\") " pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"
Sep 10 19:31:23.504251 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.504177    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a301d6a-8127-482e-b2f4-7780cbac60ed-workload-certs\") pod \"e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx\" (UID: \"1a301d6a-8127-482e-b2f4-7780cbac60ed\") " pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"
Sep 10 19:31:23.504316 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.504271    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a301d6a-8127-482e-b2f4-7780cbac60ed-workload-socket\") pod \"e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx\" (UID: \"1a301d6a-8127-482e-b2f4-7780cbac60ed\") " pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"
Sep 10 19:31:23.504424 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.504404    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a301d6a-8127-482e-b2f4-7780cbac60ed-istiod-ca-cert\") pod \"e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx\" (UID: \"1a301d6a-8127-482e-b2f4-7780cbac60ed\") " pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"
Sep 10 19:31:23.506111 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.506080    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a301d6a-8127-482e-b2f4-7780cbac60ed-istio-envoy\") pod \"e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx\" (UID: \"1a301d6a-8127-482e-b2f4-7780cbac60ed\") " pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"
Sep 10 19:31:23.506220 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.506187    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a301d6a-8127-482e-b2f4-7780cbac60ed-istio-podinfo\") pod \"e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx\" (UID: \"1a301d6a-8127-482e-b2f4-7780cbac60ed\") " pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"
Sep 10 19:31:23.528706 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.528672    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlqnm\" (UniqueName: \"kubernetes.io/projected/1a301d6a-8127-482e-b2f4-7780cbac60ed-kube-api-access-mlqnm\") pod \"e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx\" (UID: \"1a301d6a-8127-482e-b2f4-7780cbac60ed\") " pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"
Sep 10 19:31:23.532425 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.532400    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a301d6a-8127-482e-b2f4-7780cbac60ed-istio-token\") pod \"e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx\" (UID: \"1a301d6a-8127-482e-b2f4-7780cbac60ed\") " pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"
Sep 10 19:31:23.621271 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.621174    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"
Sep 10 19:31:23.916275 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.916251    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"]
Sep 10 19:31:23.918521 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:31:23.918495    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a301d6a_8127_482e_b2f4_7780cbac60ed.slice/crio-1edb8c92ad9932a2ba4bb6cdb969150d79ce127bda6cf1e7c450ae7075b699f4 WatchSource:0}: Error finding container 1edb8c92ad9932a2ba4bb6cdb969150d79ce127bda6cf1e7c450ae7075b699f4: Status 404 returned error can't find the container with id 1edb8c92ad9932a2ba4bb6cdb969150d79ce127bda6cf1e7c450ae7075b699f4
Sep 10 19:31:23.920698 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.920669    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:31:23.920789 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.920729    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:31:23.920829 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:23.920795    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:31:24.774805 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:24.774730    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx" event={"ID":"1a301d6a-8127-482e-b2f4-7780cbac60ed","Type":"ContainerStarted","Data":"143a50530fcc705ebf6fffaf44478654135328dcb3c25497a3c4f2d6bcc8731d"}
Sep 10 19:31:24.775213 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:24.774858    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx" event={"ID":"1a301d6a-8127-482e-b2f4-7780cbac60ed","Type":"ContainerStarted","Data":"1edb8c92ad9932a2ba4bb6cdb969150d79ce127bda6cf1e7c450ae7075b699f4"}
Sep 10 19:31:24.796327 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:24.796274    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx" podStartSLOduration=1.796258568 podStartE2EDuration="1.796258568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:31:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:31:24.794835367 +0000 UTC m=+940.014683848" watchObservedRunningTime="2026-09-10 19:31:24.796258568 +0000 UTC m=+940.016107050"
Sep 10 19:31:25.622271 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:25.622231    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"
Sep 10 19:31:25.628294 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:25.628264    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"
Sep 10 19:31:25.781186 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:25.781148    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"
Sep 10 19:31:25.782070 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:25.782051    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"
Sep 10 19:31:32.290150 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:32.290119    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-8d88a2/noten-model-8d88a2-kserve-55fd49f99f-7fws2"]
Sep 10 19:31:32.294075 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:32.294054    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-8d88a2/noten-model-8d88a2-kserve-55fd49f99f-7fws2"
Sep 10 19:31:32.296981 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:32.296956    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-8d88a2\"/\"openshift-service-ca.crt\""
Sep 10 19:31:32.296981 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:32.296975    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-8d88a2\"/\"default-dockercfg-z7lk7\""
Sep 10 19:31:32.297152 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:32.296959    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-8d88a2\"/\"kube-root-ca.crt\""
Sep 10 19:31:32.297152 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:32.296989    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-8d88a2\"/\"noten-model-8d88a2-kserve-self-signed-certs\""
Sep 10 19:31:32.302893 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:32.302869    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-8d88a2/noten-model-8d88a2-kserve-55fd49f99f-7fws2"]
Sep 10 19:31:32.392260 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:32.392228    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf76a064-79ad-4c7b-a000-7bf71dc9ddb7-dshm\") pod \"noten-model-8d88a2-kserve-55fd49f99f-7fws2\" (UID: \"cf76a064-79ad-4c7b-a000-7bf71dc9ddb7\") " pod="ai-tenant-e2e-noten-8d88a2/noten-model-8d88a2-kserve-55fd49f99f-7fws2"
Sep 10 19:31:32.392449 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:32.392292    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf76a064-79ad-4c7b-a000-7bf71dc9ddb7-model-cache\") pod \"noten-model-8d88a2-kserve-55fd49f99f-7fws2\" (UID: \"cf76a064-79ad-4c7b-a000-7bf71dc9ddb7\") " pod="ai-tenant-e2e-noten-8d88a2/noten-model-8d88a2-kserve-55fd49f99f-7fws2"
Sep 10 19:31:32.392449 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:32.392324    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf76a064-79ad-4c7b-a000-7bf71dc9ddb7-tls-certs\") pod \"noten-model-8d88a2-kserve-55fd49f99f-7fws2\" (UID: \"cf76a064-79ad-4c7b-a000-7bf71dc9ddb7\") " pod="ai-tenant-e2e-noten-8d88a2/noten-model-8d88a2-kserve-55fd49f99f-7fws2"
Sep 10 19:31:32.392449 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:32.392374    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf76a064-79ad-4c7b-a000-7bf71dc9ddb7-home\") pod \"noten-model-8d88a2-kserve-55fd49f99f-7fws2\" (UID: \"cf76a064-79ad-4c7b-a000-7bf71dc9ddb7\") " pod="ai-tenant-e2e-noten-8d88a2/noten-model-8d88a2-kserve-55fd49f99f-7fws2"
Sep 10 19:31:32.392449 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:32.392412    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf76a064-79ad-4c7b-a000-7bf71dc9ddb7-tmp-dir\") pod \"noten-model-8d88a2-kserve-55fd49f99f-7fws2\" (UID: \"cf76a064-79ad-4c7b-a000-7bf71dc9ddb7\") " pod="ai-tenant-e2e-noten-8d88a2/noten-model-8d88a2-kserve-55fd49f99f-7fws2"
Sep 10 19:31:32.392449 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:32.392438    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldj52\" (UniqueName: \"kubernetes.io/projected/cf76a064-79ad-4c7b-a000-7bf71dc9ddb7-kube-api-access-ldj52\") pod \"noten-model-8d88a2-kserve-55fd49f99f-7fws2\" (UID: \"cf76a064-79ad-4c7b-a000-7bf71dc9ddb7\") " pod="ai-tenant-e2e-noten-8d88a2/noten-model-8d88a2-kserve-55fd49f99f-7fws2"
Sep 10 19:31:32.493713 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:32.493677    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf76a064-79ad-4c7b-a000-7bf71dc9ddb7-dshm\") pod \"noten-model-8d88a2-kserve-55fd49f99f-7fws2\" (UID: \"cf76a064-79ad-4c7b-a000-7bf71dc9ddb7\") " pod="ai-tenant-e2e-noten-8d88a2/noten-model-8d88a2-kserve-55fd49f99f-7fws2"
Sep 10 19:31:32.493923 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:32.493733    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf76a064-79ad-4c7b-a000-7bf71dc9ddb7-model-cache\") pod \"noten-model-8d88a2-kserve-55fd49f99f-7fws2\" (UID: \"cf76a064-79ad-4c7b-a000-7bf71dc9ddb7\") " pod="ai-tenant-e2e-noten-8d88a2/noten-model-8d88a2-kserve-55fd49f99f-7fws2"
Sep 10 19:31:32.493923 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:32.493768    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf76a064-79ad-4c7b-a000-7bf71dc9ddb7-tls-certs\") pod \"noten-model-8d88a2-kserve-55fd49f99f-7fws2\" (UID: \"cf76a064-79ad-4c7b-a000-7bf71dc9ddb7\") " pod="ai-tenant-e2e-noten-8d88a2/noten-model-8d88a2-kserve-55fd49f99f-7fws2"
Sep 10 19:31:32.493923 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:32.493819    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf76a064-79ad-4c7b-a000-7bf71dc9ddb7-home\") pod \"noten-model-8d88a2-kserve-55fd49f99f-7fws2\" (UID: \"cf76a064-79ad-4c7b-a000-7bf71dc9ddb7\") " pod="ai-tenant-e2e-noten-8d88a2/noten-model-8d88a2-kserve-55fd49f99f-7fws2"
Sep 10 19:31:32.493923 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:32.493850    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf76a064-79ad-4c7b-a000-7bf71dc9ddb7-tmp-dir\") pod \"noten-model-8d88a2-kserve-55fd49f99f-7fws2\" (UID: \"cf76a064-79ad-4c7b-a000-7bf71dc9ddb7\") " pod="ai-tenant-e2e-noten-8d88a2/noten-model-8d88a2-kserve-55fd49f99f-7fws2"
Sep 10 19:31:32.493923 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:32.493882    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldj52\" (UniqueName: \"kubernetes.io/projected/cf76a064-79ad-4c7b-a000-7bf71dc9ddb7-kube-api-access-ldj52\") pod \"noten-model-8d88a2-kserve-55fd49f99f-7fws2\" (UID: \"cf76a064-79ad-4c7b-a000-7bf71dc9ddb7\") " pod="ai-tenant-e2e-noten-8d88a2/noten-model-8d88a2-kserve-55fd49f99f-7fws2"
Sep 10 19:31:32.494193 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:32.494169    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf76a064-79ad-4c7b-a000-7bf71dc9ddb7-model-cache\") pod \"noten-model-8d88a2-kserve-55fd49f99f-7fws2\" (UID: \"cf76a064-79ad-4c7b-a000-7bf71dc9ddb7\") " pod="ai-tenant-e2e-noten-8d88a2/noten-model-8d88a2-kserve-55fd49f99f-7fws2"
Sep 10 19:31:32.494253 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:32.494178    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf76a064-79ad-4c7b-a000-7bf71dc9ddb7-home\") pod \"noten-model-8d88a2-kserve-55fd49f99f-7fws2\" (UID: \"cf76a064-79ad-4c7b-a000-7bf71dc9ddb7\") " pod="ai-tenant-e2e-noten-8d88a2/noten-model-8d88a2-kserve-55fd49f99f-7fws2"
Sep 10 19:31:32.494320 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:32.494296    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf76a064-79ad-4c7b-a000-7bf71dc9ddb7-tmp-dir\") pod \"noten-model-8d88a2-kserve-55fd49f99f-7fws2\" (UID: \"cf76a064-79ad-4c7b-a000-7bf71dc9ddb7\") " pod="ai-tenant-e2e-noten-8d88a2/noten-model-8d88a2-kserve-55fd49f99f-7fws2"
Sep 10 19:31:32.496254 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:32.496233    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf76a064-79ad-4c7b-a000-7bf71dc9ddb7-dshm\") pod \"noten-model-8d88a2-kserve-55fd49f99f-7fws2\" (UID: \"cf76a064-79ad-4c7b-a000-7bf71dc9ddb7\") " pod="ai-tenant-e2e-noten-8d88a2/noten-model-8d88a2-kserve-55fd49f99f-7fws2"
Sep 10 19:31:32.496410 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:32.496392    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf76a064-79ad-4c7b-a000-7bf71dc9ddb7-tls-certs\") pod \"noten-model-8d88a2-kserve-55fd49f99f-7fws2\" (UID: \"cf76a064-79ad-4c7b-a000-7bf71dc9ddb7\") " pod="ai-tenant-e2e-noten-8d88a2/noten-model-8d88a2-kserve-55fd49f99f-7fws2"
Sep 10 19:31:32.502680 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:32.502658    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldj52\" (UniqueName: \"kubernetes.io/projected/cf76a064-79ad-4c7b-a000-7bf71dc9ddb7-kube-api-access-ldj52\") pod \"noten-model-8d88a2-kserve-55fd49f99f-7fws2\" (UID: \"cf76a064-79ad-4c7b-a000-7bf71dc9ddb7\") " pod="ai-tenant-e2e-noten-8d88a2/noten-model-8d88a2-kserve-55fd49f99f-7fws2"
Sep 10 19:31:32.604704 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:32.604613    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-8d88a2/noten-model-8d88a2-kserve-55fd49f99f-7fws2"
Sep 10 19:31:32.878242 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:32.878213    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-8d88a2/noten-model-8d88a2-kserve-55fd49f99f-7fws2"]
Sep 10 19:31:32.879395 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:31:32.879369    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcf76a064_79ad_4c7b_a000_7bf71dc9ddb7.slice/crio-e0a97217fac06d729c1a095c9632acf53676278ca02f8af0b42f460b58f23ab7 WatchSource:0}: Error finding container e0a97217fac06d729c1a095c9632acf53676278ca02f8af0b42f460b58f23ab7: Status 404 returned error can't find the container with id e0a97217fac06d729c1a095c9632acf53676278ca02f8af0b42f460b58f23ab7
Sep 10 19:31:33.830511 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:33.830474    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-8d88a2/noten-model-8d88a2-kserve-55fd49f99f-7fws2" event={"ID":"cf76a064-79ad-4c7b-a000-7bf71dc9ddb7","Type":"ContainerStarted","Data":"88beb34d6d5f60cb5ceaa8a67fca5013e42207326a836dc11e066be218173edb"}
Sep 10 19:31:33.830930 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:33.830521    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-8d88a2/noten-model-8d88a2-kserve-55fd49f99f-7fws2" event={"ID":"cf76a064-79ad-4c7b-a000-7bf71dc9ddb7","Type":"ContainerStarted","Data":"e0a97217fac06d729c1a095c9632acf53676278ca02f8af0b42f460b58f23ab7"}
Sep 10 19:31:33.848120 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:33.848065    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-8d88a2/noten-model-8d88a2-kserve-55fd49f99f-7fws2" podStartSLOduration=1.8480483300000001 podStartE2EDuration="1.84804833s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:31:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:31:33.846917756 +0000 UTC m=+949.066766238" watchObservedRunningTime="2026-09-10 19:31:33.84804833 +0000 UTC m=+949.067896811"
Sep 10 19:31:42.605438 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:42.605402    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-8d88a2/noten-model-8d88a2-kserve-55fd49f99f-7fws2"
Sep 10 19:31:42.605877 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:42.605456    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-8d88a2/noten-model-8d88a2-kserve-55fd49f99f-7fws2"
Sep 10 19:31:42.618176 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:42.618149    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-8d88a2/noten-model-8d88a2-kserve-55fd49f99f-7fws2"
Sep 10 19:31:42.905195 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:31:42.905158    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-8d88a2/noten-model-8d88a2-kserve-55fd49f99f-7fws2"
Sep 10 19:32:04.168696 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:04.168655    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-8d88a2/noten-model-8d88a2-kserve-55fd49f99f-7fws2"]
Sep 10 19:32:04.278728 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:04.278692    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"]
Sep 10 19:32:04.278958 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:04.278926    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx" podUID="1a301d6a-8127-482e-b2f4-7780cbac60ed" containerName="istio-proxy" containerID="cri-o://143a50530fcc705ebf6fffaf44478654135328dcb3c25497a3c4f2d6bcc8731d" gracePeriod=30
Sep 10 19:32:09.529049 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:09.529024    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"
Sep 10 19:32:09.658699 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:09.658667    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a301d6a-8127-482e-b2f4-7780cbac60ed-credential-socket\") pod \"1a301d6a-8127-482e-b2f4-7780cbac60ed\" (UID: \"1a301d6a-8127-482e-b2f4-7780cbac60ed\") "
Sep 10 19:32:09.658923 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:09.658712    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a301d6a-8127-482e-b2f4-7780cbac60ed-workload-certs\") pod \"1a301d6a-8127-482e-b2f4-7780cbac60ed\" (UID: \"1a301d6a-8127-482e-b2f4-7780cbac60ed\") "
Sep 10 19:32:09.658923 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:09.658788    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a301d6a-8127-482e-b2f4-7780cbac60ed-istio-podinfo\") pod \"1a301d6a-8127-482e-b2f4-7780cbac60ed\" (UID: \"1a301d6a-8127-482e-b2f4-7780cbac60ed\") "
Sep 10 19:32:09.658923 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:09.658814    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mlqnm\" (UniqueName: \"kubernetes.io/projected/1a301d6a-8127-482e-b2f4-7780cbac60ed-kube-api-access-mlqnm\") pod \"1a301d6a-8127-482e-b2f4-7780cbac60ed\" (UID: \"1a301d6a-8127-482e-b2f4-7780cbac60ed\") "
Sep 10 19:32:09.659093 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:09.658948    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a301d6a-8127-482e-b2f4-7780cbac60ed-istio-data\") pod \"1a301d6a-8127-482e-b2f4-7780cbac60ed\" (UID: \"1a301d6a-8127-482e-b2f4-7780cbac60ed\") "
Sep 10 19:32:09.659093 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:09.658976    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a301d6a-8127-482e-b2f4-7780cbac60ed-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1a301d6a-8127-482e-b2f4-7780cbac60ed" (UID: "1a301d6a-8127-482e-b2f4-7780cbac60ed"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:32:09.659093 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:09.659053    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a301d6a-8127-482e-b2f4-7780cbac60ed-istiod-ca-cert\") pod \"1a301d6a-8127-482e-b2f4-7780cbac60ed\" (UID: \"1a301d6a-8127-482e-b2f4-7780cbac60ed\") "
Sep 10 19:32:09.659093 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:09.659087    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a301d6a-8127-482e-b2f4-7780cbac60ed-workload-socket\") pod \"1a301d6a-8127-482e-b2f4-7780cbac60ed\" (UID: \"1a301d6a-8127-482e-b2f4-7780cbac60ed\") "
Sep 10 19:32:09.659314 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:09.659093    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a301d6a-8127-482e-b2f4-7780cbac60ed-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1a301d6a-8127-482e-b2f4-7780cbac60ed" (UID: "1a301d6a-8127-482e-b2f4-7780cbac60ed"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:32:09.659314 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:09.659105    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a301d6a-8127-482e-b2f4-7780cbac60ed-istio-envoy\") pod \"1a301d6a-8127-482e-b2f4-7780cbac60ed\" (UID: \"1a301d6a-8127-482e-b2f4-7780cbac60ed\") "
Sep 10 19:32:09.659314 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:09.659179    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a301d6a-8127-482e-b2f4-7780cbac60ed-istio-token\") pod \"1a301d6a-8127-482e-b2f4-7780cbac60ed\" (UID: \"1a301d6a-8127-482e-b2f4-7780cbac60ed\") "
Sep 10 19:32:09.659314 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:09.659214    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a301d6a-8127-482e-b2f4-7780cbac60ed-istio-data" (OuterVolumeSpecName: "istio-data") pod "1a301d6a-8127-482e-b2f4-7780cbac60ed" (UID: "1a301d6a-8127-482e-b2f4-7780cbac60ed"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:32:09.659635 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:09.659333    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a301d6a-8127-482e-b2f4-7780cbac60ed-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1a301d6a-8127-482e-b2f4-7780cbac60ed" (UID: "1a301d6a-8127-482e-b2f4-7780cbac60ed"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:32:09.659635 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:09.659408    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a301d6a-8127-482e-b2f4-7780cbac60ed-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1a301d6a-8127-482e-b2f4-7780cbac60ed" (UID: "1a301d6a-8127-482e-b2f4-7780cbac60ed"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:32:09.659837 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:09.659811    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a301d6a-8127-482e-b2f4-7780cbac60ed-credential-socket\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:32:09.659904 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:09.659835    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a301d6a-8127-482e-b2f4-7780cbac60ed-workload-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:32:09.659904 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:09.659852    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a301d6a-8127-482e-b2f4-7780cbac60ed-istio-data\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:32:09.659904 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:09.659866    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a301d6a-8127-482e-b2f4-7780cbac60ed-istiod-ca-cert\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:32:09.659904 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:09.659879    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a301d6a-8127-482e-b2f4-7780cbac60ed-workload-socket\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:32:09.661270 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:09.661236    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a301d6a-8127-482e-b2f4-7780cbac60ed-kube-api-access-mlqnm" (OuterVolumeSpecName: "kube-api-access-mlqnm") pod "1a301d6a-8127-482e-b2f4-7780cbac60ed" (UID: "1a301d6a-8127-482e-b2f4-7780cbac60ed"). InnerVolumeSpecName "kube-api-access-mlqnm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:32:09.661270 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:09.661257    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a301d6a-8127-482e-b2f4-7780cbac60ed-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1a301d6a-8127-482e-b2f4-7780cbac60ed" (UID: "1a301d6a-8127-482e-b2f4-7780cbac60ed"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:32:09.661270 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:09.661236    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1a301d6a-8127-482e-b2f4-7780cbac60ed-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1a301d6a-8127-482e-b2f4-7780cbac60ed" (UID: "1a301d6a-8127-482e-b2f4-7780cbac60ed"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:32:09.661441 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:09.661317    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a301d6a-8127-482e-b2f4-7780cbac60ed-istio-token" (OuterVolumeSpecName: "istio-token") pod "1a301d6a-8127-482e-b2f4-7780cbac60ed" (UID: "1a301d6a-8127-482e-b2f4-7780cbac60ed"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:32:09.760381 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:09.760343    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a301d6a-8127-482e-b2f4-7780cbac60ed-istio-podinfo\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:32:09.760381 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:09.760373    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mlqnm\" (UniqueName: \"kubernetes.io/projected/1a301d6a-8127-482e-b2f4-7780cbac60ed-kube-api-access-mlqnm\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:32:09.760381 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:09.760383    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a301d6a-8127-482e-b2f4-7780cbac60ed-istio-envoy\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:32:09.760381 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:09.760391    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a301d6a-8127-482e-b2f4-7780cbac60ed-istio-token\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:32:10.051062 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:10.050966    2579 generic.go:356] "Generic (PLEG): container finished" podID="1a301d6a-8127-482e-b2f4-7780cbac60ed" containerID="143a50530fcc705ebf6fffaf44478654135328dcb3c25497a3c4f2d6bcc8731d" exitCode=0
Sep 10 19:32:10.051062 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:10.051029    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"
Sep 10 19:32:10.051062 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:10.051051    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx" event={"ID":"1a301d6a-8127-482e-b2f4-7780cbac60ed","Type":"ContainerDied","Data":"143a50530fcc705ebf6fffaf44478654135328dcb3c25497a3c4f2d6bcc8731d"}
Sep 10 19:32:10.051355 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:10.051108    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx" event={"ID":"1a301d6a-8127-482e-b2f4-7780cbac60ed","Type":"ContainerDied","Data":"1edb8c92ad9932a2ba4bb6cdb969150d79ce127bda6cf1e7c450ae7075b699f4"}
Sep 10 19:32:10.051355 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:10.051131    2579 scope.go:117] "RemoveContainer" containerID="143a50530fcc705ebf6fffaf44478654135328dcb3c25497a3c4f2d6bcc8731d"
Sep 10 19:32:10.069884 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:10.069861    2579 scope.go:117] "RemoveContainer" containerID="143a50530fcc705ebf6fffaf44478654135328dcb3c25497a3c4f2d6bcc8731d"
Sep 10 19:32:10.070202 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:32:10.070185    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"143a50530fcc705ebf6fffaf44478654135328dcb3c25497a3c4f2d6bcc8731d\": container with ID starting with 143a50530fcc705ebf6fffaf44478654135328dcb3c25497a3c4f2d6bcc8731d not found: ID does not exist" containerID="143a50530fcc705ebf6fffaf44478654135328dcb3c25497a3c4f2d6bcc8731d"
Sep 10 19:32:10.070257 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:10.070210    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"143a50530fcc705ebf6fffaf44478654135328dcb3c25497a3c4f2d6bcc8731d"} err="failed to get container status \"143a50530fcc705ebf6fffaf44478654135328dcb3c25497a3c4f2d6bcc8731d\": rpc error: code = NotFound desc = could not find container \"143a50530fcc705ebf6fffaf44478654135328dcb3c25497a3c4f2d6bcc8731d\": container with ID starting with 143a50530fcc705ebf6fffaf44478654135328dcb3c25497a3c4f2d6bcc8731d not found: ID does not exist"
Sep 10 19:32:10.082528 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:10.082499    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"]
Sep 10 19:32:10.088346 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:10.088316    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx"]
Sep 10 19:32:11.240492 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:11.240449    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a301d6a-8127-482e-b2f4-7780cbac60ed" path="/var/lib/kubelet/pods/1a301d6a-8127-482e-b2f4-7780cbac60ed/volumes"
Sep 10 19:32:16.075174 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:16.075142    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29"]
Sep 10 19:32:16.075632 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:16.075403    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29" podUID="fb483fb7-6eb9-4b81-ad36-b7e75ed333f0" containerName="maas-api" containerID="cri-o://3c23150cd6410b1a113229c833e653487af550e0288ece18f3a9741f7dd1bd87" gracePeriod=30
Sep 10 19:32:16.332677 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:16.332613    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29"
Sep 10 19:32:16.425614 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:16.425581    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s5pqv\" (UniqueName: \"kubernetes.io/projected/fb483fb7-6eb9-4b81-ad36-b7e75ed333f0-kube-api-access-s5pqv\") pod \"fb483fb7-6eb9-4b81-ad36-b7e75ed333f0\" (UID: \"fb483fb7-6eb9-4b81-ad36-b7e75ed333f0\") "
Sep 10 19:32:16.425829 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:16.425672    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb483fb7-6eb9-4b81-ad36-b7e75ed333f0-maas-api-tls\") pod \"fb483fb7-6eb9-4b81-ad36-b7e75ed333f0\" (UID: \"fb483fb7-6eb9-4b81-ad36-b7e75ed333f0\") "
Sep 10 19:32:16.425829 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:16.425703    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb483fb7-6eb9-4b81-ad36-b7e75ed333f0-metrics-certs\") pod \"fb483fb7-6eb9-4b81-ad36-b7e75ed333f0\" (UID: \"fb483fb7-6eb9-4b81-ad36-b7e75ed333f0\") "
Sep 10 19:32:16.425829 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:16.425764    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb483fb7-6eb9-4b81-ad36-b7e75ed333f0-openshift-service-ca\") pod \"fb483fb7-6eb9-4b81-ad36-b7e75ed333f0\" (UID: \"fb483fb7-6eb9-4b81-ad36-b7e75ed333f0\") "
Sep 10 19:32:16.426213 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:16.426180    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb483fb7-6eb9-4b81-ad36-b7e75ed333f0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fb483fb7-6eb9-4b81-ad36-b7e75ed333f0" (UID: "fb483fb7-6eb9-4b81-ad36-b7e75ed333f0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:32:16.427933 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:16.427909    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb483fb7-6eb9-4b81-ad36-b7e75ed333f0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "fb483fb7-6eb9-4b81-ad36-b7e75ed333f0" (UID: "fb483fb7-6eb9-4b81-ad36-b7e75ed333f0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:32:16.428491 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:16.428473    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb483fb7-6eb9-4b81-ad36-b7e75ed333f0-kube-api-access-s5pqv" (OuterVolumeSpecName: "kube-api-access-s5pqv") pod "fb483fb7-6eb9-4b81-ad36-b7e75ed333f0" (UID: "fb483fb7-6eb9-4b81-ad36-b7e75ed333f0"). InnerVolumeSpecName "kube-api-access-s5pqv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:32:16.428491 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:16.428478    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb483fb7-6eb9-4b81-ad36-b7e75ed333f0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fb483fb7-6eb9-4b81-ad36-b7e75ed333f0" (UID: "fb483fb7-6eb9-4b81-ad36-b7e75ed333f0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:32:16.526704 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:16.526666    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5pqv\" (UniqueName: \"kubernetes.io/projected/fb483fb7-6eb9-4b81-ad36-b7e75ed333f0-kube-api-access-s5pqv\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:32:16.526704 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:16.526702    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb483fb7-6eb9-4b81-ad36-b7e75ed333f0-maas-api-tls\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:32:16.526950 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:16.526717    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb483fb7-6eb9-4b81-ad36-b7e75ed333f0-metrics-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:32:16.526950 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:16.526725    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb483fb7-6eb9-4b81-ad36-b7e75ed333f0-openshift-service-ca\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:32:17.098401 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:17.098363    2579 generic.go:356] "Generic (PLEG): container finished" podID="fb483fb7-6eb9-4b81-ad36-b7e75ed333f0" containerID="3c23150cd6410b1a113229c833e653487af550e0288ece18f3a9741f7dd1bd87" exitCode=0
Sep 10 19:32:17.098809 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:17.098423    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29"
Sep 10 19:32:17.098809 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:17.098442    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29" event={"ID":"fb483fb7-6eb9-4b81-ad36-b7e75ed333f0","Type":"ContainerDied","Data":"3c23150cd6410b1a113229c833e653487af550e0288ece18f3a9741f7dd1bd87"}
Sep 10 19:32:17.098809 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:17.098530    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29" event={"ID":"fb483fb7-6eb9-4b81-ad36-b7e75ed333f0","Type":"ContainerDied","Data":"17e93b740d2c18c5760a22b30e9264d848a241dcd6cfd91236c38b1091947eff"}
Sep 10 19:32:17.098809 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:17.098558    2579 scope.go:117] "RemoveContainer" containerID="3c23150cd6410b1a113229c833e653487af550e0288ece18f3a9741f7dd1bd87"
Sep 10 19:32:17.108676 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:17.108657    2579 scope.go:117] "RemoveContainer" containerID="3c23150cd6410b1a113229c833e653487af550e0288ece18f3a9741f7dd1bd87"
Sep 10 19:32:17.108950 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:32:17.108933    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c23150cd6410b1a113229c833e653487af550e0288ece18f3a9741f7dd1bd87\": container with ID starting with 3c23150cd6410b1a113229c833e653487af550e0288ece18f3a9741f7dd1bd87 not found: ID does not exist" containerID="3c23150cd6410b1a113229c833e653487af550e0288ece18f3a9741f7dd1bd87"
Sep 10 19:32:17.109018 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:17.108959    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c23150cd6410b1a113229c833e653487af550e0288ece18f3a9741f7dd1bd87"} err="failed to get container status \"3c23150cd6410b1a113229c833e653487af550e0288ece18f3a9741f7dd1bd87\": rpc error: code = NotFound desc = could not find container \"3c23150cd6410b1a113229c833e653487af550e0288ece18f3a9741f7dd1bd87\": container with ID starting with 3c23150cd6410b1a113229c833e653487af550e0288ece18f3a9741f7dd1bd87 not found: ID does not exist"
Sep 10 19:32:17.121232 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:17.121207    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29"]
Sep 10 19:32:17.125957 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:17.125940    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8300b9-76d78dfb8f-lsc29"]
Sep 10 19:32:17.239894 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:17.239858    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb483fb7-6eb9-4b81-ad36-b7e75ed333f0" path="/var/lib/kubelet/pods/fb483fb7-6eb9-4b81-ad36-b7e75ed333f0/volumes"
Sep 10 19:32:19.190114 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:19.190068    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-8d88a2/noten-model-8d88a2-kserve-55fd49f99f-7fws2" podUID="cf76a064-79ad-4c7b-a000-7bf71dc9ddb7" containerName="main" containerID="cri-o://88beb34d6d5f60cb5ceaa8a67fca5013e42207326a836dc11e066be218173edb" gracePeriod=45
Sep 10 19:32:19.447967 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:19.447911    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-8d88a2/noten-model-8d88a2-kserve-55fd49f99f-7fws2"
Sep 10 19:32:19.556388 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:19.556358    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ldj52\" (UniqueName: \"kubernetes.io/projected/cf76a064-79ad-4c7b-a000-7bf71dc9ddb7-kube-api-access-ldj52\") pod \"cf76a064-79ad-4c7b-a000-7bf71dc9ddb7\" (UID: \"cf76a064-79ad-4c7b-a000-7bf71dc9ddb7\") "
Sep 10 19:32:19.556560 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:19.556406    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf76a064-79ad-4c7b-a000-7bf71dc9ddb7-tmp-dir\") pod \"cf76a064-79ad-4c7b-a000-7bf71dc9ddb7\" (UID: \"cf76a064-79ad-4c7b-a000-7bf71dc9ddb7\") "
Sep 10 19:32:19.556560 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:19.556468    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf76a064-79ad-4c7b-a000-7bf71dc9ddb7-tls-certs\") pod \"cf76a064-79ad-4c7b-a000-7bf71dc9ddb7\" (UID: \"cf76a064-79ad-4c7b-a000-7bf71dc9ddb7\") "
Sep 10 19:32:19.556560 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:19.556491    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf76a064-79ad-4c7b-a000-7bf71dc9ddb7-dshm\") pod \"cf76a064-79ad-4c7b-a000-7bf71dc9ddb7\" (UID: \"cf76a064-79ad-4c7b-a000-7bf71dc9ddb7\") "
Sep 10 19:32:19.556560 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:19.556537    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf76a064-79ad-4c7b-a000-7bf71dc9ddb7-home\") pod \"cf76a064-79ad-4c7b-a000-7bf71dc9ddb7\" (UID: \"cf76a064-79ad-4c7b-a000-7bf71dc9ddb7\") "
Sep 10 19:32:19.556779 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:19.556634    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf76a064-79ad-4c7b-a000-7bf71dc9ddb7-model-cache\") pod \"cf76a064-79ad-4c7b-a000-7bf71dc9ddb7\" (UID: \"cf76a064-79ad-4c7b-a000-7bf71dc9ddb7\") "
Sep 10 19:32:19.556854 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:19.556784    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf76a064-79ad-4c7b-a000-7bf71dc9ddb7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cf76a064-79ad-4c7b-a000-7bf71dc9ddb7" (UID: "cf76a064-79ad-4c7b-a000-7bf71dc9ddb7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:32:19.556967 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:19.556916    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf76a064-79ad-4c7b-a000-7bf71dc9ddb7-home" (OuterVolumeSpecName: "home") pod "cf76a064-79ad-4c7b-a000-7bf71dc9ddb7" (UID: "cf76a064-79ad-4c7b-a000-7bf71dc9ddb7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:32:19.556967 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:19.556943    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf76a064-79ad-4c7b-a000-7bf71dc9ddb7-model-cache" (OuterVolumeSpecName: "model-cache") pod "cf76a064-79ad-4c7b-a000-7bf71dc9ddb7" (UID: "cf76a064-79ad-4c7b-a000-7bf71dc9ddb7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:32:19.557139 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:19.557080    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf76a064-79ad-4c7b-a000-7bf71dc9ddb7-home\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:32:19.557139 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:19.557101    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf76a064-79ad-4c7b-a000-7bf71dc9ddb7-model-cache\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:32:19.557139 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:19.557118    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf76a064-79ad-4c7b-a000-7bf71dc9ddb7-tmp-dir\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:32:19.558819 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:19.558796    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf76a064-79ad-4c7b-a000-7bf71dc9ddb7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cf76a064-79ad-4c7b-a000-7bf71dc9ddb7" (UID: "cf76a064-79ad-4c7b-a000-7bf71dc9ddb7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:32:19.558819 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:19.558802    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf76a064-79ad-4c7b-a000-7bf71dc9ddb7-dshm" (OuterVolumeSpecName: "dshm") pod "cf76a064-79ad-4c7b-a000-7bf71dc9ddb7" (UID: "cf76a064-79ad-4c7b-a000-7bf71dc9ddb7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:32:19.558933 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:19.558918    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf76a064-79ad-4c7b-a000-7bf71dc9ddb7-kube-api-access-ldj52" (OuterVolumeSpecName: "kube-api-access-ldj52") pod "cf76a064-79ad-4c7b-a000-7bf71dc9ddb7" (UID: "cf76a064-79ad-4c7b-a000-7bf71dc9ddb7"). InnerVolumeSpecName "kube-api-access-ldj52". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:32:19.657984 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:19.657954    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf76a064-79ad-4c7b-a000-7bf71dc9ddb7-tls-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:32:19.657984 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:19.657977    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf76a064-79ad-4c7b-a000-7bf71dc9ddb7-dshm\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:32:19.657984 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:19.657986    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldj52\" (UniqueName: \"kubernetes.io/projected/cf76a064-79ad-4c7b-a000-7bf71dc9ddb7-kube-api-access-ldj52\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:32:20.121183 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:20.121151    2579 generic.go:356] "Generic (PLEG): container finished" podID="cf76a064-79ad-4c7b-a000-7bf71dc9ddb7" containerID="88beb34d6d5f60cb5ceaa8a67fca5013e42207326a836dc11e066be218173edb" exitCode=0
Sep 10 19:32:20.121395 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:20.121216    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-8d88a2/noten-model-8d88a2-kserve-55fd49f99f-7fws2"
Sep 10 19:32:20.121395 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:20.121238    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-8d88a2/noten-model-8d88a2-kserve-55fd49f99f-7fws2" event={"ID":"cf76a064-79ad-4c7b-a000-7bf71dc9ddb7","Type":"ContainerDied","Data":"88beb34d6d5f60cb5ceaa8a67fca5013e42207326a836dc11e066be218173edb"}
Sep 10 19:32:20.121395 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:20.121319    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-8d88a2/noten-model-8d88a2-kserve-55fd49f99f-7fws2" event={"ID":"cf76a064-79ad-4c7b-a000-7bf71dc9ddb7","Type":"ContainerDied","Data":"e0a97217fac06d729c1a095c9632acf53676278ca02f8af0b42f460b58f23ab7"}
Sep 10 19:32:20.121395 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:20.121354    2579 scope.go:117] "RemoveContainer" containerID="88beb34d6d5f60cb5ceaa8a67fca5013e42207326a836dc11e066be218173edb"
Sep 10 19:32:20.131618 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:20.131588    2579 scope.go:117] "RemoveContainer" containerID="88beb34d6d5f60cb5ceaa8a67fca5013e42207326a836dc11e066be218173edb"
Sep 10 19:32:20.131951 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:32:20.131932    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88beb34d6d5f60cb5ceaa8a67fca5013e42207326a836dc11e066be218173edb\": container with ID starting with 88beb34d6d5f60cb5ceaa8a67fca5013e42207326a836dc11e066be218173edb not found: ID does not exist" containerID="88beb34d6d5f60cb5ceaa8a67fca5013e42207326a836dc11e066be218173edb"
Sep 10 19:32:20.132020 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:20.131960    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88beb34d6d5f60cb5ceaa8a67fca5013e42207326a836dc11e066be218173edb"} err="failed to get container status \"88beb34d6d5f60cb5ceaa8a67fca5013e42207326a836dc11e066be218173edb\": rpc error: code = NotFound desc = could not find container \"88beb34d6d5f60cb5ceaa8a67fca5013e42207326a836dc11e066be218173edb\": container with ID starting with 88beb34d6d5f60cb5ceaa8a67fca5013e42207326a836dc11e066be218173edb not found: ID does not exist"
Sep 10 19:32:20.147419 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:20.147391    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-8d88a2/noten-model-8d88a2-kserve-55fd49f99f-7fws2"]
Sep 10 19:32:20.152133 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:20.152114    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-8d88a2/noten-model-8d88a2-kserve-55fd49f99f-7fws2"]
Sep 10 19:32:20.220159 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:20.220131    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9d2e2715-784b7b4f88-97274"]
Sep 10 19:32:20.220856 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:20.220832    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb483fb7-6eb9-4b81-ad36-b7e75ed333f0" containerName="maas-api"
Sep 10 19:32:20.220856 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:20.220855    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb483fb7-6eb9-4b81-ad36-b7e75ed333f0" containerName="maas-api"
Sep 10 19:32:20.220856 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:20.220871    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a301d6a-8127-482e-b2f4-7780cbac60ed" containerName="istio-proxy"
Sep 10 19:32:20.221183 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:20.220882    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a301d6a-8127-482e-b2f4-7780cbac60ed" containerName="istio-proxy"
Sep 10 19:32:20.221183 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:20.220924    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf76a064-79ad-4c7b-a000-7bf71dc9ddb7" containerName="main"
Sep 10 19:32:20.221183 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:20.220932    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf76a064-79ad-4c7b-a000-7bf71dc9ddb7" containerName="main"
Sep 10 19:32:20.221183 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:20.221041    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf76a064-79ad-4c7b-a000-7bf71dc9ddb7" containerName="main"
Sep 10 19:32:20.221183 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:20.221052    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb483fb7-6eb9-4b81-ad36-b7e75ed333f0" containerName="maas-api"
Sep 10 19:32:20.221183 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:20.221069    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a301d6a-8127-482e-b2f4-7780cbac60ed" containerName="istio-proxy"
Sep 10 19:32:20.231254 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:20.227321    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d2e2715-784b7b4f88-97274"
Sep 10 19:32:20.231254 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:20.229914    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-9d2e2715-serving-cert\""
Sep 10 19:32:20.238459 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:20.238287    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9d2e2715-784b7b4f88-97274"]
Sep 10 19:32:20.366469 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:20.366427    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d748b0d-ab6e-44aa-b87b-cdd477bc4047-metrics-certs\") pod \"maas-api-e2e-mt-9d2e2715-784b7b4f88-97274\" (UID: \"0d748b0d-ab6e-44aa-b87b-cdd477bc4047\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d2e2715-784b7b4f88-97274"
Sep 10 19:32:20.366632 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:20.366510    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tt6b\" (UniqueName: \"kubernetes.io/projected/0d748b0d-ab6e-44aa-b87b-cdd477bc4047-kube-api-access-7tt6b\") pod \"maas-api-e2e-mt-9d2e2715-784b7b4f88-97274\" (UID: \"0d748b0d-ab6e-44aa-b87b-cdd477bc4047\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d2e2715-784b7b4f88-97274"
Sep 10 19:32:20.366632 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:20.366541    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d748b0d-ab6e-44aa-b87b-cdd477bc4047-maas-api-tls\") pod \"maas-api-e2e-mt-9d2e2715-784b7b4f88-97274\" (UID: \"0d748b0d-ab6e-44aa-b87b-cdd477bc4047\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d2e2715-784b7b4f88-97274"
Sep 10 19:32:20.366632 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:20.366580    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d748b0d-ab6e-44aa-b87b-cdd477bc4047-openshift-service-ca\") pod \"maas-api-e2e-mt-9d2e2715-784b7b4f88-97274\" (UID: \"0d748b0d-ab6e-44aa-b87b-cdd477bc4047\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d2e2715-784b7b4f88-97274"
Sep 10 19:32:20.467369 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:20.467330    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tt6b\" (UniqueName: \"kubernetes.io/projected/0d748b0d-ab6e-44aa-b87b-cdd477bc4047-kube-api-access-7tt6b\") pod \"maas-api-e2e-mt-9d2e2715-784b7b4f88-97274\" (UID: \"0d748b0d-ab6e-44aa-b87b-cdd477bc4047\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d2e2715-784b7b4f88-97274"
Sep 10 19:32:20.467574 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:20.467380    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d748b0d-ab6e-44aa-b87b-cdd477bc4047-maas-api-tls\") pod \"maas-api-e2e-mt-9d2e2715-784b7b4f88-97274\" (UID: \"0d748b0d-ab6e-44aa-b87b-cdd477bc4047\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d2e2715-784b7b4f88-97274"
Sep 10 19:32:20.467574 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:20.467428    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d748b0d-ab6e-44aa-b87b-cdd477bc4047-openshift-service-ca\") pod \"maas-api-e2e-mt-9d2e2715-784b7b4f88-97274\" (UID: \"0d748b0d-ab6e-44aa-b87b-cdd477bc4047\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d2e2715-784b7b4f88-97274"
Sep 10 19:32:20.467574 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:32:20.467567    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-9d2e2715-serving-cert: secret "maas-api-e2e-mt-9d2e2715-serving-cert" not found
Sep 10 19:32:20.467794 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:20.467590    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d748b0d-ab6e-44aa-b87b-cdd477bc4047-metrics-certs\") pod \"maas-api-e2e-mt-9d2e2715-784b7b4f88-97274\" (UID: \"0d748b0d-ab6e-44aa-b87b-cdd477bc4047\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d2e2715-784b7b4f88-97274"
Sep 10 19:32:20.467794 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:32:20.467651    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d748b0d-ab6e-44aa-b87b-cdd477bc4047-maas-api-tls podName:0d748b0d-ab6e-44aa-b87b-cdd477bc4047 nodeName:}" failed. No retries permitted until 2026-09-10 19:32:20.967627211 +0000 UTC m=+996.187475676 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0d748b0d-ab6e-44aa-b87b-cdd477bc4047-maas-api-tls") pod "maas-api-e2e-mt-9d2e2715-784b7b4f88-97274" (UID: "0d748b0d-ab6e-44aa-b87b-cdd477bc4047") : secret "maas-api-e2e-mt-9d2e2715-serving-cert" not found
Sep 10 19:32:20.468199 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:20.468175    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d748b0d-ab6e-44aa-b87b-cdd477bc4047-openshift-service-ca\") pod \"maas-api-e2e-mt-9d2e2715-784b7b4f88-97274\" (UID: \"0d748b0d-ab6e-44aa-b87b-cdd477bc4047\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d2e2715-784b7b4f88-97274"
Sep 10 19:32:20.470184 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:20.470160    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d748b0d-ab6e-44aa-b87b-cdd477bc4047-metrics-certs\") pod \"maas-api-e2e-mt-9d2e2715-784b7b4f88-97274\" (UID: \"0d748b0d-ab6e-44aa-b87b-cdd477bc4047\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d2e2715-784b7b4f88-97274"
Sep 10 19:32:20.475777 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:20.475741    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tt6b\" (UniqueName: \"kubernetes.io/projected/0d748b0d-ab6e-44aa-b87b-cdd477bc4047-kube-api-access-7tt6b\") pod \"maas-api-e2e-mt-9d2e2715-784b7b4f88-97274\" (UID: \"0d748b0d-ab6e-44aa-b87b-cdd477bc4047\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d2e2715-784b7b4f88-97274"
Sep 10 19:32:20.972117 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:20.972081    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d748b0d-ab6e-44aa-b87b-cdd477bc4047-maas-api-tls\") pod \"maas-api-e2e-mt-9d2e2715-784b7b4f88-97274\" (UID: \"0d748b0d-ab6e-44aa-b87b-cdd477bc4047\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d2e2715-784b7b4f88-97274"
Sep 10 19:32:20.974527 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:20.974506    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d748b0d-ab6e-44aa-b87b-cdd477bc4047-maas-api-tls\") pod \"maas-api-e2e-mt-9d2e2715-784b7b4f88-97274\" (UID: \"0d748b0d-ab6e-44aa-b87b-cdd477bc4047\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d2e2715-784b7b4f88-97274"
Sep 10 19:32:21.143982 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:21.143949    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d2e2715-784b7b4f88-97274"
Sep 10 19:32:21.240369 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:21.240299    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf76a064-79ad-4c7b-a000-7bf71dc9ddb7" path="/var/lib/kubelet/pods/cf76a064-79ad-4c7b-a000-7bf71dc9ddb7/volumes"
Sep 10 19:32:21.420338 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:21.420314    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9d2e2715-784b7b4f88-97274"]
Sep 10 19:32:21.421672 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:32:21.421647    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d748b0d_ab6e_44aa_b87b_cdd477bc4047.slice/crio-4a99532631415eb21f98afb729b4c71a66d33a9b531d8958c6f5540a8b8c18c9 WatchSource:0}: Error finding container 4a99532631415eb21f98afb729b4c71a66d33a9b531d8958c6f5540a8b8c18c9: Status 404 returned error can't find the container with id 4a99532631415eb21f98afb729b4c71a66d33a9b531d8958c6f5540a8b8c18c9
Sep 10 19:32:22.138108 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:22.138068    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d2e2715-784b7b4f88-97274" event={"ID":"0d748b0d-ab6e-44aa-b87b-cdd477bc4047","Type":"ContainerStarted","Data":"25f4eff58a6743ea05f34e57cc328e86931d90a9ba3e9760d2359aecffb7ad98"}
Sep 10 19:32:22.138295 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:22.138136    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d2e2715-784b7b4f88-97274" event={"ID":"0d748b0d-ab6e-44aa-b87b-cdd477bc4047","Type":"ContainerStarted","Data":"4a99532631415eb21f98afb729b4c71a66d33a9b531d8958c6f5540a8b8c18c9"}
Sep 10 19:32:22.138295 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:22.138257    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d2e2715-784b7b4f88-97274"
Sep 10 19:32:22.158474 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:22.158425    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d2e2715-784b7b4f88-97274" podStartSLOduration=1.791730815 podStartE2EDuration="2.158410614s" totalImagesPullingTime="366.679799ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:32:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:32:22.154960883 +0000 UTC m=+997.374809363" watchObservedRunningTime="2026-09-10 19:32:22.158410614 +0000 UTC m=+997.378259094"
Sep 10 19:32:23.300365 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:23.300336    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"]
Sep 10 19:32:23.300803 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:23.300595    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc" podUID="6e405b89-cc1d-4169-a326-070959a02f66" containerName="istio-proxy" containerID="cri-o://eb50015b36c5da678c80829aaa8ee50d31a3e3cde288b7bca5d0fc7b23baa3fc" gracePeriod=30
Sep 10 19:32:25.188719 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:25.188680    2579 patch_prober.go:28] interesting pod/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 19:32:25.189140 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:25.188738    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc" podUID="6e405b89-cc1d-4169-a326-070959a02f66" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 19:32:28.149626 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:28.149589    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d2e2715-784b7b4f88-97274"
Sep 10 19:32:28.557115 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:28.557089    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:32:28.652891 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:28.652862    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e405b89-cc1d-4169-a326-070959a02f66-workload-certs\") pod \"6e405b89-cc1d-4169-a326-070959a02f66\" (UID: \"6e405b89-cc1d-4169-a326-070959a02f66\") "
Sep 10 19:32:28.653061 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:28.652916    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r6zss\" (UniqueName: \"kubernetes.io/projected/6e405b89-cc1d-4169-a326-070959a02f66-kube-api-access-r6zss\") pod \"6e405b89-cc1d-4169-a326-070959a02f66\" (UID: \"6e405b89-cc1d-4169-a326-070959a02f66\") "
Sep 10 19:32:28.653061 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:28.652941    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e405b89-cc1d-4169-a326-070959a02f66-credential-socket\") pod \"6e405b89-cc1d-4169-a326-070959a02f66\" (UID: \"6e405b89-cc1d-4169-a326-070959a02f66\") "
Sep 10 19:32:28.653061 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:28.652964    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e405b89-cc1d-4169-a326-070959a02f66-workload-socket\") pod \"6e405b89-cc1d-4169-a326-070959a02f66\" (UID: \"6e405b89-cc1d-4169-a326-070959a02f66\") "
Sep 10 19:32:28.653061 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:28.653014    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e405b89-cc1d-4169-a326-070959a02f66-istio-data\") pod \"6e405b89-cc1d-4169-a326-070959a02f66\" (UID: \"6e405b89-cc1d-4169-a326-070959a02f66\") "
Sep 10 19:32:28.653277 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:28.653072    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e405b89-cc1d-4169-a326-070959a02f66-istio-podinfo\") pod \"6e405b89-cc1d-4169-a326-070959a02f66\" (UID: \"6e405b89-cc1d-4169-a326-070959a02f66\") "
Sep 10 19:32:28.653277 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:28.653097    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e405b89-cc1d-4169-a326-070959a02f66-istio-token\") pod \"6e405b89-cc1d-4169-a326-070959a02f66\" (UID: \"6e405b89-cc1d-4169-a326-070959a02f66\") "
Sep 10 19:32:28.653277 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:28.653153    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e405b89-cc1d-4169-a326-070959a02f66-istio-envoy\") pod \"6e405b89-cc1d-4169-a326-070959a02f66\" (UID: \"6e405b89-cc1d-4169-a326-070959a02f66\") "
Sep 10 19:32:28.653277 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:28.653154    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e405b89-cc1d-4169-a326-070959a02f66-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6e405b89-cc1d-4169-a326-070959a02f66" (UID: "6e405b89-cc1d-4169-a326-070959a02f66"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:32:28.653277 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:28.653199    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e405b89-cc1d-4169-a326-070959a02f66-istiod-ca-cert\") pod \"6e405b89-cc1d-4169-a326-070959a02f66\" (UID: \"6e405b89-cc1d-4169-a326-070959a02f66\") "
Sep 10 19:32:28.653552 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:28.653278    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e405b89-cc1d-4169-a326-070959a02f66-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6e405b89-cc1d-4169-a326-070959a02f66" (UID: "6e405b89-cc1d-4169-a326-070959a02f66"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:32:28.653552 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:28.653301    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e405b89-cc1d-4169-a326-070959a02f66-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6e405b89-cc1d-4169-a326-070959a02f66" (UID: "6e405b89-cc1d-4169-a326-070959a02f66"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:32:28.653658 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:28.653558    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e405b89-cc1d-4169-a326-070959a02f66-workload-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:32:28.653658 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:28.653566    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e405b89-cc1d-4169-a326-070959a02f66-istio-data" (OuterVolumeSpecName: "istio-data") pod "6e405b89-cc1d-4169-a326-070959a02f66" (UID: "6e405b89-cc1d-4169-a326-070959a02f66"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:32:28.653658 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:28.653578    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e405b89-cc1d-4169-a326-070959a02f66-credential-socket\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:32:28.653658 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:28.653603    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e405b89-cc1d-4169-a326-070959a02f66-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6e405b89-cc1d-4169-a326-070959a02f66" (UID: "6e405b89-cc1d-4169-a326-070959a02f66"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:32:28.653658 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:28.653611    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e405b89-cc1d-4169-a326-070959a02f66-workload-socket\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:32:28.655399 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:28.655374    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e405b89-cc1d-4169-a326-070959a02f66-istio-token" (OuterVolumeSpecName: "istio-token") pod "6e405b89-cc1d-4169-a326-070959a02f66" (UID: "6e405b89-cc1d-4169-a326-070959a02f66"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:32:28.655399 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:28.655369    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e405b89-cc1d-4169-a326-070959a02f66-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6e405b89-cc1d-4169-a326-070959a02f66" (UID: "6e405b89-cc1d-4169-a326-070959a02f66"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:32:28.655823 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:28.655803    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6e405b89-cc1d-4169-a326-070959a02f66-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6e405b89-cc1d-4169-a326-070959a02f66" (UID: "6e405b89-cc1d-4169-a326-070959a02f66"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:32:28.655823 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:28.655811    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e405b89-cc1d-4169-a326-070959a02f66-kube-api-access-r6zss" (OuterVolumeSpecName: "kube-api-access-r6zss") pod "6e405b89-cc1d-4169-a326-070959a02f66" (UID: "6e405b89-cc1d-4169-a326-070959a02f66"). InnerVolumeSpecName "kube-api-access-r6zss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:32:28.755158 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:28.755070    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e405b89-cc1d-4169-a326-070959a02f66-istio-envoy\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:32:28.755158 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:28.755097    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e405b89-cc1d-4169-a326-070959a02f66-istiod-ca-cert\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:32:28.755158 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:28.755111    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6zss\" (UniqueName: \"kubernetes.io/projected/6e405b89-cc1d-4169-a326-070959a02f66-kube-api-access-r6zss\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:32:28.755158 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:28.755122    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e405b89-cc1d-4169-a326-070959a02f66-istio-data\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:32:28.755158 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:28.755132    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e405b89-cc1d-4169-a326-070959a02f66-istio-podinfo\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:32:28.755158 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:28.755142    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e405b89-cc1d-4169-a326-070959a02f66-istio-token\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:32:29.181195 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:29.181158    2579 generic.go:356] "Generic (PLEG): container finished" podID="6e405b89-cc1d-4169-a326-070959a02f66" containerID="eb50015b36c5da678c80829aaa8ee50d31a3e3cde288b7bca5d0fc7b23baa3fc" exitCode=0
Sep 10 19:32:29.181642 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:29.181203    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"
Sep 10 19:32:29.181642 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:29.181238    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc" event={"ID":"6e405b89-cc1d-4169-a326-070959a02f66","Type":"ContainerDied","Data":"eb50015b36c5da678c80829aaa8ee50d31a3e3cde288b7bca5d0fc7b23baa3fc"}
Sep 10 19:32:29.181642 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:29.181298    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc" event={"ID":"6e405b89-cc1d-4169-a326-070959a02f66","Type":"ContainerDied","Data":"0a95c08c672ab34b75a67681e280449ad62551f43fceb4e817848ee325480eec"}
Sep 10 19:32:29.182193 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:29.181329    2579 scope.go:117] "RemoveContainer" containerID="eb50015b36c5da678c80829aaa8ee50d31a3e3cde288b7bca5d0fc7b23baa3fc"
Sep 10 19:32:29.192341 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:29.192322    2579 scope.go:117] "RemoveContainer" containerID="eb9fef4c1d2b1b124c17c29676f718401ccf0636caae07cb4e47bdcc9995285e"
Sep 10 19:32:29.200717 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:29.200702    2579 scope.go:117] "RemoveContainer" containerID="eb50015b36c5da678c80829aaa8ee50d31a3e3cde288b7bca5d0fc7b23baa3fc"
Sep 10 19:32:29.201010 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:32:29.200989    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb50015b36c5da678c80829aaa8ee50d31a3e3cde288b7bca5d0fc7b23baa3fc\": container with ID starting with eb50015b36c5da678c80829aaa8ee50d31a3e3cde288b7bca5d0fc7b23baa3fc not found: ID does not exist" containerID="eb50015b36c5da678c80829aaa8ee50d31a3e3cde288b7bca5d0fc7b23baa3fc"
Sep 10 19:32:29.201048 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:29.201019    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb50015b36c5da678c80829aaa8ee50d31a3e3cde288b7bca5d0fc7b23baa3fc"} err="failed to get container status \"eb50015b36c5da678c80829aaa8ee50d31a3e3cde288b7bca5d0fc7b23baa3fc\": rpc error: code = NotFound desc = could not find container \"eb50015b36c5da678c80829aaa8ee50d31a3e3cde288b7bca5d0fc7b23baa3fc\": container with ID starting with eb50015b36c5da678c80829aaa8ee50d31a3e3cde288b7bca5d0fc7b23baa3fc not found: ID does not exist"
Sep 10 19:32:29.201048 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:29.201034    2579 scope.go:117] "RemoveContainer" containerID="eb9fef4c1d2b1b124c17c29676f718401ccf0636caae07cb4e47bdcc9995285e"
Sep 10 19:32:29.201284 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:32:29.201267    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb9fef4c1d2b1b124c17c29676f718401ccf0636caae07cb4e47bdcc9995285e\": container with ID starting with eb9fef4c1d2b1b124c17c29676f718401ccf0636caae07cb4e47bdcc9995285e not found: ID does not exist" containerID="eb9fef4c1d2b1b124c17c29676f718401ccf0636caae07cb4e47bdcc9995285e"
Sep 10 19:32:29.201324 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:29.201291    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb9fef4c1d2b1b124c17c29676f718401ccf0636caae07cb4e47bdcc9995285e"} err="failed to get container status \"eb9fef4c1d2b1b124c17c29676f718401ccf0636caae07cb4e47bdcc9995285e\": rpc error: code = NotFound desc = could not find container \"eb9fef4c1d2b1b124c17c29676f718401ccf0636caae07cb4e47bdcc9995285e\": container with ID starting with eb9fef4c1d2b1b124c17c29676f718401ccf0636caae07cb4e47bdcc9995285e not found: ID does not exist"
Sep 10 19:32:29.206602 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:29.206579    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"]
Sep 10 19:32:29.210157 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:29.210135    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc"]
Sep 10 19:32:29.240141 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:29.240121    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e405b89-cc1d-4169-a326-070959a02f66" path="/var/lib/kubelet/pods/6e405b89-cc1d-4169-a326-070959a02f66/volumes"
Sep 10 19:32:34.329289 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:34.329256    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb"]
Sep 10 19:32:34.329893 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:34.329472    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb" podUID="3ef42da1-2dd3-473b-9e8b-7cf390d640a3" containerName="maas-api" containerID="cri-o://2073afa8099436382b414be72309823891399b99080d9ed0e5d4d7a77e8b27b2" gracePeriod=30
Sep 10 19:32:34.582297 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:34.582235    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb"
Sep 10 19:32:34.604595 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:34.604564    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ef42da1-2dd3-473b-9e8b-7cf390d640a3-openshift-service-ca\") pod \"3ef42da1-2dd3-473b-9e8b-7cf390d640a3\" (UID: \"3ef42da1-2dd3-473b-9e8b-7cf390d640a3\") "
Sep 10 19:32:34.604595 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:34.604597    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ef42da1-2dd3-473b-9e8b-7cf390d640a3-maas-api-tls\") pod \"3ef42da1-2dd3-473b-9e8b-7cf390d640a3\" (UID: \"3ef42da1-2dd3-473b-9e8b-7cf390d640a3\") "
Sep 10 19:32:34.604853 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:34.604640    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t5nzk\" (UniqueName: \"kubernetes.io/projected/3ef42da1-2dd3-473b-9e8b-7cf390d640a3-kube-api-access-t5nzk\") pod \"3ef42da1-2dd3-473b-9e8b-7cf390d640a3\" (UID: \"3ef42da1-2dd3-473b-9e8b-7cf390d640a3\") "
Sep 10 19:32:34.604853 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:34.604713    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ef42da1-2dd3-473b-9e8b-7cf390d640a3-metrics-certs\") pod \"3ef42da1-2dd3-473b-9e8b-7cf390d640a3\" (UID: \"3ef42da1-2dd3-473b-9e8b-7cf390d640a3\") "
Sep 10 19:32:34.604956 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:34.604935    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ef42da1-2dd3-473b-9e8b-7cf390d640a3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3ef42da1-2dd3-473b-9e8b-7cf390d640a3" (UID: "3ef42da1-2dd3-473b-9e8b-7cf390d640a3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:32:34.605044 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:34.605030    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ef42da1-2dd3-473b-9e8b-7cf390d640a3-openshift-service-ca\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:32:34.607641 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:34.607592    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ef42da1-2dd3-473b-9e8b-7cf390d640a3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3ef42da1-2dd3-473b-9e8b-7cf390d640a3" (UID: "3ef42da1-2dd3-473b-9e8b-7cf390d640a3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:32:34.607641 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:34.607604    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ef42da1-2dd3-473b-9e8b-7cf390d640a3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3ef42da1-2dd3-473b-9e8b-7cf390d640a3" (UID: "3ef42da1-2dd3-473b-9e8b-7cf390d640a3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:32:34.607641 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:34.607594    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ef42da1-2dd3-473b-9e8b-7cf390d640a3-kube-api-access-t5nzk" (OuterVolumeSpecName: "kube-api-access-t5nzk") pod "3ef42da1-2dd3-473b-9e8b-7cf390d640a3" (UID: "3ef42da1-2dd3-473b-9e8b-7cf390d640a3"). InnerVolumeSpecName "kube-api-access-t5nzk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:32:34.706081 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:34.706049    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ef42da1-2dd3-473b-9e8b-7cf390d640a3-maas-api-tls\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:32:34.706081 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:34.706076    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t5nzk\" (UniqueName: \"kubernetes.io/projected/3ef42da1-2dd3-473b-9e8b-7cf390d640a3-kube-api-access-t5nzk\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:32:34.706081 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:34.706086    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ef42da1-2dd3-473b-9e8b-7cf390d640a3-metrics-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:32:35.220142 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:35.220111    2579 generic.go:356] "Generic (PLEG): container finished" podID="3ef42da1-2dd3-473b-9e8b-7cf390d640a3" containerID="2073afa8099436382b414be72309823891399b99080d9ed0e5d4d7a77e8b27b2" exitCode=0
Sep 10 19:32:35.220445 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:35.220163    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb"
Sep 10 19:32:35.220445 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:35.220199    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb" event={"ID":"3ef42da1-2dd3-473b-9e8b-7cf390d640a3","Type":"ContainerDied","Data":"2073afa8099436382b414be72309823891399b99080d9ed0e5d4d7a77e8b27b2"}
Sep 10 19:32:35.220445 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:35.220252    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb" event={"ID":"3ef42da1-2dd3-473b-9e8b-7cf390d640a3","Type":"ContainerDied","Data":"e98003093732c3182fa5eff86db4463b0cf3ff01d8d78c209526286114788522"}
Sep 10 19:32:35.220445 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:35.220275    2579 scope.go:117] "RemoveContainer" containerID="2073afa8099436382b414be72309823891399b99080d9ed0e5d4d7a77e8b27b2"
Sep 10 19:32:35.230616 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:35.230601    2579 scope.go:117] "RemoveContainer" containerID="2073afa8099436382b414be72309823891399b99080d9ed0e5d4d7a77e8b27b2"
Sep 10 19:32:35.230901 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:32:35.230880    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2073afa8099436382b414be72309823891399b99080d9ed0e5d4d7a77e8b27b2\": container with ID starting with 2073afa8099436382b414be72309823891399b99080d9ed0e5d4d7a77e8b27b2 not found: ID does not exist" containerID="2073afa8099436382b414be72309823891399b99080d9ed0e5d4d7a77e8b27b2"
Sep 10 19:32:35.230968 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:35.230908    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2073afa8099436382b414be72309823891399b99080d9ed0e5d4d7a77e8b27b2"} err="failed to get container status \"2073afa8099436382b414be72309823891399b99080d9ed0e5d4d7a77e8b27b2\": rpc error: code = NotFound desc = could not find container \"2073afa8099436382b414be72309823891399b99080d9ed0e5d4d7a77e8b27b2\": container with ID starting with 2073afa8099436382b414be72309823891399b99080d9ed0e5d4d7a77e8b27b2 not found: ID does not exist"
Sep 10 19:32:35.247382 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:35.247358    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb"]
Sep 10 19:32:35.247861 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:35.247841    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4247c7-6f5455b7f5-w87fb"]
Sep 10 19:32:37.238769 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:37.238724    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ef42da1-2dd3-473b-9e8b-7cf390d640a3" path="/var/lib/kubelet/pods/3ef42da1-2dd3-473b-9e8b-7cf390d640a3/volumes"
Sep 10 19:32:42.493265 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.493235    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"]
Sep 10 19:32:42.493703 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.493634    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e405b89-cc1d-4169-a326-070959a02f66" containerName="istio-proxy"
Sep 10 19:32:42.493703 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.493645    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e405b89-cc1d-4169-a326-070959a02f66" containerName="istio-proxy"
Sep 10 19:32:42.493703 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.493658    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ef42da1-2dd3-473b-9e8b-7cf390d640a3" containerName="maas-api"
Sep 10 19:32:42.493703 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.493663    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ef42da1-2dd3-473b-9e8b-7cf390d640a3" containerName="maas-api"
Sep 10 19:32:42.493703 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.493673    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e405b89-cc1d-4169-a326-070959a02f66" containerName="istio-proxy"
Sep 10 19:32:42.493703 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.493679    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e405b89-cc1d-4169-a326-070959a02f66" containerName="istio-proxy"
Sep 10 19:32:42.494044 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.493768    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ef42da1-2dd3-473b-9e8b-7cf390d640a3" containerName="maas-api"
Sep 10 19:32:42.494044 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.493780    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e405b89-cc1d-4169-a326-070959a02f66" containerName="istio-proxy"
Sep 10 19:32:42.494044 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.493786    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e405b89-cc1d-4169-a326-070959a02f66" containerName="istio-proxy"
Sep 10 19:32:42.503763 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.503713    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:32:42.506515 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.506492    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-14a8d1-openshift-default-dockercfg-5zbd6\""
Sep 10 19:32:42.510365 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.510335    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"]
Sep 10 19:32:42.575179 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.575140    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-workload-socket\") pod \"e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89\" (UID: \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\") " pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:32:42.575343 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.575198    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-istio-podinfo\") pod \"e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89\" (UID: \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\") " pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:32:42.575343 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.575322    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-istio-data\") pod \"e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89\" (UID: \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\") " pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:32:42.575477 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.575361    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-workload-certs\") pod \"e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89\" (UID: \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\") " pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:32:42.575477 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.575382    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-credential-socket\") pod \"e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89\" (UID: \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\") " pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:32:42.575477 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.575454    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-istio-envoy\") pod \"e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89\" (UID: \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\") " pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:32:42.575642 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.575538    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-istiod-ca-cert\") pod \"e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89\" (UID: \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\") " pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:32:42.575642 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.575576    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-istio-token\") pod \"e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89\" (UID: \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\") " pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:32:42.575642 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.575613    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs2x5\" (UniqueName: \"kubernetes.io/projected/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-kube-api-access-bs2x5\") pod \"e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89\" (UID: \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\") " pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:32:42.676461 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.676425    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-istio-data\") pod \"e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89\" (UID: \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\") " pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:32:42.676461 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.676463    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-workload-certs\") pod \"e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89\" (UID: \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\") " pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:32:42.676721 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.676481    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-credential-socket\") pod \"e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89\" (UID: \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\") " pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:32:42.676721 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.676514    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-istio-envoy\") pod \"e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89\" (UID: \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\") " pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:32:42.676721 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.676581    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-istiod-ca-cert\") pod \"e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89\" (UID: \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\") " pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:32:42.676721 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.676613    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-istio-token\") pod \"e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89\" (UID: \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\") " pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:32:42.676721 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.676642    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bs2x5\" (UniqueName: \"kubernetes.io/projected/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-kube-api-access-bs2x5\") pod \"e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89\" (UID: \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\") " pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:32:42.676721 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.676672    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-workload-socket\") pod \"e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89\" (UID: \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\") " pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:32:42.676721 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.676698    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-istio-podinfo\") pod \"e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89\" (UID: \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\") " pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:32:42.677123 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.676890    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-istio-data\") pod \"e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89\" (UID: \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\") " pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:32:42.677123 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.677028    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-credential-socket\") pod \"e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89\" (UID: \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\") " pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:32:42.677123 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.677108    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-workload-certs\") pod \"e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89\" (UID: \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\") " pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:32:42.677236 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.677182    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-workload-socket\") pod \"e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89\" (UID: \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\") " pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:32:42.677384 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.677364    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-istiod-ca-cert\") pod \"e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89\" (UID: \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\") " pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:32:42.679121 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.679088    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-istio-envoy\") pod \"e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89\" (UID: \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\") " pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:32:42.679350 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.679331    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-istio-podinfo\") pod \"e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89\" (UID: \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\") " pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:32:42.689259 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.689226    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs2x5\" (UniqueName: \"kubernetes.io/projected/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-kube-api-access-bs2x5\") pod \"e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89\" (UID: \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\") " pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:32:42.689789 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.689739    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-istio-token\") pod \"e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89\" (UID: \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\") " pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:32:42.816702 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:42.816607    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:32:43.097047 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:43.097014    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"]
Sep 10 19:32:43.098566 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:32:43.098541    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a5aff68_9e9c_4e0e_bc9e_a64170d0a9f0.slice/crio-7043b585b59dca9506164ea7aafe5971e1ccc8bef472a7fb0e47c411e9279a99 WatchSource:0}: Error finding container 7043b585b59dca9506164ea7aafe5971e1ccc8bef472a7fb0e47c411e9279a99: Status 404 returned error can't find the container with id 7043b585b59dca9506164ea7aafe5971e1ccc8bef472a7fb0e47c411e9279a99
Sep 10 19:32:43.100797 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:43.100743    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:32:43.100883 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:43.100833    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:32:43.100883 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:43.100859    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:32:43.267938 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:43.267900    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89" event={"ID":"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0","Type":"ContainerStarted","Data":"479477f7c313c3b751a1321bf6850eeb3d12b21e7870c278d76fe9066f6d1e22"}
Sep 10 19:32:43.268127 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:43.267964    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89" event={"ID":"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0","Type":"ContainerStarted","Data":"7043b585b59dca9506164ea7aafe5971e1ccc8bef472a7fb0e47c411e9279a99"}
Sep 10 19:32:43.291880 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:43.291832    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89" podStartSLOduration=1.291816595 podStartE2EDuration="1.291816595s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:32:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:32:43.288333903 +0000 UTC m=+1018.508182385" watchObservedRunningTime="2026-09-10 19:32:43.291816595 +0000 UTC m=+1018.511665076"
Sep 10 19:32:43.816907 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:43.816868    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:32:44.822383 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:44.822343    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:32:45.279570 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:45.279542    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:32:45.280659 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:45.280638    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:32:46.415442 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:46.415412    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj"]
Sep 10 19:32:46.420638 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:46.420613    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj"
Sep 10 19:32:46.428103 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:46.427955    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-14a8d1-serving-cert\""
Sep 10 19:32:46.439228 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:46.439197    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj"]
Sep 10 19:32:46.513712 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:46.513675    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a6d5e92-b3fb-401f-890d-32c1d7672be9-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj\" (UID: \"5a6d5e92-b3fb-401f-890d-32c1d7672be9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj"
Sep 10 19:32:46.513906 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:46.513721    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a6d5e92-b3fb-401f-890d-32c1d7672be9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj\" (UID: \"5a6d5e92-b3fb-401f-890d-32c1d7672be9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj"
Sep 10 19:32:46.513906 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:46.513775    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5a6d5e92-b3fb-401f-890d-32c1d7672be9-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj\" (UID: \"5a6d5e92-b3fb-401f-890d-32c1d7672be9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj"
Sep 10 19:32:46.513906 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:46.513814    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v65x\" (UniqueName: \"kubernetes.io/projected/5a6d5e92-b3fb-401f-890d-32c1d7672be9-kube-api-access-8v65x\") pod \"maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj\" (UID: \"5a6d5e92-b3fb-401f-890d-32c1d7672be9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj"
Sep 10 19:32:46.614948 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:46.614917    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a6d5e92-b3fb-401f-890d-32c1d7672be9-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj\" (UID: \"5a6d5e92-b3fb-401f-890d-32c1d7672be9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj"
Sep 10 19:32:46.614948 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:46.614953    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a6d5e92-b3fb-401f-890d-32c1d7672be9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj\" (UID: \"5a6d5e92-b3fb-401f-890d-32c1d7672be9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj"
Sep 10 19:32:46.615217 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:46.614970    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5a6d5e92-b3fb-401f-890d-32c1d7672be9-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj\" (UID: \"5a6d5e92-b3fb-401f-890d-32c1d7672be9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj"
Sep 10 19:32:46.615217 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:46.614990    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v65x\" (UniqueName: \"kubernetes.io/projected/5a6d5e92-b3fb-401f-890d-32c1d7672be9-kube-api-access-8v65x\") pod \"maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj\" (UID: \"5a6d5e92-b3fb-401f-890d-32c1d7672be9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj"
Sep 10 19:32:46.615217 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:32:46.615088    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14a8d1-serving-cert: secret "maas-api-e2e-shared-a-w3-14a8d1-serving-cert" not found
Sep 10 19:32:46.615217 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:32:46.615184    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a6d5e92-b3fb-401f-890d-32c1d7672be9-maas-api-tls podName:5a6d5e92-b3fb-401f-890d-32c1d7672be9 nodeName:}" failed. No retries permitted until 2026-09-10 19:32:47.115161233 +0000 UTC m=+1022.335009706 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5a6d5e92-b3fb-401f-890d-32c1d7672be9-maas-api-tls") pod "maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj" (UID: "5a6d5e92-b3fb-401f-890d-32c1d7672be9") : secret "maas-api-e2e-shared-a-w3-14a8d1-serving-cert" not found
Sep 10 19:32:46.615618 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:46.615591    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a6d5e92-b3fb-401f-890d-32c1d7672be9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj\" (UID: \"5a6d5e92-b3fb-401f-890d-32c1d7672be9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj"
Sep 10 19:32:46.617624 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:46.617601    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5a6d5e92-b3fb-401f-890d-32c1d7672be9-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj\" (UID: \"5a6d5e92-b3fb-401f-890d-32c1d7672be9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj"
Sep 10 19:32:46.623192 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:46.623168    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v65x\" (UniqueName: \"kubernetes.io/projected/5a6d5e92-b3fb-401f-890d-32c1d7672be9-kube-api-access-8v65x\") pod \"maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj\" (UID: \"5a6d5e92-b3fb-401f-890d-32c1d7672be9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj"
Sep 10 19:32:47.120571 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:47.120531    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a6d5e92-b3fb-401f-890d-32c1d7672be9-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj\" (UID: \"5a6d5e92-b3fb-401f-890d-32c1d7672be9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj"
Sep 10 19:32:47.122994 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:47.122971    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a6d5e92-b3fb-401f-890d-32c1d7672be9-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj\" (UID: \"5a6d5e92-b3fb-401f-890d-32c1d7672be9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj"
Sep 10 19:32:47.347128 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:47.347090    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj"
Sep 10 19:32:47.625349 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:47.625320    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj"]
Sep 10 19:32:47.626953 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:32:47.626927    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a6d5e92_b3fb_401f_890d_32c1d7672be9.slice/crio-a64b0b775e9a43f3687093cc5df76dec028bdbb77713bafe73335555485e3293 WatchSource:0}: Error finding container a64b0b775e9a43f3687093cc5df76dec028bdbb77713bafe73335555485e3293: Status 404 returned error can't find the container with id a64b0b775e9a43f3687093cc5df76dec028bdbb77713bafe73335555485e3293
Sep 10 19:32:47.628995 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:47.628974    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 19:32:48.297510 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:48.297410    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj" event={"ID":"5a6d5e92-b3fb-401f-890d-32c1d7672be9","Type":"ContainerStarted","Data":"318b6035afbdc510771241fc08934deabc8208ad16a016c5d11be724511f9e05"}
Sep 10 19:32:48.297510 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:48.297473    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj" event={"ID":"5a6d5e92-b3fb-401f-890d-32c1d7672be9","Type":"ContainerStarted","Data":"a64b0b775e9a43f3687093cc5df76dec028bdbb77713bafe73335555485e3293"}
Sep 10 19:32:48.297702 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:48.297527    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj"
Sep 10 19:32:48.316866 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:48.316817    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj" podStartSLOduration=1.967867509 podStartE2EDuration="2.316798149s" totalImagesPullingTime="348.93064ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:32:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:32:48.316123905 +0000 UTC m=+1023.535972386" watchObservedRunningTime="2026-09-10 19:32:48.316798149 +0000 UTC m=+1023.536646629"
Sep 10 19:32:54.309710 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:32:54.309679    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj"
Sep 10 19:33:00.280785 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:00.280727    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Sep 10 19:33:00.281223 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:00.280810    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89" podUID="0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Sep 10 19:33:00.372072 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:00.372037    2579 generic.go:356] "Generic (PLEG): container finished" podID="0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0" containerID="479477f7c313c3b751a1321bf6850eeb3d12b21e7870c278d76fe9066f6d1e22" exitCode=0
Sep 10 19:33:00.372259 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:00.372117    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89" event={"ID":"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0","Type":"ContainerDied","Data":"479477f7c313c3b751a1321bf6850eeb3d12b21e7870c278d76fe9066f6d1e22"}
Sep 10 19:33:00.372590 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:00.372572    2579 scope.go:117] "RemoveContainer" containerID="479477f7c313c3b751a1321bf6850eeb3d12b21e7870c278d76fe9066f6d1e22"
Sep 10 19:33:00.373720 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:00.373695    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:33:00.373817 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:00.373791    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:33:00.373873 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:00.373823    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:33:01.382378 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:01.382348    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89" event={"ID":"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0","Type":"ContainerStarted","Data":"68248b55e153c45fe636be1dcbc63f178538ee29629bd16c4e1b197defe5a3e0"}
Sep 10 19:33:01.817490 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:01.817392    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:33:01.817636 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:01.817558    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Sep 10 19:33:01.817636 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:01.817602    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89" podUID="0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Sep 10 19:33:02.475332 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:02.475302    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9d2e2715-784b7b4f88-97274"]
Sep 10 19:33:02.476768 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:02.476470    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d2e2715-784b7b4f88-97274" podUID="0d748b0d-ab6e-44aa-b87b-cdd477bc4047" containerName="maas-api" containerID="cri-o://25f4eff58a6743ea05f34e57cc328e86931d90a9ba3e9760d2359aecffb7ad98" gracePeriod=30
Sep 10 19:33:02.774913 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:02.774886    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d2e2715-784b7b4f88-97274"
Sep 10 19:33:02.817446 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:02.817412    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Sep 10 19:33:02.817636 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:02.817463    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89" podUID="0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Sep 10 19:33:02.889833 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:02.889798    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7tt6b\" (UniqueName: \"kubernetes.io/projected/0d748b0d-ab6e-44aa-b87b-cdd477bc4047-kube-api-access-7tt6b\") pod \"0d748b0d-ab6e-44aa-b87b-cdd477bc4047\" (UID: \"0d748b0d-ab6e-44aa-b87b-cdd477bc4047\") "
Sep 10 19:33:02.890001 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:02.889888    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d748b0d-ab6e-44aa-b87b-cdd477bc4047-maas-api-tls\") pod \"0d748b0d-ab6e-44aa-b87b-cdd477bc4047\" (UID: \"0d748b0d-ab6e-44aa-b87b-cdd477bc4047\") "
Sep 10 19:33:02.890001 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:02.889907    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d748b0d-ab6e-44aa-b87b-cdd477bc4047-openshift-service-ca\") pod \"0d748b0d-ab6e-44aa-b87b-cdd477bc4047\" (UID: \"0d748b0d-ab6e-44aa-b87b-cdd477bc4047\") "
Sep 10 19:33:02.890001 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:02.889944    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d748b0d-ab6e-44aa-b87b-cdd477bc4047-metrics-certs\") pod \"0d748b0d-ab6e-44aa-b87b-cdd477bc4047\" (UID: \"0d748b0d-ab6e-44aa-b87b-cdd477bc4047\") "
Sep 10 19:33:02.890289 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:02.890249    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d748b0d-ab6e-44aa-b87b-cdd477bc4047-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0d748b0d-ab6e-44aa-b87b-cdd477bc4047" (UID: "0d748b0d-ab6e-44aa-b87b-cdd477bc4047"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:33:02.892122 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:02.892094    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d748b0d-ab6e-44aa-b87b-cdd477bc4047-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0d748b0d-ab6e-44aa-b87b-cdd477bc4047" (UID: "0d748b0d-ab6e-44aa-b87b-cdd477bc4047"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:33:02.892200 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:02.892115    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d748b0d-ab6e-44aa-b87b-cdd477bc4047-kube-api-access-7tt6b" (OuterVolumeSpecName: "kube-api-access-7tt6b") pod "0d748b0d-ab6e-44aa-b87b-cdd477bc4047" (UID: "0d748b0d-ab6e-44aa-b87b-cdd477bc4047"). InnerVolumeSpecName "kube-api-access-7tt6b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:33:02.892200 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:02.892158    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d748b0d-ab6e-44aa-b87b-cdd477bc4047-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0d748b0d-ab6e-44aa-b87b-cdd477bc4047" (UID: "0d748b0d-ab6e-44aa-b87b-cdd477bc4047"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:33:02.991257 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:02.991172    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7tt6b\" (UniqueName: \"kubernetes.io/projected/0d748b0d-ab6e-44aa-b87b-cdd477bc4047-kube-api-access-7tt6b\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:33:02.991257 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:02.991199    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d748b0d-ab6e-44aa-b87b-cdd477bc4047-maas-api-tls\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:33:02.991257 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:02.991210    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d748b0d-ab6e-44aa-b87b-cdd477bc4047-openshift-service-ca\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:33:02.991257 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:02.991218    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d748b0d-ab6e-44aa-b87b-cdd477bc4047-metrics-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:33:03.395359 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:03.395322    2579 generic.go:356] "Generic (PLEG): container finished" podID="0d748b0d-ab6e-44aa-b87b-cdd477bc4047" containerID="25f4eff58a6743ea05f34e57cc328e86931d90a9ba3e9760d2359aecffb7ad98" exitCode=0
Sep 10 19:33:03.395534 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:03.395393    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d2e2715-784b7b4f88-97274"
Sep 10 19:33:03.395534 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:03.395396    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d2e2715-784b7b4f88-97274" event={"ID":"0d748b0d-ab6e-44aa-b87b-cdd477bc4047","Type":"ContainerDied","Data":"25f4eff58a6743ea05f34e57cc328e86931d90a9ba3e9760d2359aecffb7ad98"}
Sep 10 19:33:03.395534 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:03.395511    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9d2e2715-784b7b4f88-97274" event={"ID":"0d748b0d-ab6e-44aa-b87b-cdd477bc4047","Type":"ContainerDied","Data":"4a99532631415eb21f98afb729b4c71a66d33a9b531d8958c6f5540a8b8c18c9"}
Sep 10 19:33:03.395534 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:03.395533    2579 scope.go:117] "RemoveContainer" containerID="25f4eff58a6743ea05f34e57cc328e86931d90a9ba3e9760d2359aecffb7ad98"
Sep 10 19:33:03.405296 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:03.405284    2579 scope.go:117] "RemoveContainer" containerID="25f4eff58a6743ea05f34e57cc328e86931d90a9ba3e9760d2359aecffb7ad98"
Sep 10 19:33:03.405582 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:33:03.405555    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25f4eff58a6743ea05f34e57cc328e86931d90a9ba3e9760d2359aecffb7ad98\": container with ID starting with 25f4eff58a6743ea05f34e57cc328e86931d90a9ba3e9760d2359aecffb7ad98 not found: ID does not exist" containerID="25f4eff58a6743ea05f34e57cc328e86931d90a9ba3e9760d2359aecffb7ad98"
Sep 10 19:33:03.405647 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:03.405594    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25f4eff58a6743ea05f34e57cc328e86931d90a9ba3e9760d2359aecffb7ad98"} err="failed to get container status \"25f4eff58a6743ea05f34e57cc328e86931d90a9ba3e9760d2359aecffb7ad98\": rpc error: code = NotFound desc = could not find container \"25f4eff58a6743ea05f34e57cc328e86931d90a9ba3e9760d2359aecffb7ad98\": container with ID starting with 25f4eff58a6743ea05f34e57cc328e86931d90a9ba3e9760d2359aecffb7ad98 not found: ID does not exist"
Sep 10 19:33:03.413220 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:03.413199    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9d2e2715-784b7b4f88-97274"]
Sep 10 19:33:03.417468 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:03.417446    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9d2e2715-784b7b4f88-97274"]
Sep 10 19:33:03.817296 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:03.817266    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Sep 10 19:33:03.817667 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:03.817316    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89" podUID="0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Sep 10 19:33:04.817822 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:04.817788    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Sep 10 19:33:04.818226 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:04.817840    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89" podUID="0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Sep 10 19:33:05.240085 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:05.240054    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d748b0d-ab6e-44aa-b87b-cdd477bc4047" path="/var/lib/kubelet/pods/0d748b0d-ab6e-44aa-b87b-cdd477bc4047/volumes"
Sep 10 19:33:05.817665 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:05.817623    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Sep 10 19:33:05.817884 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:05.817718    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89" podUID="0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Sep 10 19:33:06.821441 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:06.821410    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:33:06.822043 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:06.821816    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:33:06.822580 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:06.822563    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:33:14.608493 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:14.608451    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg"]
Sep 10 19:33:14.609030 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:14.609006    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d748b0d-ab6e-44aa-b87b-cdd477bc4047" containerName="maas-api"
Sep 10 19:33:14.609030 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:14.609028    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d748b0d-ab6e-44aa-b87b-cdd477bc4047" containerName="maas-api"
Sep 10 19:33:14.609175 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:14.609120    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d748b0d-ab6e-44aa-b87b-cdd477bc4047" containerName="maas-api"
Sep 10 19:33:14.612184 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:14.612165    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg"
Sep 10 19:33:14.614381 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:14.614346    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-cab4ee-serving-cert\""
Sep 10 19:33:14.626933 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:14.626907    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg"]
Sep 10 19:33:14.705289 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:14.705255    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c431193-2d5f-4789-8abf-35a6ab2f02e8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg\" (UID: \"7c431193-2d5f-4789-8abf-35a6ab2f02e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg"
Sep 10 19:33:14.705470 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:14.705352    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c431193-2d5f-4789-8abf-35a6ab2f02e8-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg\" (UID: \"7c431193-2d5f-4789-8abf-35a6ab2f02e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg"
Sep 10 19:33:14.705531 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:14.705468    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rvlj\" (UniqueName: \"kubernetes.io/projected/7c431193-2d5f-4789-8abf-35a6ab2f02e8-kube-api-access-8rvlj\") pod \"maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg\" (UID: \"7c431193-2d5f-4789-8abf-35a6ab2f02e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg"
Sep 10 19:33:14.705531 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:14.705500    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c431193-2d5f-4789-8abf-35a6ab2f02e8-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg\" (UID: \"7c431193-2d5f-4789-8abf-35a6ab2f02e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg"
Sep 10 19:33:14.806588 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:14.806553    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rvlj\" (UniqueName: \"kubernetes.io/projected/7c431193-2d5f-4789-8abf-35a6ab2f02e8-kube-api-access-8rvlj\") pod \"maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg\" (UID: \"7c431193-2d5f-4789-8abf-35a6ab2f02e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg"
Sep 10 19:33:14.806588 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:14.806589    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c431193-2d5f-4789-8abf-35a6ab2f02e8-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg\" (UID: \"7c431193-2d5f-4789-8abf-35a6ab2f02e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg"
Sep 10 19:33:14.806848 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:14.806620    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c431193-2d5f-4789-8abf-35a6ab2f02e8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg\" (UID: \"7c431193-2d5f-4789-8abf-35a6ab2f02e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg"
Sep 10 19:33:14.806848 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:14.806672    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c431193-2d5f-4789-8abf-35a6ab2f02e8-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg\" (UID: \"7c431193-2d5f-4789-8abf-35a6ab2f02e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg"
Sep 10 19:33:14.807463 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:14.807434    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c431193-2d5f-4789-8abf-35a6ab2f02e8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg\" (UID: \"7c431193-2d5f-4789-8abf-35a6ab2f02e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg"
Sep 10 19:33:14.809330 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:14.809304    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c431193-2d5f-4789-8abf-35a6ab2f02e8-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg\" (UID: \"7c431193-2d5f-4789-8abf-35a6ab2f02e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg"
Sep 10 19:33:14.809430 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:14.809343    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c431193-2d5f-4789-8abf-35a6ab2f02e8-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg\" (UID: \"7c431193-2d5f-4789-8abf-35a6ab2f02e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg"
Sep 10 19:33:14.814840 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:14.814814    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rvlj\" (UniqueName: \"kubernetes.io/projected/7c431193-2d5f-4789-8abf-35a6ab2f02e8-kube-api-access-8rvlj\") pod \"maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg\" (UID: \"7c431193-2d5f-4789-8abf-35a6ab2f02e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg"
Sep 10 19:33:14.924226 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:14.924191    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg"
Sep 10 19:33:15.199710 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:15.199682    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg"]
Sep 10 19:33:15.201533 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:33:15.201488    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c431193_2d5f_4789_8abf_35a6ab2f02e8.slice/crio-8ceb456fa1c8be34e6ab11141371455e892d1c771d5855b8dafb74dc3a6db4f9 WatchSource:0}: Error finding container 8ceb456fa1c8be34e6ab11141371455e892d1c771d5855b8dafb74dc3a6db4f9: Status 404 returned error can't find the container with id 8ceb456fa1c8be34e6ab11141371455e892d1c771d5855b8dafb74dc3a6db4f9
Sep 10 19:33:15.480598 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:15.480499    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg" event={"ID":"7c431193-2d5f-4789-8abf-35a6ab2f02e8","Type":"ContainerStarted","Data":"8ceb456fa1c8be34e6ab11141371455e892d1c771d5855b8dafb74dc3a6db4f9"}
Sep 10 19:33:16.487707 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:16.487671    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg" event={"ID":"7c431193-2d5f-4789-8abf-35a6ab2f02e8","Type":"ContainerStarted","Data":"f18aece6a86e29e7b4ad4b24b36b2aaccf2baded50cc2a2d412af7b7515a865d"}
Sep 10 19:33:16.488119 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:16.487728    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg"
Sep 10 19:33:16.506011 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:16.505962    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg" podStartSLOduration=2.119812205 podStartE2EDuration="2.505946491s" totalImagesPullingTime="386.134286ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:33:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:33:16.504884042 +0000 UTC m=+1051.724732523" watchObservedRunningTime="2026-09-10 19:33:16.505946491 +0000 UTC m=+1051.725794972"
Sep 10 19:33:22.501535 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:22.501507    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg"
Sep 10 19:33:29.597354 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:29.597317    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl"]
Sep 10 19:33:29.601360 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:29.601339    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl"
Sep 10 19:33:29.603513 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:29.603489    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-8dd4a3-serving-cert\""
Sep 10 19:33:29.614200 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:29.614176    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl"]
Sep 10 19:33:29.647741 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:29.647712    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eef2f9d1-abbd-40f4-ae09-3352e50affed-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl\" (UID: \"eef2f9d1-abbd-40f4-ae09-3352e50affed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl"
Sep 10 19:33:29.647899 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:29.647777    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eef2f9d1-abbd-40f4-ae09-3352e50affed-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl\" (UID: \"eef2f9d1-abbd-40f4-ae09-3352e50affed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl"
Sep 10 19:33:29.647899 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:29.647835    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnghb\" (UniqueName: \"kubernetes.io/projected/eef2f9d1-abbd-40f4-ae09-3352e50affed-kube-api-access-jnghb\") pod \"maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl\" (UID: \"eef2f9d1-abbd-40f4-ae09-3352e50affed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl"
Sep 10 19:33:29.647899 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:29.647885    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eef2f9d1-abbd-40f4-ae09-3352e50affed-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl\" (UID: \"eef2f9d1-abbd-40f4-ae09-3352e50affed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl"
Sep 10 19:33:29.748936 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:29.748901    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnghb\" (UniqueName: \"kubernetes.io/projected/eef2f9d1-abbd-40f4-ae09-3352e50affed-kube-api-access-jnghb\") pod \"maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl\" (UID: \"eef2f9d1-abbd-40f4-ae09-3352e50affed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl"
Sep 10 19:33:29.749112 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:29.748982    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eef2f9d1-abbd-40f4-ae09-3352e50affed-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl\" (UID: \"eef2f9d1-abbd-40f4-ae09-3352e50affed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl"
Sep 10 19:33:29.749112 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:29.749022    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eef2f9d1-abbd-40f4-ae09-3352e50affed-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl\" (UID: \"eef2f9d1-abbd-40f4-ae09-3352e50affed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl"
Sep 10 19:33:29.749112 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:29.749056    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eef2f9d1-abbd-40f4-ae09-3352e50affed-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl\" (UID: \"eef2f9d1-abbd-40f4-ae09-3352e50affed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl"
Sep 10 19:33:29.749225 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:33:29.749150    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8dd4a3-serving-cert: secret "maas-api-e2e-shared-a-w2-8dd4a3-serving-cert" not found
Sep 10 19:33:29.749260 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:33:29.749238    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eef2f9d1-abbd-40f4-ae09-3352e50affed-maas-api-tls podName:eef2f9d1-abbd-40f4-ae09-3352e50affed nodeName:}" failed. No retries permitted until 2026-09-10 19:33:30.249213477 +0000 UTC m=+1065.469061950 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/eef2f9d1-abbd-40f4-ae09-3352e50affed-maas-api-tls") pod "maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl" (UID: "eef2f9d1-abbd-40f4-ae09-3352e50affed") : secret "maas-api-e2e-shared-a-w2-8dd4a3-serving-cert" not found
Sep 10 19:33:29.749621 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:29.749599    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eef2f9d1-abbd-40f4-ae09-3352e50affed-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl\" (UID: \"eef2f9d1-abbd-40f4-ae09-3352e50affed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl"
Sep 10 19:33:29.751731 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:29.751713    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eef2f9d1-abbd-40f4-ae09-3352e50affed-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl\" (UID: \"eef2f9d1-abbd-40f4-ae09-3352e50affed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl"
Sep 10 19:33:29.758080 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:29.758057    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnghb\" (UniqueName: \"kubernetes.io/projected/eef2f9d1-abbd-40f4-ae09-3352e50affed-kube-api-access-jnghb\") pod \"maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl\" (UID: \"eef2f9d1-abbd-40f4-ae09-3352e50affed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl"
Sep 10 19:33:30.254783 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:30.254675    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eef2f9d1-abbd-40f4-ae09-3352e50affed-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl\" (UID: \"eef2f9d1-abbd-40f4-ae09-3352e50affed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl"
Sep 10 19:33:30.257390 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:30.257363    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eef2f9d1-abbd-40f4-ae09-3352e50affed-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl\" (UID: \"eef2f9d1-abbd-40f4-ae09-3352e50affed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl"
Sep 10 19:33:30.514045 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:30.513971    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl"
Sep 10 19:33:30.788355 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:30.788332    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl"]
Sep 10 19:33:30.789808 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:33:30.789786    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeef2f9d1_abbd_40f4_ae09_3352e50affed.slice/crio-a8e966860a52ee0d4dfe3f43500ac71ab3c5f2c76fddd76896fd81cfb67663fe WatchSource:0}: Error finding container a8e966860a52ee0d4dfe3f43500ac71ab3c5f2c76fddd76896fd81cfb67663fe: Status 404 returned error can't find the container with id a8e966860a52ee0d4dfe3f43500ac71ab3c5f2c76fddd76896fd81cfb67663fe
Sep 10 19:33:31.590766 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:31.590708    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl" event={"ID":"eef2f9d1-abbd-40f4-ae09-3352e50affed","Type":"ContainerStarted","Data":"ce44c610b720759fb12feafd32b7b9449cf9d8f7dac3b76b460986d6479a10b9"}
Sep 10 19:33:31.590931 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:31.590791    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl" event={"ID":"eef2f9d1-abbd-40f4-ae09-3352e50affed","Type":"ContainerStarted","Data":"a8e966860a52ee0d4dfe3f43500ac71ab3c5f2c76fddd76896fd81cfb67663fe"}
Sep 10 19:33:31.591014 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:31.590972    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl"
Sep 10 19:33:31.610636 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:31.610584    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl" podStartSLOduration=1.902491774 podStartE2EDuration="2.610565261s" totalImagesPullingTime="708.073487ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:33:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:33:31.608648513 +0000 UTC m=+1066.828496994" watchObservedRunningTime="2026-09-10 19:33:31.610565261 +0000 UTC m=+1066.830413742"
Sep 10 19:33:37.602180 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:37.602100    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl"
Sep 10 19:33:54.048640 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.048611    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"]
Sep 10 19:33:54.052398 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.052383    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:33:54.055572 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.055545    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-4bc8b3-openshift-default-dockercfg-fmsr5\""
Sep 10 19:33:54.068956 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.068931    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"]
Sep 10 19:33:54.090830 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.090804    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-credential-socket\") pod \"e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx\" (UID: \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\") " pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:33:54.090960 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.090841    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-istio-token\") pod \"e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx\" (UID: \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\") " pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:33:54.091023 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.090998    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-istio-podinfo\") pod \"e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx\" (UID: \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\") " pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:33:54.091100 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.091084    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-workload-socket\") pod \"e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx\" (UID: \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\") " pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:33:54.091137 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.091109    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwknr\" (UniqueName: \"kubernetes.io/projected/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-kube-api-access-lwknr\") pod \"e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx\" (UID: \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\") " pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:33:54.091137 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.091131    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-istio-envoy\") pod \"e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx\" (UID: \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\") " pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:33:54.091205 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.091151    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-istio-data\") pod \"e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx\" (UID: \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\") " pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:33:54.091205 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.091180    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-istiod-ca-cert\") pod \"e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx\" (UID: \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\") " pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:33:54.091269 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.091211    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-workload-certs\") pod \"e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx\" (UID: \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\") " pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:33:54.191973 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.191942    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-istio-podinfo\") pod \"e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx\" (UID: \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\") " pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:33:54.192185 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.192003    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-workload-socket\") pod \"e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx\" (UID: \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\") " pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:33:54.192185 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.192020    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwknr\" (UniqueName: \"kubernetes.io/projected/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-kube-api-access-lwknr\") pod \"e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx\" (UID: \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\") " pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:33:54.192185 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.192039    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-istio-envoy\") pod \"e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx\" (UID: \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\") " pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:33:54.192185 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.192057    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-istio-data\") pod \"e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx\" (UID: \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\") " pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:33:54.192185 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.192078    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-istiod-ca-cert\") pod \"e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx\" (UID: \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\") " pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:33:54.192185 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.192104    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-workload-certs\") pod \"e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx\" (UID: \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\") " pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:33:54.192185 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.192136    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-credential-socket\") pod \"e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx\" (UID: \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\") " pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:33:54.192185 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.192168    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-istio-token\") pod \"e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx\" (UID: \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\") " pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:33:54.192611 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.192507    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-istio-data\") pod \"e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx\" (UID: \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\") " pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:33:54.192611 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.192550    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-workload-socket\") pod \"e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx\" (UID: \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\") " pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:33:54.192727 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.192668    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-workload-certs\") pod \"e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx\" (UID: \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\") " pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:33:54.192727 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.192672    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-credential-socket\") pod \"e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx\" (UID: \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\") " pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:33:54.192940 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.192918    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-istiod-ca-cert\") pod \"e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx\" (UID: \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\") " pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:33:54.194604 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.194585    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-istio-envoy\") pod \"e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx\" (UID: \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\") " pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:33:54.194708 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.194617    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-istio-podinfo\") pod \"e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx\" (UID: \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\") " pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:33:54.200852 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.200827    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwknr\" (UniqueName: \"kubernetes.io/projected/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-kube-api-access-lwknr\") pod \"e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx\" (UID: \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\") " pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:33:54.201227 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.201211    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-istio-token\") pod \"e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx\" (UID: \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\") " pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:33:54.362818 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.362711    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:33:54.644433 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.644405    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"]
Sep 10 19:33:54.645218 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:33:54.645193    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bf44155_0d1f_4b0b_a6f6_bfe7706e0488.slice/crio-27e7cc7214fd3f7f16df64b17f8da6bfb925320adc5f661f2bfbd8fe77557c2a WatchSource:0}: Error finding container 27e7cc7214fd3f7f16df64b17f8da6bfb925320adc5f661f2bfbd8fe77557c2a: Status 404 returned error can't find the container with id 27e7cc7214fd3f7f16df64b17f8da6bfb925320adc5f661f2bfbd8fe77557c2a
Sep 10 19:33:54.647658 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.647628    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:33:54.647782 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.647706    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:33:54.647782 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.647773    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:33:54.736390 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.736346    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx" event={"ID":"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488","Type":"ContainerStarted","Data":"24429ba0b0f3b38293c814d64b39fa53cb3121a06bf5450a5f1bdea6bd2f5b74"}
Sep 10 19:33:54.736601 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.736420    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx" event={"ID":"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488","Type":"ContainerStarted","Data":"27e7cc7214fd3f7f16df64b17f8da6bfb925320adc5f661f2bfbd8fe77557c2a"}
Sep 10 19:33:54.757047 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:54.756996    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx" podStartSLOduration=0.756979733 podStartE2EDuration="756.979733ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:33:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:33:54.754294226 +0000 UTC m=+1089.974142707" watchObservedRunningTime="2026-09-10 19:33:54.756979733 +0000 UTC m=+1089.976828287"
Sep 10 19:33:55.363513 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:55.363481    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:33:55.369162 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:55.369142    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:33:55.742364 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:55.742329    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:33:55.743421 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:55.743403    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:33:57.801694 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:57.801660    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc"]
Sep 10 19:33:57.806259 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:57.806230    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc"
Sep 10 19:33:57.808608 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:57.808554    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-4bc8b3-serving-cert\""
Sep 10 19:33:57.817189 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:57.816882    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc"]
Sep 10 19:33:57.822273 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:57.822250    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd180728-3a63-41d1-896a-32e024c7a046-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc\" (UID: \"cd180728-3a63-41d1-896a-32e024c7a046\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc"
Sep 10 19:33:57.822511 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:57.822461    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqq9q\" (UniqueName: \"kubernetes.io/projected/cd180728-3a63-41d1-896a-32e024c7a046-kube-api-access-vqq9q\") pod \"maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc\" (UID: \"cd180728-3a63-41d1-896a-32e024c7a046\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc"
Sep 10 19:33:57.822611 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:57.822590    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd180728-3a63-41d1-896a-32e024c7a046-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc\" (UID: \"cd180728-3a63-41d1-896a-32e024c7a046\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc"
Sep 10 19:33:57.822676 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:57.822657    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd180728-3a63-41d1-896a-32e024c7a046-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc\" (UID: \"cd180728-3a63-41d1-896a-32e024c7a046\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc"
Sep 10 19:33:57.923313 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:57.923283    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqq9q\" (UniqueName: \"kubernetes.io/projected/cd180728-3a63-41d1-896a-32e024c7a046-kube-api-access-vqq9q\") pod \"maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc\" (UID: \"cd180728-3a63-41d1-896a-32e024c7a046\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc"
Sep 10 19:33:57.923512 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:57.923326    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd180728-3a63-41d1-896a-32e024c7a046-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc\" (UID: \"cd180728-3a63-41d1-896a-32e024c7a046\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc"
Sep 10 19:33:57.923512 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:57.923357    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd180728-3a63-41d1-896a-32e024c7a046-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc\" (UID: \"cd180728-3a63-41d1-896a-32e024c7a046\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc"
Sep 10 19:33:57.923512 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:57.923483    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd180728-3a63-41d1-896a-32e024c7a046-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc\" (UID: \"cd180728-3a63-41d1-896a-32e024c7a046\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc"
Sep 10 19:33:57.924210 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:57.924186    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd180728-3a63-41d1-896a-32e024c7a046-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc\" (UID: \"cd180728-3a63-41d1-896a-32e024c7a046\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc"
Sep 10 19:33:57.925914 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:57.925890    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd180728-3a63-41d1-896a-32e024c7a046-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc\" (UID: \"cd180728-3a63-41d1-896a-32e024c7a046\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc"
Sep 10 19:33:57.926001 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:57.925942    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd180728-3a63-41d1-896a-32e024c7a046-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc\" (UID: \"cd180728-3a63-41d1-896a-32e024c7a046\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc"
Sep 10 19:33:57.931597 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:57.931578    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqq9q\" (UniqueName: \"kubernetes.io/projected/cd180728-3a63-41d1-896a-32e024c7a046-kube-api-access-vqq9q\") pod \"maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc\" (UID: \"cd180728-3a63-41d1-896a-32e024c7a046\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc"
Sep 10 19:33:58.122795 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:58.122687    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc"
Sep 10 19:33:58.402337 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:58.402312    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc"]
Sep 10 19:33:58.403069 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:33:58.403047    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd180728_3a63_41d1_896a_32e024c7a046.slice/crio-42a61dc81261023e57e37823f5e46439d122c3dc6b2869578e476aa04357c7ec WatchSource:0}: Error finding container 42a61dc81261023e57e37823f5e46439d122c3dc6b2869578e476aa04357c7ec: Status 404 returned error can't find the container with id 42a61dc81261023e57e37823f5e46439d122c3dc6b2869578e476aa04357c7ec
Sep 10 19:33:58.761713 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:58.761688    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc" event={"ID":"cd180728-3a63-41d1-896a-32e024c7a046","Type":"ContainerStarted","Data":"42a61dc81261023e57e37823f5e46439d122c3dc6b2869578e476aa04357c7ec"}
Sep 10 19:33:59.769365 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:59.769329    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc" event={"ID":"cd180728-3a63-41d1-896a-32e024c7a046","Type":"ContainerStarted","Data":"6f01629044bd3886e1e0066d933ca812739061ecf4645da85cc806c97e655865"}
Sep 10 19:33:59.769794 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:59.769519    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc"
Sep 10 19:33:59.789516 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:33:59.789469    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc" podStartSLOduration=2.438776614 podStartE2EDuration="2.789452533s" totalImagesPullingTime="350.675919ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:33:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:33:59.786243856 +0000 UTC m=+1095.006092339" watchObservedRunningTime="2026-09-10 19:33:59.789452533 +0000 UTC m=+1095.009301014"
Sep 10 19:34:05.781552 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:05.781517    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc"
Sep 10 19:34:11.784928 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:11.784885    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 19:34:11.785364 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:11.784956    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx" podUID="7bf44155-0d1f-4b0b-a6f6-bfe7706e0488" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 19:34:13.864118 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:13.864083    2579 generic.go:356] "Generic (PLEG): container finished" podID="7bf44155-0d1f-4b0b-a6f6-bfe7706e0488" containerID="24429ba0b0f3b38293c814d64b39fa53cb3121a06bf5450a5f1bdea6bd2f5b74" exitCode=0
Sep 10 19:34:13.864571 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:13.864159    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx" event={"ID":"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488","Type":"ContainerDied","Data":"24429ba0b0f3b38293c814d64b39fa53cb3121a06bf5450a5f1bdea6bd2f5b74"}
Sep 10 19:34:13.864619 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:13.864589    2579 scope.go:117] "RemoveContainer" containerID="24429ba0b0f3b38293c814d64b39fa53cb3121a06bf5450a5f1bdea6bd2f5b74"
Sep 10 19:34:13.865715 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:13.865689    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:34:13.865847 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:13.865797    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:34:13.865913 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:13.865848    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 19:34:14.877002 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:14.876961    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx" event={"ID":"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488","Type":"ContainerStarted","Data":"9982704f58e48d542bddc1a32c49facffef34cd7a2e8814374465eb0bc4ff248"}
Sep 10 19:34:15.363720 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:15.363682    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:34:15.364200 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:15.364137    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Sep 10 19:34:15.364679 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:15.364618    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx" podUID="7bf44155-0d1f-4b0b-a6f6-bfe7706e0488" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Sep 10 19:34:16.364127 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:16.364081    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Sep 10 19:34:16.364646 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:16.364151    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx" podUID="7bf44155-0d1f-4b0b-a6f6-bfe7706e0488" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Sep 10 19:34:17.364045 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:17.364000    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Sep 10 19:34:17.364260 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:17.364083    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx" podUID="7bf44155-0d1f-4b0b-a6f6-bfe7706e0488" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Sep 10 19:34:18.282658 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:18.282618    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-14a8d1/auth-test-model-14a8d1-kserve-645b87cf5f-4qv95"]
Sep 10 19:34:18.286762 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:18.286727    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-14a8d1/auth-test-model-14a8d1-kserve-645b87cf5f-4qv95"
Sep 10 19:34:18.289328 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:18.289302    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-14a8d1\"/\"kube-root-ca.crt\""
Sep 10 19:34:18.289764 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:18.289733    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-14a8d1\"/\"openshift-service-ca.crt\""
Sep 10 19:34:18.290101 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:18.290087    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-14a8d1\"/\"default-dockercfg-s9krw\""
Sep 10 19:34:18.290284 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:18.290258    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-14a8d1\"/\"auth-test-model-14a8d1-kserve-self-signed-certs\""
Sep 10 19:34:18.302413 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:18.302389    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14a8d1/auth-test-model-14a8d1-kserve-645b87cf5f-4qv95"]
Sep 10 19:34:18.330629 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:18.330594    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/001ddfc0-33e8-47de-9d4f-af35bbf7928c-home\") pod \"auth-test-model-14a8d1-kserve-645b87cf5f-4qv95\" (UID: \"001ddfc0-33e8-47de-9d4f-af35bbf7928c\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/auth-test-model-14a8d1-kserve-645b87cf5f-4qv95"
Sep 10 19:34:18.330832 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:18.330639    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/001ddfc0-33e8-47de-9d4f-af35bbf7928c-model-cache\") pod \"auth-test-model-14a8d1-kserve-645b87cf5f-4qv95\" (UID: \"001ddfc0-33e8-47de-9d4f-af35bbf7928c\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/auth-test-model-14a8d1-kserve-645b87cf5f-4qv95"
Sep 10 19:34:18.330832 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:18.330667    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/001ddfc0-33e8-47de-9d4f-af35bbf7928c-dshm\") pod \"auth-test-model-14a8d1-kserve-645b87cf5f-4qv95\" (UID: \"001ddfc0-33e8-47de-9d4f-af35bbf7928c\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/auth-test-model-14a8d1-kserve-645b87cf5f-4qv95"
Sep 10 19:34:18.330981 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:18.330837    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/001ddfc0-33e8-47de-9d4f-af35bbf7928c-tls-certs\") pod \"auth-test-model-14a8d1-kserve-645b87cf5f-4qv95\" (UID: \"001ddfc0-33e8-47de-9d4f-af35bbf7928c\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/auth-test-model-14a8d1-kserve-645b87cf5f-4qv95"
Sep 10 19:34:18.330981 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:18.330930    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bblcn\" (UniqueName: \"kubernetes.io/projected/001ddfc0-33e8-47de-9d4f-af35bbf7928c-kube-api-access-bblcn\") pod \"auth-test-model-14a8d1-kserve-645b87cf5f-4qv95\" (UID: \"001ddfc0-33e8-47de-9d4f-af35bbf7928c\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/auth-test-model-14a8d1-kserve-645b87cf5f-4qv95"
Sep 10 19:34:18.330981 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:18.330971    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/001ddfc0-33e8-47de-9d4f-af35bbf7928c-tmp-dir\") pod \"auth-test-model-14a8d1-kserve-645b87cf5f-4qv95\" (UID: \"001ddfc0-33e8-47de-9d4f-af35bbf7928c\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/auth-test-model-14a8d1-kserve-645b87cf5f-4qv95"
Sep 10 19:34:18.366776 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:18.365183    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Sep 10 19:34:18.366776 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:18.365265    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx" podUID="7bf44155-0d1f-4b0b-a6f6-bfe7706e0488" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Sep 10 19:34:18.432234 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:18.432190    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/001ddfc0-33e8-47de-9d4f-af35bbf7928c-home\") pod \"auth-test-model-14a8d1-kserve-645b87cf5f-4qv95\" (UID: \"001ddfc0-33e8-47de-9d4f-af35bbf7928c\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/auth-test-model-14a8d1-kserve-645b87cf5f-4qv95"
Sep 10 19:34:18.432436 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:18.432246    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/001ddfc0-33e8-47de-9d4f-af35bbf7928c-model-cache\") pod \"auth-test-model-14a8d1-kserve-645b87cf5f-4qv95\" (UID: \"001ddfc0-33e8-47de-9d4f-af35bbf7928c\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/auth-test-model-14a8d1-kserve-645b87cf5f-4qv95"
Sep 10 19:34:18.432436 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:18.432270    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/001ddfc0-33e8-47de-9d4f-af35bbf7928c-dshm\") pod \"auth-test-model-14a8d1-kserve-645b87cf5f-4qv95\" (UID: \"001ddfc0-33e8-47de-9d4f-af35bbf7928c\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/auth-test-model-14a8d1-kserve-645b87cf5f-4qv95"
Sep 10 19:34:18.432436 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:18.432331    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/001ddfc0-33e8-47de-9d4f-af35bbf7928c-tls-certs\") pod \"auth-test-model-14a8d1-kserve-645b87cf5f-4qv95\" (UID: \"001ddfc0-33e8-47de-9d4f-af35bbf7928c\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/auth-test-model-14a8d1-kserve-645b87cf5f-4qv95"
Sep 10 19:34:18.432436 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:18.432375    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bblcn\" (UniqueName: \"kubernetes.io/projected/001ddfc0-33e8-47de-9d4f-af35bbf7928c-kube-api-access-bblcn\") pod \"auth-test-model-14a8d1-kserve-645b87cf5f-4qv95\" (UID: \"001ddfc0-33e8-47de-9d4f-af35bbf7928c\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/auth-test-model-14a8d1-kserve-645b87cf5f-4qv95"
Sep 10 19:34:18.432436 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:18.432413    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/001ddfc0-33e8-47de-9d4f-af35bbf7928c-tmp-dir\") pod \"auth-test-model-14a8d1-kserve-645b87cf5f-4qv95\" (UID: \"001ddfc0-33e8-47de-9d4f-af35bbf7928c\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/auth-test-model-14a8d1-kserve-645b87cf5f-4qv95"
Sep 10 19:34:18.432738 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:18.432603    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/001ddfc0-33e8-47de-9d4f-af35bbf7928c-home\") pod \"auth-test-model-14a8d1-kserve-645b87cf5f-4qv95\" (UID: \"001ddfc0-33e8-47de-9d4f-af35bbf7928c\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/auth-test-model-14a8d1-kserve-645b87cf5f-4qv95"
Sep 10 19:34:18.432738 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:18.432659    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/001ddfc0-33e8-47de-9d4f-af35bbf7928c-model-cache\") pod \"auth-test-model-14a8d1-kserve-645b87cf5f-4qv95\" (UID: \"001ddfc0-33e8-47de-9d4f-af35bbf7928c\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/auth-test-model-14a8d1-kserve-645b87cf5f-4qv95"
Sep 10 19:34:18.432877 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:18.432808    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/001ddfc0-33e8-47de-9d4f-af35bbf7928c-tmp-dir\") pod \"auth-test-model-14a8d1-kserve-645b87cf5f-4qv95\" (UID: \"001ddfc0-33e8-47de-9d4f-af35bbf7928c\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/auth-test-model-14a8d1-kserve-645b87cf5f-4qv95"
Sep 10 19:34:18.434762 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:18.434725    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/001ddfc0-33e8-47de-9d4f-af35bbf7928c-dshm\") pod \"auth-test-model-14a8d1-kserve-645b87cf5f-4qv95\" (UID: \"001ddfc0-33e8-47de-9d4f-af35bbf7928c\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/auth-test-model-14a8d1-kserve-645b87cf5f-4qv95"
Sep 10 19:34:18.435051 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:18.435035    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/001ddfc0-33e8-47de-9d4f-af35bbf7928c-tls-certs\") pod \"auth-test-model-14a8d1-kserve-645b87cf5f-4qv95\" (UID: \"001ddfc0-33e8-47de-9d4f-af35bbf7928c\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/auth-test-model-14a8d1-kserve-645b87cf5f-4qv95"
Sep 10 19:34:18.440635 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:18.440609    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bblcn\" (UniqueName: \"kubernetes.io/projected/001ddfc0-33e8-47de-9d4f-af35bbf7928c-kube-api-access-bblcn\") pod \"auth-test-model-14a8d1-kserve-645b87cf5f-4qv95\" (UID: \"001ddfc0-33e8-47de-9d4f-af35bbf7928c\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/auth-test-model-14a8d1-kserve-645b87cf5f-4qv95"
Sep 10 19:34:18.601183 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:18.601102    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-14a8d1/auth-test-model-14a8d1-kserve-645b87cf5f-4qv95"
Sep 10 19:34:18.895015 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:18.894990    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14a8d1/auth-test-model-14a8d1-kserve-645b87cf5f-4qv95"]
Sep 10 19:34:18.896684 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:34:18.896653    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod001ddfc0_33e8_47de_9d4f_af35bbf7928c.slice/crio-b30d8632dc6779628419af4979878891671a672e57e4917c94858e6cb4fd843e WatchSource:0}: Error finding container b30d8632dc6779628419af4979878891671a672e57e4917c94858e6cb4fd843e: Status 404 returned error can't find the container with id b30d8632dc6779628419af4979878891671a672e57e4917c94858e6cb4fd843e
Sep 10 19:34:19.363195 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:19.363098    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Sep 10 19:34:19.363431 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:19.363403    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx" podUID="7bf44155-0d1f-4b0b-a6f6-bfe7706e0488" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Sep 10 19:34:19.910481 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:19.910450    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-14a8d1/auth-test-model-14a8d1-kserve-645b87cf5f-4qv95" event={"ID":"001ddfc0-33e8-47de-9d4f-af35bbf7928c","Type":"ContainerStarted","Data":"c61795227566beb386f0ba033ce2dfbebde4748fb735c402fd17f1313aec1457"}
Sep 10 19:34:19.910889 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:19.910491    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-14a8d1/auth-test-model-14a8d1-kserve-645b87cf5f-4qv95" event={"ID":"001ddfc0-33e8-47de-9d4f-af35bbf7928c","Type":"ContainerStarted","Data":"b30d8632dc6779628419af4979878891671a672e57e4917c94858e6cb4fd843e"}
Sep 10 19:34:19.929389 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:19.929335    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-14a8d1/auth-test-model-14a8d1-kserve-645b87cf5f-4qv95" podStartSLOduration=1.929317052 podStartE2EDuration="1.929317052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:34:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:34:19.926580764 +0000 UTC m=+1115.146429260" watchObservedRunningTime="2026-09-10 19:34:19.929317052 +0000 UTC m=+1115.149165543"
Sep 10 19:34:20.367464 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:20.367382    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:34:20.367701 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:20.367678    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:34:20.368349 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:20.368330    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:34:28.602166 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:28.602130    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-14a8d1/auth-test-model-14a8d1-kserve-645b87cf5f-4qv95"
Sep 10 19:34:28.602166 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:28.602175    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-14a8d1/auth-test-model-14a8d1-kserve-645b87cf5f-4qv95"
Sep 10 19:34:28.614371 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:28.614338    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-14a8d1/auth-test-model-14a8d1-kserve-645b87cf5f-4qv95"
Sep 10 19:34:28.979444 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:28.979407    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-14a8d1/auth-test-model-14a8d1-kserve-645b87cf5f-4qv95"
Sep 10 19:34:37.217238 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:37.217199    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv"]
Sep 10 19:34:37.222146 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:37.222125    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv"
Sep 10 19:34:37.224868 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:37.224844    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-c615f98e-serving-cert\""
Sep 10 19:34:37.241367 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:37.241343    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv"]
Sep 10 19:34:37.327624 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:37.327589    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d62b7953-3748-4eb1-ba18-af06fed86281-openshift-service-ca\") pod \"maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv\" (UID: \"d62b7953-3748-4eb1-ba18-af06fed86281\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv"
Sep 10 19:34:37.327802 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:37.327679    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d62b7953-3748-4eb1-ba18-af06fed86281-metrics-certs\") pod \"maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv\" (UID: \"d62b7953-3748-4eb1-ba18-af06fed86281\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv"
Sep 10 19:34:37.327802 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:37.327705    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d62b7953-3748-4eb1-ba18-af06fed86281-maas-api-tls\") pod \"maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv\" (UID: \"d62b7953-3748-4eb1-ba18-af06fed86281\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv"
Sep 10 19:34:37.327906 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:37.327821    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g65dv\" (UniqueName: \"kubernetes.io/projected/d62b7953-3748-4eb1-ba18-af06fed86281-kube-api-access-g65dv\") pod \"maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv\" (UID: \"d62b7953-3748-4eb1-ba18-af06fed86281\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv"
Sep 10 19:34:37.429113 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:37.429041    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g65dv\" (UniqueName: \"kubernetes.io/projected/d62b7953-3748-4eb1-ba18-af06fed86281-kube-api-access-g65dv\") pod \"maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv\" (UID: \"d62b7953-3748-4eb1-ba18-af06fed86281\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv"
Sep 10 19:34:37.429113 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:37.429099    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d62b7953-3748-4eb1-ba18-af06fed86281-openshift-service-ca\") pod \"maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv\" (UID: \"d62b7953-3748-4eb1-ba18-af06fed86281\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv"
Sep 10 19:34:37.429426 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:37.429170    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d62b7953-3748-4eb1-ba18-af06fed86281-metrics-certs\") pod \"maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv\" (UID: \"d62b7953-3748-4eb1-ba18-af06fed86281\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv"
Sep 10 19:34:37.429426 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:37.429199    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d62b7953-3748-4eb1-ba18-af06fed86281-maas-api-tls\") pod \"maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv\" (UID: \"d62b7953-3748-4eb1-ba18-af06fed86281\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv"
Sep 10 19:34:37.429962 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:37.429934    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d62b7953-3748-4eb1-ba18-af06fed86281-openshift-service-ca\") pod \"maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv\" (UID: \"d62b7953-3748-4eb1-ba18-af06fed86281\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv"
Sep 10 19:34:37.431895 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:37.431873    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d62b7953-3748-4eb1-ba18-af06fed86281-metrics-certs\") pod \"maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv\" (UID: \"d62b7953-3748-4eb1-ba18-af06fed86281\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv"
Sep 10 19:34:37.432001 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:37.431952    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d62b7953-3748-4eb1-ba18-af06fed86281-maas-api-tls\") pod \"maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv\" (UID: \"d62b7953-3748-4eb1-ba18-af06fed86281\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv"
Sep 10 19:34:37.440906 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:37.440886    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g65dv\" (UniqueName: \"kubernetes.io/projected/d62b7953-3748-4eb1-ba18-af06fed86281-kube-api-access-g65dv\") pod \"maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv\" (UID: \"d62b7953-3748-4eb1-ba18-af06fed86281\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv"
Sep 10 19:34:37.536723 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:37.536644    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv"
Sep 10 19:34:37.813674 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:37.813643    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv"]
Sep 10 19:34:37.814406 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:34:37.814375    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd62b7953_3748_4eb1_ba18_af06fed86281.slice/crio-d74da46d78d1500b9f2c6c8392beb31b085d732aef217a6f2624406f1b520fc6 WatchSource:0}: Error finding container d74da46d78d1500b9f2c6c8392beb31b085d732aef217a6f2624406f1b520fc6: Status 404 returned error can't find the container with id d74da46d78d1500b9f2c6c8392beb31b085d732aef217a6f2624406f1b520fc6
Sep 10 19:34:38.023918 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:38.023880    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv" event={"ID":"d62b7953-3748-4eb1-ba18-af06fed86281","Type":"ContainerStarted","Data":"d74da46d78d1500b9f2c6c8392beb31b085d732aef217a6f2624406f1b520fc6"}
Sep 10 19:34:39.031633 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:39.031594    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv" event={"ID":"d62b7953-3748-4eb1-ba18-af06fed86281","Type":"ContainerStarted","Data":"af7b57df2d57e4998073357fe38e6fcf5965dff500f10ee670d41ebd2921f450"}
Sep 10 19:34:39.031997 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:39.031661    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv"
Sep 10 19:34:39.048958 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:39.048912    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv" podStartSLOduration=1.709002301 podStartE2EDuration="2.048897775s" totalImagesPullingTime="339.895474ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:34:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:34:39.047241773 +0000 UTC m=+1134.267090274" watchObservedRunningTime="2026-09-10 19:34:39.048897775 +0000 UTC m=+1134.268746255"
Sep 10 19:34:45.043841 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:34:45.043813    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv"
Sep 10 19:35:45.240276 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:35:45.240248    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bmdg9_9817270c-8629-44c4-ab8f-f3982e1f0e2f/console-operator/1.log"
Sep 10 19:35:45.244028 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:35:45.244005    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bmdg9_9817270c-8629-44c4-ab8f-f3982e1f0e2f/console-operator/1.log"
Sep 10 19:35:45.245629 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:35:45.245607    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8r66s_2826f8ec-78f6-4cab-879e-47a21854ed5a/ovn-acl-logging/0.log"
Sep 10 19:35:45.249205 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:35:45.249184    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8r66s_2826f8ec-78f6-4cab-879e-47a21854ed5a/ovn-acl-logging/0.log"
Sep 10 19:35:55.898686 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:35:55.898651    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv"]
Sep 10 19:35:55.899227 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:35:55.898946    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv" podUID="d62b7953-3748-4eb1-ba18-af06fed86281" containerName="maas-api" containerID="cri-o://af7b57df2d57e4998073357fe38e6fcf5965dff500f10ee670d41ebd2921f450" gracePeriod=30
Sep 10 19:35:56.167049 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:35:56.166980    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv"
Sep 10 19:35:56.223987 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:35:56.223939    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d62b7953-3748-4eb1-ba18-af06fed86281-metrics-certs\") pod \"d62b7953-3748-4eb1-ba18-af06fed86281\" (UID: \"d62b7953-3748-4eb1-ba18-af06fed86281\") "
Sep 10 19:35:56.224175 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:35:56.224044    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g65dv\" (UniqueName: \"kubernetes.io/projected/d62b7953-3748-4eb1-ba18-af06fed86281-kube-api-access-g65dv\") pod \"d62b7953-3748-4eb1-ba18-af06fed86281\" (UID: \"d62b7953-3748-4eb1-ba18-af06fed86281\") "
Sep 10 19:35:56.224175 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:35:56.224126    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d62b7953-3748-4eb1-ba18-af06fed86281-maas-api-tls\") pod \"d62b7953-3748-4eb1-ba18-af06fed86281\" (UID: \"d62b7953-3748-4eb1-ba18-af06fed86281\") "
Sep 10 19:35:56.224298 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:35:56.224260    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d62b7953-3748-4eb1-ba18-af06fed86281-openshift-service-ca\") pod \"d62b7953-3748-4eb1-ba18-af06fed86281\" (UID: \"d62b7953-3748-4eb1-ba18-af06fed86281\") "
Sep 10 19:35:56.224842 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:35:56.224799    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d62b7953-3748-4eb1-ba18-af06fed86281-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d62b7953-3748-4eb1-ba18-af06fed86281" (UID: "d62b7953-3748-4eb1-ba18-af06fed86281"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:35:56.226720 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:35:56.226695    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d62b7953-3748-4eb1-ba18-af06fed86281-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d62b7953-3748-4eb1-ba18-af06fed86281" (UID: "d62b7953-3748-4eb1-ba18-af06fed86281"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:35:56.226964 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:35:56.226945    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d62b7953-3748-4eb1-ba18-af06fed86281-kube-api-access-g65dv" (OuterVolumeSpecName: "kube-api-access-g65dv") pod "d62b7953-3748-4eb1-ba18-af06fed86281" (UID: "d62b7953-3748-4eb1-ba18-af06fed86281"). InnerVolumeSpecName "kube-api-access-g65dv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:35:56.227043 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:35:56.226991    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d62b7953-3748-4eb1-ba18-af06fed86281-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d62b7953-3748-4eb1-ba18-af06fed86281" (UID: "d62b7953-3748-4eb1-ba18-af06fed86281"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:35:56.326370 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:35:56.326331    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d62b7953-3748-4eb1-ba18-af06fed86281-maas-api-tls\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:35:56.326370 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:35:56.326369    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d62b7953-3748-4eb1-ba18-af06fed86281-openshift-service-ca\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:35:56.326627 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:35:56.326385    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d62b7953-3748-4eb1-ba18-af06fed86281-metrics-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:35:56.326627 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:35:56.326399    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g65dv\" (UniqueName: \"kubernetes.io/projected/d62b7953-3748-4eb1-ba18-af06fed86281-kube-api-access-g65dv\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:35:56.520347 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:35:56.520251    2579 generic.go:356] "Generic (PLEG): container finished" podID="d62b7953-3748-4eb1-ba18-af06fed86281" containerID="af7b57df2d57e4998073357fe38e6fcf5965dff500f10ee670d41ebd2921f450" exitCode=0
Sep 10 19:35:56.520347 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:35:56.520316    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv"
Sep 10 19:35:56.520347 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:35:56.520332    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv" event={"ID":"d62b7953-3748-4eb1-ba18-af06fed86281","Type":"ContainerDied","Data":"af7b57df2d57e4998073357fe38e6fcf5965dff500f10ee670d41ebd2921f450"}
Sep 10 19:35:56.520632 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:35:56.520408    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv" event={"ID":"d62b7953-3748-4eb1-ba18-af06fed86281","Type":"ContainerDied","Data":"d74da46d78d1500b9f2c6c8392beb31b085d732aef217a6f2624406f1b520fc6"}
Sep 10 19:35:56.520632 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:35:56.520430    2579 scope.go:117] "RemoveContainer" containerID="af7b57df2d57e4998073357fe38e6fcf5965dff500f10ee670d41ebd2921f450"
Sep 10 19:35:56.535884 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:35:56.535596    2579 scope.go:117] "RemoveContainer" containerID="af7b57df2d57e4998073357fe38e6fcf5965dff500f10ee670d41ebd2921f450"
Sep 10 19:35:56.536071 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:35:56.536043    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af7b57df2d57e4998073357fe38e6fcf5965dff500f10ee670d41ebd2921f450\": container with ID starting with af7b57df2d57e4998073357fe38e6fcf5965dff500f10ee670d41ebd2921f450 not found: ID does not exist" containerID="af7b57df2d57e4998073357fe38e6fcf5965dff500f10ee670d41ebd2921f450"
Sep 10 19:35:56.536276 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:35:56.536228    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af7b57df2d57e4998073357fe38e6fcf5965dff500f10ee670d41ebd2921f450"} err="failed to get container status \"af7b57df2d57e4998073357fe38e6fcf5965dff500f10ee670d41ebd2921f450\": rpc error: code = NotFound desc = could not find container \"af7b57df2d57e4998073357fe38e6fcf5965dff500f10ee670d41ebd2921f450\": container with ID starting with af7b57df2d57e4998073357fe38e6fcf5965dff500f10ee670d41ebd2921f450 not found: ID does not exist"
Sep 10 19:35:56.559348 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:35:56.559316    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv"]
Sep 10 19:35:56.565284 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:35:56.565257    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c615f98e-6b66b7d788-nvqnv"]
Sep 10 19:35:57.240621 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:35:57.240571    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d62b7953-3748-4eb1-ba18-af06fed86281" path="/var/lib/kubelet/pods/d62b7953-3748-4eb1-ba18-af06fed86281/volumes"
Sep 10 19:37:12.692305 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:12.692260    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14a8d1/auth-test-model-14a8d1-kserve-645b87cf5f-4qv95"]
Sep 10 19:37:27.714466 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:27.714419    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-14a8d1/auth-test-model-14a8d1-kserve-645b87cf5f-4qv95" podUID="001ddfc0-33e8-47de-9d4f-af35bbf7928c" containerName="main" containerID="cri-o://c61795227566beb386f0ba033ce2dfbebde4748fb735c402fd17f1313aec1457" gracePeriod=45
Sep 10 19:37:27.971699 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:27.971632    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-14a8d1/auth-test-model-14a8d1-kserve-645b87cf5f-4qv95"
Sep 10 19:37:28.069325 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:28.069292    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/001ddfc0-33e8-47de-9d4f-af35bbf7928c-tls-certs\") pod \"001ddfc0-33e8-47de-9d4f-af35bbf7928c\" (UID: \"001ddfc0-33e8-47de-9d4f-af35bbf7928c\") "
Sep 10 19:37:28.069501 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:28.069377    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/001ddfc0-33e8-47de-9d4f-af35bbf7928c-dshm\") pod \"001ddfc0-33e8-47de-9d4f-af35bbf7928c\" (UID: \"001ddfc0-33e8-47de-9d4f-af35bbf7928c\") "
Sep 10 19:37:28.069501 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:28.069455    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bblcn\" (UniqueName: \"kubernetes.io/projected/001ddfc0-33e8-47de-9d4f-af35bbf7928c-kube-api-access-bblcn\") pod \"001ddfc0-33e8-47de-9d4f-af35bbf7928c\" (UID: \"001ddfc0-33e8-47de-9d4f-af35bbf7928c\") "
Sep 10 19:37:28.069501 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:28.069486    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/001ddfc0-33e8-47de-9d4f-af35bbf7928c-model-cache\") pod \"001ddfc0-33e8-47de-9d4f-af35bbf7928c\" (UID: \"001ddfc0-33e8-47de-9d4f-af35bbf7928c\") "
Sep 10 19:37:28.069794 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:28.069522    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/001ddfc0-33e8-47de-9d4f-af35bbf7928c-tmp-dir\") pod \"001ddfc0-33e8-47de-9d4f-af35bbf7928c\" (UID: \"001ddfc0-33e8-47de-9d4f-af35bbf7928c\") "
Sep 10 19:37:28.069794 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:28.069560    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/001ddfc0-33e8-47de-9d4f-af35bbf7928c-home\") pod \"001ddfc0-33e8-47de-9d4f-af35bbf7928c\" (UID: \"001ddfc0-33e8-47de-9d4f-af35bbf7928c\") "
Sep 10 19:37:28.069901 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:28.069787    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/001ddfc0-33e8-47de-9d4f-af35bbf7928c-model-cache" (OuterVolumeSpecName: "model-cache") pod "001ddfc0-33e8-47de-9d4f-af35bbf7928c" (UID: "001ddfc0-33e8-47de-9d4f-af35bbf7928c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:37:28.069901 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:28.069806    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/001ddfc0-33e8-47de-9d4f-af35bbf7928c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "001ddfc0-33e8-47de-9d4f-af35bbf7928c" (UID: "001ddfc0-33e8-47de-9d4f-af35bbf7928c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:37:28.070019 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:28.069898    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/001ddfc0-33e8-47de-9d4f-af35bbf7928c-home" (OuterVolumeSpecName: "home") pod "001ddfc0-33e8-47de-9d4f-af35bbf7928c" (UID: "001ddfc0-33e8-47de-9d4f-af35bbf7928c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:37:28.070019 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:28.069994    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/001ddfc0-33e8-47de-9d4f-af35bbf7928c-home\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:37:28.070019 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:28.070005    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/001ddfc0-33e8-47de-9d4f-af35bbf7928c-model-cache\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:37:28.070019 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:28.070016    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/001ddfc0-33e8-47de-9d4f-af35bbf7928c-tmp-dir\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:37:28.071782 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:28.071761    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/001ddfc0-33e8-47de-9d4f-af35bbf7928c-dshm" (OuterVolumeSpecName: "dshm") pod "001ddfc0-33e8-47de-9d4f-af35bbf7928c" (UID: "001ddfc0-33e8-47de-9d4f-af35bbf7928c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:37:28.071901 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:28.071817    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/001ddfc0-33e8-47de-9d4f-af35bbf7928c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "001ddfc0-33e8-47de-9d4f-af35bbf7928c" (UID: "001ddfc0-33e8-47de-9d4f-af35bbf7928c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:37:28.071901 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:28.071860    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/001ddfc0-33e8-47de-9d4f-af35bbf7928c-kube-api-access-bblcn" (OuterVolumeSpecName: "kube-api-access-bblcn") pod "001ddfc0-33e8-47de-9d4f-af35bbf7928c" (UID: "001ddfc0-33e8-47de-9d4f-af35bbf7928c"). InnerVolumeSpecName "kube-api-access-bblcn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:37:28.088190 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:28.088025    2579 generic.go:356] "Generic (PLEG): container finished" podID="001ddfc0-33e8-47de-9d4f-af35bbf7928c" containerID="c61795227566beb386f0ba033ce2dfbebde4748fb735c402fd17f1313aec1457" exitCode=0
Sep 10 19:37:28.088339 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:28.088164    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-14a8d1/auth-test-model-14a8d1-kserve-645b87cf5f-4qv95" event={"ID":"001ddfc0-33e8-47de-9d4f-af35bbf7928c","Type":"ContainerDied","Data":"c61795227566beb386f0ba033ce2dfbebde4748fb735c402fd17f1313aec1457"}
Sep 10 19:37:28.088339 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:28.088316    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-14a8d1/auth-test-model-14a8d1-kserve-645b87cf5f-4qv95" event={"ID":"001ddfc0-33e8-47de-9d4f-af35bbf7928c","Type":"ContainerDied","Data":"b30d8632dc6779628419af4979878891671a672e57e4917c94858e6cb4fd843e"}
Sep 10 19:37:28.088428 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:28.088174    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-14a8d1/auth-test-model-14a8d1-kserve-645b87cf5f-4qv95"
Sep 10 19:37:28.088428 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:28.088343    2579 scope.go:117] "RemoveContainer" containerID="c61795227566beb386f0ba033ce2dfbebde4748fb735c402fd17f1313aec1457"
Sep 10 19:37:28.099284 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:28.099265    2579 scope.go:117] "RemoveContainer" containerID="c61795227566beb386f0ba033ce2dfbebde4748fb735c402fd17f1313aec1457"
Sep 10 19:37:28.099597 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:37:28.099579    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c61795227566beb386f0ba033ce2dfbebde4748fb735c402fd17f1313aec1457\": container with ID starting with c61795227566beb386f0ba033ce2dfbebde4748fb735c402fd17f1313aec1457 not found: ID does not exist" containerID="c61795227566beb386f0ba033ce2dfbebde4748fb735c402fd17f1313aec1457"
Sep 10 19:37:28.099640 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:28.099607    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c61795227566beb386f0ba033ce2dfbebde4748fb735c402fd17f1313aec1457"} err="failed to get container status \"c61795227566beb386f0ba033ce2dfbebde4748fb735c402fd17f1313aec1457\": rpc error: code = NotFound desc = could not find container \"c61795227566beb386f0ba033ce2dfbebde4748fb735c402fd17f1313aec1457\": container with ID starting with c61795227566beb386f0ba033ce2dfbebde4748fb735c402fd17f1313aec1457 not found: ID does not exist"
Sep 10 19:37:28.111952 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:28.111925    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14a8d1/auth-test-model-14a8d1-kserve-645b87cf5f-4qv95"]
Sep 10 19:37:28.115865 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:28.115841    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14a8d1/auth-test-model-14a8d1-kserve-645b87cf5f-4qv95"]
Sep 10 19:37:28.170810 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:28.170730    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bblcn\" (UniqueName: \"kubernetes.io/projected/001ddfc0-33e8-47de-9d4f-af35bbf7928c-kube-api-access-bblcn\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:37:28.170810 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:28.170797    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/001ddfc0-33e8-47de-9d4f-af35bbf7928c-tls-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:37:28.170810 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:28.170815    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/001ddfc0-33e8-47de-9d4f-af35bbf7928c-dshm\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:37:29.239809 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:29.239772    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="001ddfc0-33e8-47de-9d4f-af35bbf7928c" path="/var/lib/kubelet/pods/001ddfc0-33e8-47de-9d4f-af35bbf7928c/volumes"
Sep 10 19:37:53.629938 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:53.629890    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-439275-6796f6fcb-dr5pm"]
Sep 10 19:37:53.630689 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:53.630669    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="001ddfc0-33e8-47de-9d4f-af35bbf7928c" containerName="main"
Sep 10 19:37:53.630740 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:53.630695    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="001ddfc0-33e8-47de-9d4f-af35bbf7928c" containerName="main"
Sep 10 19:37:53.630740 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:53.630717    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d62b7953-3748-4eb1-ba18-af06fed86281" containerName="maas-api"
Sep 10 19:37:53.630740 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:53.630726    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d62b7953-3748-4eb1-ba18-af06fed86281" containerName="maas-api"
Sep 10 19:37:53.630870 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:53.630862    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="001ddfc0-33e8-47de-9d4f-af35bbf7928c" containerName="main"
Sep 10 19:37:53.630906 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:53.630883    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d62b7953-3748-4eb1-ba18-af06fed86281" containerName="maas-api"
Sep 10 19:37:53.636274 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:53.636251    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-439275-6796f6fcb-dr5pm"
Sep 10 19:37:53.638830 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:53.638546    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-439275-serving-cert\""
Sep 10 19:37:53.642165 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:53.642126    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-439275-6796f6fcb-dr5pm"]
Sep 10 19:37:53.730168 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:53.730126    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/381e2ae5-0ff2-4752-86fd-6b4b420a7a20-maas-api-tls\") pod \"maas-api-e2e-api-a-439275-6796f6fcb-dr5pm\" (UID: \"381e2ae5-0ff2-4752-86fd-6b4b420a7a20\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-439275-6796f6fcb-dr5pm"
Sep 10 19:37:53.730382 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:53.730193    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xk7lc\" (UniqueName: \"kubernetes.io/projected/381e2ae5-0ff2-4752-86fd-6b4b420a7a20-kube-api-access-xk7lc\") pod \"maas-api-e2e-api-a-439275-6796f6fcb-dr5pm\" (UID: \"381e2ae5-0ff2-4752-86fd-6b4b420a7a20\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-439275-6796f6fcb-dr5pm"
Sep 10 19:37:53.730382 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:53.730251    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/381e2ae5-0ff2-4752-86fd-6b4b420a7a20-metrics-certs\") pod \"maas-api-e2e-api-a-439275-6796f6fcb-dr5pm\" (UID: \"381e2ae5-0ff2-4752-86fd-6b4b420a7a20\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-439275-6796f6fcb-dr5pm"
Sep 10 19:37:53.730382 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:53.730326    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/381e2ae5-0ff2-4752-86fd-6b4b420a7a20-openshift-service-ca\") pod \"maas-api-e2e-api-a-439275-6796f6fcb-dr5pm\" (UID: \"381e2ae5-0ff2-4752-86fd-6b4b420a7a20\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-439275-6796f6fcb-dr5pm"
Sep 10 19:37:53.831054 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:53.831022    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/381e2ae5-0ff2-4752-86fd-6b4b420a7a20-metrics-certs\") pod \"maas-api-e2e-api-a-439275-6796f6fcb-dr5pm\" (UID: \"381e2ae5-0ff2-4752-86fd-6b4b420a7a20\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-439275-6796f6fcb-dr5pm"
Sep 10 19:37:53.831220 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:53.831075    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/381e2ae5-0ff2-4752-86fd-6b4b420a7a20-openshift-service-ca\") pod \"maas-api-e2e-api-a-439275-6796f6fcb-dr5pm\" (UID: \"381e2ae5-0ff2-4752-86fd-6b4b420a7a20\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-439275-6796f6fcb-dr5pm"
Sep 10 19:37:53.831220 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:53.831156    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/381e2ae5-0ff2-4752-86fd-6b4b420a7a20-maas-api-tls\") pod \"maas-api-e2e-api-a-439275-6796f6fcb-dr5pm\" (UID: \"381e2ae5-0ff2-4752-86fd-6b4b420a7a20\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-439275-6796f6fcb-dr5pm"
Sep 10 19:37:53.831334 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:53.831219    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xk7lc\" (UniqueName: \"kubernetes.io/projected/381e2ae5-0ff2-4752-86fd-6b4b420a7a20-kube-api-access-xk7lc\") pod \"maas-api-e2e-api-a-439275-6796f6fcb-dr5pm\" (UID: \"381e2ae5-0ff2-4752-86fd-6b4b420a7a20\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-439275-6796f6fcb-dr5pm"
Sep 10 19:37:53.831994 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:53.831965    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/381e2ae5-0ff2-4752-86fd-6b4b420a7a20-openshift-service-ca\") pod \"maas-api-e2e-api-a-439275-6796f6fcb-dr5pm\" (UID: \"381e2ae5-0ff2-4752-86fd-6b4b420a7a20\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-439275-6796f6fcb-dr5pm"
Sep 10 19:37:53.833876 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:53.833848    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/381e2ae5-0ff2-4752-86fd-6b4b420a7a20-maas-api-tls\") pod \"maas-api-e2e-api-a-439275-6796f6fcb-dr5pm\" (UID: \"381e2ae5-0ff2-4752-86fd-6b4b420a7a20\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-439275-6796f6fcb-dr5pm"
Sep 10 19:37:53.834002 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:53.833954    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/381e2ae5-0ff2-4752-86fd-6b4b420a7a20-metrics-certs\") pod \"maas-api-e2e-api-a-439275-6796f6fcb-dr5pm\" (UID: \"381e2ae5-0ff2-4752-86fd-6b4b420a7a20\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-439275-6796f6fcb-dr5pm"
Sep 10 19:37:53.838613 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:53.838582    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xk7lc\" (UniqueName: \"kubernetes.io/projected/381e2ae5-0ff2-4752-86fd-6b4b420a7a20-kube-api-access-xk7lc\") pod \"maas-api-e2e-api-a-439275-6796f6fcb-dr5pm\" (UID: \"381e2ae5-0ff2-4752-86fd-6b4b420a7a20\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-439275-6796f6fcb-dr5pm"
Sep 10 19:37:53.952543 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:53.952498    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-439275-6796f6fcb-dr5pm"
Sep 10 19:37:54.234677 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:54.234652    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-439275-6796f6fcb-dr5pm"]
Sep 10 19:37:54.236677 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:37:54.236651    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod381e2ae5_0ff2_4752_86fd_6b4b420a7a20.slice/crio-dfce2cf3cc1a2e07f88eed34a2d4fbe88494bdba86c2c3d445bb7c03ac7bbf6d WatchSource:0}: Error finding container dfce2cf3cc1a2e07f88eed34a2d4fbe88494bdba86c2c3d445bb7c03ac7bbf6d: Status 404 returned error can't find the container with id dfce2cf3cc1a2e07f88eed34a2d4fbe88494bdba86c2c3d445bb7c03ac7bbf6d
Sep 10 19:37:54.238149 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:54.238129    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 19:37:54.247282 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:54.247253    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-439275-6796f6fcb-dr5pm" event={"ID":"381e2ae5-0ff2-4752-86fd-6b4b420a7a20","Type":"ContainerStarted","Data":"dfce2cf3cc1a2e07f88eed34a2d4fbe88494bdba86c2c3d445bb7c03ac7bbf6d"}
Sep 10 19:37:55.254811 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:55.254775    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-439275-6796f6fcb-dr5pm" event={"ID":"381e2ae5-0ff2-4752-86fd-6b4b420a7a20","Type":"ContainerStarted","Data":"9e6962ea150918fcb66e6b3298226743ba479e62f21359f06319d78c94e1b46c"}
Sep 10 19:37:55.255196 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:55.254934    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-439275-6796f6fcb-dr5pm"
Sep 10 19:37:55.272314 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:37:55.272259    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-439275-6796f6fcb-dr5pm" podStartSLOduration=1.922378559 podStartE2EDuration="2.272241887s" totalImagesPullingTime="349.863328ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:37:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:37:55.271202469 +0000 UTC m=+1330.491050953" watchObservedRunningTime="2026-09-10 19:37:55.272241887 +0000 UTC m=+1330.492090368"
Sep 10 19:38:01.267710 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:01.267677    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-439275-6796f6fcb-dr5pm"
Sep 10 19:38:21.765090 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:21.764801    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-cab4ee/sub-test-model-cab4ee-kserve-84566d678c-gc2bh"]
Sep 10 19:38:21.768811 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:21.768789    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-cab4ee/sub-test-model-cab4ee-kserve-84566d678c-gc2bh"
Sep 10 19:38:21.771346 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:21.771325    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-cab4ee\"/\"openshift-service-ca.crt\""
Sep 10 19:38:21.771475 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:21.771328    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-cab4ee\"/\"default-dockercfg-z99gm\""
Sep 10 19:38:21.771475 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:21.771366    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-cab4ee\"/\"sub-test-model-cab4ee-kserve-self-signed-certs\""
Sep 10 19:38:21.771475 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:21.771356    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-cab4ee\"/\"kube-root-ca.crt\""
Sep 10 19:38:21.777741 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:21.777720    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-cab4ee/sub-test-model-cab4ee-kserve-84566d678c-gc2bh"]
Sep 10 19:38:21.807676 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:21.807654    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d24b496-c358-4c7d-a44b-b9f1c0059563-tls-certs\") pod \"sub-test-model-cab4ee-kserve-84566d678c-gc2bh\" (UID: \"1d24b496-c358-4c7d-a44b-b9f1c0059563\") " pod="ai-tenant-e2e-shared-b-w3-cab4ee/sub-test-model-cab4ee-kserve-84566d678c-gc2bh"
Sep 10 19:38:21.807810 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:21.807692    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n492r\" (UniqueName: \"kubernetes.io/projected/1d24b496-c358-4c7d-a44b-b9f1c0059563-kube-api-access-n492r\") pod \"sub-test-model-cab4ee-kserve-84566d678c-gc2bh\" (UID: \"1d24b496-c358-4c7d-a44b-b9f1c0059563\") " pod="ai-tenant-e2e-shared-b-w3-cab4ee/sub-test-model-cab4ee-kserve-84566d678c-gc2bh"
Sep 10 19:38:21.807810 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:21.807720    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d24b496-c358-4c7d-a44b-b9f1c0059563-tmp-dir\") pod \"sub-test-model-cab4ee-kserve-84566d678c-gc2bh\" (UID: \"1d24b496-c358-4c7d-a44b-b9f1c0059563\") " pod="ai-tenant-e2e-shared-b-w3-cab4ee/sub-test-model-cab4ee-kserve-84566d678c-gc2bh"
Sep 10 19:38:21.807810 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:21.807793    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d24b496-c358-4c7d-a44b-b9f1c0059563-dshm\") pod \"sub-test-model-cab4ee-kserve-84566d678c-gc2bh\" (UID: \"1d24b496-c358-4c7d-a44b-b9f1c0059563\") " pod="ai-tenant-e2e-shared-b-w3-cab4ee/sub-test-model-cab4ee-kserve-84566d678c-gc2bh"
Sep 10 19:38:21.807929 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:21.807849    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d24b496-c358-4c7d-a44b-b9f1c0059563-home\") pod \"sub-test-model-cab4ee-kserve-84566d678c-gc2bh\" (UID: \"1d24b496-c358-4c7d-a44b-b9f1c0059563\") " pod="ai-tenant-e2e-shared-b-w3-cab4ee/sub-test-model-cab4ee-kserve-84566d678c-gc2bh"
Sep 10 19:38:21.807929 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:21.807870    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d24b496-c358-4c7d-a44b-b9f1c0059563-model-cache\") pod \"sub-test-model-cab4ee-kserve-84566d678c-gc2bh\" (UID: \"1d24b496-c358-4c7d-a44b-b9f1c0059563\") " pod="ai-tenant-e2e-shared-b-w3-cab4ee/sub-test-model-cab4ee-kserve-84566d678c-gc2bh"
Sep 10 19:38:21.908724 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:21.908690    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d24b496-c358-4c7d-a44b-b9f1c0059563-tls-certs\") pod \"sub-test-model-cab4ee-kserve-84566d678c-gc2bh\" (UID: \"1d24b496-c358-4c7d-a44b-b9f1c0059563\") " pod="ai-tenant-e2e-shared-b-w3-cab4ee/sub-test-model-cab4ee-kserve-84566d678c-gc2bh"
Sep 10 19:38:21.908905 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:21.908729    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n492r\" (UniqueName: \"kubernetes.io/projected/1d24b496-c358-4c7d-a44b-b9f1c0059563-kube-api-access-n492r\") pod \"sub-test-model-cab4ee-kserve-84566d678c-gc2bh\" (UID: \"1d24b496-c358-4c7d-a44b-b9f1c0059563\") " pod="ai-tenant-e2e-shared-b-w3-cab4ee/sub-test-model-cab4ee-kserve-84566d678c-gc2bh"
Sep 10 19:38:21.908905 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:21.908793    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d24b496-c358-4c7d-a44b-b9f1c0059563-tmp-dir\") pod \"sub-test-model-cab4ee-kserve-84566d678c-gc2bh\" (UID: \"1d24b496-c358-4c7d-a44b-b9f1c0059563\") " pod="ai-tenant-e2e-shared-b-w3-cab4ee/sub-test-model-cab4ee-kserve-84566d678c-gc2bh"
Sep 10 19:38:21.908905 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:21.908823    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d24b496-c358-4c7d-a44b-b9f1c0059563-dshm\") pod \"sub-test-model-cab4ee-kserve-84566d678c-gc2bh\" (UID: \"1d24b496-c358-4c7d-a44b-b9f1c0059563\") " pod="ai-tenant-e2e-shared-b-w3-cab4ee/sub-test-model-cab4ee-kserve-84566d678c-gc2bh"
Sep 10 19:38:21.908905 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:21.908893    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d24b496-c358-4c7d-a44b-b9f1c0059563-home\") pod \"sub-test-model-cab4ee-kserve-84566d678c-gc2bh\" (UID: \"1d24b496-c358-4c7d-a44b-b9f1c0059563\") " pod="ai-tenant-e2e-shared-b-w3-cab4ee/sub-test-model-cab4ee-kserve-84566d678c-gc2bh"
Sep 10 19:38:21.909074 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:21.908915    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d24b496-c358-4c7d-a44b-b9f1c0059563-model-cache\") pod \"sub-test-model-cab4ee-kserve-84566d678c-gc2bh\" (UID: \"1d24b496-c358-4c7d-a44b-b9f1c0059563\") " pod="ai-tenant-e2e-shared-b-w3-cab4ee/sub-test-model-cab4ee-kserve-84566d678c-gc2bh"
Sep 10 19:38:21.909183 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:21.909160    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d24b496-c358-4c7d-a44b-b9f1c0059563-tmp-dir\") pod \"sub-test-model-cab4ee-kserve-84566d678c-gc2bh\" (UID: \"1d24b496-c358-4c7d-a44b-b9f1c0059563\") " pod="ai-tenant-e2e-shared-b-w3-cab4ee/sub-test-model-cab4ee-kserve-84566d678c-gc2bh"
Sep 10 19:38:21.909243 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:21.909231    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d24b496-c358-4c7d-a44b-b9f1c0059563-home\") pod \"sub-test-model-cab4ee-kserve-84566d678c-gc2bh\" (UID: \"1d24b496-c358-4c7d-a44b-b9f1c0059563\") " pod="ai-tenant-e2e-shared-b-w3-cab4ee/sub-test-model-cab4ee-kserve-84566d678c-gc2bh"
Sep 10 19:38:21.909302 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:21.909272    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d24b496-c358-4c7d-a44b-b9f1c0059563-model-cache\") pod \"sub-test-model-cab4ee-kserve-84566d678c-gc2bh\" (UID: \"1d24b496-c358-4c7d-a44b-b9f1c0059563\") " pod="ai-tenant-e2e-shared-b-w3-cab4ee/sub-test-model-cab4ee-kserve-84566d678c-gc2bh"
Sep 10 19:38:21.911092 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:21.911074    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d24b496-c358-4c7d-a44b-b9f1c0059563-dshm\") pod \"sub-test-model-cab4ee-kserve-84566d678c-gc2bh\" (UID: \"1d24b496-c358-4c7d-a44b-b9f1c0059563\") " pod="ai-tenant-e2e-shared-b-w3-cab4ee/sub-test-model-cab4ee-kserve-84566d678c-gc2bh"
Sep 10 19:38:21.911500 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:21.911481    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d24b496-c358-4c7d-a44b-b9f1c0059563-tls-certs\") pod \"sub-test-model-cab4ee-kserve-84566d678c-gc2bh\" (UID: \"1d24b496-c358-4c7d-a44b-b9f1c0059563\") " pod="ai-tenant-e2e-shared-b-w3-cab4ee/sub-test-model-cab4ee-kserve-84566d678c-gc2bh"
Sep 10 19:38:21.916466 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:21.916442    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n492r\" (UniqueName: \"kubernetes.io/projected/1d24b496-c358-4c7d-a44b-b9f1c0059563-kube-api-access-n492r\") pod \"sub-test-model-cab4ee-kserve-84566d678c-gc2bh\" (UID: \"1d24b496-c358-4c7d-a44b-b9f1c0059563\") " pod="ai-tenant-e2e-shared-b-w3-cab4ee/sub-test-model-cab4ee-kserve-84566d678c-gc2bh"
Sep 10 19:38:22.080296 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:22.080206    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-cab4ee/sub-test-model-cab4ee-kserve-84566d678c-gc2bh"
Sep 10 19:38:22.302347 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:22.302289    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn"]
Sep 10 19:38:22.306308 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:22.306287    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn"
Sep 10 19:38:22.308431 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:22.308410    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-cbac1f-serving-cert\""
Sep 10 19:38:22.316108 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:22.315992    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn"]
Sep 10 19:38:22.362647 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:22.362616    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-cab4ee/sub-test-model-cab4ee-kserve-84566d678c-gc2bh"]
Sep 10 19:38:22.363712 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:38:22.363690    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1d24b496_c358_4c7d_a44b_b9f1c0059563.slice/crio-5d9f4cfd37d0bddb911f4c3501aef30dcde0de6d7d1f99aa02fb54cee9ce7821 WatchSource:0}: Error finding container 5d9f4cfd37d0bddb911f4c3501aef30dcde0de6d7d1f99aa02fb54cee9ce7821: Status 404 returned error can't find the container with id 5d9f4cfd37d0bddb911f4c3501aef30dcde0de6d7d1f99aa02fb54cee9ce7821
Sep 10 19:38:22.414513 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:22.414474    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c094e69e-89a0-4b31-a3c5-a31c89283ce0-metrics-certs\") pod \"maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn\" (UID: \"c094e69e-89a0-4b31-a3c5-a31c89283ce0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn"
Sep 10 19:38:22.414635 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:22.414541    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c094e69e-89a0-4b31-a3c5-a31c89283ce0-maas-api-tls\") pod \"maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn\" (UID: \"c094e69e-89a0-4b31-a3c5-a31c89283ce0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn"
Sep 10 19:38:22.414635 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:22.414603    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hkmh\" (UniqueName: \"kubernetes.io/projected/c094e69e-89a0-4b31-a3c5-a31c89283ce0-kube-api-access-4hkmh\") pod \"maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn\" (UID: \"c094e69e-89a0-4b31-a3c5-a31c89283ce0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn"
Sep 10 19:38:22.414715 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:22.414639    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c094e69e-89a0-4b31-a3c5-a31c89283ce0-openshift-service-ca\") pod \"maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn\" (UID: \"c094e69e-89a0-4b31-a3c5-a31c89283ce0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn"
Sep 10 19:38:22.422992 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:22.422965    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-cab4ee/sub-test-model-cab4ee-kserve-84566d678c-gc2bh" event={"ID":"1d24b496-c358-4c7d-a44b-b9f1c0059563","Type":"ContainerStarted","Data":"5d9f4cfd37d0bddb911f4c3501aef30dcde0de6d7d1f99aa02fb54cee9ce7821"}
Sep 10 19:38:22.515383 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:22.515349    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c094e69e-89a0-4b31-a3c5-a31c89283ce0-maas-api-tls\") pod \"maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn\" (UID: \"c094e69e-89a0-4b31-a3c5-a31c89283ce0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn"
Sep 10 19:38:22.515556 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:22.515418    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hkmh\" (UniqueName: \"kubernetes.io/projected/c094e69e-89a0-4b31-a3c5-a31c89283ce0-kube-api-access-4hkmh\") pod \"maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn\" (UID: \"c094e69e-89a0-4b31-a3c5-a31c89283ce0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn"
Sep 10 19:38:22.515556 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:22.515447    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c094e69e-89a0-4b31-a3c5-a31c89283ce0-openshift-service-ca\") pod \"maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn\" (UID: \"c094e69e-89a0-4b31-a3c5-a31c89283ce0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn"
Sep 10 19:38:22.515556 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:38:22.515496    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-cbac1f-serving-cert: secret "maas-api-e2e-api-b-cbac1f-serving-cert" not found
Sep 10 19:38:22.515556 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:22.515541    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c094e69e-89a0-4b31-a3c5-a31c89283ce0-metrics-certs\") pod \"maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn\" (UID: \"c094e69e-89a0-4b31-a3c5-a31c89283ce0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn"
Sep 10 19:38:22.515714 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:38:22.515570    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c094e69e-89a0-4b31-a3c5-a31c89283ce0-maas-api-tls podName:c094e69e-89a0-4b31-a3c5-a31c89283ce0 nodeName:}" failed. No retries permitted until 2026-09-10 19:38:23.015554108 +0000 UTC m=+1358.235402570 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c094e69e-89a0-4b31-a3c5-a31c89283ce0-maas-api-tls") pod "maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn" (UID: "c094e69e-89a0-4b31-a3c5-a31c89283ce0") : secret "maas-api-e2e-api-b-cbac1f-serving-cert" not found
Sep 10 19:38:22.516253 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:22.516237    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c094e69e-89a0-4b31-a3c5-a31c89283ce0-openshift-service-ca\") pod \"maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn\" (UID: \"c094e69e-89a0-4b31-a3c5-a31c89283ce0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn"
Sep 10 19:38:22.518189 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:22.518172    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c094e69e-89a0-4b31-a3c5-a31c89283ce0-metrics-certs\") pod \"maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn\" (UID: \"c094e69e-89a0-4b31-a3c5-a31c89283ce0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn"
Sep 10 19:38:22.526036 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:22.526012    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hkmh\" (UniqueName: \"kubernetes.io/projected/c094e69e-89a0-4b31-a3c5-a31c89283ce0-kube-api-access-4hkmh\") pod \"maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn\" (UID: \"c094e69e-89a0-4b31-a3c5-a31c89283ce0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn"
Sep 10 19:38:23.021976 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:23.021942    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c094e69e-89a0-4b31-a3c5-a31c89283ce0-maas-api-tls\") pod \"maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn\" (UID: \"c094e69e-89a0-4b31-a3c5-a31c89283ce0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn"
Sep 10 19:38:23.024687 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:23.024656    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c094e69e-89a0-4b31-a3c5-a31c89283ce0-maas-api-tls\") pod \"maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn\" (UID: \"c094e69e-89a0-4b31-a3c5-a31c89283ce0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn"
Sep 10 19:38:23.223863 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:23.223837    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn"
Sep 10 19:38:23.431025 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:23.430988    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-cab4ee/sub-test-model-cab4ee-kserve-84566d678c-gc2bh" event={"ID":"1d24b496-c358-4c7d-a44b-b9f1c0059563","Type":"ContainerStarted","Data":"893608062d70011273f597cc1ac6d1c0404e395cc09192cef0f42627e73f38ab"}
Sep 10 19:38:23.448202 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:23.448093    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-cab4ee/sub-test-model-cab4ee-kserve-84566d678c-gc2bh" podStartSLOduration=2.448071611 podStartE2EDuration="2.448071611s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:38:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:38:23.447525681 +0000 UTC m=+1358.667374162" watchObservedRunningTime="2026-09-10 19:38:23.448071611 +0000 UTC m=+1358.667920106"
Sep 10 19:38:23.502926 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:23.502897    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn"]
Sep 10 19:38:23.503457 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:38:23.503435    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc094e69e_89a0_4b31_a3c5_a31c89283ce0.slice/crio-680db5bcdffe211c030decc68bd000ab58340272ccaffea1c5160e536f96c552 WatchSource:0}: Error finding container 680db5bcdffe211c030decc68bd000ab58340272ccaffea1c5160e536f96c552: Status 404 returned error can't find the container with id 680db5bcdffe211c030decc68bd000ab58340272ccaffea1c5160e536f96c552
Sep 10 19:38:24.439200 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:24.439157    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn" event={"ID":"c094e69e-89a0-4b31-a3c5-a31c89283ce0","Type":"ContainerStarted","Data":"2ec8db6864a1f00d378a848342444f21e71ca7b80611795a55143ba65d36f350"}
Sep 10 19:38:24.439648 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:24.439400    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn"
Sep 10 19:38:24.439648 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:24.439443    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn" event={"ID":"c094e69e-89a0-4b31-a3c5-a31c89283ce0","Type":"ContainerStarted","Data":"680db5bcdffe211c030decc68bd000ab58340272ccaffea1c5160e536f96c552"}
Sep 10 19:38:24.457407 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:24.457361    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn" podStartSLOduration=1.714002329 podStartE2EDuration="2.457342502s" totalImagesPullingTime="743.340173ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:38:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:38:24.454889434 +0000 UTC m=+1359.674737916" watchObservedRunningTime="2026-09-10 19:38:24.457342502 +0000 UTC m=+1359.677190983"
Sep 10 19:38:30.451037 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:30.451006    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn"
Sep 10 19:38:32.080903 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:32.080873    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-cab4ee/sub-test-model-cab4ee-kserve-84566d678c-gc2bh"
Sep 10 19:38:32.080903 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:32.080913    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-cab4ee/sub-test-model-cab4ee-kserve-84566d678c-gc2bh"
Sep 10 19:38:32.093189 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:32.093163    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-cab4ee/sub-test-model-cab4ee-kserve-84566d678c-gc2bh"
Sep 10 19:38:32.500396 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:38:32.500370    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-cab4ee/sub-test-model-cab4ee-kserve-84566d678c-gc2bh"
Sep 10 19:39:04.884291 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:04.884260    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-439275-6796f6fcb-dr5pm"]
Sep 10 19:39:04.884891 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:04.884504    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-439275-6796f6fcb-dr5pm" podUID="381e2ae5-0ff2-4752-86fd-6b4b420a7a20" containerName="maas-api" containerID="cri-o://9e6962ea150918fcb66e6b3298226743ba479e62f21359f06319d78c94e1b46c" gracePeriod=30
Sep 10 19:39:05.139510 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:05.139449    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-439275-6796f6fcb-dr5pm"
Sep 10 19:39:05.269037 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:05.268973    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xk7lc\" (UniqueName: \"kubernetes.io/projected/381e2ae5-0ff2-4752-86fd-6b4b420a7a20-kube-api-access-xk7lc\") pod \"381e2ae5-0ff2-4752-86fd-6b4b420a7a20\" (UID: \"381e2ae5-0ff2-4752-86fd-6b4b420a7a20\") "
Sep 10 19:39:05.269037 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:05.269040    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/381e2ae5-0ff2-4752-86fd-6b4b420a7a20-maas-api-tls\") pod \"381e2ae5-0ff2-4752-86fd-6b4b420a7a20\" (UID: \"381e2ae5-0ff2-4752-86fd-6b4b420a7a20\") "
Sep 10 19:39:05.273273 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:05.269105    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/381e2ae5-0ff2-4752-86fd-6b4b420a7a20-metrics-certs\") pod \"381e2ae5-0ff2-4752-86fd-6b4b420a7a20\" (UID: \"381e2ae5-0ff2-4752-86fd-6b4b420a7a20\") "
Sep 10 19:39:05.273273 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:05.269257    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/381e2ae5-0ff2-4752-86fd-6b4b420a7a20-openshift-service-ca\") pod \"381e2ae5-0ff2-4752-86fd-6b4b420a7a20\" (UID: \"381e2ae5-0ff2-4752-86fd-6b4b420a7a20\") "
Sep 10 19:39:05.273273 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:05.269948    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/381e2ae5-0ff2-4752-86fd-6b4b420a7a20-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "381e2ae5-0ff2-4752-86fd-6b4b420a7a20" (UID: "381e2ae5-0ff2-4752-86fd-6b4b420a7a20"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:39:05.273273 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:05.273197    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/381e2ae5-0ff2-4752-86fd-6b4b420a7a20-openshift-service-ca\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:39:05.275615 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:05.275581    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/381e2ae5-0ff2-4752-86fd-6b4b420a7a20-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "381e2ae5-0ff2-4752-86fd-6b4b420a7a20" (UID: "381e2ae5-0ff2-4752-86fd-6b4b420a7a20"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:39:05.275736 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:05.275665    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/381e2ae5-0ff2-4752-86fd-6b4b420a7a20-kube-api-access-xk7lc" (OuterVolumeSpecName: "kube-api-access-xk7lc") pod "381e2ae5-0ff2-4752-86fd-6b4b420a7a20" (UID: "381e2ae5-0ff2-4752-86fd-6b4b420a7a20"). InnerVolumeSpecName "kube-api-access-xk7lc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:39:05.276855 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:05.276825    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/381e2ae5-0ff2-4752-86fd-6b4b420a7a20-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "381e2ae5-0ff2-4752-86fd-6b4b420a7a20" (UID: "381e2ae5-0ff2-4752-86fd-6b4b420a7a20"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:39:05.374444 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:05.374376    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/381e2ae5-0ff2-4752-86fd-6b4b420a7a20-maas-api-tls\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:39:05.374444 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:05.374406    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/381e2ae5-0ff2-4752-86fd-6b4b420a7a20-metrics-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:39:05.374444 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:05.374419    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xk7lc\" (UniqueName: \"kubernetes.io/projected/381e2ae5-0ff2-4752-86fd-6b4b420a7a20-kube-api-access-xk7lc\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:39:05.698555 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:05.698520    2579 generic.go:356] "Generic (PLEG): container finished" podID="381e2ae5-0ff2-4752-86fd-6b4b420a7a20" containerID="9e6962ea150918fcb66e6b3298226743ba479e62f21359f06319d78c94e1b46c" exitCode=0
Sep 10 19:39:05.698773 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:05.698575    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-439275-6796f6fcb-dr5pm"
Sep 10 19:39:05.698773 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:05.698608    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-439275-6796f6fcb-dr5pm" event={"ID":"381e2ae5-0ff2-4752-86fd-6b4b420a7a20","Type":"ContainerDied","Data":"9e6962ea150918fcb66e6b3298226743ba479e62f21359f06319d78c94e1b46c"}
Sep 10 19:39:05.698773 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:05.698660    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-439275-6796f6fcb-dr5pm" event={"ID":"381e2ae5-0ff2-4752-86fd-6b4b420a7a20","Type":"ContainerDied","Data":"dfce2cf3cc1a2e07f88eed34a2d4fbe88494bdba86c2c3d445bb7c03ac7bbf6d"}
Sep 10 19:39:05.698773 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:05.698682    2579 scope.go:117] "RemoveContainer" containerID="9e6962ea150918fcb66e6b3298226743ba479e62f21359f06319d78c94e1b46c"
Sep 10 19:39:05.708621 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:05.708599    2579 scope.go:117] "RemoveContainer" containerID="9e6962ea150918fcb66e6b3298226743ba479e62f21359f06319d78c94e1b46c"
Sep 10 19:39:05.708930 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:39:05.708908    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e6962ea150918fcb66e6b3298226743ba479e62f21359f06319d78c94e1b46c\": container with ID starting with 9e6962ea150918fcb66e6b3298226743ba479e62f21359f06319d78c94e1b46c not found: ID does not exist" containerID="9e6962ea150918fcb66e6b3298226743ba479e62f21359f06319d78c94e1b46c"
Sep 10 19:39:05.709013 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:05.708936    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e6962ea150918fcb66e6b3298226743ba479e62f21359f06319d78c94e1b46c"} err="failed to get container status \"9e6962ea150918fcb66e6b3298226743ba479e62f21359f06319d78c94e1b46c\": rpc error: code = NotFound desc = could not find container \"9e6962ea150918fcb66e6b3298226743ba479e62f21359f06319d78c94e1b46c\": container with ID starting with 9e6962ea150918fcb66e6b3298226743ba479e62f21359f06319d78c94e1b46c not found: ID does not exist"
Sep 10 19:39:05.720582 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:05.720558    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-439275-6796f6fcb-dr5pm"]
Sep 10 19:39:05.726112 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:05.726088    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-439275-6796f6fcb-dr5pm"]
Sep 10 19:39:07.239721 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:07.239689    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="381e2ae5-0ff2-4752-86fd-6b4b420a7a20" path="/var/lib/kubelet/pods/381e2ae5-0ff2-4752-86fd-6b4b420a7a20/volumes"
Sep 10 19:39:25.973025 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:25.972432    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-cab4ee/sub-test-model-cab4ee-kserve-84566d678c-gc2bh"]
Sep 10 19:39:27.875699 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:27.875665    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn"]
Sep 10 19:39:27.876310 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:27.875927    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn" podUID="c094e69e-89a0-4b31-a3c5-a31c89283ce0" containerName="maas-api" containerID="cri-o://2ec8db6864a1f00d378a848342444f21e71ca7b80611795a55143ba65d36f350" gracePeriod=30
Sep 10 19:39:28.139135 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:28.139114    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn"
Sep 10 19:39:28.207033 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:28.207000    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c094e69e-89a0-4b31-a3c5-a31c89283ce0-maas-api-tls\") pod \"c094e69e-89a0-4b31-a3c5-a31c89283ce0\" (UID: \"c094e69e-89a0-4b31-a3c5-a31c89283ce0\") "
Sep 10 19:39:28.207202 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:28.207073    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c094e69e-89a0-4b31-a3c5-a31c89283ce0-openshift-service-ca\") pod \"c094e69e-89a0-4b31-a3c5-a31c89283ce0\" (UID: \"c094e69e-89a0-4b31-a3c5-a31c89283ce0\") "
Sep 10 19:39:28.207202 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:28.207167    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c094e69e-89a0-4b31-a3c5-a31c89283ce0-metrics-certs\") pod \"c094e69e-89a0-4b31-a3c5-a31c89283ce0\" (UID: \"c094e69e-89a0-4b31-a3c5-a31c89283ce0\") "
Sep 10 19:39:28.207277 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:28.207211    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4hkmh\" (UniqueName: \"kubernetes.io/projected/c094e69e-89a0-4b31-a3c5-a31c89283ce0-kube-api-access-4hkmh\") pod \"c094e69e-89a0-4b31-a3c5-a31c89283ce0\" (UID: \"c094e69e-89a0-4b31-a3c5-a31c89283ce0\") "
Sep 10 19:39:28.207491 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:28.207454    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c094e69e-89a0-4b31-a3c5-a31c89283ce0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c094e69e-89a0-4b31-a3c5-a31c89283ce0" (UID: "c094e69e-89a0-4b31-a3c5-a31c89283ce0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:39:28.209408 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:28.209385    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c094e69e-89a0-4b31-a3c5-a31c89283ce0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c094e69e-89a0-4b31-a3c5-a31c89283ce0" (UID: "c094e69e-89a0-4b31-a3c5-a31c89283ce0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:39:28.209408 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:28.209394    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c094e69e-89a0-4b31-a3c5-a31c89283ce0-kube-api-access-4hkmh" (OuterVolumeSpecName: "kube-api-access-4hkmh") pod "c094e69e-89a0-4b31-a3c5-a31c89283ce0" (UID: "c094e69e-89a0-4b31-a3c5-a31c89283ce0"). InnerVolumeSpecName "kube-api-access-4hkmh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:39:28.209523 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:28.209460    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c094e69e-89a0-4b31-a3c5-a31c89283ce0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c094e69e-89a0-4b31-a3c5-a31c89283ce0" (UID: "c094e69e-89a0-4b31-a3c5-a31c89283ce0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:39:28.308843 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:28.308801    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c094e69e-89a0-4b31-a3c5-a31c89283ce0-maas-api-tls\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:39:28.309049 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:28.308902    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c094e69e-89a0-4b31-a3c5-a31c89283ce0-openshift-service-ca\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:39:28.309049 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:28.309016    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c094e69e-89a0-4b31-a3c5-a31c89283ce0-metrics-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:39:28.309049 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:28.309034    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4hkmh\" (UniqueName: \"kubernetes.io/projected/c094e69e-89a0-4b31-a3c5-a31c89283ce0-kube-api-access-4hkmh\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:39:28.846030 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:28.845996    2579 generic.go:356] "Generic (PLEG): container finished" podID="c094e69e-89a0-4b31-a3c5-a31c89283ce0" containerID="2ec8db6864a1f00d378a848342444f21e71ca7b80611795a55143ba65d36f350" exitCode=0
Sep 10 19:39:28.846290 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:28.846045    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn"
Sep 10 19:39:28.846290 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:28.846083    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn" event={"ID":"c094e69e-89a0-4b31-a3c5-a31c89283ce0","Type":"ContainerDied","Data":"2ec8db6864a1f00d378a848342444f21e71ca7b80611795a55143ba65d36f350"}
Sep 10 19:39:28.846290 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:28.846137    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn" event={"ID":"c094e69e-89a0-4b31-a3c5-a31c89283ce0","Type":"ContainerDied","Data":"680db5bcdffe211c030decc68bd000ab58340272ccaffea1c5160e536f96c552"}
Sep 10 19:39:28.846290 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:28.846161    2579 scope.go:117] "RemoveContainer" containerID="2ec8db6864a1f00d378a848342444f21e71ca7b80611795a55143ba65d36f350"
Sep 10 19:39:28.856673 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:28.856655    2579 scope.go:117] "RemoveContainer" containerID="2ec8db6864a1f00d378a848342444f21e71ca7b80611795a55143ba65d36f350"
Sep 10 19:39:28.857068 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:39:28.857048    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ec8db6864a1f00d378a848342444f21e71ca7b80611795a55143ba65d36f350\": container with ID starting with 2ec8db6864a1f00d378a848342444f21e71ca7b80611795a55143ba65d36f350 not found: ID does not exist" containerID="2ec8db6864a1f00d378a848342444f21e71ca7b80611795a55143ba65d36f350"
Sep 10 19:39:28.857136 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:28.857076    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ec8db6864a1f00d378a848342444f21e71ca7b80611795a55143ba65d36f350"} err="failed to get container status \"2ec8db6864a1f00d378a848342444f21e71ca7b80611795a55143ba65d36f350\": rpc error: code = NotFound desc = could not find container \"2ec8db6864a1f00d378a848342444f21e71ca7b80611795a55143ba65d36f350\": container with ID starting with 2ec8db6864a1f00d378a848342444f21e71ca7b80611795a55143ba65d36f350 not found: ID does not exist"
Sep 10 19:39:28.868192 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:28.868170    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn"]
Sep 10 19:39:28.872132 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:28.872108    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-cbac1f-6b6cd5db6f-jp4bn"]
Sep 10 19:39:29.239796 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:29.239732    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c094e69e-89a0-4b31-a3c5-a31c89283ce0" path="/var/lib/kubelet/pods/c094e69e-89a0-4b31-a3c5-a31c89283ce0/volumes"
Sep 10 19:39:34.284733 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:34.284703    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-14a8d1/rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8"]
Sep 10 19:39:34.285280 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:34.285266    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="381e2ae5-0ff2-4752-86fd-6b4b420a7a20" containerName="maas-api"
Sep 10 19:39:34.285326 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:34.285284    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="381e2ae5-0ff2-4752-86fd-6b4b420a7a20" containerName="maas-api"
Sep 10 19:39:34.285326 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:34.285295    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c094e69e-89a0-4b31-a3c5-a31c89283ce0" containerName="maas-api"
Sep 10 19:39:34.285326 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:34.285301    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c094e69e-89a0-4b31-a3c5-a31c89283ce0" containerName="maas-api"
Sep 10 19:39:34.285472 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:34.285393    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c094e69e-89a0-4b31-a3c5-a31c89283ce0" containerName="maas-api"
Sep 10 19:39:34.285472 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:34.285404    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="381e2ae5-0ff2-4752-86fd-6b4b420a7a20" containerName="maas-api"
Sep 10 19:39:34.289922 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:34.289900    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-14a8d1/rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8"
Sep 10 19:39:34.292810 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:34.292781    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-14a8d1\"/\"default-dockercfg-s9krw\""
Sep 10 19:39:34.293227 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:34.292864    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-14a8d1\"/\"openshift-service-ca.crt\""
Sep 10 19:39:34.293227 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:34.292966    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-14a8d1\"/\"kube-root-ca.crt\""
Sep 10 19:39:34.293227 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:34.293092    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-14a8d1\"/\"rate-test-model-14a8d1-kserve-self-signed-certs\""
Sep 10 19:39:34.299362 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:34.299341    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14a8d1/rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8"]
Sep 10 19:39:34.365947 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:34.365912    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/854458d7-857b-43b1-baab-cd9ec3cbc675-tls-certs\") pod \"rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8\" (UID: \"854458d7-857b-43b1-baab-cd9ec3cbc675\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8"
Sep 10 19:39:34.365947 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:34.365956    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/854458d7-857b-43b1-baab-cd9ec3cbc675-home\") pod \"rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8\" (UID: \"854458d7-857b-43b1-baab-cd9ec3cbc675\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8"
Sep 10 19:39:34.366166 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:34.365985    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/854458d7-857b-43b1-baab-cd9ec3cbc675-tmp-dir\") pod \"rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8\" (UID: \"854458d7-857b-43b1-baab-cd9ec3cbc675\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8"
Sep 10 19:39:34.366166 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:34.366043    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/854458d7-857b-43b1-baab-cd9ec3cbc675-model-cache\") pod \"rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8\" (UID: \"854458d7-857b-43b1-baab-cd9ec3cbc675\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8"
Sep 10 19:39:34.366166 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:34.366096    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dckj6\" (UniqueName: \"kubernetes.io/projected/854458d7-857b-43b1-baab-cd9ec3cbc675-kube-api-access-dckj6\") pod \"rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8\" (UID: \"854458d7-857b-43b1-baab-cd9ec3cbc675\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8"
Sep 10 19:39:34.366267 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:34.366220    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/854458d7-857b-43b1-baab-cd9ec3cbc675-dshm\") pod \"rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8\" (UID: \"854458d7-857b-43b1-baab-cd9ec3cbc675\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8"
Sep 10 19:39:34.467611 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:34.467575    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/854458d7-857b-43b1-baab-cd9ec3cbc675-dshm\") pod \"rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8\" (UID: \"854458d7-857b-43b1-baab-cd9ec3cbc675\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8"
Sep 10 19:39:34.467850 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:34.467625    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/854458d7-857b-43b1-baab-cd9ec3cbc675-tls-certs\") pod \"rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8\" (UID: \"854458d7-857b-43b1-baab-cd9ec3cbc675\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8"
Sep 10 19:39:34.467850 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:34.467642    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/854458d7-857b-43b1-baab-cd9ec3cbc675-home\") pod \"rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8\" (UID: \"854458d7-857b-43b1-baab-cd9ec3cbc675\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8"
Sep 10 19:39:34.467850 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:34.467660    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/854458d7-857b-43b1-baab-cd9ec3cbc675-tmp-dir\") pod \"rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8\" (UID: \"854458d7-857b-43b1-baab-cd9ec3cbc675\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8"
Sep 10 19:39:34.467850 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:34.467682    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/854458d7-857b-43b1-baab-cd9ec3cbc675-model-cache\") pod \"rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8\" (UID: \"854458d7-857b-43b1-baab-cd9ec3cbc675\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8"
Sep 10 19:39:34.467850 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:34.467718    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dckj6\" (UniqueName: \"kubernetes.io/projected/854458d7-857b-43b1-baab-cd9ec3cbc675-kube-api-access-dckj6\") pod \"rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8\" (UID: \"854458d7-857b-43b1-baab-cd9ec3cbc675\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8"
Sep 10 19:39:34.468104 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:34.468088    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/854458d7-857b-43b1-baab-cd9ec3cbc675-home\") pod \"rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8\" (UID: \"854458d7-857b-43b1-baab-cd9ec3cbc675\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8"
Sep 10 19:39:34.468162 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:34.468108    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/854458d7-857b-43b1-baab-cd9ec3cbc675-model-cache\") pod \"rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8\" (UID: \"854458d7-857b-43b1-baab-cd9ec3cbc675\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8"
Sep 10 19:39:34.468206 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:34.468179    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/854458d7-857b-43b1-baab-cd9ec3cbc675-tmp-dir\") pod \"rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8\" (UID: \"854458d7-857b-43b1-baab-cd9ec3cbc675\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8"
Sep 10 19:39:34.470150 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:34.470126    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/854458d7-857b-43b1-baab-cd9ec3cbc675-dshm\") pod \"rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8\" (UID: \"854458d7-857b-43b1-baab-cd9ec3cbc675\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8"
Sep 10 19:39:34.470352 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:34.470334    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/854458d7-857b-43b1-baab-cd9ec3cbc675-tls-certs\") pod \"rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8\" (UID: \"854458d7-857b-43b1-baab-cd9ec3cbc675\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8"
Sep 10 19:39:34.475283 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:34.475260    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dckj6\" (UniqueName: \"kubernetes.io/projected/854458d7-857b-43b1-baab-cd9ec3cbc675-kube-api-access-dckj6\") pod \"rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8\" (UID: \"854458d7-857b-43b1-baab-cd9ec3cbc675\") " pod="ai-tenant-e2e-shared-a-w3-14a8d1/rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8"
Sep 10 19:39:34.601617 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:34.601524    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-14a8d1/rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8"
Sep 10 19:39:34.885422 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:34.885391    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14a8d1/rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8"]
Sep 10 19:39:35.898101 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:35.898064    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-14a8d1/rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8" event={"ID":"854458d7-857b-43b1-baab-cd9ec3cbc675","Type":"ContainerStarted","Data":"9e4a5b9d76f31e212ece0c49e979f52552138f8f7272dc2c05766c51c78f01af"}
Sep 10 19:39:35.898571 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:35.898109    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-14a8d1/rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8" event={"ID":"854458d7-857b-43b1-baab-cd9ec3cbc675","Type":"ContainerStarted","Data":"522f89d73a9afee4970a3ee9790be1acca6fa3ca46b441da30c53c1ce0ddd677"}
Sep 10 19:39:35.915578 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:35.915532    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-14a8d1/rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8" podStartSLOduration=1.915514955 podStartE2EDuration="1.915514955s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:39:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:39:35.914471531 +0000 UTC m=+1431.134320017" watchObservedRunningTime="2026-09-10 19:39:35.915514955 +0000 UTC m=+1431.135363440"
Sep 10 19:39:40.995824 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:40.995783    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-cab4ee/sub-test-model-cab4ee-kserve-84566d678c-gc2bh" podUID="1d24b496-c358-4c7d-a44b-b9f1c0059563" containerName="main" containerID="cri-o://893608062d70011273f597cc1ac6d1c0404e395cc09192cef0f42627e73f38ab" gracePeriod=45
Sep 10 19:39:41.247575 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:41.247514    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-cab4ee/sub-test-model-cab4ee-kserve-84566d678c-gc2bh"
Sep 10 19:39:41.339001 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:41.338970    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d24b496-c358-4c7d-a44b-b9f1c0059563-tls-certs\") pod \"1d24b496-c358-4c7d-a44b-b9f1c0059563\" (UID: \"1d24b496-c358-4c7d-a44b-b9f1c0059563\") "
Sep 10 19:39:41.339202 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:41.339025    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d24b496-c358-4c7d-a44b-b9f1c0059563-dshm\") pod \"1d24b496-c358-4c7d-a44b-b9f1c0059563\" (UID: \"1d24b496-c358-4c7d-a44b-b9f1c0059563\") "
Sep 10 19:39:41.339202 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:41.339053    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d24b496-c358-4c7d-a44b-b9f1c0059563-model-cache\") pod \"1d24b496-c358-4c7d-a44b-b9f1c0059563\" (UID: \"1d24b496-c358-4c7d-a44b-b9f1c0059563\") "
Sep 10 19:39:41.339202 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:41.339083    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n492r\" (UniqueName: \"kubernetes.io/projected/1d24b496-c358-4c7d-a44b-b9f1c0059563-kube-api-access-n492r\") pod \"1d24b496-c358-4c7d-a44b-b9f1c0059563\" (UID: \"1d24b496-c358-4c7d-a44b-b9f1c0059563\") "
Sep 10 19:39:41.339202 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:41.339105    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d24b496-c358-4c7d-a44b-b9f1c0059563-home\") pod \"1d24b496-c358-4c7d-a44b-b9f1c0059563\" (UID: \"1d24b496-c358-4c7d-a44b-b9f1c0059563\") "
Sep 10 19:39:41.339202 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:41.339126    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d24b496-c358-4c7d-a44b-b9f1c0059563-tmp-dir\") pod \"1d24b496-c358-4c7d-a44b-b9f1c0059563\" (UID: \"1d24b496-c358-4c7d-a44b-b9f1c0059563\") "
Sep 10 19:39:41.339487 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:41.339391    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d24b496-c358-4c7d-a44b-b9f1c0059563-model-cache" (OuterVolumeSpecName: "model-cache") pod "1d24b496-c358-4c7d-a44b-b9f1c0059563" (UID: "1d24b496-c358-4c7d-a44b-b9f1c0059563"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:41.339487 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:41.339421    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d24b496-c358-4c7d-a44b-b9f1c0059563-home" (OuterVolumeSpecName: "home") pod "1d24b496-c358-4c7d-a44b-b9f1c0059563" (UID: "1d24b496-c358-4c7d-a44b-b9f1c0059563"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:41.339487 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:41.339447    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d24b496-c358-4c7d-a44b-b9f1c0059563-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1d24b496-c358-4c7d-a44b-b9f1c0059563" (UID: "1d24b496-c358-4c7d-a44b-b9f1c0059563"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:41.339658 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:41.339617    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d24b496-c358-4c7d-a44b-b9f1c0059563-model-cache\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:39:41.339658 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:41.339630    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d24b496-c358-4c7d-a44b-b9f1c0059563-home\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:39:41.339658 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:41.339642    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d24b496-c358-4c7d-a44b-b9f1c0059563-tmp-dir\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:39:41.341522 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:41.341501    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d24b496-c358-4c7d-a44b-b9f1c0059563-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1d24b496-c358-4c7d-a44b-b9f1c0059563" (UID: "1d24b496-c358-4c7d-a44b-b9f1c0059563"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:39:41.341522 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:41.341507    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d24b496-c358-4c7d-a44b-b9f1c0059563-dshm" (OuterVolumeSpecName: "dshm") pod "1d24b496-c358-4c7d-a44b-b9f1c0059563" (UID: "1d24b496-c358-4c7d-a44b-b9f1c0059563"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:41.341658 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:41.341498    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d24b496-c358-4c7d-a44b-b9f1c0059563-kube-api-access-n492r" (OuterVolumeSpecName: "kube-api-access-n492r") pod "1d24b496-c358-4c7d-a44b-b9f1c0059563" (UID: "1d24b496-c358-4c7d-a44b-b9f1c0059563"). InnerVolumeSpecName "kube-api-access-n492r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:39:41.440817 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:41.440784    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d24b496-c358-4c7d-a44b-b9f1c0059563-tls-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:39:41.440817 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:41.440812    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d24b496-c358-4c7d-a44b-b9f1c0059563-dshm\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:39:41.440817 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:41.440821    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n492r\" (UniqueName: \"kubernetes.io/projected/1d24b496-c358-4c7d-a44b-b9f1c0059563-kube-api-access-n492r\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:39:41.940943 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:41.940912    2579 generic.go:356] "Generic (PLEG): container finished" podID="1d24b496-c358-4c7d-a44b-b9f1c0059563" containerID="893608062d70011273f597cc1ac6d1c0404e395cc09192cef0f42627e73f38ab" exitCode=0
Sep 10 19:39:41.941103 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:41.940956    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-cab4ee/sub-test-model-cab4ee-kserve-84566d678c-gc2bh"
Sep 10 19:39:41.941103 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:41.941014    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-cab4ee/sub-test-model-cab4ee-kserve-84566d678c-gc2bh" event={"ID":"1d24b496-c358-4c7d-a44b-b9f1c0059563","Type":"ContainerDied","Data":"893608062d70011273f597cc1ac6d1c0404e395cc09192cef0f42627e73f38ab"}
Sep 10 19:39:41.941103 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:41.941064    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-cab4ee/sub-test-model-cab4ee-kserve-84566d678c-gc2bh" event={"ID":"1d24b496-c358-4c7d-a44b-b9f1c0059563","Type":"ContainerDied","Data":"5d9f4cfd37d0bddb911f4c3501aef30dcde0de6d7d1f99aa02fb54cee9ce7821"}
Sep 10 19:39:41.941103 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:41.941091    2579 scope.go:117] "RemoveContainer" containerID="893608062d70011273f597cc1ac6d1c0404e395cc09192cef0f42627e73f38ab"
Sep 10 19:39:41.953157 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:41.953135    2579 scope.go:117] "RemoveContainer" containerID="893608062d70011273f597cc1ac6d1c0404e395cc09192cef0f42627e73f38ab"
Sep 10 19:39:41.953539 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:39:41.953508    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"893608062d70011273f597cc1ac6d1c0404e395cc09192cef0f42627e73f38ab\": container with ID starting with 893608062d70011273f597cc1ac6d1c0404e395cc09192cef0f42627e73f38ab not found: ID does not exist" containerID="893608062d70011273f597cc1ac6d1c0404e395cc09192cef0f42627e73f38ab"
Sep 10 19:39:41.953668 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:41.953550    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"893608062d70011273f597cc1ac6d1c0404e395cc09192cef0f42627e73f38ab"} err="failed to get container status \"893608062d70011273f597cc1ac6d1c0404e395cc09192cef0f42627e73f38ab\": rpc error: code = NotFound desc = could not find container \"893608062d70011273f597cc1ac6d1c0404e395cc09192cef0f42627e73f38ab\": container with ID starting with 893608062d70011273f597cc1ac6d1c0404e395cc09192cef0f42627e73f38ab not found: ID does not exist"
Sep 10 19:39:41.966771 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:41.966714    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-cab4ee/sub-test-model-cab4ee-kserve-84566d678c-gc2bh"]
Sep 10 19:39:41.971638 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:41.971605    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-cab4ee/sub-test-model-cab4ee-kserve-84566d678c-gc2bh"]
Sep 10 19:39:43.239832 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:43.239800    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d24b496-c358-4c7d-a44b-b9f1c0059563" path="/var/lib/kubelet/pods/1d24b496-c358-4c7d-a44b-b9f1c0059563/volumes"
Sep 10 19:39:44.602366 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:44.602329    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-14a8d1/rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8"
Sep 10 19:39:44.602824 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:44.602379    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-14a8d1/rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8"
Sep 10 19:39:44.615543 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:44.615517    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-14a8d1/rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8"
Sep 10 19:39:44.972622 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:44.972588    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-14a8d1/rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8"
Sep 10 19:39:45.656853 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:45.656817    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl"]
Sep 10 19:39:45.657260 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:45.657062    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl" podUID="eef2f9d1-abbd-40f4-ae09-3352e50affed" containerName="maas-api" containerID="cri-o://ce44c610b720759fb12feafd32b7b9449cf9d8f7dac3b76b460986d6479a10b9" gracePeriod=30
Sep 10 19:39:45.928773 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:45.928729    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl"
Sep 10 19:39:45.968900 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:45.968866    2579 generic.go:356] "Generic (PLEG): container finished" podID="eef2f9d1-abbd-40f4-ae09-3352e50affed" containerID="ce44c610b720759fb12feafd32b7b9449cf9d8f7dac3b76b460986d6479a10b9" exitCode=0
Sep 10 19:39:45.969076 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:45.968918    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl"
Sep 10 19:39:45.969076 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:45.968948    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl" event={"ID":"eef2f9d1-abbd-40f4-ae09-3352e50affed","Type":"ContainerDied","Data":"ce44c610b720759fb12feafd32b7b9449cf9d8f7dac3b76b460986d6479a10b9"}
Sep 10 19:39:45.969076 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:45.969023    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl" event={"ID":"eef2f9d1-abbd-40f4-ae09-3352e50affed","Type":"ContainerDied","Data":"a8e966860a52ee0d4dfe3f43500ac71ab3c5f2c76fddd76896fd81cfb67663fe"}
Sep 10 19:39:45.969076 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:45.969055    2579 scope.go:117] "RemoveContainer" containerID="ce44c610b720759fb12feafd32b7b9449cf9d8f7dac3b76b460986d6479a10b9"
Sep 10 19:39:45.981790 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:45.981741    2579 scope.go:117] "RemoveContainer" containerID="ce44c610b720759fb12feafd32b7b9449cf9d8f7dac3b76b460986d6479a10b9"
Sep 10 19:39:45.982111 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:39:45.982089    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce44c610b720759fb12feafd32b7b9449cf9d8f7dac3b76b460986d6479a10b9\": container with ID starting with ce44c610b720759fb12feafd32b7b9449cf9d8f7dac3b76b460986d6479a10b9 not found: ID does not exist" containerID="ce44c610b720759fb12feafd32b7b9449cf9d8f7dac3b76b460986d6479a10b9"
Sep 10 19:39:45.982198 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:45.982125    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce44c610b720759fb12feafd32b7b9449cf9d8f7dac3b76b460986d6479a10b9"} err="failed to get container status \"ce44c610b720759fb12feafd32b7b9449cf9d8f7dac3b76b460986d6479a10b9\": rpc error: code = NotFound desc = could not find container \"ce44c610b720759fb12feafd32b7b9449cf9d8f7dac3b76b460986d6479a10b9\": container with ID starting with ce44c610b720759fb12feafd32b7b9449cf9d8f7dac3b76b460986d6479a10b9 not found: ID does not exist"
Sep 10 19:39:46.083590 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:46.083494    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eef2f9d1-abbd-40f4-ae09-3352e50affed-maas-api-tls\") pod \"eef2f9d1-abbd-40f4-ae09-3352e50affed\" (UID: \"eef2f9d1-abbd-40f4-ae09-3352e50affed\") "
Sep 10 19:39:46.083590 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:46.083542    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jnghb\" (UniqueName: \"kubernetes.io/projected/eef2f9d1-abbd-40f4-ae09-3352e50affed-kube-api-access-jnghb\") pod \"eef2f9d1-abbd-40f4-ae09-3352e50affed\" (UID: \"eef2f9d1-abbd-40f4-ae09-3352e50affed\") "
Sep 10 19:39:46.083590 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:46.083574    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eef2f9d1-abbd-40f4-ae09-3352e50affed-openshift-service-ca\") pod \"eef2f9d1-abbd-40f4-ae09-3352e50affed\" (UID: \"eef2f9d1-abbd-40f4-ae09-3352e50affed\") "
Sep 10 19:39:46.083921 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:46.083684    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eef2f9d1-abbd-40f4-ae09-3352e50affed-metrics-certs\") pod \"eef2f9d1-abbd-40f4-ae09-3352e50affed\" (UID: \"eef2f9d1-abbd-40f4-ae09-3352e50affed\") "
Sep 10 19:39:46.084060 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:46.083978    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eef2f9d1-abbd-40f4-ae09-3352e50affed-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "eef2f9d1-abbd-40f4-ae09-3352e50affed" (UID: "eef2f9d1-abbd-40f4-ae09-3352e50affed"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:39:46.084290 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:46.084256    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eef2f9d1-abbd-40f4-ae09-3352e50affed-openshift-service-ca\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:39:46.085913 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:46.085890    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eef2f9d1-abbd-40f4-ae09-3352e50affed-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "eef2f9d1-abbd-40f4-ae09-3352e50affed" (UID: "eef2f9d1-abbd-40f4-ae09-3352e50affed"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:39:46.085999 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:46.085980    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eef2f9d1-abbd-40f4-ae09-3352e50affed-kube-api-access-jnghb" (OuterVolumeSpecName: "kube-api-access-jnghb") pod "eef2f9d1-abbd-40f4-ae09-3352e50affed" (UID: "eef2f9d1-abbd-40f4-ae09-3352e50affed"). InnerVolumeSpecName "kube-api-access-jnghb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:39:46.086067 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:46.086051    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eef2f9d1-abbd-40f4-ae09-3352e50affed-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "eef2f9d1-abbd-40f4-ae09-3352e50affed" (UID: "eef2f9d1-abbd-40f4-ae09-3352e50affed"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:39:46.184894 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:46.184861    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eef2f9d1-abbd-40f4-ae09-3352e50affed-maas-api-tls\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:39:46.184894 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:46.184888    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnghb\" (UniqueName: \"kubernetes.io/projected/eef2f9d1-abbd-40f4-ae09-3352e50affed-kube-api-access-jnghb\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:39:46.184894 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:46.184899    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eef2f9d1-abbd-40f4-ae09-3352e50affed-metrics-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:39:46.292983 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:46.292940    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl"]
Sep 10 19:39:46.296144 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:46.296114    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8dd4a3-86fdc6b99-rfthl"]
Sep 10 19:39:47.240105 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:39:47.240071    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eef2f9d1-abbd-40f4-ae09-3352e50affed" path="/var/lib/kubelet/pods/eef2f9d1-abbd-40f4-ae09-3352e50affed/volumes"
Sep 10 19:40:04.111700 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:04.111663    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc"]
Sep 10 19:40:04.112262 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:04.111934    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc" podUID="cd180728-3a63-41d1-896a-32e024c7a046" containerName="maas-api" containerID="cri-o://6f01629044bd3886e1e0066d933ca812739061ecf4645da85cc806c97e655865" gracePeriod=30
Sep 10 19:40:04.388948 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:04.388788    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc"
Sep 10 19:40:04.459414 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:04.459384    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vqq9q\" (UniqueName: \"kubernetes.io/projected/cd180728-3a63-41d1-896a-32e024c7a046-kube-api-access-vqq9q\") pod \"cd180728-3a63-41d1-896a-32e024c7a046\" (UID: \"cd180728-3a63-41d1-896a-32e024c7a046\") "
Sep 10 19:40:04.459578 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:04.459462    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd180728-3a63-41d1-896a-32e024c7a046-openshift-service-ca\") pod \"cd180728-3a63-41d1-896a-32e024c7a046\" (UID: \"cd180728-3a63-41d1-896a-32e024c7a046\") "
Sep 10 19:40:04.459578 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:04.459506    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd180728-3a63-41d1-896a-32e024c7a046-maas-api-tls\") pod \"cd180728-3a63-41d1-896a-32e024c7a046\" (UID: \"cd180728-3a63-41d1-896a-32e024c7a046\") "
Sep 10 19:40:04.459578 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:04.459573    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd180728-3a63-41d1-896a-32e024c7a046-metrics-certs\") pod \"cd180728-3a63-41d1-896a-32e024c7a046\" (UID: \"cd180728-3a63-41d1-896a-32e024c7a046\") "
Sep 10 19:40:04.459876 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:04.459843    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd180728-3a63-41d1-896a-32e024c7a046-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cd180728-3a63-41d1-896a-32e024c7a046" (UID: "cd180728-3a63-41d1-896a-32e024c7a046"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:40:04.461705 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:04.461676    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd180728-3a63-41d1-896a-32e024c7a046-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cd180728-3a63-41d1-896a-32e024c7a046" (UID: "cd180728-3a63-41d1-896a-32e024c7a046"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:40:04.461880 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:04.461703    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd180728-3a63-41d1-896a-32e024c7a046-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "cd180728-3a63-41d1-896a-32e024c7a046" (UID: "cd180728-3a63-41d1-896a-32e024c7a046"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:40:04.461880 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:04.461703    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd180728-3a63-41d1-896a-32e024c7a046-kube-api-access-vqq9q" (OuterVolumeSpecName: "kube-api-access-vqq9q") pod "cd180728-3a63-41d1-896a-32e024c7a046" (UID: "cd180728-3a63-41d1-896a-32e024c7a046"). InnerVolumeSpecName "kube-api-access-vqq9q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:40:04.560329 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:04.560290    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd180728-3a63-41d1-896a-32e024c7a046-metrics-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:40:04.560329 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:04.560322    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vqq9q\" (UniqueName: \"kubernetes.io/projected/cd180728-3a63-41d1-896a-32e024c7a046-kube-api-access-vqq9q\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:40:04.560329 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:04.560332    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd180728-3a63-41d1-896a-32e024c7a046-openshift-service-ca\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:40:04.560582 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:04.560343    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd180728-3a63-41d1-896a-32e024c7a046-maas-api-tls\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:40:05.085996 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:05.085964    2579 generic.go:356] "Generic (PLEG): container finished" podID="cd180728-3a63-41d1-896a-32e024c7a046" containerID="6f01629044bd3886e1e0066d933ca812739061ecf4645da85cc806c97e655865" exitCode=0
Sep 10 19:40:05.086227 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:05.086010    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc"
Sep 10 19:40:05.086227 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:05.086054    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc" event={"ID":"cd180728-3a63-41d1-896a-32e024c7a046","Type":"ContainerDied","Data":"6f01629044bd3886e1e0066d933ca812739061ecf4645da85cc806c97e655865"}
Sep 10 19:40:05.086227 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:05.086147    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc" event={"ID":"cd180728-3a63-41d1-896a-32e024c7a046","Type":"ContainerDied","Data":"42a61dc81261023e57e37823f5e46439d122c3dc6b2869578e476aa04357c7ec"}
Sep 10 19:40:05.086227 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:05.086180    2579 scope.go:117] "RemoveContainer" containerID="6f01629044bd3886e1e0066d933ca812739061ecf4645da85cc806c97e655865"
Sep 10 19:40:05.096554 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:05.096514    2579 scope.go:117] "RemoveContainer" containerID="6f01629044bd3886e1e0066d933ca812739061ecf4645da85cc806c97e655865"
Sep 10 19:40:05.096832 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:40:05.096813    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f01629044bd3886e1e0066d933ca812739061ecf4645da85cc806c97e655865\": container with ID starting with 6f01629044bd3886e1e0066d933ca812739061ecf4645da85cc806c97e655865 not found: ID does not exist" containerID="6f01629044bd3886e1e0066d933ca812739061ecf4645da85cc806c97e655865"
Sep 10 19:40:05.096885 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:05.096842    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f01629044bd3886e1e0066d933ca812739061ecf4645da85cc806c97e655865"} err="failed to get container status \"6f01629044bd3886e1e0066d933ca812739061ecf4645da85cc806c97e655865\": rpc error: code = NotFound desc = could not find container \"6f01629044bd3886e1e0066d933ca812739061ecf4645da85cc806c97e655865\": container with ID starting with 6f01629044bd3886e1e0066d933ca812739061ecf4645da85cc806c97e655865 not found: ID does not exist"
Sep 10 19:40:05.114572 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:05.114547    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc"]
Sep 10 19:40:05.118437 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:05.118418    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4bc8b3-74cb5fd55d-smbrc"]
Sep 10 19:40:05.240364 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:05.240332    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd180728-3a63-41d1-896a-32e024c7a046" path="/var/lib/kubelet/pods/cd180728-3a63-41d1-896a-32e024c7a046/volumes"
Sep 10 19:40:10.925782 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:10.925721    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"]
Sep 10 19:40:10.926218 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:10.926073    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx" podUID="7bf44155-0d1f-4b0b-a6f6-bfe7706e0488" containerName="istio-proxy" containerID="cri-o://9982704f58e48d542bddc1a32c49facffef34cd7a2e8814374465eb0bc4ff248" gracePeriod=30
Sep 10 19:40:16.159480 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:16.159443    2579 generic.go:356] "Generic (PLEG): container finished" podID="7bf44155-0d1f-4b0b-a6f6-bfe7706e0488" containerID="9982704f58e48d542bddc1a32c49facffef34cd7a2e8814374465eb0bc4ff248" exitCode=0
Sep 10 19:40:16.159865 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:16.159513    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx" event={"ID":"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488","Type":"ContainerDied","Data":"9982704f58e48d542bddc1a32c49facffef34cd7a2e8814374465eb0bc4ff248"}
Sep 10 19:40:16.159865 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:16.159599    2579 scope.go:117] "RemoveContainer" containerID="24429ba0b0f3b38293c814d64b39fa53cb3121a06bf5450a5f1bdea6bd2f5b74"
Sep 10 19:40:16.179555 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:16.179537    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:40:16.277144 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:16.277061    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-istio-podinfo\") pod \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\" (UID: \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\") "
Sep 10 19:40:16.277144 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:16.277125    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-istiod-ca-cert\") pod \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\" (UID: \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\") "
Sep 10 19:40:16.277366 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:16.277175    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-credential-socket\") pod \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\" (UID: \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\") "
Sep 10 19:40:16.277366 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:16.277203    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lwknr\" (UniqueName: \"kubernetes.io/projected/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-kube-api-access-lwknr\") pod \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\" (UID: \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\") "
Sep 10 19:40:16.277366 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:16.277248    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-workload-certs\") pod \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\" (UID: \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\") "
Sep 10 19:40:16.277366 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:16.277306    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-istio-token\") pod \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\" (UID: \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\") "
Sep 10 19:40:16.277577 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:16.277426    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-istio-data\") pod \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\" (UID: \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\") "
Sep 10 19:40:16.277577 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:16.277472    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7bf44155-0d1f-4b0b-a6f6-bfe7706e0488" (UID: "7bf44155-0d1f-4b0b-a6f6-bfe7706e0488"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:40:16.277577 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:16.277497    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-workload-socket\") pod \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\" (UID: \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\") "
Sep 10 19:40:16.277577 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:16.277569    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-istio-envoy\") pod \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\" (UID: \"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488\") "
Sep 10 19:40:16.277850 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:16.277577    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7bf44155-0d1f-4b0b-a6f6-bfe7706e0488" (UID: "7bf44155-0d1f-4b0b-a6f6-bfe7706e0488"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:40:16.277850 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:16.277612    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7bf44155-0d1f-4b0b-a6f6-bfe7706e0488" (UID: "7bf44155-0d1f-4b0b-a6f6-bfe7706e0488"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:40:16.278032 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:16.277857    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-istio-data" (OuterVolumeSpecName: "istio-data") pod "7bf44155-0d1f-4b0b-a6f6-bfe7706e0488" (UID: "7bf44155-0d1f-4b0b-a6f6-bfe7706e0488"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:40:16.278032 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:16.277994    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-istiod-ca-cert\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:40:16.278032 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:16.278016    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-credential-socket\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:40:16.278032 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:16.278034    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-workload-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:40:16.278381 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:16.278086    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7bf44155-0d1f-4b0b-a6f6-bfe7706e0488" (UID: "7bf44155-0d1f-4b0b-a6f6-bfe7706e0488"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:40:16.279458 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:16.279433    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7bf44155-0d1f-4b0b-a6f6-bfe7706e0488" (UID: "7bf44155-0d1f-4b0b-a6f6-bfe7706e0488"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:40:16.279618 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:16.279595    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-istio-token" (OuterVolumeSpecName: "istio-token") pod "7bf44155-0d1f-4b0b-a6f6-bfe7706e0488" (UID: "7bf44155-0d1f-4b0b-a6f6-bfe7706e0488"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:40:16.279618 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:16.279605    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-kube-api-access-lwknr" (OuterVolumeSpecName: "kube-api-access-lwknr") pod "7bf44155-0d1f-4b0b-a6f6-bfe7706e0488" (UID: "7bf44155-0d1f-4b0b-a6f6-bfe7706e0488"). InnerVolumeSpecName "kube-api-access-lwknr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:40:16.279919 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:16.279848    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7bf44155-0d1f-4b0b-a6f6-bfe7706e0488" (UID: "7bf44155-0d1f-4b0b-a6f6-bfe7706e0488"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:40:16.379205 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:16.379168    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwknr\" (UniqueName: \"kubernetes.io/projected/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-kube-api-access-lwknr\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:40:16.379205 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:16.379200    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-istio-token\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:40:16.379205 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:16.379209    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-istio-data\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:40:16.379205 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:16.379218    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-workload-socket\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:40:16.379473 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:16.379225    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-istio-envoy\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:40:16.379473 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:16.379233    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488-istio-podinfo\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:40:17.167280 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:17.167249    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"
Sep 10 19:40:17.167741 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:17.167288    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx" event={"ID":"7bf44155-0d1f-4b0b-a6f6-bfe7706e0488","Type":"ContainerDied","Data":"27e7cc7214fd3f7f16df64b17f8da6bfb925320adc5f661f2bfbd8fe77557c2a"}
Sep 10 19:40:17.167741 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:17.167343    2579 scope.go:117] "RemoveContainer" containerID="9982704f58e48d542bddc1a32c49facffef34cd7a2e8814374465eb0bc4ff248"
Sep 10 19:40:17.199772 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:17.199717    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"]
Sep 10 19:40:17.203619 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:17.203592    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx"]
Sep 10 19:40:17.239887 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:17.239855    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bf44155-0d1f-4b0b-a6f6-bfe7706e0488" path="/var/lib/kubelet/pods/7bf44155-0d1f-4b0b-a6f6-bfe7706e0488/volumes"
Sep 10 19:40:45.282013 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:45.281985    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bmdg9_9817270c-8629-44c4-ab8f-f3982e1f0e2f/console-operator/1.log"
Sep 10 19:40:45.287514 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:45.287488    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8r66s_2826f8ec-78f6-4cab-879e-47a21854ed5a/ovn-acl-logging/0.log"
Sep 10 19:40:45.288653 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:45.288629    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bmdg9_9817270c-8629-44c4-ab8f-f3982e1f0e2f/console-operator/1.log"
Sep 10 19:40:45.293957 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:45.293939    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8r66s_2826f8ec-78f6-4cab-879e-47a21854ed5a/ovn-acl-logging/0.log"
Sep 10 19:40:49.385028 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:49.384996    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14a8d1/rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8"]
Sep 10 19:40:55.180257 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:55.180220    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx"]
Sep 10 19:40:55.180928 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:55.180907    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eef2f9d1-abbd-40f4-ae09-3352e50affed" containerName="maas-api"
Sep 10 19:40:55.180928 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:55.180930    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="eef2f9d1-abbd-40f4-ae09-3352e50affed" containerName="maas-api"
Sep 10 19:40:55.181109 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:55.180940    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d24b496-c358-4c7d-a44b-b9f1c0059563" containerName="main"
Sep 10 19:40:55.181109 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:55.180948    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d24b496-c358-4c7d-a44b-b9f1c0059563" containerName="main"
Sep 10 19:40:55.181109 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:55.180973    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd180728-3a63-41d1-896a-32e024c7a046" containerName="maas-api"
Sep 10 19:40:55.181109 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:55.180981    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd180728-3a63-41d1-896a-32e024c7a046" containerName="maas-api"
Sep 10 19:40:55.181109 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:55.181002    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bf44155-0d1f-4b0b-a6f6-bfe7706e0488" containerName="istio-proxy"
Sep 10 19:40:55.181109 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:55.181009    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bf44155-0d1f-4b0b-a6f6-bfe7706e0488" containerName="istio-proxy"
Sep 10 19:40:55.181109 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:55.181020    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bf44155-0d1f-4b0b-a6f6-bfe7706e0488" containerName="istio-proxy"
Sep 10 19:40:55.181109 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:55.181027    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bf44155-0d1f-4b0b-a6f6-bfe7706e0488" containerName="istio-proxy"
Sep 10 19:40:55.181490 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:55.181135    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d24b496-c358-4c7d-a44b-b9f1c0059563" containerName="main"
Sep 10 19:40:55.181490 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:55.181149    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd180728-3a63-41d1-896a-32e024c7a046" containerName="maas-api"
Sep 10 19:40:55.181490 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:55.181159    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bf44155-0d1f-4b0b-a6f6-bfe7706e0488" containerName="istio-proxy"
Sep 10 19:40:55.181490 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:55.181170    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bf44155-0d1f-4b0b-a6f6-bfe7706e0488" containerName="istio-proxy"
Sep 10 19:40:55.181490 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:55.181178    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="eef2f9d1-abbd-40f4-ae09-3352e50affed" containerName="maas-api"
Sep 10 19:40:55.185862 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:55.185839    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx"
Sep 10 19:40:55.188346 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:55.188324    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-df863b-serving-cert\""
Sep 10 19:40:55.199991 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:55.199965    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx"]
Sep 10 19:40:55.253298 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:55.253256    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cg28g\" (UniqueName: \"kubernetes.io/projected/678724f6-2a61-40e3-ba09-ea8099e5e3fc-kube-api-access-cg28g\") pod \"maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx\" (UID: \"678724f6-2a61-40e3-ba09-ea8099e5e3fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx"
Sep 10 19:40:55.253298 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:55.253304    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/678724f6-2a61-40e3-ba09-ea8099e5e3fc-metrics-certs\") pod \"maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx\" (UID: \"678724f6-2a61-40e3-ba09-ea8099e5e3fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx"
Sep 10 19:40:55.253530 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:55.253344    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/678724f6-2a61-40e3-ba09-ea8099e5e3fc-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx\" (UID: \"678724f6-2a61-40e3-ba09-ea8099e5e3fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx"
Sep 10 19:40:55.253530 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:55.253452    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/678724f6-2a61-40e3-ba09-ea8099e5e3fc-maas-api-tls\") pod \"maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx\" (UID: \"678724f6-2a61-40e3-ba09-ea8099e5e3fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx"
Sep 10 19:40:55.354283 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:55.354245    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/678724f6-2a61-40e3-ba09-ea8099e5e3fc-maas-api-tls\") pod \"maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx\" (UID: \"678724f6-2a61-40e3-ba09-ea8099e5e3fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx"
Sep 10 19:40:55.354448 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:55.354333    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg28g\" (UniqueName: \"kubernetes.io/projected/678724f6-2a61-40e3-ba09-ea8099e5e3fc-kube-api-access-cg28g\") pod \"maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx\" (UID: \"678724f6-2a61-40e3-ba09-ea8099e5e3fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx"
Sep 10 19:40:55.354448 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:55.354359    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/678724f6-2a61-40e3-ba09-ea8099e5e3fc-metrics-certs\") pod \"maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx\" (UID: \"678724f6-2a61-40e3-ba09-ea8099e5e3fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx"
Sep 10 19:40:55.354448 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:55.354381    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/678724f6-2a61-40e3-ba09-ea8099e5e3fc-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx\" (UID: \"678724f6-2a61-40e3-ba09-ea8099e5e3fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx"
Sep 10 19:40:55.354448 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:40:55.354401    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-df863b-serving-cert: secret "maas-api-e2e-ipp-a-df863b-serving-cert" not found
Sep 10 19:40:55.354637 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:40:55.354489    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/678724f6-2a61-40e3-ba09-ea8099e5e3fc-maas-api-tls podName:678724f6-2a61-40e3-ba09-ea8099e5e3fc nodeName:}" failed. No retries permitted until 2026-09-10 19:40:55.854466178 +0000 UTC m=+1511.074314640 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/678724f6-2a61-40e3-ba09-ea8099e5e3fc-maas-api-tls") pod "maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx" (UID: "678724f6-2a61-40e3-ba09-ea8099e5e3fc") : secret "maas-api-e2e-ipp-a-df863b-serving-cert" not found
Sep 10 19:40:55.354996 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:55.354979    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/678724f6-2a61-40e3-ba09-ea8099e5e3fc-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx\" (UID: \"678724f6-2a61-40e3-ba09-ea8099e5e3fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx"
Sep 10 19:40:55.357021 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:55.357004    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/678724f6-2a61-40e3-ba09-ea8099e5e3fc-metrics-certs\") pod \"maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx\" (UID: \"678724f6-2a61-40e3-ba09-ea8099e5e3fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx"
Sep 10 19:40:55.363379 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:55.363359    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cg28g\" (UniqueName: \"kubernetes.io/projected/678724f6-2a61-40e3-ba09-ea8099e5e3fc-kube-api-access-cg28g\") pod \"maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx\" (UID: \"678724f6-2a61-40e3-ba09-ea8099e5e3fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx"
Sep 10 19:40:55.859517 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:55.859483    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/678724f6-2a61-40e3-ba09-ea8099e5e3fc-maas-api-tls\") pod \"maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx\" (UID: \"678724f6-2a61-40e3-ba09-ea8099e5e3fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx"
Sep 10 19:40:55.862128 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:55.862102    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/678724f6-2a61-40e3-ba09-ea8099e5e3fc-maas-api-tls\") pod \"maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx\" (UID: \"678724f6-2a61-40e3-ba09-ea8099e5e3fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx"
Sep 10 19:40:56.101379 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:56.101351    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx"
Sep 10 19:40:56.392463 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:56.392425    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx"]
Sep 10 19:40:56.394634 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:40:56.394605    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod678724f6_2a61_40e3_ba09_ea8099e5e3fc.slice/crio-8abbc0cdae4a1ec534dc40c3cb23b256225bdbb50229d57e73153e40365b26fd WatchSource:0}: Error finding container 8abbc0cdae4a1ec534dc40c3cb23b256225bdbb50229d57e73153e40365b26fd: Status 404 returned error can't find the container with id 8abbc0cdae4a1ec534dc40c3cb23b256225bdbb50229d57e73153e40365b26fd
Sep 10 19:40:56.414983 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:56.414948    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx" event={"ID":"678724f6-2a61-40e3-ba09-ea8099e5e3fc","Type":"ContainerStarted","Data":"8abbc0cdae4a1ec534dc40c3cb23b256225bdbb50229d57e73153e40365b26fd"}
Sep 10 19:40:57.422509 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:57.422465    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx" event={"ID":"678724f6-2a61-40e3-ba09-ea8099e5e3fc","Type":"ContainerStarted","Data":"d22e270e71891160d832a0eae4082e6a77feafc4461ab27b63ef07ffeda97bfe"}
Sep 10 19:40:57.422929 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:57.422553    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx"
Sep 10 19:40:57.443070 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:40:57.443029    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx" podStartSLOduration=1.9771743370000001 podStartE2EDuration="2.443013983s" totalImagesPullingTime="465.839646ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:40:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:40:57.44184302 +0000 UTC m=+1512.661691500" watchObservedRunningTime="2026-09-10 19:40:57.443013983 +0000 UTC m=+1512.662862466"
Sep 10 19:41:03.434679 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:03.434649    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx"
Sep 10 19:41:04.405052 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:04.405010    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-14a8d1/rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8" podUID="854458d7-857b-43b1-baab-cd9ec3cbc675" containerName="main" containerID="cri-o://9e4a5b9d76f31e212ece0c49e979f52552138f8f7272dc2c05766c51c78f01af" gracePeriod=45
Sep 10 19:41:04.647213 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:04.647189    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-14a8d1/rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8"
Sep 10 19:41:04.750194 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:04.750113    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/854458d7-857b-43b1-baab-cd9ec3cbc675-model-cache\") pod \"854458d7-857b-43b1-baab-cd9ec3cbc675\" (UID: \"854458d7-857b-43b1-baab-cd9ec3cbc675\") "
Sep 10 19:41:04.750194 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:04.750150    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dckj6\" (UniqueName: \"kubernetes.io/projected/854458d7-857b-43b1-baab-cd9ec3cbc675-kube-api-access-dckj6\") pod \"854458d7-857b-43b1-baab-cd9ec3cbc675\" (UID: \"854458d7-857b-43b1-baab-cd9ec3cbc675\") "
Sep 10 19:41:04.750194 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:04.750184    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/854458d7-857b-43b1-baab-cd9ec3cbc675-tmp-dir\") pod \"854458d7-857b-43b1-baab-cd9ec3cbc675\" (UID: \"854458d7-857b-43b1-baab-cd9ec3cbc675\") "
Sep 10 19:41:04.750461 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:04.750269    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/854458d7-857b-43b1-baab-cd9ec3cbc675-home\") pod \"854458d7-857b-43b1-baab-cd9ec3cbc675\" (UID: \"854458d7-857b-43b1-baab-cd9ec3cbc675\") "
Sep 10 19:41:04.750461 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:04.750327    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/854458d7-857b-43b1-baab-cd9ec3cbc675-dshm\") pod \"854458d7-857b-43b1-baab-cd9ec3cbc675\" (UID: \"854458d7-857b-43b1-baab-cd9ec3cbc675\") "
Sep 10 19:41:04.750461 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:04.750392    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/854458d7-857b-43b1-baab-cd9ec3cbc675-tls-certs\") pod \"854458d7-857b-43b1-baab-cd9ec3cbc675\" (UID: \"854458d7-857b-43b1-baab-cd9ec3cbc675\") "
Sep 10 19:41:04.750461 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:04.750436    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/854458d7-857b-43b1-baab-cd9ec3cbc675-model-cache" (OuterVolumeSpecName: "model-cache") pod "854458d7-857b-43b1-baab-cd9ec3cbc675" (UID: "854458d7-857b-43b1-baab-cd9ec3cbc675"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:41:04.750684 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:04.750497    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/854458d7-857b-43b1-baab-cd9ec3cbc675-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "854458d7-857b-43b1-baab-cd9ec3cbc675" (UID: "854458d7-857b-43b1-baab-cd9ec3cbc675"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:41:04.750684 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:04.750505    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/854458d7-857b-43b1-baab-cd9ec3cbc675-home" (OuterVolumeSpecName: "home") pod "854458d7-857b-43b1-baab-cd9ec3cbc675" (UID: "854458d7-857b-43b1-baab-cd9ec3cbc675"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:41:04.750819 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:04.750793    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/854458d7-857b-43b1-baab-cd9ec3cbc675-model-cache\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:41:04.750819 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:04.750809    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/854458d7-857b-43b1-baab-cd9ec3cbc675-tmp-dir\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:41:04.750914 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:04.750824    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/854458d7-857b-43b1-baab-cd9ec3cbc675-home\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:41:04.752445 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:04.752417    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/854458d7-857b-43b1-baab-cd9ec3cbc675-dshm" (OuterVolumeSpecName: "dshm") pod "854458d7-857b-43b1-baab-cd9ec3cbc675" (UID: "854458d7-857b-43b1-baab-cd9ec3cbc675"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:41:04.752445 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:04.752437    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/854458d7-857b-43b1-baab-cd9ec3cbc675-kube-api-access-dckj6" (OuterVolumeSpecName: "kube-api-access-dckj6") pod "854458d7-857b-43b1-baab-cd9ec3cbc675" (UID: "854458d7-857b-43b1-baab-cd9ec3cbc675"). InnerVolumeSpecName "kube-api-access-dckj6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:41:04.760308 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:04.752589    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/854458d7-857b-43b1-baab-cd9ec3cbc675-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "854458d7-857b-43b1-baab-cd9ec3cbc675" (UID: "854458d7-857b-43b1-baab-cd9ec3cbc675"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:41:04.851824 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:04.851787    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/854458d7-857b-43b1-baab-cd9ec3cbc675-tls-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:41:04.851824 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:04.851817    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dckj6\" (UniqueName: \"kubernetes.io/projected/854458d7-857b-43b1-baab-cd9ec3cbc675-kube-api-access-dckj6\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:41:04.851824 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:04.851826    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/854458d7-857b-43b1-baab-cd9ec3cbc675-dshm\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:41:05.483587 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:05.483550    2579 generic.go:356] "Generic (PLEG): container finished" podID="854458d7-857b-43b1-baab-cd9ec3cbc675" containerID="9e4a5b9d76f31e212ece0c49e979f52552138f8f7272dc2c05766c51c78f01af" exitCode=0
Sep 10 19:41:05.483808 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:05.483607    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-14a8d1/rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8"
Sep 10 19:41:05.483808 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:05.483642    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-14a8d1/rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8" event={"ID":"854458d7-857b-43b1-baab-cd9ec3cbc675","Type":"ContainerDied","Data":"9e4a5b9d76f31e212ece0c49e979f52552138f8f7272dc2c05766c51c78f01af"}
Sep 10 19:41:05.483808 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:05.483687    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-14a8d1/rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8" event={"ID":"854458d7-857b-43b1-baab-cd9ec3cbc675","Type":"ContainerDied","Data":"522f89d73a9afee4970a3ee9790be1acca6fa3ca46b441da30c53c1ce0ddd677"}
Sep 10 19:41:05.483808 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:05.483707    2579 scope.go:117] "RemoveContainer" containerID="9e4a5b9d76f31e212ece0c49e979f52552138f8f7272dc2c05766c51c78f01af"
Sep 10 19:41:05.492622 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:05.492601    2579 scope.go:117] "RemoveContainer" containerID="9e4a5b9d76f31e212ece0c49e979f52552138f8f7272dc2c05766c51c78f01af"
Sep 10 19:41:05.492982 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:41:05.492961    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e4a5b9d76f31e212ece0c49e979f52552138f8f7272dc2c05766c51c78f01af\": container with ID starting with 9e4a5b9d76f31e212ece0c49e979f52552138f8f7272dc2c05766c51c78f01af not found: ID does not exist" containerID="9e4a5b9d76f31e212ece0c49e979f52552138f8f7272dc2c05766c51c78f01af"
Sep 10 19:41:05.493031 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:05.492992    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e4a5b9d76f31e212ece0c49e979f52552138f8f7272dc2c05766c51c78f01af"} err="failed to get container status \"9e4a5b9d76f31e212ece0c49e979f52552138f8f7272dc2c05766c51c78f01af\": rpc error: code = NotFound desc = could not find container \"9e4a5b9d76f31e212ece0c49e979f52552138f8f7272dc2c05766c51c78f01af\": container with ID starting with 9e4a5b9d76f31e212ece0c49e979f52552138f8f7272dc2c05766c51c78f01af not found: ID does not exist"
Sep 10 19:41:05.500521 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:05.500470    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14a8d1/rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8"]
Sep 10 19:41:05.502986 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:05.502968    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14a8d1/rate-test-model-14a8d1-kserve-79b6dbf4c9-kcxj8"]
Sep 10 19:41:07.239846 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:07.239741    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="854458d7-857b-43b1-baab-cd9ec3cbc675" path="/var/lib/kubelet/pods/854458d7-857b-43b1-baab-cd9ec3cbc675/volumes"
Sep 10 19:41:24.306089 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:24.306054    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6"]
Sep 10 19:41:24.306713 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:24.306692    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="854458d7-857b-43b1-baab-cd9ec3cbc675" containerName="main"
Sep 10 19:41:24.306830 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:24.306716    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="854458d7-857b-43b1-baab-cd9ec3cbc675" containerName="main"
Sep 10 19:41:24.306892 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:24.306868    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="854458d7-857b-43b1-baab-cd9ec3cbc675" containerName="main"
Sep 10 19:41:24.310336 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:24.310314    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6"
Sep 10 19:41:24.313798 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:24.313776    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-3765cf-serving-cert\""
Sep 10 19:41:24.324542 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:24.324510    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6"]
Sep 10 19:41:24.449076 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:24.449040    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2028475e-bf7c-4451-82da-70fd2b0ccf26-maas-api-tls\") pod \"maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6\" (UID: \"2028475e-bf7c-4451-82da-70fd2b0ccf26\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6"
Sep 10 19:41:24.449277 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:24.449086    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2028475e-bf7c-4451-82da-70fd2b0ccf26-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6\" (UID: \"2028475e-bf7c-4451-82da-70fd2b0ccf26\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6"
Sep 10 19:41:24.449277 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:24.449166    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66kgg\" (UniqueName: \"kubernetes.io/projected/2028475e-bf7c-4451-82da-70fd2b0ccf26-kube-api-access-66kgg\") pod \"maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6\" (UID: \"2028475e-bf7c-4451-82da-70fd2b0ccf26\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6"
Sep 10 19:41:24.449277 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:24.449251    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2028475e-bf7c-4451-82da-70fd2b0ccf26-metrics-certs\") pod \"maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6\" (UID: \"2028475e-bf7c-4451-82da-70fd2b0ccf26\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6"
Sep 10 19:41:24.550605 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:24.550570    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2028475e-bf7c-4451-82da-70fd2b0ccf26-maas-api-tls\") pod \"maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6\" (UID: \"2028475e-bf7c-4451-82da-70fd2b0ccf26\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6"
Sep 10 19:41:24.550847 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:24.550613    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2028475e-bf7c-4451-82da-70fd2b0ccf26-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6\" (UID: \"2028475e-bf7c-4451-82da-70fd2b0ccf26\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6"
Sep 10 19:41:24.550847 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:24.550737    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66kgg\" (UniqueName: \"kubernetes.io/projected/2028475e-bf7c-4451-82da-70fd2b0ccf26-kube-api-access-66kgg\") pod \"maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6\" (UID: \"2028475e-bf7c-4451-82da-70fd2b0ccf26\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6"
Sep 10 19:41:24.550847 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:24.550832    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2028475e-bf7c-4451-82da-70fd2b0ccf26-metrics-certs\") pod \"maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6\" (UID: \"2028475e-bf7c-4451-82da-70fd2b0ccf26\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6"
Sep 10 19:41:24.551370 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:24.551344    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2028475e-bf7c-4451-82da-70fd2b0ccf26-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6\" (UID: \"2028475e-bf7c-4451-82da-70fd2b0ccf26\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6"
Sep 10 19:41:24.553381 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:24.553352    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2028475e-bf7c-4451-82da-70fd2b0ccf26-metrics-certs\") pod \"maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6\" (UID: \"2028475e-bf7c-4451-82da-70fd2b0ccf26\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6"
Sep 10 19:41:24.553538 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:24.553520    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2028475e-bf7c-4451-82da-70fd2b0ccf26-maas-api-tls\") pod \"maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6\" (UID: \"2028475e-bf7c-4451-82da-70fd2b0ccf26\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6"
Sep 10 19:41:24.559133 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:24.559107    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66kgg\" (UniqueName: \"kubernetes.io/projected/2028475e-bf7c-4451-82da-70fd2b0ccf26-kube-api-access-66kgg\") pod \"maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6\" (UID: \"2028475e-bf7c-4451-82da-70fd2b0ccf26\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6"
Sep 10 19:41:24.620909 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:24.620874    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6"
Sep 10 19:41:24.902868 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:24.902834    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6"]
Sep 10 19:41:24.907291 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:41:24.907232    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2028475e_bf7c_4451_82da_70fd2b0ccf26.slice/crio-131e8b87f32cd328da7de1983ba32f4b579b107d17302289b3c01038671a7012 WatchSource:0}: Error finding container 131e8b87f32cd328da7de1983ba32f4b579b107d17302289b3c01038671a7012: Status 404 returned error can't find the container with id 131e8b87f32cd328da7de1983ba32f4b579b107d17302289b3c01038671a7012
Sep 10 19:41:25.601488 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:25.601380    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6" event={"ID":"2028475e-bf7c-4451-82da-70fd2b0ccf26","Type":"ContainerStarted","Data":"fd69804f084dc99a6c1dbea2dd4196cdd5dc381ddb2e5023abfc1460ae3f54e3"}
Sep 10 19:41:25.601488 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:25.601460    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6" event={"ID":"2028475e-bf7c-4451-82da-70fd2b0ccf26","Type":"ContainerStarted","Data":"131e8b87f32cd328da7de1983ba32f4b579b107d17302289b3c01038671a7012"}
Sep 10 19:41:25.601488 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:25.601488    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6"
Sep 10 19:41:25.620120 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:25.620078    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6" podStartSLOduration=1.231299456 podStartE2EDuration="1.620063274s" totalImagesPullingTime="388.763818ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:41:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:41:25.618657302 +0000 UTC m=+1540.838505786" watchObservedRunningTime="2026-09-10 19:41:25.620063274 +0000 UTC m=+1540.839911755"
Sep 10 19:41:31.613000 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:41:31.612968    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6"
Sep 10 19:42:03.192022 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:03.191989    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-df863b/ipp-route-df863b-kserve-c687f8dc-5rqht"]
Sep 10 19:42:03.197085 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:03.197066    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-df863b/ipp-route-df863b-kserve-c687f8dc-5rqht"
Sep 10 19:42:03.199189 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:03.199162    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-df863b\"/\"openshift-service-ca.crt\""
Sep 10 19:42:03.199803 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:03.199785    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-df863b\"/\"kube-root-ca.crt\""
Sep 10 19:42:03.199880 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:03.199785    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-df863b\"/\"default-dockercfg-8fhmc\""
Sep 10 19:42:03.199880 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:03.199851    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-df863b\"/\"ipp-route-df863b-kserve-self-signed-certs\""
Sep 10 19:42:03.205899 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:03.205876    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-df863b/ipp-route-df863b-kserve-c687f8dc-5rqht"]
Sep 10 19:42:03.326121 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:03.326085    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00407951-e8b0-4d42-be73-436f35e3d385-tls-certs\") pod \"ipp-route-df863b-kserve-c687f8dc-5rqht\" (UID: \"00407951-e8b0-4d42-be73-436f35e3d385\") " pod="ai-tenant-e2e-ipp-a-df863b/ipp-route-df863b-kserve-c687f8dc-5rqht"
Sep 10 19:42:03.326314 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:03.326179    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00407951-e8b0-4d42-be73-436f35e3d385-dshm\") pod \"ipp-route-df863b-kserve-c687f8dc-5rqht\" (UID: \"00407951-e8b0-4d42-be73-436f35e3d385\") " pod="ai-tenant-e2e-ipp-a-df863b/ipp-route-df863b-kserve-c687f8dc-5rqht"
Sep 10 19:42:03.326314 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:03.326284    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtj5t\" (UniqueName: \"kubernetes.io/projected/00407951-e8b0-4d42-be73-436f35e3d385-kube-api-access-jtj5t\") pod \"ipp-route-df863b-kserve-c687f8dc-5rqht\" (UID: \"00407951-e8b0-4d42-be73-436f35e3d385\") " pod="ai-tenant-e2e-ipp-a-df863b/ipp-route-df863b-kserve-c687f8dc-5rqht"
Sep 10 19:42:03.326421 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:03.326334    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00407951-e8b0-4d42-be73-436f35e3d385-home\") pod \"ipp-route-df863b-kserve-c687f8dc-5rqht\" (UID: \"00407951-e8b0-4d42-be73-436f35e3d385\") " pod="ai-tenant-e2e-ipp-a-df863b/ipp-route-df863b-kserve-c687f8dc-5rqht"
Sep 10 19:42:03.326421 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:03.326360    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00407951-e8b0-4d42-be73-436f35e3d385-model-cache\") pod \"ipp-route-df863b-kserve-c687f8dc-5rqht\" (UID: \"00407951-e8b0-4d42-be73-436f35e3d385\") " pod="ai-tenant-e2e-ipp-a-df863b/ipp-route-df863b-kserve-c687f8dc-5rqht"
Sep 10 19:42:03.326421 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:03.326379    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00407951-e8b0-4d42-be73-436f35e3d385-tmp-dir\") pod \"ipp-route-df863b-kserve-c687f8dc-5rqht\" (UID: \"00407951-e8b0-4d42-be73-436f35e3d385\") " pod="ai-tenant-e2e-ipp-a-df863b/ipp-route-df863b-kserve-c687f8dc-5rqht"
Sep 10 19:42:03.427473 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:03.427438    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00407951-e8b0-4d42-be73-436f35e3d385-home\") pod \"ipp-route-df863b-kserve-c687f8dc-5rqht\" (UID: \"00407951-e8b0-4d42-be73-436f35e3d385\") " pod="ai-tenant-e2e-ipp-a-df863b/ipp-route-df863b-kserve-c687f8dc-5rqht"
Sep 10 19:42:03.427473 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:03.427479    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00407951-e8b0-4d42-be73-436f35e3d385-model-cache\") pod \"ipp-route-df863b-kserve-c687f8dc-5rqht\" (UID: \"00407951-e8b0-4d42-be73-436f35e3d385\") " pod="ai-tenant-e2e-ipp-a-df863b/ipp-route-df863b-kserve-c687f8dc-5rqht"
Sep 10 19:42:03.427712 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:03.427497    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00407951-e8b0-4d42-be73-436f35e3d385-tmp-dir\") pod \"ipp-route-df863b-kserve-c687f8dc-5rqht\" (UID: \"00407951-e8b0-4d42-be73-436f35e3d385\") " pod="ai-tenant-e2e-ipp-a-df863b/ipp-route-df863b-kserve-c687f8dc-5rqht"
Sep 10 19:42:03.427712 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:03.427631    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00407951-e8b0-4d42-be73-436f35e3d385-tls-certs\") pod \"ipp-route-df863b-kserve-c687f8dc-5rqht\" (UID: \"00407951-e8b0-4d42-be73-436f35e3d385\") " pod="ai-tenant-e2e-ipp-a-df863b/ipp-route-df863b-kserve-c687f8dc-5rqht"
Sep 10 19:42:03.427869 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:03.427733    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00407951-e8b0-4d42-be73-436f35e3d385-dshm\") pod \"ipp-route-df863b-kserve-c687f8dc-5rqht\" (UID: \"00407951-e8b0-4d42-be73-436f35e3d385\") " pod="ai-tenant-e2e-ipp-a-df863b/ipp-route-df863b-kserve-c687f8dc-5rqht"
Sep 10 19:42:03.427869 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:03.427857    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtj5t\" (UniqueName: \"kubernetes.io/projected/00407951-e8b0-4d42-be73-436f35e3d385-kube-api-access-jtj5t\") pod \"ipp-route-df863b-kserve-c687f8dc-5rqht\" (UID: \"00407951-e8b0-4d42-be73-436f35e3d385\") " pod="ai-tenant-e2e-ipp-a-df863b/ipp-route-df863b-kserve-c687f8dc-5rqht"
Sep 10 19:42:03.427964 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:03.427875    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00407951-e8b0-4d42-be73-436f35e3d385-home\") pod \"ipp-route-df863b-kserve-c687f8dc-5rqht\" (UID: \"00407951-e8b0-4d42-be73-436f35e3d385\") " pod="ai-tenant-e2e-ipp-a-df863b/ipp-route-df863b-kserve-c687f8dc-5rqht"
Sep 10 19:42:03.427964 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:03.427920    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00407951-e8b0-4d42-be73-436f35e3d385-model-cache\") pod \"ipp-route-df863b-kserve-c687f8dc-5rqht\" (UID: \"00407951-e8b0-4d42-be73-436f35e3d385\") " pod="ai-tenant-e2e-ipp-a-df863b/ipp-route-df863b-kserve-c687f8dc-5rqht"
Sep 10 19:42:03.427964 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:03.427955    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00407951-e8b0-4d42-be73-436f35e3d385-tmp-dir\") pod \"ipp-route-df863b-kserve-c687f8dc-5rqht\" (UID: \"00407951-e8b0-4d42-be73-436f35e3d385\") " pod="ai-tenant-e2e-ipp-a-df863b/ipp-route-df863b-kserve-c687f8dc-5rqht"
Sep 10 19:42:03.430209 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:03.430187    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00407951-e8b0-4d42-be73-436f35e3d385-dshm\") pod \"ipp-route-df863b-kserve-c687f8dc-5rqht\" (UID: \"00407951-e8b0-4d42-be73-436f35e3d385\") " pod="ai-tenant-e2e-ipp-a-df863b/ipp-route-df863b-kserve-c687f8dc-5rqht"
Sep 10 19:42:03.430418 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:03.430400    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00407951-e8b0-4d42-be73-436f35e3d385-tls-certs\") pod \"ipp-route-df863b-kserve-c687f8dc-5rqht\" (UID: \"00407951-e8b0-4d42-be73-436f35e3d385\") " pod="ai-tenant-e2e-ipp-a-df863b/ipp-route-df863b-kserve-c687f8dc-5rqht"
Sep 10 19:42:03.437563 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:03.437543    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtj5t\" (UniqueName: \"kubernetes.io/projected/00407951-e8b0-4d42-be73-436f35e3d385-kube-api-access-jtj5t\") pod \"ipp-route-df863b-kserve-c687f8dc-5rqht\" (UID: \"00407951-e8b0-4d42-be73-436f35e3d385\") " pod="ai-tenant-e2e-ipp-a-df863b/ipp-route-df863b-kserve-c687f8dc-5rqht"
Sep 10 19:42:03.506645 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:03.506557    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-df863b/ipp-route-df863b-kserve-c687f8dc-5rqht"
Sep 10 19:42:03.782954 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:03.782926    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-df863b/ipp-route-df863b-kserve-c687f8dc-5rqht"]
Sep 10 19:42:03.783875 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:42:03.783848    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod00407951_e8b0_4d42_be73_436f35e3d385.slice/crio-aa4306f392cb44e843359ab2e05c414ff70be42b5551fd6f980313ed70fd6a3b WatchSource:0}: Error finding container aa4306f392cb44e843359ab2e05c414ff70be42b5551fd6f980313ed70fd6a3b: Status 404 returned error can't find the container with id aa4306f392cb44e843359ab2e05c414ff70be42b5551fd6f980313ed70fd6a3b
Sep 10 19:42:03.815223 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:03.815197    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-df863b/ipp-route-df863b-kserve-c687f8dc-5rqht" event={"ID":"00407951-e8b0-4d42-be73-436f35e3d385","Type":"ContainerStarted","Data":"aa4306f392cb44e843359ab2e05c414ff70be42b5551fd6f980313ed70fd6a3b"}
Sep 10 19:42:04.822280 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:04.822243    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-df863b/ipp-route-df863b-kserve-c687f8dc-5rqht" event={"ID":"00407951-e8b0-4d42-be73-436f35e3d385","Type":"ContainerStarted","Data":"fbdef1764c550e3fc07974dea366be037028128886194c46e0393661e167934d"}
Sep 10 19:42:04.839795 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:04.839729    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-df863b/ipp-route-df863b-kserve-c687f8dc-5rqht" podStartSLOduration=1.839714795 podStartE2EDuration="1.839714795s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:42:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:42:04.837945926 +0000 UTC m=+1580.057794531" watchObservedRunningTime="2026-09-10 19:42:04.839714795 +0000 UTC m=+1580.059563328"
Sep 10 19:42:13.507141 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:13.507104    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-df863b/ipp-route-df863b-kserve-c687f8dc-5rqht"
Sep 10 19:42:13.507510 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:13.507150    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-df863b/ipp-route-df863b-kserve-c687f8dc-5rqht"
Sep 10 19:42:13.524336 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:13.524304    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-df863b/ipp-route-df863b-kserve-c687f8dc-5rqht"
Sep 10 19:42:13.890017 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:13.889983    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-df863b/ipp-route-df863b-kserve-c687f8dc-5rqht"
Sep 10 19:42:34.709938 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:34.709906    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn"]
Sep 10 19:42:34.713806 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:34.713785    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn"
Sep 10 19:42:34.715948 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:34.715924    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-8a4013-serving-cert\""
Sep 10 19:42:34.725937 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:34.725911    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn"]
Sep 10 19:42:34.835247 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:34.835216    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41d2b767-335b-41bf-9e03-004b433653d8-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn\" (UID: \"41d2b767-335b-41bf-9e03-004b433653d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn"
Sep 10 19:42:34.835441 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:34.835278    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hspzp\" (UniqueName: \"kubernetes.io/projected/41d2b767-335b-41bf-9e03-004b433653d8-kube-api-access-hspzp\") pod \"maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn\" (UID: \"41d2b767-335b-41bf-9e03-004b433653d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn"
Sep 10 19:42:34.835441 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:34.835315    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41d2b767-335b-41bf-9e03-004b433653d8-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn\" (UID: \"41d2b767-335b-41bf-9e03-004b433653d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn"
Sep 10 19:42:34.835441 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:34.835386    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41d2b767-335b-41bf-9e03-004b433653d8-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn\" (UID: \"41d2b767-335b-41bf-9e03-004b433653d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn"
Sep 10 19:42:34.936033 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:34.936002    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41d2b767-335b-41bf-9e03-004b433653d8-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn\" (UID: \"41d2b767-335b-41bf-9e03-004b433653d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn"
Sep 10 19:42:34.936170 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:34.936135    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41d2b767-335b-41bf-9e03-004b433653d8-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn\" (UID: \"41d2b767-335b-41bf-9e03-004b433653d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn"
Sep 10 19:42:34.936224 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:34.936166    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hspzp\" (UniqueName: \"kubernetes.io/projected/41d2b767-335b-41bf-9e03-004b433653d8-kube-api-access-hspzp\") pod \"maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn\" (UID: \"41d2b767-335b-41bf-9e03-004b433653d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn"
Sep 10 19:42:34.936224 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:34.936202    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41d2b767-335b-41bf-9e03-004b433653d8-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn\" (UID: \"41d2b767-335b-41bf-9e03-004b433653d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn"
Sep 10 19:42:34.936296 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:42:34.936273    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a4013-serving-cert: secret "maas-api-e2e-ipp-cleanup-8a4013-serving-cert" not found
Sep 10 19:42:34.936347 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:42:34.936336    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41d2b767-335b-41bf-9e03-004b433653d8-maas-api-tls podName:41d2b767-335b-41bf-9e03-004b433653d8 nodeName:}" failed. No retries permitted until 2026-09-10 19:42:35.436320485 +0000 UTC m=+1610.656168972 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/41d2b767-335b-41bf-9e03-004b433653d8-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn" (UID: "41d2b767-335b-41bf-9e03-004b433653d8") : secret "maas-api-e2e-ipp-cleanup-8a4013-serving-cert" not found
Sep 10 19:42:34.936812 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:34.936794    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41d2b767-335b-41bf-9e03-004b433653d8-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn\" (UID: \"41d2b767-335b-41bf-9e03-004b433653d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn"
Sep 10 19:42:34.938575 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:34.938558    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41d2b767-335b-41bf-9e03-004b433653d8-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn\" (UID: \"41d2b767-335b-41bf-9e03-004b433653d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn"
Sep 10 19:42:34.945090 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:34.945070    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hspzp\" (UniqueName: \"kubernetes.io/projected/41d2b767-335b-41bf-9e03-004b433653d8-kube-api-access-hspzp\") pod \"maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn\" (UID: \"41d2b767-335b-41bf-9e03-004b433653d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn"
Sep 10 19:42:35.444842 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:35.444806    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41d2b767-335b-41bf-9e03-004b433653d8-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn\" (UID: \"41d2b767-335b-41bf-9e03-004b433653d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn"
Sep 10 19:42:35.447379 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:35.447348    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41d2b767-335b-41bf-9e03-004b433653d8-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn\" (UID: \"41d2b767-335b-41bf-9e03-004b433653d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn"
Sep 10 19:42:35.627252 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:35.627211    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn"
Sep 10 19:42:35.907327 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:35.907301    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn"]
Sep 10 19:42:35.908628 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:42:35.908604    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41d2b767_335b_41bf_9e03_004b433653d8.slice/crio-7fc833b54606678f7f1311bfe605493a59314a847f6b328bc09e68822b853f85 WatchSource:0}: Error finding container 7fc833b54606678f7f1311bfe605493a59314a847f6b328bc09e68822b853f85: Status 404 returned error can't find the container with id 7fc833b54606678f7f1311bfe605493a59314a847f6b328bc09e68822b853f85
Sep 10 19:42:36.012361 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:36.012325    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn" event={"ID":"41d2b767-335b-41bf-9e03-004b433653d8","Type":"ContainerStarted","Data":"7fc833b54606678f7f1311bfe605493a59314a847f6b328bc09e68822b853f85"}
Sep 10 19:42:37.020889 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:37.020851    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn" event={"ID":"41d2b767-335b-41bf-9e03-004b433653d8","Type":"ContainerStarted","Data":"733b46f0355d16e474a8466b5f7ef968159601a51904d2b3b2feb0ebb43bfc56"}
Sep 10 19:42:37.021341 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:37.020913    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn"
Sep 10 19:42:37.038731 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:37.038676    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn" podStartSLOduration=2.699515658 podStartE2EDuration="3.038659989s" totalImagesPullingTime="339.144331ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:42:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:42:37.036123248 +0000 UTC m=+1612.255971732" watchObservedRunningTime="2026-09-10 19:42:37.038659989 +0000 UTC m=+1612.258508516"
Sep 10 19:42:43.032668 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:42:43.032639    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn"
Sep 10 19:43:05.461535 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:05.461499    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn"]
Sep 10 19:43:05.461969 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:05.461707    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn" podUID="41d2b767-335b-41bf-9e03-004b433653d8" containerName="maas-api" containerID="cri-o://733b46f0355d16e474a8466b5f7ef968159601a51904d2b3b2feb0ebb43bfc56" gracePeriod=30
Sep 10 19:43:05.725319 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:05.725253    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn"
Sep 10 19:43:05.843484 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:05.843449    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41d2b767-335b-41bf-9e03-004b433653d8-openshift-service-ca\") pod \"41d2b767-335b-41bf-9e03-004b433653d8\" (UID: \"41d2b767-335b-41bf-9e03-004b433653d8\") "
Sep 10 19:43:05.843808 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:05.843546    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41d2b767-335b-41bf-9e03-004b433653d8-metrics-certs\") pod \"41d2b767-335b-41bf-9e03-004b433653d8\" (UID: \"41d2b767-335b-41bf-9e03-004b433653d8\") "
Sep 10 19:43:05.843808 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:05.843616    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41d2b767-335b-41bf-9e03-004b433653d8-maas-api-tls\") pod \"41d2b767-335b-41bf-9e03-004b433653d8\" (UID: \"41d2b767-335b-41bf-9e03-004b433653d8\") "
Sep 10 19:43:05.843808 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:05.843673    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hspzp\" (UniqueName: \"kubernetes.io/projected/41d2b767-335b-41bf-9e03-004b433653d8-kube-api-access-hspzp\") pod \"41d2b767-335b-41bf-9e03-004b433653d8\" (UID: \"41d2b767-335b-41bf-9e03-004b433653d8\") "
Sep 10 19:43:05.844021 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:05.843858    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41d2b767-335b-41bf-9e03-004b433653d8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "41d2b767-335b-41bf-9e03-004b433653d8" (UID: "41d2b767-335b-41bf-9e03-004b433653d8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:43:05.844021 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:05.843984    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41d2b767-335b-41bf-9e03-004b433653d8-openshift-service-ca\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:43:05.845856 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:05.845828    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41d2b767-335b-41bf-9e03-004b433653d8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "41d2b767-335b-41bf-9e03-004b433653d8" (UID: "41d2b767-335b-41bf-9e03-004b433653d8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:43:05.845856 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:05.845848    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41d2b767-335b-41bf-9e03-004b433653d8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "41d2b767-335b-41bf-9e03-004b433653d8" (UID: "41d2b767-335b-41bf-9e03-004b433653d8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:43:05.846093 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:05.845945    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41d2b767-335b-41bf-9e03-004b433653d8-kube-api-access-hspzp" (OuterVolumeSpecName: "kube-api-access-hspzp") pod "41d2b767-335b-41bf-9e03-004b433653d8" (UID: "41d2b767-335b-41bf-9e03-004b433653d8"). InnerVolumeSpecName "kube-api-access-hspzp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:43:05.944578 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:05.944540    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41d2b767-335b-41bf-9e03-004b433653d8-maas-api-tls\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:43:05.944578 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:05.944570    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hspzp\" (UniqueName: \"kubernetes.io/projected/41d2b767-335b-41bf-9e03-004b433653d8-kube-api-access-hspzp\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:43:05.944578 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:05.944580    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41d2b767-335b-41bf-9e03-004b433653d8-metrics-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:43:06.208029 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:06.207982    2579 generic.go:356] "Generic (PLEG): container finished" podID="41d2b767-335b-41bf-9e03-004b433653d8" containerID="733b46f0355d16e474a8466b5f7ef968159601a51904d2b3b2feb0ebb43bfc56" exitCode=0
Sep 10 19:43:06.208029 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:06.208028    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn"
Sep 10 19:43:06.208283 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:06.208045    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn" event={"ID":"41d2b767-335b-41bf-9e03-004b433653d8","Type":"ContainerDied","Data":"733b46f0355d16e474a8466b5f7ef968159601a51904d2b3b2feb0ebb43bfc56"}
Sep 10 19:43:06.208283 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:06.208100    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn" event={"ID":"41d2b767-335b-41bf-9e03-004b433653d8","Type":"ContainerDied","Data":"7fc833b54606678f7f1311bfe605493a59314a847f6b328bc09e68822b853f85"}
Sep 10 19:43:06.208283 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:06.208124    2579 scope.go:117] "RemoveContainer" containerID="733b46f0355d16e474a8466b5f7ef968159601a51904d2b3b2feb0ebb43bfc56"
Sep 10 19:43:06.218372 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:06.218354    2579 scope.go:117] "RemoveContainer" containerID="733b46f0355d16e474a8466b5f7ef968159601a51904d2b3b2feb0ebb43bfc56"
Sep 10 19:43:06.218661 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:43:06.218639    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"733b46f0355d16e474a8466b5f7ef968159601a51904d2b3b2feb0ebb43bfc56\": container with ID starting with 733b46f0355d16e474a8466b5f7ef968159601a51904d2b3b2feb0ebb43bfc56 not found: ID does not exist" containerID="733b46f0355d16e474a8466b5f7ef968159601a51904d2b3b2feb0ebb43bfc56"
Sep 10 19:43:06.218725 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:06.218670    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"733b46f0355d16e474a8466b5f7ef968159601a51904d2b3b2feb0ebb43bfc56"} err="failed to get container status \"733b46f0355d16e474a8466b5f7ef968159601a51904d2b3b2feb0ebb43bfc56\": rpc error: code = NotFound desc = could not find container \"733b46f0355d16e474a8466b5f7ef968159601a51904d2b3b2feb0ebb43bfc56\": container with ID starting with 733b46f0355d16e474a8466b5f7ef968159601a51904d2b3b2feb0ebb43bfc56 not found: ID does not exist"
Sep 10 19:43:06.231120 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:06.231088    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn"]
Sep 10 19:43:06.237232 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:06.237209    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8a4013-7746988d87-bsnwn"]
Sep 10 19:43:07.240011 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:07.239971    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41d2b767-335b-41bf-9e03-004b433653d8" path="/var/lib/kubelet/pods/41d2b767-335b-41bf-9e03-004b433653d8/volumes"
Sep 10 19:43:32.474677 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:32.474639    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx"]
Sep 10 19:43:32.475099 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:32.474833    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx" podUID="678724f6-2a61-40e3-ba09-ea8099e5e3fc" containerName="maas-api" containerID="cri-o://d22e270e71891160d832a0eae4082e6a77feafc4461ab27b63ef07ffeda97bfe" gracePeriod=30
Sep 10 19:43:32.731196 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:32.731129    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx"
Sep 10 19:43:32.806397 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:32.806363    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/678724f6-2a61-40e3-ba09-ea8099e5e3fc-openshift-service-ca\") pod \"678724f6-2a61-40e3-ba09-ea8099e5e3fc\" (UID: \"678724f6-2a61-40e3-ba09-ea8099e5e3fc\") "
Sep 10 19:43:32.806559 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:32.806431    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cg28g\" (UniqueName: \"kubernetes.io/projected/678724f6-2a61-40e3-ba09-ea8099e5e3fc-kube-api-access-cg28g\") pod \"678724f6-2a61-40e3-ba09-ea8099e5e3fc\" (UID: \"678724f6-2a61-40e3-ba09-ea8099e5e3fc\") "
Sep 10 19:43:32.806559 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:32.806449    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/678724f6-2a61-40e3-ba09-ea8099e5e3fc-metrics-certs\") pod \"678724f6-2a61-40e3-ba09-ea8099e5e3fc\" (UID: \"678724f6-2a61-40e3-ba09-ea8099e5e3fc\") "
Sep 10 19:43:32.806559 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:32.806489    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/678724f6-2a61-40e3-ba09-ea8099e5e3fc-maas-api-tls\") pod \"678724f6-2a61-40e3-ba09-ea8099e5e3fc\" (UID: \"678724f6-2a61-40e3-ba09-ea8099e5e3fc\") "
Sep 10 19:43:32.806859 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:32.806831    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/678724f6-2a61-40e3-ba09-ea8099e5e3fc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "678724f6-2a61-40e3-ba09-ea8099e5e3fc" (UID: "678724f6-2a61-40e3-ba09-ea8099e5e3fc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:43:32.808806 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:32.808775    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/678724f6-2a61-40e3-ba09-ea8099e5e3fc-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "678724f6-2a61-40e3-ba09-ea8099e5e3fc" (UID: "678724f6-2a61-40e3-ba09-ea8099e5e3fc"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:43:32.808921 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:32.808826    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/678724f6-2a61-40e3-ba09-ea8099e5e3fc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "678724f6-2a61-40e3-ba09-ea8099e5e3fc" (UID: "678724f6-2a61-40e3-ba09-ea8099e5e3fc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:43:32.808921 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:32.808858    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/678724f6-2a61-40e3-ba09-ea8099e5e3fc-kube-api-access-cg28g" (OuterVolumeSpecName: "kube-api-access-cg28g") pod "678724f6-2a61-40e3-ba09-ea8099e5e3fc" (UID: "678724f6-2a61-40e3-ba09-ea8099e5e3fc"). InnerVolumeSpecName "kube-api-access-cg28g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:43:32.907830 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:32.907788    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/678724f6-2a61-40e3-ba09-ea8099e5e3fc-maas-api-tls\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:43:32.907830 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:32.907822    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/678724f6-2a61-40e3-ba09-ea8099e5e3fc-openshift-service-ca\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:43:32.907830 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:32.907833    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cg28g\" (UniqueName: \"kubernetes.io/projected/678724f6-2a61-40e3-ba09-ea8099e5e3fc-kube-api-access-cg28g\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:43:32.907830 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:32.907841    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/678724f6-2a61-40e3-ba09-ea8099e5e3fc-metrics-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:43:33.367136 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:33.367100    2579 generic.go:356] "Generic (PLEG): container finished" podID="678724f6-2a61-40e3-ba09-ea8099e5e3fc" containerID="d22e270e71891160d832a0eae4082e6a77feafc4461ab27b63ef07ffeda97bfe" exitCode=0
Sep 10 19:43:33.367305 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:33.367171    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx"
Sep 10 19:43:33.367305 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:33.367187    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx" event={"ID":"678724f6-2a61-40e3-ba09-ea8099e5e3fc","Type":"ContainerDied","Data":"d22e270e71891160d832a0eae4082e6a77feafc4461ab27b63ef07ffeda97bfe"}
Sep 10 19:43:33.367305 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:33.367238    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx" event={"ID":"678724f6-2a61-40e3-ba09-ea8099e5e3fc","Type":"ContainerDied","Data":"8abbc0cdae4a1ec534dc40c3cb23b256225bdbb50229d57e73153e40365b26fd"}
Sep 10 19:43:33.367305 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:33.367265    2579 scope.go:117] "RemoveContainer" containerID="d22e270e71891160d832a0eae4082e6a77feafc4461ab27b63ef07ffeda97bfe"
Sep 10 19:43:33.376517 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:33.376491    2579 scope.go:117] "RemoveContainer" containerID="d22e270e71891160d832a0eae4082e6a77feafc4461ab27b63ef07ffeda97bfe"
Sep 10 19:43:33.376816 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:43:33.376796    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d22e270e71891160d832a0eae4082e6a77feafc4461ab27b63ef07ffeda97bfe\": container with ID starting with d22e270e71891160d832a0eae4082e6a77feafc4461ab27b63ef07ffeda97bfe not found: ID does not exist" containerID="d22e270e71891160d832a0eae4082e6a77feafc4461ab27b63ef07ffeda97bfe"
Sep 10 19:43:33.376871 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:33.376824    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d22e270e71891160d832a0eae4082e6a77feafc4461ab27b63ef07ffeda97bfe"} err="failed to get container status \"d22e270e71891160d832a0eae4082e6a77feafc4461ab27b63ef07ffeda97bfe\": rpc error: code = NotFound desc = could not find container \"d22e270e71891160d832a0eae4082e6a77feafc4461ab27b63ef07ffeda97bfe\": container with ID starting with d22e270e71891160d832a0eae4082e6a77feafc4461ab27b63ef07ffeda97bfe not found: ID does not exist"
Sep 10 19:43:33.384627 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:33.384605    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx"]
Sep 10 19:43:33.387345 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:33.387325    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-df863b-54f89f6f4-86jfx"]
Sep 10 19:43:35.239987 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:35.239957    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="678724f6-2a61-40e3-ba09-ea8099e5e3fc" path="/var/lib/kubelet/pods/678724f6-2a61-40e3-ba09-ea8099e5e3fc/volumes"
Sep 10 19:43:38.151094 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:38.151054    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-df863b/ipp-route-df863b-kserve-c687f8dc-5rqht"]
Sep 10 19:43:53.171651 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:53.171600    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-df863b/ipp-route-df863b-kserve-c687f8dc-5rqht" podUID="00407951-e8b0-4d42-be73-436f35e3d385" containerName="main" containerID="cri-o://fbdef1764c550e3fc07974dea366be037028128886194c46e0393661e167934d" gracePeriod=45
Sep 10 19:43:53.434963 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:53.434899    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-df863b/ipp-route-df863b-kserve-c687f8dc-5rqht"
Sep 10 19:43:53.489592 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:53.489556    2579 generic.go:356] "Generic (PLEG): container finished" podID="00407951-e8b0-4d42-be73-436f35e3d385" containerID="fbdef1764c550e3fc07974dea366be037028128886194c46e0393661e167934d" exitCode=0
Sep 10 19:43:53.489794 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:53.489650    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-df863b/ipp-route-df863b-kserve-c687f8dc-5rqht"
Sep 10 19:43:53.489794 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:53.489692    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-df863b/ipp-route-df863b-kserve-c687f8dc-5rqht" event={"ID":"00407951-e8b0-4d42-be73-436f35e3d385","Type":"ContainerDied","Data":"fbdef1764c550e3fc07974dea366be037028128886194c46e0393661e167934d"}
Sep 10 19:43:53.489794 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:53.489777    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-df863b/ipp-route-df863b-kserve-c687f8dc-5rqht" event={"ID":"00407951-e8b0-4d42-be73-436f35e3d385","Type":"ContainerDied","Data":"aa4306f392cb44e843359ab2e05c414ff70be42b5551fd6f980313ed70fd6a3b"}
Sep 10 19:43:53.489928 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:53.489806    2579 scope.go:117] "RemoveContainer" containerID="fbdef1764c550e3fc07974dea366be037028128886194c46e0393661e167934d"
Sep 10 19:43:53.499181 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:53.499165    2579 scope.go:117] "RemoveContainer" containerID="fbdef1764c550e3fc07974dea366be037028128886194c46e0393661e167934d"
Sep 10 19:43:53.499459 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:43:53.499441    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbdef1764c550e3fc07974dea366be037028128886194c46e0393661e167934d\": container with ID starting with fbdef1764c550e3fc07974dea366be037028128886194c46e0393661e167934d not found: ID does not exist" containerID="fbdef1764c550e3fc07974dea366be037028128886194c46e0393661e167934d"
Sep 10 19:43:53.499517 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:53.499467    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbdef1764c550e3fc07974dea366be037028128886194c46e0393661e167934d"} err="failed to get container status \"fbdef1764c550e3fc07974dea366be037028128886194c46e0393661e167934d\": rpc error: code = NotFound desc = could not find container \"fbdef1764c550e3fc07974dea366be037028128886194c46e0393661e167934d\": container with ID starting with fbdef1764c550e3fc07974dea366be037028128886194c46e0393661e167934d not found: ID does not exist"
Sep 10 19:43:53.507268 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:53.507252    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00407951-e8b0-4d42-be73-436f35e3d385-model-cache\") pod \"00407951-e8b0-4d42-be73-436f35e3d385\" (UID: \"00407951-e8b0-4d42-be73-436f35e3d385\") "
Sep 10 19:43:53.507354 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:53.507294    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00407951-e8b0-4d42-be73-436f35e3d385-dshm\") pod \"00407951-e8b0-4d42-be73-436f35e3d385\" (UID: \"00407951-e8b0-4d42-be73-436f35e3d385\") "
Sep 10 19:43:53.507418 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:53.507378    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jtj5t\" (UniqueName: \"kubernetes.io/projected/00407951-e8b0-4d42-be73-436f35e3d385-kube-api-access-jtj5t\") pod \"00407951-e8b0-4d42-be73-436f35e3d385\" (UID: \"00407951-e8b0-4d42-be73-436f35e3d385\") "
Sep 10 19:43:53.507418 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:53.507409    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00407951-e8b0-4d42-be73-436f35e3d385-home\") pod \"00407951-e8b0-4d42-be73-436f35e3d385\" (UID: \"00407951-e8b0-4d42-be73-436f35e3d385\") "
Sep 10 19:43:53.507526 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:53.507477    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00407951-e8b0-4d42-be73-436f35e3d385-tls-certs\") pod \"00407951-e8b0-4d42-be73-436f35e3d385\" (UID: \"00407951-e8b0-4d42-be73-436f35e3d385\") "
Sep 10 19:43:53.507580 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:53.507529    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00407951-e8b0-4d42-be73-436f35e3d385-tmp-dir\") pod \"00407951-e8b0-4d42-be73-436f35e3d385\" (UID: \"00407951-e8b0-4d42-be73-436f35e3d385\") "
Sep 10 19:43:53.507580 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:53.507532    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00407951-e8b0-4d42-be73-436f35e3d385-model-cache" (OuterVolumeSpecName: "model-cache") pod "00407951-e8b0-4d42-be73-436f35e3d385" (UID: "00407951-e8b0-4d42-be73-436f35e3d385"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:43:53.507769 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:53.507724    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00407951-e8b0-4d42-be73-436f35e3d385-home" (OuterVolumeSpecName: "home") pod "00407951-e8b0-4d42-be73-436f35e3d385" (UID: "00407951-e8b0-4d42-be73-436f35e3d385"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:43:53.507884 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:53.507853    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00407951-e8b0-4d42-be73-436f35e3d385-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "00407951-e8b0-4d42-be73-436f35e3d385" (UID: "00407951-e8b0-4d42-be73-436f35e3d385"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:43:53.507997 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:53.507979    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00407951-e8b0-4d42-be73-436f35e3d385-home\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:43:53.508060 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:53.508002    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00407951-e8b0-4d42-be73-436f35e3d385-tmp-dir\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:43:53.508060 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:53.508016    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00407951-e8b0-4d42-be73-436f35e3d385-model-cache\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:43:53.509715 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:53.509679    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00407951-e8b0-4d42-be73-436f35e3d385-dshm" (OuterVolumeSpecName: "dshm") pod "00407951-e8b0-4d42-be73-436f35e3d385" (UID: "00407951-e8b0-4d42-be73-436f35e3d385"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:43:53.509860 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:53.509768    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00407951-e8b0-4d42-be73-436f35e3d385-kube-api-access-jtj5t" (OuterVolumeSpecName: "kube-api-access-jtj5t") pod "00407951-e8b0-4d42-be73-436f35e3d385" (UID: "00407951-e8b0-4d42-be73-436f35e3d385"). InnerVolumeSpecName "kube-api-access-jtj5t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:43:53.509860 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:53.509788    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00407951-e8b0-4d42-be73-436f35e3d385-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "00407951-e8b0-4d42-be73-436f35e3d385" (UID: "00407951-e8b0-4d42-be73-436f35e3d385"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:43:53.608604 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:53.608573    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00407951-e8b0-4d42-be73-436f35e3d385-dshm\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:43:53.608604 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:53.608600    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtj5t\" (UniqueName: \"kubernetes.io/projected/00407951-e8b0-4d42-be73-436f35e3d385-kube-api-access-jtj5t\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:43:53.608604 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:53.608610    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00407951-e8b0-4d42-be73-436f35e3d385-tls-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:43:53.814254 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:53.814225    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-df863b/ipp-route-df863b-kserve-c687f8dc-5rqht"]
Sep 10 19:43:53.822041 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:53.822015    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-df863b/ipp-route-df863b-kserve-c687f8dc-5rqht"]
Sep 10 19:43:55.241397 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:43:55.241364    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00407951-e8b0-4d42-be73-436f35e3d385" path="/var/lib/kubelet/pods/00407951-e8b0-4d42-be73-436f35e3d385/volumes"
Sep 10 19:44:38.549093 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:38.549059    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6"]
Sep 10 19:44:38.551743 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:38.549348    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6" podUID="2028475e-bf7c-4451-82da-70fd2b0ccf26" containerName="maas-api" containerID="cri-o://fd69804f084dc99a6c1dbea2dd4196cdd5dc381ddb2e5023abfc1460ae3f54e3" gracePeriod=30
Sep 10 19:44:38.768359 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:38.768330    2579 generic.go:356] "Generic (PLEG): container finished" podID="2028475e-bf7c-4451-82da-70fd2b0ccf26" containerID="fd69804f084dc99a6c1dbea2dd4196cdd5dc381ddb2e5023abfc1460ae3f54e3" exitCode=0
Sep 10 19:44:38.768511 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:38.768376    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6" event={"ID":"2028475e-bf7c-4451-82da-70fd2b0ccf26","Type":"ContainerDied","Data":"fd69804f084dc99a6c1dbea2dd4196cdd5dc381ddb2e5023abfc1460ae3f54e3"}
Sep 10 19:44:38.809597 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:38.809532    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6"
Sep 10 19:44:38.952407 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:38.952370    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66kgg\" (UniqueName: \"kubernetes.io/projected/2028475e-bf7c-4451-82da-70fd2b0ccf26-kube-api-access-66kgg\") pod \"2028475e-bf7c-4451-82da-70fd2b0ccf26\" (UID: \"2028475e-bf7c-4451-82da-70fd2b0ccf26\") "
Sep 10 19:44:38.952621 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:38.952449    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2028475e-bf7c-4451-82da-70fd2b0ccf26-openshift-service-ca\") pod \"2028475e-bf7c-4451-82da-70fd2b0ccf26\" (UID: \"2028475e-bf7c-4451-82da-70fd2b0ccf26\") "
Sep 10 19:44:38.952621 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:38.952478    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2028475e-bf7c-4451-82da-70fd2b0ccf26-metrics-certs\") pod \"2028475e-bf7c-4451-82da-70fd2b0ccf26\" (UID: \"2028475e-bf7c-4451-82da-70fd2b0ccf26\") "
Sep 10 19:44:38.952621 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:38.952581    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2028475e-bf7c-4451-82da-70fd2b0ccf26-maas-api-tls\") pod \"2028475e-bf7c-4451-82da-70fd2b0ccf26\" (UID: \"2028475e-bf7c-4451-82da-70fd2b0ccf26\") "
Sep 10 19:44:38.952920 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:38.952878    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2028475e-bf7c-4451-82da-70fd2b0ccf26-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2028475e-bf7c-4451-82da-70fd2b0ccf26" (UID: "2028475e-bf7c-4451-82da-70fd2b0ccf26"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:44:38.953044 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:38.953025    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2028475e-bf7c-4451-82da-70fd2b0ccf26-openshift-service-ca\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:44:38.955548 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:38.955519    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2028475e-bf7c-4451-82da-70fd2b0ccf26-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2028475e-bf7c-4451-82da-70fd2b0ccf26" (UID: "2028475e-bf7c-4451-82da-70fd2b0ccf26"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:44:38.957316 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:38.957233    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2028475e-bf7c-4451-82da-70fd2b0ccf26-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2028475e-bf7c-4451-82da-70fd2b0ccf26" (UID: "2028475e-bf7c-4451-82da-70fd2b0ccf26"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:44:38.957965 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:38.957939    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2028475e-bf7c-4451-82da-70fd2b0ccf26-kube-api-access-66kgg" (OuterVolumeSpecName: "kube-api-access-66kgg") pod "2028475e-bf7c-4451-82da-70fd2b0ccf26" (UID: "2028475e-bf7c-4451-82da-70fd2b0ccf26"). InnerVolumeSpecName "kube-api-access-66kgg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:44:39.054543 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:39.054504    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66kgg\" (UniqueName: \"kubernetes.io/projected/2028475e-bf7c-4451-82da-70fd2b0ccf26-kube-api-access-66kgg\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:44:39.054543 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:39.054536    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2028475e-bf7c-4451-82da-70fd2b0ccf26-metrics-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:44:39.054543 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:39.054544    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2028475e-bf7c-4451-82da-70fd2b0ccf26-maas-api-tls\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:44:39.775811 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:39.775780    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6"
Sep 10 19:44:39.776311 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:39.775713    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6" event={"ID":"2028475e-bf7c-4451-82da-70fd2b0ccf26","Type":"ContainerDied","Data":"131e8b87f32cd328da7de1983ba32f4b579b107d17302289b3c01038671a7012"}
Sep 10 19:44:39.776311 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:39.775904    2579 scope.go:117] "RemoveContainer" containerID="fd69804f084dc99a6c1dbea2dd4196cdd5dc381ddb2e5023abfc1460ae3f54e3"
Sep 10 19:44:39.796928 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:39.796903    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6"]
Sep 10 19:44:39.800549 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:39.800526    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-3765cf-5f7d54f54b-hknh6"]
Sep 10 19:44:41.239800 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:41.239731    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2028475e-bf7c-4451-82da-70fd2b0ccf26" path="/var/lib/kubelet/pods/2028475e-bf7c-4451-82da-70fd2b0ccf26/volumes"
Sep 10 19:44:56.336863 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:56.336819    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj"]
Sep 10 19:44:56.337465 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:56.337109    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj" podUID="5a6d5e92-b3fb-401f-890d-32c1d7672be9" containerName="maas-api" containerID="cri-o://318b6035afbdc510771241fc08934deabc8208ad16a016c5d11be724511f9e05" gracePeriod=30
Sep 10 19:44:56.601043 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:56.600975    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj"
Sep 10 19:44:56.612471 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:56.612445    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a6d5e92-b3fb-401f-890d-32c1d7672be9-maas-api-tls\") pod \"5a6d5e92-b3fb-401f-890d-32c1d7672be9\" (UID: \"5a6d5e92-b3fb-401f-890d-32c1d7672be9\") "
Sep 10 19:44:56.612664 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:56.612478    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8v65x\" (UniqueName: \"kubernetes.io/projected/5a6d5e92-b3fb-401f-890d-32c1d7672be9-kube-api-access-8v65x\") pod \"5a6d5e92-b3fb-401f-890d-32c1d7672be9\" (UID: \"5a6d5e92-b3fb-401f-890d-32c1d7672be9\") "
Sep 10 19:44:56.612664 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:56.612503    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5a6d5e92-b3fb-401f-890d-32c1d7672be9-metrics-certs\") pod \"5a6d5e92-b3fb-401f-890d-32c1d7672be9\" (UID: \"5a6d5e92-b3fb-401f-890d-32c1d7672be9\") "
Sep 10 19:44:56.612664 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:56.612574    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a6d5e92-b3fb-401f-890d-32c1d7672be9-openshift-service-ca\") pod \"5a6d5e92-b3fb-401f-890d-32c1d7672be9\" (UID: \"5a6d5e92-b3fb-401f-890d-32c1d7672be9\") "
Sep 10 19:44:56.613035 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:56.613000    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a6d5e92-b3fb-401f-890d-32c1d7672be9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5a6d5e92-b3fb-401f-890d-32c1d7672be9" (UID: "5a6d5e92-b3fb-401f-890d-32c1d7672be9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:44:56.614971 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:56.614933    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a6d5e92-b3fb-401f-890d-32c1d7672be9-kube-api-access-8v65x" (OuterVolumeSpecName: "kube-api-access-8v65x") pod "5a6d5e92-b3fb-401f-890d-32c1d7672be9" (UID: "5a6d5e92-b3fb-401f-890d-32c1d7672be9"). InnerVolumeSpecName "kube-api-access-8v65x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:44:56.615311 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:56.615288    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a6d5e92-b3fb-401f-890d-32c1d7672be9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5a6d5e92-b3fb-401f-890d-32c1d7672be9" (UID: "5a6d5e92-b3fb-401f-890d-32c1d7672be9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:44:56.615400 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:56.615327    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a6d5e92-b3fb-401f-890d-32c1d7672be9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5a6d5e92-b3fb-401f-890d-32c1d7672be9" (UID: "5a6d5e92-b3fb-401f-890d-32c1d7672be9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:44:56.713934 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:56.713902    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a6d5e92-b3fb-401f-890d-32c1d7672be9-maas-api-tls\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:44:56.713934 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:56.713930    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8v65x\" (UniqueName: \"kubernetes.io/projected/5a6d5e92-b3fb-401f-890d-32c1d7672be9-kube-api-access-8v65x\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:44:56.713934 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:56.713943    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5a6d5e92-b3fb-401f-890d-32c1d7672be9-metrics-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:44:56.714155 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:56.713952    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a6d5e92-b3fb-401f-890d-32c1d7672be9-openshift-service-ca\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:44:56.876970 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:56.876875    2579 generic.go:356] "Generic (PLEG): container finished" podID="5a6d5e92-b3fb-401f-890d-32c1d7672be9" containerID="318b6035afbdc510771241fc08934deabc8208ad16a016c5d11be724511f9e05" exitCode=0
Sep 10 19:44:56.876970 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:56.876925    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj"
Sep 10 19:44:56.876970 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:56.876956    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj" event={"ID":"5a6d5e92-b3fb-401f-890d-32c1d7672be9","Type":"ContainerDied","Data":"318b6035afbdc510771241fc08934deabc8208ad16a016c5d11be724511f9e05"}
Sep 10 19:44:56.877208 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:56.877010    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj" event={"ID":"5a6d5e92-b3fb-401f-890d-32c1d7672be9","Type":"ContainerDied","Data":"a64b0b775e9a43f3687093cc5df76dec028bdbb77713bafe73335555485e3293"}
Sep 10 19:44:56.877208 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:56.877036    2579 scope.go:117] "RemoveContainer" containerID="318b6035afbdc510771241fc08934deabc8208ad16a016c5d11be724511f9e05"
Sep 10 19:44:56.887851 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:56.887826    2579 scope.go:117] "RemoveContainer" containerID="318b6035afbdc510771241fc08934deabc8208ad16a016c5d11be724511f9e05"
Sep 10 19:44:56.888210 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:44:56.888178    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"318b6035afbdc510771241fc08934deabc8208ad16a016c5d11be724511f9e05\": container with ID starting with 318b6035afbdc510771241fc08934deabc8208ad16a016c5d11be724511f9e05 not found: ID does not exist" containerID="318b6035afbdc510771241fc08934deabc8208ad16a016c5d11be724511f9e05"
Sep 10 19:44:56.888357 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:56.888220    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"318b6035afbdc510771241fc08934deabc8208ad16a016c5d11be724511f9e05"} err="failed to get container status \"318b6035afbdc510771241fc08934deabc8208ad16a016c5d11be724511f9e05\": rpc error: code = NotFound desc = could not find container \"318b6035afbdc510771241fc08934deabc8208ad16a016c5d11be724511f9e05\": container with ID starting with 318b6035afbdc510771241fc08934deabc8208ad16a016c5d11be724511f9e05 not found: ID does not exist"
Sep 10 19:44:56.899730 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:56.899700    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj"]
Sep 10 19:44:56.904249 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:56.904224    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14a8d1-9b6964c5c-5w6rj"]
Sep 10 19:44:57.240160 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:44:57.240126    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a6d5e92-b3fb-401f-890d-32c1d7672be9" path="/var/lib/kubelet/pods/5a6d5e92-b3fb-401f-890d-32c1d7672be9/volumes"
Sep 10 19:45:03.445601 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:03.445567    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"]
Sep 10 19:45:03.446097 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:03.445861    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89" podUID="0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0" containerName="istio-proxy" containerID="cri-o://68248b55e153c45fe636be1dcbc63f178538ee29629bd16c4e1b197defe5a3e0" gracePeriod=30
Sep 10 19:45:06.824355 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:06.824321    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 19:45:06.824740 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:06.824372    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89" podUID="0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 19:45:08.702694 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.702670    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:45:08.838820 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.838715    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-istio-data\") pod \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\" (UID: \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\") "
Sep 10 19:45:08.838820 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.838790    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-istiod-ca-cert\") pod \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\" (UID: \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\") "
Sep 10 19:45:08.839011 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.838843    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-credential-socket\") pod \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\" (UID: \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\") "
Sep 10 19:45:08.839011 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.838908    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-istio-token\") pod \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\" (UID: \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\") "
Sep 10 19:45:08.839011 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.838972    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bs2x5\" (UniqueName: \"kubernetes.io/projected/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-kube-api-access-bs2x5\") pod \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\" (UID: \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\") "
Sep 10 19:45:08.839011 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.839001    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-istio-podinfo\") pod \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\" (UID: \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\") "
Sep 10 19:45:08.839213 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.839039    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-workload-socket\") pod \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\" (UID: \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\") "
Sep 10 19:45:08.839213 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.839039    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-istio-data" (OuterVolumeSpecName: "istio-data") pod "0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0" (UID: "0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:45:08.839213 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.839079    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-istio-envoy\") pod \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\" (UID: \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\") "
Sep 10 19:45:08.839213 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.839104    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-workload-certs\") pod \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\" (UID: \"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0\") "
Sep 10 19:45:08.839213 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.839163    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0" (UID: "0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:45:08.839457 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.839285    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0" (UID: "0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:45:08.839457 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.839316    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0" (UID: "0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:45:08.839565 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.839479    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-istio-data\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:45:08.839565 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.839496    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-istiod-ca-cert\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:45:08.839565 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.839509    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-credential-socket\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:45:08.839565 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.839520    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-workload-socket\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:45:08.839565 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.839518    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0" (UID: "0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:45:08.841330 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.841301    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-istio-token" (OuterVolumeSpecName: "istio-token") pod "0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0" (UID: "0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:45:08.841330 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.841315    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-kube-api-access-bs2x5" (OuterVolumeSpecName: "kube-api-access-bs2x5") pod "0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0" (UID: "0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0"). InnerVolumeSpecName "kube-api-access-bs2x5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:45:08.841535 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.841354    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0" (UID: "0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:45:08.841535 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.841391    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0" (UID: "0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:45:08.940291 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.940255    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-istio-token\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:45:08.940291 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.940286    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bs2x5\" (UniqueName: \"kubernetes.io/projected/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-kube-api-access-bs2x5\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:45:08.940291 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.940297    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-istio-podinfo\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:45:08.940541 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.940308    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-istio-envoy\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:45:08.940541 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.940316    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0-workload-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:45:08.954985 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.954951    2579 generic.go:356] "Generic (PLEG): container finished" podID="0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0" containerID="68248b55e153c45fe636be1dcbc63f178538ee29629bd16c4e1b197defe5a3e0" exitCode=0
Sep 10 19:45:08.955155 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.955016    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"
Sep 10 19:45:08.955155 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.955042    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89" event={"ID":"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0","Type":"ContainerDied","Data":"68248b55e153c45fe636be1dcbc63f178538ee29629bd16c4e1b197defe5a3e0"}
Sep 10 19:45:08.955244 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.955168    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89" event={"ID":"0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0","Type":"ContainerDied","Data":"7043b585b59dca9506164ea7aafe5971e1ccc8bef472a7fb0e47c411e9279a99"}
Sep 10 19:45:08.955244 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.955211    2579 scope.go:117] "RemoveContainer" containerID="68248b55e153c45fe636be1dcbc63f178538ee29629bd16c4e1b197defe5a3e0"
Sep 10 19:45:08.965307 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.965288    2579 scope.go:117] "RemoveContainer" containerID="479477f7c313c3b751a1321bf6850eeb3d12b21e7870c278d76fe9066f6d1e22"
Sep 10 19:45:08.974739 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.974718    2579 scope.go:117] "RemoveContainer" containerID="68248b55e153c45fe636be1dcbc63f178538ee29629bd16c4e1b197defe5a3e0"
Sep 10 19:45:08.975130 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:45:08.975103    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68248b55e153c45fe636be1dcbc63f178538ee29629bd16c4e1b197defe5a3e0\": container with ID starting with 68248b55e153c45fe636be1dcbc63f178538ee29629bd16c4e1b197defe5a3e0 not found: ID does not exist" containerID="68248b55e153c45fe636be1dcbc63f178538ee29629bd16c4e1b197defe5a3e0"
Sep 10 19:45:08.975227 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.975140    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68248b55e153c45fe636be1dcbc63f178538ee29629bd16c4e1b197defe5a3e0"} err="failed to get container status \"68248b55e153c45fe636be1dcbc63f178538ee29629bd16c4e1b197defe5a3e0\": rpc error: code = NotFound desc = could not find container \"68248b55e153c45fe636be1dcbc63f178538ee29629bd16c4e1b197defe5a3e0\": container with ID starting with 68248b55e153c45fe636be1dcbc63f178538ee29629bd16c4e1b197defe5a3e0 not found: ID does not exist"
Sep 10 19:45:08.975227 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.975161    2579 scope.go:117] "RemoveContainer" containerID="479477f7c313c3b751a1321bf6850eeb3d12b21e7870c278d76fe9066f6d1e22"
Sep 10 19:45:08.975408 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:45:08.975390    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"479477f7c313c3b751a1321bf6850eeb3d12b21e7870c278d76fe9066f6d1e22\": container with ID starting with 479477f7c313c3b751a1321bf6850eeb3d12b21e7870c278d76fe9066f6d1e22 not found: ID does not exist" containerID="479477f7c313c3b751a1321bf6850eeb3d12b21e7870c278d76fe9066f6d1e22"
Sep 10 19:45:08.975449 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.975414    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"479477f7c313c3b751a1321bf6850eeb3d12b21e7870c278d76fe9066f6d1e22"} err="failed to get container status \"479477f7c313c3b751a1321bf6850eeb3d12b21e7870c278d76fe9066f6d1e22\": rpc error: code = NotFound desc = could not find container \"479477f7c313c3b751a1321bf6850eeb3d12b21e7870c278d76fe9066f6d1e22\": container with ID starting with 479477f7c313c3b751a1321bf6850eeb3d12b21e7870c278d76fe9066f6d1e22 not found: ID does not exist"
Sep 10 19:45:08.976780 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.976735    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"]
Sep 10 19:45:08.981538 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:08.981517    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89"]
Sep 10 19:45:09.241705 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:09.241673    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0" path="/var/lib/kubelet/pods/0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0/volumes"
Sep 10 19:45:14.400411 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:14.400378    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg"]
Sep 10 19:45:14.400853 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:14.400659    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg" podUID="7c431193-2d5f-4789-8abf-35a6ab2f02e8" containerName="maas-api" containerID="cri-o://f18aece6a86e29e7b4ad4b24b36b2aaccf2baded50cc2a2d412af7b7515a865d" gracePeriod=30
Sep 10 19:45:14.659161 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:14.659096    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg"
Sep 10 19:45:14.798693 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:14.798664    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c431193-2d5f-4789-8abf-35a6ab2f02e8-openshift-service-ca\") pod \"7c431193-2d5f-4789-8abf-35a6ab2f02e8\" (UID: \"7c431193-2d5f-4789-8abf-35a6ab2f02e8\") "
Sep 10 19:45:14.798923 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:14.798709    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8rvlj\" (UniqueName: \"kubernetes.io/projected/7c431193-2d5f-4789-8abf-35a6ab2f02e8-kube-api-access-8rvlj\") pod \"7c431193-2d5f-4789-8abf-35a6ab2f02e8\" (UID: \"7c431193-2d5f-4789-8abf-35a6ab2f02e8\") "
Sep 10 19:45:14.798923 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:14.798839    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c431193-2d5f-4789-8abf-35a6ab2f02e8-metrics-certs\") pod \"7c431193-2d5f-4789-8abf-35a6ab2f02e8\" (UID: \"7c431193-2d5f-4789-8abf-35a6ab2f02e8\") "
Sep 10 19:45:14.798923 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:14.798866    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c431193-2d5f-4789-8abf-35a6ab2f02e8-maas-api-tls\") pod \"7c431193-2d5f-4789-8abf-35a6ab2f02e8\" (UID: \"7c431193-2d5f-4789-8abf-35a6ab2f02e8\") "
Sep 10 19:45:14.799162 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:14.799133    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c431193-2d5f-4789-8abf-35a6ab2f02e8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7c431193-2d5f-4789-8abf-35a6ab2f02e8" (UID: "7c431193-2d5f-4789-8abf-35a6ab2f02e8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:45:14.801149 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:14.801118    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c431193-2d5f-4789-8abf-35a6ab2f02e8-kube-api-access-8rvlj" (OuterVolumeSpecName: "kube-api-access-8rvlj") pod "7c431193-2d5f-4789-8abf-35a6ab2f02e8" (UID: "7c431193-2d5f-4789-8abf-35a6ab2f02e8"). InnerVolumeSpecName "kube-api-access-8rvlj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:45:14.801312 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:14.801168    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c431193-2d5f-4789-8abf-35a6ab2f02e8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7c431193-2d5f-4789-8abf-35a6ab2f02e8" (UID: "7c431193-2d5f-4789-8abf-35a6ab2f02e8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:45:14.801453 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:14.801432    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c431193-2d5f-4789-8abf-35a6ab2f02e8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7c431193-2d5f-4789-8abf-35a6ab2f02e8" (UID: "7c431193-2d5f-4789-8abf-35a6ab2f02e8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:45:14.900156 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:14.900116    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c431193-2d5f-4789-8abf-35a6ab2f02e8-metrics-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:45:14.900156 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:14.900148    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c431193-2d5f-4789-8abf-35a6ab2f02e8-maas-api-tls\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:45:14.900156 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:14.900158    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c431193-2d5f-4789-8abf-35a6ab2f02e8-openshift-service-ca\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:45:14.900156 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:14.900168    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rvlj\" (UniqueName: \"kubernetes.io/projected/7c431193-2d5f-4789-8abf-35a6ab2f02e8-kube-api-access-8rvlj\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:45:14.992636 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:14.992540    2579 generic.go:356] "Generic (PLEG): container finished" podID="7c431193-2d5f-4789-8abf-35a6ab2f02e8" containerID="f18aece6a86e29e7b4ad4b24b36b2aaccf2baded50cc2a2d412af7b7515a865d" exitCode=0
Sep 10 19:45:14.992636 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:14.992584    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg"
Sep 10 19:45:14.992636 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:14.992626    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg" event={"ID":"7c431193-2d5f-4789-8abf-35a6ab2f02e8","Type":"ContainerDied","Data":"f18aece6a86e29e7b4ad4b24b36b2aaccf2baded50cc2a2d412af7b7515a865d"}
Sep 10 19:45:14.992944 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:14.992699    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg" event={"ID":"7c431193-2d5f-4789-8abf-35a6ab2f02e8","Type":"ContainerDied","Data":"8ceb456fa1c8be34e6ab11141371455e892d1c771d5855b8dafb74dc3a6db4f9"}
Sep 10 19:45:14.992944 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:14.992722    2579 scope.go:117] "RemoveContainer" containerID="f18aece6a86e29e7b4ad4b24b36b2aaccf2baded50cc2a2d412af7b7515a865d"
Sep 10 19:45:15.003828 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:15.003808    2579 scope.go:117] "RemoveContainer" containerID="f18aece6a86e29e7b4ad4b24b36b2aaccf2baded50cc2a2d412af7b7515a865d"
Sep 10 19:45:15.004136 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:45:15.004115    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f18aece6a86e29e7b4ad4b24b36b2aaccf2baded50cc2a2d412af7b7515a865d\": container with ID starting with f18aece6a86e29e7b4ad4b24b36b2aaccf2baded50cc2a2d412af7b7515a865d not found: ID does not exist" containerID="f18aece6a86e29e7b4ad4b24b36b2aaccf2baded50cc2a2d412af7b7515a865d"
Sep 10 19:45:15.004208 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:15.004149    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f18aece6a86e29e7b4ad4b24b36b2aaccf2baded50cc2a2d412af7b7515a865d"} err="failed to get container status \"f18aece6a86e29e7b4ad4b24b36b2aaccf2baded50cc2a2d412af7b7515a865d\": rpc error: code = NotFound desc = could not find container \"f18aece6a86e29e7b4ad4b24b36b2aaccf2baded50cc2a2d412af7b7515a865d\": container with ID starting with f18aece6a86e29e7b4ad4b24b36b2aaccf2baded50cc2a2d412af7b7515a865d not found: ID does not exist"
Sep 10 19:45:15.014482 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:15.014457    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg"]
Sep 10 19:45:15.017067 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:15.017045    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-cab4ee-5449cd8467-v4fvg"]
Sep 10 19:45:15.240288 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:15.240255    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c431193-2d5f-4789-8abf-35a6ab2f02e8" path="/var/lib/kubelet/pods/7c431193-2d5f-4789-8abf-35a6ab2f02e8/volumes"
Sep 10 19:45:24.004797 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:24.004741    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-77c474c76b-h9hhf"]
Sep 10 19:45:24.005194 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:24.004980    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-77c474c76b-h9hhf" podUID="6738a3a0-7730-4f3a-8251-f28b713bd83a" containerName="manager" containerID="cri-o://def77155daa1e6517ffe74a0564a53796a807585c2723bdf7bc6e954df7ec71a" gracePeriod=30
Sep 10 19:45:24.262537 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:24.262471    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77c474c76b-h9hhf"
Sep 10 19:45:24.394493 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:24.394464    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6738a3a0-7730-4f3a-8251-f28b713bd83a-metrics-certs\") pod \"6738a3a0-7730-4f3a-8251-f28b713bd83a\" (UID: \"6738a3a0-7730-4f3a-8251-f28b713bd83a\") "
Sep 10 19:45:24.394667 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:24.394518    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7ffj9\" (UniqueName: \"kubernetes.io/projected/6738a3a0-7730-4f3a-8251-f28b713bd83a-kube-api-access-7ffj9\") pod \"6738a3a0-7730-4f3a-8251-f28b713bd83a\" (UID: \"6738a3a0-7730-4f3a-8251-f28b713bd83a\") "
Sep 10 19:45:24.394667 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:24.394628    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6738a3a0-7730-4f3a-8251-f28b713bd83a-webhook-cert\") pod \"6738a3a0-7730-4f3a-8251-f28b713bd83a\" (UID: \"6738a3a0-7730-4f3a-8251-f28b713bd83a\") "
Sep 10 19:45:24.397031 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:24.397004    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6738a3a0-7730-4f3a-8251-f28b713bd83a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6738a3a0-7730-4f3a-8251-f28b713bd83a" (UID: "6738a3a0-7730-4f3a-8251-f28b713bd83a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:45:24.397228 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:24.397215    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6738a3a0-7730-4f3a-8251-f28b713bd83a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6738a3a0-7730-4f3a-8251-f28b713bd83a" (UID: "6738a3a0-7730-4f3a-8251-f28b713bd83a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:45:24.397337 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:24.397317    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6738a3a0-7730-4f3a-8251-f28b713bd83a-kube-api-access-7ffj9" (OuterVolumeSpecName: "kube-api-access-7ffj9") pod "6738a3a0-7730-4f3a-8251-f28b713bd83a" (UID: "6738a3a0-7730-4f3a-8251-f28b713bd83a"). InnerVolumeSpecName "kube-api-access-7ffj9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:45:24.496399 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:24.496353    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6738a3a0-7730-4f3a-8251-f28b713bd83a-metrics-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:45:24.496399 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:24.496393    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ffj9\" (UniqueName: \"kubernetes.io/projected/6738a3a0-7730-4f3a-8251-f28b713bd83a-kube-api-access-7ffj9\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:45:24.496399 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:24.496403    2579 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6738a3a0-7730-4f3a-8251-f28b713bd83a-webhook-cert\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:45:25.058840 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:25.058806    2579 generic.go:356] "Generic (PLEG): container finished" podID="6738a3a0-7730-4f3a-8251-f28b713bd83a" containerID="def77155daa1e6517ffe74a0564a53796a807585c2723bdf7bc6e954df7ec71a" exitCode=0
Sep 10 19:45:25.059280 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:25.058852    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77c474c76b-h9hhf"
Sep 10 19:45:25.059280 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:25.058893    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77c474c76b-h9hhf" event={"ID":"6738a3a0-7730-4f3a-8251-f28b713bd83a","Type":"ContainerDied","Data":"def77155daa1e6517ffe74a0564a53796a807585c2723bdf7bc6e954df7ec71a"}
Sep 10 19:45:25.059280 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:25.058945    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77c474c76b-h9hhf" event={"ID":"6738a3a0-7730-4f3a-8251-f28b713bd83a","Type":"ContainerDied","Data":"8bae8b1845e4a4d31a56ce5334995b593a113bfb0b51cc382d0f2b673511e74d"}
Sep 10 19:45:25.059280 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:25.058968    2579 scope.go:117] "RemoveContainer" containerID="def77155daa1e6517ffe74a0564a53796a807585c2723bdf7bc6e954df7ec71a"
Sep 10 19:45:25.070327 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:25.070300    2579 scope.go:117] "RemoveContainer" containerID="def77155daa1e6517ffe74a0564a53796a807585c2723bdf7bc6e954df7ec71a"
Sep 10 19:45:25.070670 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:45:25.070644    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"def77155daa1e6517ffe74a0564a53796a807585c2723bdf7bc6e954df7ec71a\": container with ID starting with def77155daa1e6517ffe74a0564a53796a807585c2723bdf7bc6e954df7ec71a not found: ID does not exist" containerID="def77155daa1e6517ffe74a0564a53796a807585c2723bdf7bc6e954df7ec71a"
Sep 10 19:45:25.070779 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:25.070678    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"def77155daa1e6517ffe74a0564a53796a807585c2723bdf7bc6e954df7ec71a"} err="failed to get container status \"def77155daa1e6517ffe74a0564a53796a807585c2723bdf7bc6e954df7ec71a\": rpc error: code = NotFound desc = could not find container \"def77155daa1e6517ffe74a0564a53796a807585c2723bdf7bc6e954df7ec71a\": container with ID starting with def77155daa1e6517ffe74a0564a53796a807585c2723bdf7bc6e954df7ec71a not found: ID does not exist"
Sep 10 19:45:25.081265 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:25.081239    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-77c474c76b-h9hhf"]
Sep 10 19:45:25.086918 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:25.086893    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-77c474c76b-h9hhf"]
Sep 10 19:45:25.239809 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:25.239775    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6738a3a0-7730-4f3a-8251-f28b713bd83a" path="/var/lib/kubelet/pods/6738a3a0-7730-4f3a-8251-f28b713bd83a/volumes"
Sep 10 19:45:45.322519 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:45.322491    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bmdg9_9817270c-8629-44c4-ab8f-f3982e1f0e2f/console-operator/1.log"
Sep 10 19:45:45.327856 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:45.327829    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8r66s_2826f8ec-78f6-4cab-879e-47a21854ed5a/ovn-acl-logging/0.log"
Sep 10 19:45:45.331070 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:45.331051    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bmdg9_9817270c-8629-44c4-ab8f-f3982e1f0e2f/console-operator/1.log"
Sep 10 19:45:45.335561 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:45:45.335542    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8r66s_2826f8ec-78f6-4cab-879e-47a21854ed5a/ovn-acl-logging/0.log"
Sep 10 19:46:25.886216 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:25.886152    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-77c474c76b-qbzg6"]
Sep 10 19:46:25.886715 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:25.886698    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2028475e-bf7c-4451-82da-70fd2b0ccf26" containerName="maas-api"
Sep 10 19:46:25.886787 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:25.886718    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2028475e-bf7c-4451-82da-70fd2b0ccf26" containerName="maas-api"
Sep 10 19:46:25.886787 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:25.886743    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0" containerName="istio-proxy"
Sep 10 19:46:25.886787 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:25.886765    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0" containerName="istio-proxy"
Sep 10 19:46:25.886787 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:25.886781    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0" containerName="istio-proxy"
Sep 10 19:46:25.886787 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:25.886787    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0" containerName="istio-proxy"
Sep 10 19:46:25.886947 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:25.886798    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6738a3a0-7730-4f3a-8251-f28b713bd83a" containerName="manager"
Sep 10 19:46:25.886947 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:25.886803    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6738a3a0-7730-4f3a-8251-f28b713bd83a" containerName="manager"
Sep 10 19:46:25.886947 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:25.886813    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c431193-2d5f-4789-8abf-35a6ab2f02e8" containerName="maas-api"
Sep 10 19:46:25.886947 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:25.886822    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c431193-2d5f-4789-8abf-35a6ab2f02e8" containerName="maas-api"
Sep 10 19:46:25.886947 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:25.886834    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="678724f6-2a61-40e3-ba09-ea8099e5e3fc" containerName="maas-api"
Sep 10 19:46:25.886947 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:25.886839    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="678724f6-2a61-40e3-ba09-ea8099e5e3fc" containerName="maas-api"
Sep 10 19:46:25.886947 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:25.886845    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a6d5e92-b3fb-401f-890d-32c1d7672be9" containerName="maas-api"
Sep 10 19:46:25.886947 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:25.886850    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a6d5e92-b3fb-401f-890d-32c1d7672be9" containerName="maas-api"
Sep 10 19:46:25.886947 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:25.886857    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00407951-e8b0-4d42-be73-436f35e3d385" containerName="main"
Sep 10 19:46:25.886947 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:25.886861    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="00407951-e8b0-4d42-be73-436f35e3d385" containerName="main"
Sep 10 19:46:25.886947 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:25.886866    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41d2b767-335b-41bf-9e03-004b433653d8" containerName="maas-api"
Sep 10 19:46:25.886947 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:25.886871    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="41d2b767-335b-41bf-9e03-004b433653d8" containerName="maas-api"
Sep 10 19:46:25.886947 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:25.886948    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="41d2b767-335b-41bf-9e03-004b433653d8" containerName="maas-api"
Sep 10 19:46:25.887326 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:25.886958    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0" containerName="istio-proxy"
Sep 10 19:46:25.887326 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:25.886963    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="678724f6-2a61-40e3-ba09-ea8099e5e3fc" containerName="maas-api"
Sep 10 19:46:25.887326 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:25.886972    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6738a3a0-7730-4f3a-8251-f28b713bd83a" containerName="manager"
Sep 10 19:46:25.887326 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:25.886981    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a5aff68-9e9c-4e0e-bc9e-a64170d0a9f0" containerName="istio-proxy"
Sep 10 19:46:25.887326 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:25.886989    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="00407951-e8b0-4d42-be73-436f35e3d385" containerName="main"
Sep 10 19:46:25.887326 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:25.886994    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a6d5e92-b3fb-401f-890d-32c1d7672be9" containerName="maas-api"
Sep 10 19:46:25.887326 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:25.886999    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2028475e-bf7c-4451-82da-70fd2b0ccf26" containerName="maas-api"
Sep 10 19:46:25.887326 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:25.887003    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c431193-2d5f-4789-8abf-35a6ab2f02e8" containerName="maas-api"
Sep 10 19:46:25.890156 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:25.890140    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77c474c76b-qbzg6"
Sep 10 19:46:25.895451 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:25.895432    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 19:46:25.895600 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:25.895489    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 19:46:25.895600 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:25.895534    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-d7nmk\""
Sep 10 19:46:25.895821 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:25.895808    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 19:46:25.904300 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:25.904275    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77c474c76b-qbzg6"]
Sep 10 19:46:25.962274 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:25.962238    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01265a89-e033-4840-878b-657a8717a722-metrics-certs\") pod \"maas-controller-77c474c76b-qbzg6\" (UID: \"01265a89-e033-4840-878b-657a8717a722\") " pod="opendatahub/maas-controller-77c474c76b-qbzg6"
Sep 10 19:46:25.962441 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:25.962358    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9gg5\" (UniqueName: \"kubernetes.io/projected/01265a89-e033-4840-878b-657a8717a722-kube-api-access-d9gg5\") pod \"maas-controller-77c474c76b-qbzg6\" (UID: \"01265a89-e033-4840-878b-657a8717a722\") " pod="opendatahub/maas-controller-77c474c76b-qbzg6"
Sep 10 19:46:25.962441 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:25.962391    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/01265a89-e033-4840-878b-657a8717a722-webhook-cert\") pod \"maas-controller-77c474c76b-qbzg6\" (UID: \"01265a89-e033-4840-878b-657a8717a722\") " pod="opendatahub/maas-controller-77c474c76b-qbzg6"
Sep 10 19:46:26.063719 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:26.063667    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9gg5\" (UniqueName: \"kubernetes.io/projected/01265a89-e033-4840-878b-657a8717a722-kube-api-access-d9gg5\") pod \"maas-controller-77c474c76b-qbzg6\" (UID: \"01265a89-e033-4840-878b-657a8717a722\") " pod="opendatahub/maas-controller-77c474c76b-qbzg6"
Sep 10 19:46:26.063719 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:26.063725    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/01265a89-e033-4840-878b-657a8717a722-webhook-cert\") pod \"maas-controller-77c474c76b-qbzg6\" (UID: \"01265a89-e033-4840-878b-657a8717a722\") " pod="opendatahub/maas-controller-77c474c76b-qbzg6"
Sep 10 19:46:26.063945 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:26.063796    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01265a89-e033-4840-878b-657a8717a722-metrics-certs\") pod \"maas-controller-77c474c76b-qbzg6\" (UID: \"01265a89-e033-4840-878b-657a8717a722\") " pod="opendatahub/maas-controller-77c474c76b-qbzg6"
Sep 10 19:46:26.066363 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:26.066342    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01265a89-e033-4840-878b-657a8717a722-metrics-certs\") pod \"maas-controller-77c474c76b-qbzg6\" (UID: \"01265a89-e033-4840-878b-657a8717a722\") " pod="opendatahub/maas-controller-77c474c76b-qbzg6"
Sep 10 19:46:26.066463 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:26.066445    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/01265a89-e033-4840-878b-657a8717a722-webhook-cert\") pod \"maas-controller-77c474c76b-qbzg6\" (UID: \"01265a89-e033-4840-878b-657a8717a722\") " pod="opendatahub/maas-controller-77c474c76b-qbzg6"
Sep 10 19:46:26.073574 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:26.073554    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9gg5\" (UniqueName: \"kubernetes.io/projected/01265a89-e033-4840-878b-657a8717a722-kube-api-access-d9gg5\") pod \"maas-controller-77c474c76b-qbzg6\" (UID: \"01265a89-e033-4840-878b-657a8717a722\") " pod="opendatahub/maas-controller-77c474c76b-qbzg6"
Sep 10 19:46:26.200774 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:26.200667    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77c474c76b-qbzg6"
Sep 10 19:46:26.490269 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:26.490233    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77c474c76b-qbzg6"]
Sep 10 19:46:26.491641 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:46:26.491612    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01265a89_e033_4840_878b_657a8717a722.slice/crio-64f08c34b53d2b2b38594339a42ae11a648dc7a9d413dede04771c52fa2bb499 WatchSource:0}: Error finding container 64f08c34b53d2b2b38594339a42ae11a648dc7a9d413dede04771c52fa2bb499: Status 404 returned error can't find the container with id 64f08c34b53d2b2b38594339a42ae11a648dc7a9d413dede04771c52fa2bb499
Sep 10 19:46:26.493065 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:26.493050    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 19:46:27.425288 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:27.425249    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77c474c76b-qbzg6" event={"ID":"01265a89-e033-4840-878b-657a8717a722","Type":"ContainerStarted","Data":"8a037d5f3dcf128cb84de5ecad3a3ade3d645467286813564de93408795f63b5"}
Sep 10 19:46:27.425812 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:27.425309    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77c474c76b-qbzg6" event={"ID":"01265a89-e033-4840-878b-657a8717a722","Type":"ContainerStarted","Data":"64f08c34b53d2b2b38594339a42ae11a648dc7a9d413dede04771c52fa2bb499"}
Sep 10 19:46:27.425812 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:27.425464    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-77c474c76b-qbzg6"
Sep 10 19:46:27.456304 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:27.456247    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-77c474c76b-qbzg6" podStartSLOduration=1.906889536 podStartE2EDuration="2.456228921s" totalImagesPullingTime="549.339385ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:46:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:46:27.453309773 +0000 UTC m=+1842.673158254" watchObservedRunningTime="2026-09-10 19:46:27.456228921 +0000 UTC m=+1842.676077402"
Sep 10 19:46:38.433003 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:38.432971    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-77c474c76b-qbzg6"
Sep 10 19:46:47.794927 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:47.794884    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-77c474c76b-qbzg6"]
Sep 10 19:46:47.795390 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:47.795142    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-77c474c76b-qbzg6" podUID="01265a89-e033-4840-878b-657a8717a722" containerName="manager" containerID="cri-o://8a037d5f3dcf128cb84de5ecad3a3ade3d645467286813564de93408795f63b5" gracePeriod=30
Sep 10 19:46:48.043136 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:48.043107    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77c474c76b-qbzg6"
Sep 10 19:46:48.176361 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:48.176327    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/01265a89-e033-4840-878b-657a8717a722-webhook-cert\") pod \"01265a89-e033-4840-878b-657a8717a722\" (UID: \"01265a89-e033-4840-878b-657a8717a722\") "
Sep 10 19:46:48.176556 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:48.176379    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01265a89-e033-4840-878b-657a8717a722-metrics-certs\") pod \"01265a89-e033-4840-878b-657a8717a722\" (UID: \"01265a89-e033-4840-878b-657a8717a722\") "
Sep 10 19:46:48.176556 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:48.176407    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9gg5\" (UniqueName: \"kubernetes.io/projected/01265a89-e033-4840-878b-657a8717a722-kube-api-access-d9gg5\") pod \"01265a89-e033-4840-878b-657a8717a722\" (UID: \"01265a89-e033-4840-878b-657a8717a722\") "
Sep 10 19:46:48.178623 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:48.178593    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01265a89-e033-4840-878b-657a8717a722-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "01265a89-e033-4840-878b-657a8717a722" (UID: "01265a89-e033-4840-878b-657a8717a722"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:46:48.178794 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:48.178652    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01265a89-e033-4840-878b-657a8717a722-kube-api-access-d9gg5" (OuterVolumeSpecName: "kube-api-access-d9gg5") pod "01265a89-e033-4840-878b-657a8717a722" (UID: "01265a89-e033-4840-878b-657a8717a722"). InnerVolumeSpecName "kube-api-access-d9gg5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:46:48.178794 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:48.178713    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01265a89-e033-4840-878b-657a8717a722-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "01265a89-e033-4840-878b-657a8717a722" (UID: "01265a89-e033-4840-878b-657a8717a722"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:46:48.277508 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:48.277473    2579 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/01265a89-e033-4840-878b-657a8717a722-webhook-cert\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:46:48.277508 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:48.277501    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01265a89-e033-4840-878b-657a8717a722-metrics-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:46:48.277508 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:48.277510    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9gg5\" (UniqueName: \"kubernetes.io/projected/01265a89-e033-4840-878b-657a8717a722-kube-api-access-d9gg5\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:46:48.552243 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:48.552148    2579 generic.go:356] "Generic (PLEG): container finished" podID="01265a89-e033-4840-878b-657a8717a722" containerID="8a037d5f3dcf128cb84de5ecad3a3ade3d645467286813564de93408795f63b5" exitCode=0
Sep 10 19:46:48.552243 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:48.552197    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77c474c76b-qbzg6"
Sep 10 19:46:48.552484 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:48.552236    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77c474c76b-qbzg6" event={"ID":"01265a89-e033-4840-878b-657a8717a722","Type":"ContainerDied","Data":"8a037d5f3dcf128cb84de5ecad3a3ade3d645467286813564de93408795f63b5"}
Sep 10 19:46:48.552484 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:48.552302    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77c474c76b-qbzg6" event={"ID":"01265a89-e033-4840-878b-657a8717a722","Type":"ContainerDied","Data":"64f08c34b53d2b2b38594339a42ae11a648dc7a9d413dede04771c52fa2bb499"}
Sep 10 19:46:48.552484 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:48.552332    2579 scope.go:117] "RemoveContainer" containerID="8a037d5f3dcf128cb84de5ecad3a3ade3d645467286813564de93408795f63b5"
Sep 10 19:46:48.565296 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:48.565275    2579 scope.go:117] "RemoveContainer" containerID="8a037d5f3dcf128cb84de5ecad3a3ade3d645467286813564de93408795f63b5"
Sep 10 19:46:48.565602 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:46:48.565581    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a037d5f3dcf128cb84de5ecad3a3ade3d645467286813564de93408795f63b5\": container with ID starting with 8a037d5f3dcf128cb84de5ecad3a3ade3d645467286813564de93408795f63b5 not found: ID does not exist" containerID="8a037d5f3dcf128cb84de5ecad3a3ade3d645467286813564de93408795f63b5"
Sep 10 19:46:48.565646 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:48.565610    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a037d5f3dcf128cb84de5ecad3a3ade3d645467286813564de93408795f63b5"} err="failed to get container status \"8a037d5f3dcf128cb84de5ecad3a3ade3d645467286813564de93408795f63b5\": rpc error: code = NotFound desc = could not find container \"8a037d5f3dcf128cb84de5ecad3a3ade3d645467286813564de93408795f63b5\": container with ID starting with 8a037d5f3dcf128cb84de5ecad3a3ade3d645467286813564de93408795f63b5 not found: ID does not exist"
Sep 10 19:46:48.577220 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:48.577194    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-77c474c76b-qbzg6"]
Sep 10 19:46:48.580332 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:48.580308    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-77c474c76b-qbzg6"]
Sep 10 19:46:49.247611 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:49.247570    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01265a89-e033-4840-878b-657a8717a722" path="/var/lib/kubelet/pods/01265a89-e033-4840-878b-657a8717a722/volumes"
Sep 10 19:46:49.394857 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:49.394826    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-77c474c76b-jhn5d"]
Sep 10 19:46:49.395227 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:49.395215    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01265a89-e033-4840-878b-657a8717a722" containerName="manager"
Sep 10 19:46:49.395272 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:49.395229    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="01265a89-e033-4840-878b-657a8717a722" containerName="manager"
Sep 10 19:46:49.395304 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:49.395295    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="01265a89-e033-4840-878b-657a8717a722" containerName="manager"
Sep 10 19:46:49.399848 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:49.399825    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77c474c76b-jhn5d"
Sep 10 19:46:49.402086 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:49.402066    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-d7nmk\""
Sep 10 19:46:49.402795 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:49.402734    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 19:46:49.402795 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:49.402738    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 19:46:49.402986 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:49.402800    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 19:46:49.411811 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:49.411788    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77c474c76b-jhn5d"]
Sep 10 19:46:49.489349 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:49.489312    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvjqb\" (UniqueName: \"kubernetes.io/projected/3aeab5a1-59dd-4975-811f-8c9eb50a3605-kube-api-access-rvjqb\") pod \"maas-controller-77c474c76b-jhn5d\" (UID: \"3aeab5a1-59dd-4975-811f-8c9eb50a3605\") " pod="opendatahub/maas-controller-77c474c76b-jhn5d"
Sep 10 19:46:49.489522 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:49.489372    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3aeab5a1-59dd-4975-811f-8c9eb50a3605-metrics-certs\") pod \"maas-controller-77c474c76b-jhn5d\" (UID: \"3aeab5a1-59dd-4975-811f-8c9eb50a3605\") " pod="opendatahub/maas-controller-77c474c76b-jhn5d"
Sep 10 19:46:49.489522 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:49.489437    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3aeab5a1-59dd-4975-811f-8c9eb50a3605-webhook-cert\") pod \"maas-controller-77c474c76b-jhn5d\" (UID: \"3aeab5a1-59dd-4975-811f-8c9eb50a3605\") " pod="opendatahub/maas-controller-77c474c76b-jhn5d"
Sep 10 19:46:49.590740 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:49.590646    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3aeab5a1-59dd-4975-811f-8c9eb50a3605-webhook-cert\") pod \"maas-controller-77c474c76b-jhn5d\" (UID: \"3aeab5a1-59dd-4975-811f-8c9eb50a3605\") " pod="opendatahub/maas-controller-77c474c76b-jhn5d"
Sep 10 19:46:49.590740 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:49.590702    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvjqb\" (UniqueName: \"kubernetes.io/projected/3aeab5a1-59dd-4975-811f-8c9eb50a3605-kube-api-access-rvjqb\") pod \"maas-controller-77c474c76b-jhn5d\" (UID: \"3aeab5a1-59dd-4975-811f-8c9eb50a3605\") " pod="opendatahub/maas-controller-77c474c76b-jhn5d"
Sep 10 19:46:49.590962 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:49.590772    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3aeab5a1-59dd-4975-811f-8c9eb50a3605-metrics-certs\") pod \"maas-controller-77c474c76b-jhn5d\" (UID: \"3aeab5a1-59dd-4975-811f-8c9eb50a3605\") " pod="opendatahub/maas-controller-77c474c76b-jhn5d"
Sep 10 19:46:49.593369 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:49.593349    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3aeab5a1-59dd-4975-811f-8c9eb50a3605-metrics-certs\") pod \"maas-controller-77c474c76b-jhn5d\" (UID: \"3aeab5a1-59dd-4975-811f-8c9eb50a3605\") " pod="opendatahub/maas-controller-77c474c76b-jhn5d"
Sep 10 19:46:49.593459 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:49.593404    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3aeab5a1-59dd-4975-811f-8c9eb50a3605-webhook-cert\") pod \"maas-controller-77c474c76b-jhn5d\" (UID: \"3aeab5a1-59dd-4975-811f-8c9eb50a3605\") " pod="opendatahub/maas-controller-77c474c76b-jhn5d"
Sep 10 19:46:49.598730 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:49.598704    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvjqb\" (UniqueName: \"kubernetes.io/projected/3aeab5a1-59dd-4975-811f-8c9eb50a3605-kube-api-access-rvjqb\") pod \"maas-controller-77c474c76b-jhn5d\" (UID: \"3aeab5a1-59dd-4975-811f-8c9eb50a3605\") " pod="opendatahub/maas-controller-77c474c76b-jhn5d"
Sep 10 19:46:49.710888 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:49.710849    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77c474c76b-jhn5d"
Sep 10 19:46:49.988095 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:49.988066    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77c474c76b-jhn5d"]
Sep 10 19:46:49.989581 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:46:49.989553    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3aeab5a1_59dd_4975_811f_8c9eb50a3605.slice/crio-8c615000d627bb8ca7cf8142441a0cbe4e33133243f0dd30c1278e45df7c2d19 WatchSource:0}: Error finding container 8c615000d627bb8ca7cf8142441a0cbe4e33133243f0dd30c1278e45df7c2d19: Status 404 returned error can't find the container with id 8c615000d627bb8ca7cf8142441a0cbe4e33133243f0dd30c1278e45df7c2d19
Sep 10 19:46:50.565593 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:50.565552    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77c474c76b-jhn5d" event={"ID":"3aeab5a1-59dd-4975-811f-8c9eb50a3605","Type":"ContainerStarted","Data":"c0e9093a378aa08c16c5176247c83f8d961c15701ce392b0e1a975a332d23b39"}
Sep 10 19:46:50.566021 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:50.565604    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77c474c76b-jhn5d" event={"ID":"3aeab5a1-59dd-4975-811f-8c9eb50a3605","Type":"ContainerStarted","Data":"8c615000d627bb8ca7cf8142441a0cbe4e33133243f0dd30c1278e45df7c2d19"}
Sep 10 19:46:50.566021 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:50.565688    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-77c474c76b-jhn5d"
Sep 10 19:46:50.590541 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:46:50.590485    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-77c474c76b-jhn5d" podStartSLOduration=1.22343321 podStartE2EDuration="1.590469543s" totalImagesPullingTime="367.036333ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:46:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:46:50.590273917 +0000 UTC m=+1865.810122399" watchObservedRunningTime="2026-09-10 19:46:50.590469543 +0000 UTC m=+1865.810318023"
Sep 10 19:47:01.573557 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:47:01.573519    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-77c474c76b-jhn5d"
Sep 10 19:49:37.487380 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:37.487344    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh"]
Sep 10 19:49:37.491122 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:37.491098    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh"
Sep 10 19:49:37.493235 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:37.493205    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-66bec08b-kserve-self-signed-certs\""
Sep 10 19:49:37.501453 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:37.501433    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh"]
Sep 10 19:49:37.549881 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:37.549842    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1146124a-e1a9-48f9-b419-4d143bca6d7a-tls-certs\") pod \"e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh\" (UID: \"1146124a-e1a9-48f9-b419-4d143bca6d7a\") " pod="llm/e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh"
Sep 10 19:49:37.550149 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:37.550116    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-992tb\" (UniqueName: \"kubernetes.io/projected/1146124a-e1a9-48f9-b419-4d143bca6d7a-kube-api-access-992tb\") pod \"e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh\" (UID: \"1146124a-e1a9-48f9-b419-4d143bca6d7a\") " pod="llm/e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh"
Sep 10 19:49:37.550288 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:37.550172    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1146124a-e1a9-48f9-b419-4d143bca6d7a-model-cache\") pod \"e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh\" (UID: \"1146124a-e1a9-48f9-b419-4d143bca6d7a\") " pod="llm/e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh"
Sep 10 19:49:37.550288 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:37.550220    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1146124a-e1a9-48f9-b419-4d143bca6d7a-tmp-dir\") pod \"e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh\" (UID: \"1146124a-e1a9-48f9-b419-4d143bca6d7a\") " pod="llm/e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh"
Sep 10 19:49:37.550288 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:37.550271    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1146124a-e1a9-48f9-b419-4d143bca6d7a-home\") pod \"e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh\" (UID: \"1146124a-e1a9-48f9-b419-4d143bca6d7a\") " pod="llm/e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh"
Sep 10 19:49:37.550463 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:37.550357    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1146124a-e1a9-48f9-b419-4d143bca6d7a-dshm\") pod \"e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh\" (UID: \"1146124a-e1a9-48f9-b419-4d143bca6d7a\") " pod="llm/e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh"
Sep 10 19:49:37.651400 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:37.651364    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1146124a-e1a9-48f9-b419-4d143bca6d7a-tmp-dir\") pod \"e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh\" (UID: \"1146124a-e1a9-48f9-b419-4d143bca6d7a\") " pod="llm/e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh"
Sep 10 19:49:37.651715 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:37.651673    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1146124a-e1a9-48f9-b419-4d143bca6d7a-home\") pod \"e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh\" (UID: \"1146124a-e1a9-48f9-b419-4d143bca6d7a\") " pod="llm/e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh"
Sep 10 19:49:37.651890 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:37.651808    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1146124a-e1a9-48f9-b419-4d143bca6d7a-dshm\") pod \"e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh\" (UID: \"1146124a-e1a9-48f9-b419-4d143bca6d7a\") " pod="llm/e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh"
Sep 10 19:49:37.651890 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:37.651842    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1146124a-e1a9-48f9-b419-4d143bca6d7a-tmp-dir\") pod \"e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh\" (UID: \"1146124a-e1a9-48f9-b419-4d143bca6d7a\") " pod="llm/e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh"
Sep 10 19:49:37.652021 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:37.651968    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1146124a-e1a9-48f9-b419-4d143bca6d7a-tls-certs\") pod \"e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh\" (UID: \"1146124a-e1a9-48f9-b419-4d143bca6d7a\") " pod="llm/e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh"
Sep 10 19:49:37.652021 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:37.652008    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-992tb\" (UniqueName: \"kubernetes.io/projected/1146124a-e1a9-48f9-b419-4d143bca6d7a-kube-api-access-992tb\") pod \"e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh\" (UID: \"1146124a-e1a9-48f9-b419-4d143bca6d7a\") " pod="llm/e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh"
Sep 10 19:49:37.652130 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:37.652051    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1146124a-e1a9-48f9-b419-4d143bca6d7a-model-cache\") pod \"e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh\" (UID: \"1146124a-e1a9-48f9-b419-4d143bca6d7a\") " pod="llm/e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh"
Sep 10 19:49:37.652213 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:37.652190    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1146124a-e1a9-48f9-b419-4d143bca6d7a-home\") pod \"e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh\" (UID: \"1146124a-e1a9-48f9-b419-4d143bca6d7a\") " pod="llm/e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh"
Sep 10 19:49:37.652389 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:37.652368    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1146124a-e1a9-48f9-b419-4d143bca6d7a-model-cache\") pod \"e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh\" (UID: \"1146124a-e1a9-48f9-b419-4d143bca6d7a\") " pod="llm/e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh"
Sep 10 19:49:37.655030 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:37.655008    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1146124a-e1a9-48f9-b419-4d143bca6d7a-tls-certs\") pod \"e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh\" (UID: \"1146124a-e1a9-48f9-b419-4d143bca6d7a\") " pod="llm/e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh"
Sep 10 19:49:37.655165 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:37.655127    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1146124a-e1a9-48f9-b419-4d143bca6d7a-dshm\") pod \"e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh\" (UID: \"1146124a-e1a9-48f9-b419-4d143bca6d7a\") " pod="llm/e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh"
Sep 10 19:49:37.660380 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:37.660355    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-992tb\" (UniqueName: \"kubernetes.io/projected/1146124a-e1a9-48f9-b419-4d143bca6d7a-kube-api-access-992tb\") pod \"e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh\" (UID: \"1146124a-e1a9-48f9-b419-4d143bca6d7a\") " pod="llm/e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh"
Sep 10 19:49:37.802016 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:37.801927    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh"
Sep 10 19:49:38.073789 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:38.073758    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh"]
Sep 10 19:49:38.075220 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:49:38.075198    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1146124a_e1a9_48f9_b419_4d143bca6d7a.slice/crio-f3f51d15eee77288948f703d5cd7b549430517adb6bd5694b53a66a2c6a437a6 WatchSource:0}: Error finding container f3f51d15eee77288948f703d5cd7b549430517adb6bd5694b53a66a2c6a437a6: Status 404 returned error can't find the container with id f3f51d15eee77288948f703d5cd7b549430517adb6bd5694b53a66a2c6a437a6
Sep 10 19:49:38.530073 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:38.530040    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh" event={"ID":"1146124a-e1a9-48f9-b419-4d143bca6d7a","Type":"ContainerStarted","Data":"95dbccc0daa18405ce4c2edda714b137e85aac5f4ab14f6f1aa04e08ad685055"}
Sep 10 19:49:38.530443 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:38.530081    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh" event={"ID":"1146124a-e1a9-48f9-b419-4d143bca6d7a","Type":"ContainerStarted","Data":"f3f51d15eee77288948f703d5cd7b549430517adb6bd5694b53a66a2c6a437a6"}
Sep 10 19:49:38.547768 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:38.547704    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh" podStartSLOduration=1.547689283 podStartE2EDuration="1.547689283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:49:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:49:38.544565292 +0000 UTC m=+2033.764413773" watchObservedRunningTime="2026-09-10 19:49:38.547689283 +0000 UTC m=+2033.767537763"
Sep 10 19:49:45.983985 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:45.983920    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch"]
Sep 10 19:49:45.987646 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:45.987630    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch"
Sep 10 19:49:45.989868 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:45.989847    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-66bec08b-kserve-self-signed-certs\""
Sep 10 19:49:45.994901 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:45.994878    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch"]
Sep 10 19:49:46.134678 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:46.134644    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ph8q5\" (UniqueName: \"kubernetes.io/projected/d1efddb7-f98d-4a7f-8a0d-8834fd17cb04-kube-api-access-ph8q5\") pod \"e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch\" (UID: \"d1efddb7-f98d-4a7f-8a0d-8834fd17cb04\") " pod="llm/e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch"
Sep 10 19:49:46.134678 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:46.134686    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1efddb7-f98d-4a7f-8a0d-8834fd17cb04-dshm\") pod \"e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch\" (UID: \"d1efddb7-f98d-4a7f-8a0d-8834fd17cb04\") " pod="llm/e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch"
Sep 10 19:49:46.134992 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:46.134720    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1efddb7-f98d-4a7f-8a0d-8834fd17cb04-tmp-dir\") pod \"e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch\" (UID: \"d1efddb7-f98d-4a7f-8a0d-8834fd17cb04\") " pod="llm/e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch"
Sep 10 19:49:46.134992 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:46.134845    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1efddb7-f98d-4a7f-8a0d-8834fd17cb04-model-cache\") pod \"e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch\" (UID: \"d1efddb7-f98d-4a7f-8a0d-8834fd17cb04\") " pod="llm/e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch"
Sep 10 19:49:46.134992 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:46.134922    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1efddb7-f98d-4a7f-8a0d-8834fd17cb04-home\") pod \"e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch\" (UID: \"d1efddb7-f98d-4a7f-8a0d-8834fd17cb04\") " pod="llm/e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch"
Sep 10 19:49:46.134992 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:46.134950    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1efddb7-f98d-4a7f-8a0d-8834fd17cb04-tls-certs\") pod \"e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch\" (UID: \"d1efddb7-f98d-4a7f-8a0d-8834fd17cb04\") " pod="llm/e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch"
Sep 10 19:49:46.235727 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:46.235639    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1efddb7-f98d-4a7f-8a0d-8834fd17cb04-home\") pod \"e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch\" (UID: \"d1efddb7-f98d-4a7f-8a0d-8834fd17cb04\") " pod="llm/e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch"
Sep 10 19:49:46.235727 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:46.235674    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1efddb7-f98d-4a7f-8a0d-8834fd17cb04-tls-certs\") pod \"e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch\" (UID: \"d1efddb7-f98d-4a7f-8a0d-8834fd17cb04\") " pod="llm/e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch"
Sep 10 19:49:46.235988 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:46.235812    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ph8q5\" (UniqueName: \"kubernetes.io/projected/d1efddb7-f98d-4a7f-8a0d-8834fd17cb04-kube-api-access-ph8q5\") pod \"e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch\" (UID: \"d1efddb7-f98d-4a7f-8a0d-8834fd17cb04\") " pod="llm/e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch"
Sep 10 19:49:46.235988 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:46.235852    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1efddb7-f98d-4a7f-8a0d-8834fd17cb04-dshm\") pod \"e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch\" (UID: \"d1efddb7-f98d-4a7f-8a0d-8834fd17cb04\") " pod="llm/e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch"
Sep 10 19:49:46.235988 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:46.235908    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1efddb7-f98d-4a7f-8a0d-8834fd17cb04-tmp-dir\") pod \"e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch\" (UID: \"d1efddb7-f98d-4a7f-8a0d-8834fd17cb04\") " pod="llm/e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch"
Sep 10 19:49:46.235988 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:46.235955    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1efddb7-f98d-4a7f-8a0d-8834fd17cb04-model-cache\") pod \"e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch\" (UID: \"d1efddb7-f98d-4a7f-8a0d-8834fd17cb04\") " pod="llm/e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch"
Sep 10 19:49:46.236198 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:46.236110    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1efddb7-f98d-4a7f-8a0d-8834fd17cb04-home\") pod \"e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch\" (UID: \"d1efddb7-f98d-4a7f-8a0d-8834fd17cb04\") " pod="llm/e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch"
Sep 10 19:49:46.236314 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:46.236293    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1efddb7-f98d-4a7f-8a0d-8834fd17cb04-tmp-dir\") pod \"e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch\" (UID: \"d1efddb7-f98d-4a7f-8a0d-8834fd17cb04\") " pod="llm/e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch"
Sep 10 19:49:46.236372 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:46.236344    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1efddb7-f98d-4a7f-8a0d-8834fd17cb04-model-cache\") pod \"e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch\" (UID: \"d1efddb7-f98d-4a7f-8a0d-8834fd17cb04\") " pod="llm/e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch"
Sep 10 19:49:46.238319 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:46.238292    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1efddb7-f98d-4a7f-8a0d-8834fd17cb04-dshm\") pod \"e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch\" (UID: \"d1efddb7-f98d-4a7f-8a0d-8834fd17cb04\") " pod="llm/e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch"
Sep 10 19:49:46.238548 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:46.238534    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1efddb7-f98d-4a7f-8a0d-8834fd17cb04-tls-certs\") pod \"e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch\" (UID: \"d1efddb7-f98d-4a7f-8a0d-8834fd17cb04\") " pod="llm/e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch"
Sep 10 19:49:46.244541 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:46.244523    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ph8q5\" (UniqueName: \"kubernetes.io/projected/d1efddb7-f98d-4a7f-8a0d-8834fd17cb04-kube-api-access-ph8q5\") pod \"e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch\" (UID: \"d1efddb7-f98d-4a7f-8a0d-8834fd17cb04\") " pod="llm/e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch"
Sep 10 19:49:46.298911 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:46.298878    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch"
Sep 10 19:49:46.574452 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:46.574420    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch"]
Sep 10 19:49:46.575928 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:49:46.575905    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd1efddb7_f98d_4a7f_8a0d_8834fd17cb04.slice/crio-e9689b0e7172aae7a83c3fc5815716adc5d42f404d24f8ff205e114d5a21e39d WatchSource:0}: Error finding container e9689b0e7172aae7a83c3fc5815716adc5d42f404d24f8ff205e114d5a21e39d: Status 404 returned error can't find the container with id e9689b0e7172aae7a83c3fc5815716adc5d42f404d24f8ff205e114d5a21e39d
Sep 10 19:49:47.582230 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:47.582191    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch" event={"ID":"d1efddb7-f98d-4a7f-8a0d-8834fd17cb04","Type":"ContainerStarted","Data":"918fdf2f7d9a6eff2e87df02c5d2f8860080c808ff609c4af1d511071e15e870"}
Sep 10 19:49:47.582230 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:47.582239    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch" event={"ID":"d1efddb7-f98d-4a7f-8a0d-8834fd17cb04","Type":"ContainerStarted","Data":"e9689b0e7172aae7a83c3fc5815716adc5d42f404d24f8ff205e114d5a21e39d"}
Sep 10 19:49:47.600569 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:47.600464    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch" podStartSLOduration=2.60045103 podStartE2EDuration="2.60045103s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:49:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:49:47.5973202 +0000 UTC m=+2042.817168680" watchObservedRunningTime="2026-09-10 19:49:47.60045103 +0000 UTC m=+2042.820299511"
Sep 10 19:49:47.802790 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:47.802740    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh"
Sep 10 19:49:47.802970 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:47.802834    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh"
Sep 10 19:49:47.815456 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:47.815430    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh"
Sep 10 19:49:48.598820 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:48.598794    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh"
Sep 10 19:49:56.299281 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:56.299249    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch"
Sep 10 19:49:56.299763 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:56.299516    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch"
Sep 10 19:49:56.312737 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:56.312705    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch"
Sep 10 19:49:56.647235 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:56.647206    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch"
Sep 10 19:49:58.748390 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:49:58.748353    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh"]
Sep 10 19:50:00.367780 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:00.367733    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch"]
Sep 10 19:50:13.769821 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:13.769694    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh" podUID="1146124a-e1a9-48f9-b419-4d143bca6d7a" containerName="main" containerID="cri-o://95dbccc0daa18405ce4c2edda714b137e85aac5f4ab14f6f1aa04e08ad685055" gracePeriod=45
Sep 10 19:50:14.030383 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:14.030326    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh"
Sep 10 19:50:14.109842 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:14.109808    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1146124a-e1a9-48f9-b419-4d143bca6d7a-tmp-dir\") pod \"1146124a-e1a9-48f9-b419-4d143bca6d7a\" (UID: \"1146124a-e1a9-48f9-b419-4d143bca6d7a\") "
Sep 10 19:50:14.110020 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:14.109865    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-992tb\" (UniqueName: \"kubernetes.io/projected/1146124a-e1a9-48f9-b419-4d143bca6d7a-kube-api-access-992tb\") pod \"1146124a-e1a9-48f9-b419-4d143bca6d7a\" (UID: \"1146124a-e1a9-48f9-b419-4d143bca6d7a\") "
Sep 10 19:50:14.110020 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:14.109883    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1146124a-e1a9-48f9-b419-4d143bca6d7a-home\") pod \"1146124a-e1a9-48f9-b419-4d143bca6d7a\" (UID: \"1146124a-e1a9-48f9-b419-4d143bca6d7a\") "
Sep 10 19:50:14.110020 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:14.109933    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1146124a-e1a9-48f9-b419-4d143bca6d7a-dshm\") pod \"1146124a-e1a9-48f9-b419-4d143bca6d7a\" (UID: \"1146124a-e1a9-48f9-b419-4d143bca6d7a\") "
Sep 10 19:50:14.110020 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:14.109960    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1146124a-e1a9-48f9-b419-4d143bca6d7a-tls-certs\") pod \"1146124a-e1a9-48f9-b419-4d143bca6d7a\" (UID: \"1146124a-e1a9-48f9-b419-4d143bca6d7a\") "
Sep 10 19:50:14.110020 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:14.109976    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1146124a-e1a9-48f9-b419-4d143bca6d7a-model-cache\") pod \"1146124a-e1a9-48f9-b419-4d143bca6d7a\" (UID: \"1146124a-e1a9-48f9-b419-4d143bca6d7a\") "
Sep 10 19:50:14.110290 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:14.110142    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1146124a-e1a9-48f9-b419-4d143bca6d7a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1146124a-e1a9-48f9-b419-4d143bca6d7a" (UID: "1146124a-e1a9-48f9-b419-4d143bca6d7a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:50:14.110290 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:14.110181    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1146124a-e1a9-48f9-b419-4d143bca6d7a-home" (OuterVolumeSpecName: "home") pod "1146124a-e1a9-48f9-b419-4d143bca6d7a" (UID: "1146124a-e1a9-48f9-b419-4d143bca6d7a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:50:14.110404 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:14.110294    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1146124a-e1a9-48f9-b419-4d143bca6d7a-model-cache" (OuterVolumeSpecName: "model-cache") pod "1146124a-e1a9-48f9-b419-4d143bca6d7a" (UID: "1146124a-e1a9-48f9-b419-4d143bca6d7a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:50:14.110460 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:14.110425    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1146124a-e1a9-48f9-b419-4d143bca6d7a-tmp-dir\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:50:14.110460 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:14.110439    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1146124a-e1a9-48f9-b419-4d143bca6d7a-home\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:50:14.110460 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:14.110454    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1146124a-e1a9-48f9-b419-4d143bca6d7a-model-cache\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:50:14.112267 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:14.112234    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1146124a-e1a9-48f9-b419-4d143bca6d7a-dshm" (OuterVolumeSpecName: "dshm") pod "1146124a-e1a9-48f9-b419-4d143bca6d7a" (UID: "1146124a-e1a9-48f9-b419-4d143bca6d7a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:50:14.112384 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:14.112275    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1146124a-e1a9-48f9-b419-4d143bca6d7a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1146124a-e1a9-48f9-b419-4d143bca6d7a" (UID: "1146124a-e1a9-48f9-b419-4d143bca6d7a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:50:14.112384 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:14.112320    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1146124a-e1a9-48f9-b419-4d143bca6d7a-kube-api-access-992tb" (OuterVolumeSpecName: "kube-api-access-992tb") pod "1146124a-e1a9-48f9-b419-4d143bca6d7a" (UID: "1146124a-e1a9-48f9-b419-4d143bca6d7a"). InnerVolumeSpecName "kube-api-access-992tb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:50:14.211606 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:14.211570    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1146124a-e1a9-48f9-b419-4d143bca6d7a-tls-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:50:14.211606 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:14.211604    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-992tb\" (UniqueName: \"kubernetes.io/projected/1146124a-e1a9-48f9-b419-4d143bca6d7a-kube-api-access-992tb\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:50:14.211606 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:14.211614    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1146124a-e1a9-48f9-b419-4d143bca6d7a-dshm\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:50:14.749287 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:14.749255    2579 generic.go:356] "Generic (PLEG): container finished" podID="1146124a-e1a9-48f9-b419-4d143bca6d7a" containerID="95dbccc0daa18405ce4c2edda714b137e85aac5f4ab14f6f1aa04e08ad685055" exitCode=0
Sep 10 19:50:14.749467 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:14.749298    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh"
Sep 10 19:50:14.749467 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:14.749346    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh" event={"ID":"1146124a-e1a9-48f9-b419-4d143bca6d7a","Type":"ContainerDied","Data":"95dbccc0daa18405ce4c2edda714b137e85aac5f4ab14f6f1aa04e08ad685055"}
Sep 10 19:50:14.749467 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:14.749389    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh" event={"ID":"1146124a-e1a9-48f9-b419-4d143bca6d7a","Type":"ContainerDied","Data":"f3f51d15eee77288948f703d5cd7b549430517adb6bd5694b53a66a2c6a437a6"}
Sep 10 19:50:14.749467 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:14.749408    2579 scope.go:117] "RemoveContainer" containerID="95dbccc0daa18405ce4c2edda714b137e85aac5f4ab14f6f1aa04e08ad685055"
Sep 10 19:50:14.759619 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:14.759599    2579 scope.go:117] "RemoveContainer" containerID="95dbccc0daa18405ce4c2edda714b137e85aac5f4ab14f6f1aa04e08ad685055"
Sep 10 19:50:14.759914 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:50:14.759896    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95dbccc0daa18405ce4c2edda714b137e85aac5f4ab14f6f1aa04e08ad685055\": container with ID starting with 95dbccc0daa18405ce4c2edda714b137e85aac5f4ab14f6f1aa04e08ad685055 not found: ID does not exist" containerID="95dbccc0daa18405ce4c2edda714b137e85aac5f4ab14f6f1aa04e08ad685055"
Sep 10 19:50:14.759983 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:14.759926    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95dbccc0daa18405ce4c2edda714b137e85aac5f4ab14f6f1aa04e08ad685055"} err="failed to get container status \"95dbccc0daa18405ce4c2edda714b137e85aac5f4ab14f6f1aa04e08ad685055\": rpc error: code = NotFound desc = could not find container \"95dbccc0daa18405ce4c2edda714b137e85aac5f4ab14f6f1aa04e08ad685055\": container with ID starting with 95dbccc0daa18405ce4c2edda714b137e85aac5f4ab14f6f1aa04e08ad685055 not found: ID does not exist"
Sep 10 19:50:14.771365 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:14.771339    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh"]
Sep 10 19:50:14.775792 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:14.775771    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-66bec08b-kserve-5784f4c6dc-nn8jh"]
Sep 10 19:50:15.240559 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:15.240529    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1146124a-e1a9-48f9-b419-4d143bca6d7a" path="/var/lib/kubelet/pods/1146124a-e1a9-48f9-b419-4d143bca6d7a/volumes"
Sep 10 19:50:15.388419 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:15.388384    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch" podUID="d1efddb7-f98d-4a7f-8a0d-8834fd17cb04" containerName="main" containerID="cri-o://918fdf2f7d9a6eff2e87df02c5d2f8860080c808ff609c4af1d511071e15e870" gracePeriod=45
Sep 10 19:50:15.641473 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:15.641445    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch"
Sep 10 19:50:15.726192 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:15.726155    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1efddb7-f98d-4a7f-8a0d-8834fd17cb04-model-cache\") pod \"d1efddb7-f98d-4a7f-8a0d-8834fd17cb04\" (UID: \"d1efddb7-f98d-4a7f-8a0d-8834fd17cb04\") "
Sep 10 19:50:15.726389 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:15.726223    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1efddb7-f98d-4a7f-8a0d-8834fd17cb04-tmp-dir\") pod \"d1efddb7-f98d-4a7f-8a0d-8834fd17cb04\" (UID: \"d1efddb7-f98d-4a7f-8a0d-8834fd17cb04\") "
Sep 10 19:50:15.726389 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:15.726286    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1efddb7-f98d-4a7f-8a0d-8834fd17cb04-dshm\") pod \"d1efddb7-f98d-4a7f-8a0d-8834fd17cb04\" (UID: \"d1efddb7-f98d-4a7f-8a0d-8834fd17cb04\") "
Sep 10 19:50:15.726389 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:15.726315    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1efddb7-f98d-4a7f-8a0d-8834fd17cb04-home\") pod \"d1efddb7-f98d-4a7f-8a0d-8834fd17cb04\" (UID: \"d1efddb7-f98d-4a7f-8a0d-8834fd17cb04\") "
Sep 10 19:50:15.726389 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:15.726371    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ph8q5\" (UniqueName: \"kubernetes.io/projected/d1efddb7-f98d-4a7f-8a0d-8834fd17cb04-kube-api-access-ph8q5\") pod \"d1efddb7-f98d-4a7f-8a0d-8834fd17cb04\" (UID: \"d1efddb7-f98d-4a7f-8a0d-8834fd17cb04\") "
Sep 10 19:50:15.726629 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:15.726438    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1efddb7-f98d-4a7f-8a0d-8834fd17cb04-tls-certs\") pod \"d1efddb7-f98d-4a7f-8a0d-8834fd17cb04\" (UID: \"d1efddb7-f98d-4a7f-8a0d-8834fd17cb04\") "
Sep 10 19:50:15.726629 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:15.726492    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1efddb7-f98d-4a7f-8a0d-8834fd17cb04-model-cache" (OuterVolumeSpecName: "model-cache") pod "d1efddb7-f98d-4a7f-8a0d-8834fd17cb04" (UID: "d1efddb7-f98d-4a7f-8a0d-8834fd17cb04"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:50:15.726629 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:15.726516    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1efddb7-f98d-4a7f-8a0d-8834fd17cb04-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d1efddb7-f98d-4a7f-8a0d-8834fd17cb04" (UID: "d1efddb7-f98d-4a7f-8a0d-8834fd17cb04"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:50:15.726629 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:15.726563    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1efddb7-f98d-4a7f-8a0d-8834fd17cb04-home" (OuterVolumeSpecName: "home") pod "d1efddb7-f98d-4a7f-8a0d-8834fd17cb04" (UID: "d1efddb7-f98d-4a7f-8a0d-8834fd17cb04"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:50:15.726869 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:15.726813    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1efddb7-f98d-4a7f-8a0d-8834fd17cb04-model-cache\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:50:15.726869 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:15.726829    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1efddb7-f98d-4a7f-8a0d-8834fd17cb04-tmp-dir\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:50:15.726869 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:15.726840    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1efddb7-f98d-4a7f-8a0d-8834fd17cb04-home\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:50:15.728563 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:15.728538    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1efddb7-f98d-4a7f-8a0d-8834fd17cb04-dshm" (OuterVolumeSpecName: "dshm") pod "d1efddb7-f98d-4a7f-8a0d-8834fd17cb04" (UID: "d1efddb7-f98d-4a7f-8a0d-8834fd17cb04"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:50:15.728673 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:15.728570    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1efddb7-f98d-4a7f-8a0d-8834fd17cb04-kube-api-access-ph8q5" (OuterVolumeSpecName: "kube-api-access-ph8q5") pod "d1efddb7-f98d-4a7f-8a0d-8834fd17cb04" (UID: "d1efddb7-f98d-4a7f-8a0d-8834fd17cb04"). InnerVolumeSpecName "kube-api-access-ph8q5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:50:15.728673 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:15.728604    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1efddb7-f98d-4a7f-8a0d-8834fd17cb04-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d1efddb7-f98d-4a7f-8a0d-8834fd17cb04" (UID: "d1efddb7-f98d-4a7f-8a0d-8834fd17cb04"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:50:15.757059 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:15.756975    2579 generic.go:356] "Generic (PLEG): container finished" podID="d1efddb7-f98d-4a7f-8a0d-8834fd17cb04" containerID="918fdf2f7d9a6eff2e87df02c5d2f8860080c808ff609c4af1d511071e15e870" exitCode=0
Sep 10 19:50:15.757059 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:15.757023    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch"
Sep 10 19:50:15.757264 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:15.757063    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch" event={"ID":"d1efddb7-f98d-4a7f-8a0d-8834fd17cb04","Type":"ContainerDied","Data":"918fdf2f7d9a6eff2e87df02c5d2f8860080c808ff609c4af1d511071e15e870"}
Sep 10 19:50:15.757264 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:15.757117    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch" event={"ID":"d1efddb7-f98d-4a7f-8a0d-8834fd17cb04","Type":"ContainerDied","Data":"e9689b0e7172aae7a83c3fc5815716adc5d42f404d24f8ff205e114d5a21e39d"}
Sep 10 19:50:15.757264 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:15.757139    2579 scope.go:117] "RemoveContainer" containerID="918fdf2f7d9a6eff2e87df02c5d2f8860080c808ff609c4af1d511071e15e870"
Sep 10 19:50:15.766944 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:15.766908    2579 scope.go:117] "RemoveContainer" containerID="918fdf2f7d9a6eff2e87df02c5d2f8860080c808ff609c4af1d511071e15e870"
Sep 10 19:50:15.767219 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:50:15.767200    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"918fdf2f7d9a6eff2e87df02c5d2f8860080c808ff609c4af1d511071e15e870\": container with ID starting with 918fdf2f7d9a6eff2e87df02c5d2f8860080c808ff609c4af1d511071e15e870 not found: ID does not exist" containerID="918fdf2f7d9a6eff2e87df02c5d2f8860080c808ff609c4af1d511071e15e870"
Sep 10 19:50:15.767282 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:15.767231    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"918fdf2f7d9a6eff2e87df02c5d2f8860080c808ff609c4af1d511071e15e870"} err="failed to get container status \"918fdf2f7d9a6eff2e87df02c5d2f8860080c808ff609c4af1d511071e15e870\": rpc error: code = NotFound desc = could not find container \"918fdf2f7d9a6eff2e87df02c5d2f8860080c808ff609c4af1d511071e15e870\": container with ID starting with 918fdf2f7d9a6eff2e87df02c5d2f8860080c808ff609c4af1d511071e15e870 not found: ID does not exist"
Sep 10 19:50:15.779048 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:15.779025    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch"]
Sep 10 19:50:15.783145 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:15.783125    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-66bec08b-kserve-67449dc6c-672ch"]
Sep 10 19:50:15.827377 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:15.827345    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ph8q5\" (UniqueName: \"kubernetes.io/projected/d1efddb7-f98d-4a7f-8a0d-8834fd17cb04-kube-api-access-ph8q5\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:50:15.827377 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:15.827367    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1efddb7-f98d-4a7f-8a0d-8834fd17cb04-tls-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:50:15.827537 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:15.827384    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1efddb7-f98d-4a7f-8a0d-8834fd17cb04-dshm\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:50:17.240157 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:17.240116    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1efddb7-f98d-4a7f-8a0d-8834fd17cb04" path="/var/lib/kubelet/pods/d1efddb7-f98d-4a7f-8a0d-8834fd17cb04/volumes"
Sep 10 19:50:19.482813 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:19.482780    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-8cb5f7c95-jvf59"]
Sep 10 19:50:19.483377 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:19.483357    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1146124a-e1a9-48f9-b419-4d143bca6d7a" containerName="main"
Sep 10 19:50:19.483458 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:19.483379    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1146124a-e1a9-48f9-b419-4d143bca6d7a" containerName="main"
Sep 10 19:50:19.483458 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:19.483391    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1efddb7-f98d-4a7f-8a0d-8834fd17cb04" containerName="main"
Sep 10 19:50:19.483458 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:19.483398    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1efddb7-f98d-4a7f-8a0d-8834fd17cb04" containerName="main"
Sep 10 19:50:19.483620 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:19.483518    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1146124a-e1a9-48f9-b419-4d143bca6d7a" containerName="main"
Sep 10 19:50:19.483620 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:19.483531    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1efddb7-f98d-4a7f-8a0d-8834fd17cb04" containerName="main"
Sep 10 19:50:19.486727 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:19.486694    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-jvf59"
Sep 10 19:50:19.511841 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:19.511802    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8cb5f7c95-jvf59"]
Sep 10 19:50:19.563096 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:19.563062    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b9348d4-d92a-4a6e-944d-1611a38bc965-openshift-service-ca\") pod \"maas-api-8cb5f7c95-jvf59\" (UID: \"6b9348d4-d92a-4a6e-944d-1611a38bc965\") " pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-jvf59"
Sep 10 19:50:19.563281 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:19.563106    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hmfq\" (UniqueName: \"kubernetes.io/projected/6b9348d4-d92a-4a6e-944d-1611a38bc965-kube-api-access-4hmfq\") pod \"maas-api-8cb5f7c95-jvf59\" (UID: \"6b9348d4-d92a-4a6e-944d-1611a38bc965\") " pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-jvf59"
Sep 10 19:50:19.563281 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:19.563148    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b9348d4-d92a-4a6e-944d-1611a38bc965-maas-api-tls\") pod \"maas-api-8cb5f7c95-jvf59\" (UID: \"6b9348d4-d92a-4a6e-944d-1611a38bc965\") " pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-jvf59"
Sep 10 19:50:19.563281 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:19.563188    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b9348d4-d92a-4a6e-944d-1611a38bc965-metrics-certs\") pod \"maas-api-8cb5f7c95-jvf59\" (UID: \"6b9348d4-d92a-4a6e-944d-1611a38bc965\") " pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-jvf59"
Sep 10 19:50:19.664719 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:19.664684    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b9348d4-d92a-4a6e-944d-1611a38bc965-openshift-service-ca\") pod \"maas-api-8cb5f7c95-jvf59\" (UID: \"6b9348d4-d92a-4a6e-944d-1611a38bc965\") " pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-jvf59"
Sep 10 19:50:19.664719 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:19.664721    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hmfq\" (UniqueName: \"kubernetes.io/projected/6b9348d4-d92a-4a6e-944d-1611a38bc965-kube-api-access-4hmfq\") pod \"maas-api-8cb5f7c95-jvf59\" (UID: \"6b9348d4-d92a-4a6e-944d-1611a38bc965\") " pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-jvf59"
Sep 10 19:50:19.664997 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:19.664783    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b9348d4-d92a-4a6e-944d-1611a38bc965-maas-api-tls\") pod \"maas-api-8cb5f7c95-jvf59\" (UID: \"6b9348d4-d92a-4a6e-944d-1611a38bc965\") " pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-jvf59"
Sep 10 19:50:19.664997 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:19.664808    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b9348d4-d92a-4a6e-944d-1611a38bc965-metrics-certs\") pod \"maas-api-8cb5f7c95-jvf59\" (UID: \"6b9348d4-d92a-4a6e-944d-1611a38bc965\") " pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-jvf59"
Sep 10 19:50:19.665444 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:19.665418    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b9348d4-d92a-4a6e-944d-1611a38bc965-openshift-service-ca\") pod \"maas-api-8cb5f7c95-jvf59\" (UID: \"6b9348d4-d92a-4a6e-944d-1611a38bc965\") " pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-jvf59"
Sep 10 19:50:19.667482 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:19.667458    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b9348d4-d92a-4a6e-944d-1611a38bc965-metrics-certs\") pod \"maas-api-8cb5f7c95-jvf59\" (UID: \"6b9348d4-d92a-4a6e-944d-1611a38bc965\") " pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-jvf59"
Sep 10 19:50:19.667583 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:19.667482    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b9348d4-d92a-4a6e-944d-1611a38bc965-maas-api-tls\") pod \"maas-api-8cb5f7c95-jvf59\" (UID: \"6b9348d4-d92a-4a6e-944d-1611a38bc965\") " pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-jvf59"
Sep 10 19:50:19.673184 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:19.673164    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hmfq\" (UniqueName: \"kubernetes.io/projected/6b9348d4-d92a-4a6e-944d-1611a38bc965-kube-api-access-4hmfq\") pod \"maas-api-8cb5f7c95-jvf59\" (UID: \"6b9348d4-d92a-4a6e-944d-1611a38bc965\") " pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-jvf59"
Sep 10 19:50:19.801060 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:19.800969    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-jvf59"
Sep 10 19:50:20.090259 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:20.090235    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8cb5f7c95-jvf59"]
Sep 10 19:50:20.789250 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:20.789153    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-jvf59" event={"ID":"6b9348d4-d92a-4a6e-944d-1611a38bc965","Type":"ContainerStarted","Data":"e65b52d7e6818318dc42be519b21169e5414347c86d3164f7261dccb1250e430"}
Sep 10 19:50:20.789250 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:20.789206    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-jvf59" event={"ID":"6b9348d4-d92a-4a6e-944d-1611a38bc965","Type":"ContainerStarted","Data":"16ccb5d82464cf0cc48ea9a041a34cb2dfb3aedec09c8baa7e037917774d3650"}
Sep 10 19:50:20.789854 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:20.789288    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-jvf59"
Sep 10 19:50:20.811589 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:20.811537    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-jvf59" podStartSLOduration=1.371443177 podStartE2EDuration="1.811518095s" totalImagesPullingTime="440.074918ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:50:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:50:20.80890266 +0000 UTC m=+2076.028751140" watchObservedRunningTime="2026-09-10 19:50:20.811518095 +0000 UTC m=+2076.031366575"
Sep 10 19:50:26.800477 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:26.800448    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-jvf59"
Sep 10 19:50:34.112308 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:34.112275    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7d4dc54f4c-wvlr7"]
Sep 10 19:50:34.112771 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:34.112481    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-wvlr7" podUID="3d2ff726-2d16-4771-b26e-4a11e90db6f5" containerName="maas-api" containerID="cri-o://48512e9bb64b1eec18611391efe314d8110b1c273b6592fc17fda3bb02201d55" gracePeriod=30
Sep 10 19:50:34.136051 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:34.136008    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-wvlr7" podUID="3d2ff726-2d16-4771-b26e-4a11e90db6f5" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.59:8443/health\": dial tcp 10.133.0.59:8443: connect: connection refused"
Sep 10 19:50:35.356776 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:35.356731    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-wvlr7"
Sep 10 19:50:35.521907 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:35.521817    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d2ff726-2d16-4771-b26e-4a11e90db6f5-openshift-service-ca\") pod \"3d2ff726-2d16-4771-b26e-4a11e90db6f5\" (UID: \"3d2ff726-2d16-4771-b26e-4a11e90db6f5\") "
Sep 10 19:50:35.522067 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:35.521954    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d2ff726-2d16-4771-b26e-4a11e90db6f5-metrics-certs\") pod \"3d2ff726-2d16-4771-b26e-4a11e90db6f5\" (UID: \"3d2ff726-2d16-4771-b26e-4a11e90db6f5\") "
Sep 10 19:50:35.522067 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:35.521986    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g2xzm\" (UniqueName: \"kubernetes.io/projected/3d2ff726-2d16-4771-b26e-4a11e90db6f5-kube-api-access-g2xzm\") pod \"3d2ff726-2d16-4771-b26e-4a11e90db6f5\" (UID: \"3d2ff726-2d16-4771-b26e-4a11e90db6f5\") "
Sep 10 19:50:35.522067 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:35.522000    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d2ff726-2d16-4771-b26e-4a11e90db6f5-maas-api-tls\") pod \"3d2ff726-2d16-4771-b26e-4a11e90db6f5\" (UID: \"3d2ff726-2d16-4771-b26e-4a11e90db6f5\") "
Sep 10 19:50:35.522236 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:35.522170    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d2ff726-2d16-4771-b26e-4a11e90db6f5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3d2ff726-2d16-4771-b26e-4a11e90db6f5" (UID: "3d2ff726-2d16-4771-b26e-4a11e90db6f5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:50:35.524318 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:35.524290    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d2ff726-2d16-4771-b26e-4a11e90db6f5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3d2ff726-2d16-4771-b26e-4a11e90db6f5" (UID: "3d2ff726-2d16-4771-b26e-4a11e90db6f5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:50:35.524318 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:35.524310    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d2ff726-2d16-4771-b26e-4a11e90db6f5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3d2ff726-2d16-4771-b26e-4a11e90db6f5" (UID: "3d2ff726-2d16-4771-b26e-4a11e90db6f5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:50:35.524469 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:35.524366    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d2ff726-2d16-4771-b26e-4a11e90db6f5-kube-api-access-g2xzm" (OuterVolumeSpecName: "kube-api-access-g2xzm") pod "3d2ff726-2d16-4771-b26e-4a11e90db6f5" (UID: "3d2ff726-2d16-4771-b26e-4a11e90db6f5"). InnerVolumeSpecName "kube-api-access-g2xzm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:50:35.623166 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:35.623126    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d2ff726-2d16-4771-b26e-4a11e90db6f5-metrics-certs\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:50:35.623166 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:35.623154    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2xzm\" (UniqueName: \"kubernetes.io/projected/3d2ff726-2d16-4771-b26e-4a11e90db6f5-kube-api-access-g2xzm\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:50:35.623166 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:35.623163    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d2ff726-2d16-4771-b26e-4a11e90db6f5-maas-api-tls\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:50:35.623166 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:35.623174    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d2ff726-2d16-4771-b26e-4a11e90db6f5-openshift-service-ca\") on node \"ip-10-0-142-20.ec2.internal\" DevicePath \"\""
Sep 10 19:50:35.884228 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:35.884193    2579 generic.go:356] "Generic (PLEG): container finished" podID="3d2ff726-2d16-4771-b26e-4a11e90db6f5" containerID="48512e9bb64b1eec18611391efe314d8110b1c273b6592fc17fda3bb02201d55" exitCode=0
Sep 10 19:50:35.884388 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:35.884240    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-wvlr7"
Sep 10 19:50:35.884388 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:35.884243    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-wvlr7" event={"ID":"3d2ff726-2d16-4771-b26e-4a11e90db6f5","Type":"ContainerDied","Data":"48512e9bb64b1eec18611391efe314d8110b1c273b6592fc17fda3bb02201d55"}
Sep 10 19:50:35.884388 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:35.884302    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7d4dc54f4c-wvlr7" event={"ID":"3d2ff726-2d16-4771-b26e-4a11e90db6f5","Type":"ContainerDied","Data":"6079a9e0086d14da8447b9b5035955b100b36a35556478f9a3d7bc7da2a6b9b8"}
Sep 10 19:50:35.884388 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:35.884325    2579 scope.go:117] "RemoveContainer" containerID="48512e9bb64b1eec18611391efe314d8110b1c273b6592fc17fda3bb02201d55"
Sep 10 19:50:35.894599 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:35.894580    2579 scope.go:117] "RemoveContainer" containerID="48512e9bb64b1eec18611391efe314d8110b1c273b6592fc17fda3bb02201d55"
Sep 10 19:50:35.894947 ip-10-0-142-20 kubenswrapper[2579]: E0910 19:50:35.894928    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48512e9bb64b1eec18611391efe314d8110b1c273b6592fc17fda3bb02201d55\": container with ID starting with 48512e9bb64b1eec18611391efe314d8110b1c273b6592fc17fda3bb02201d55 not found: ID does not exist" containerID="48512e9bb64b1eec18611391efe314d8110b1c273b6592fc17fda3bb02201d55"
Sep 10 19:50:35.895009 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:35.894957    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48512e9bb64b1eec18611391efe314d8110b1c273b6592fc17fda3bb02201d55"} err="failed to get container status \"48512e9bb64b1eec18611391efe314d8110b1c273b6592fc17fda3bb02201d55\": rpc error: code = NotFound desc = could not find container \"48512e9bb64b1eec18611391efe314d8110b1c273b6592fc17fda3bb02201d55\": container with ID starting with 48512e9bb64b1eec18611391efe314d8110b1c273b6592fc17fda3bb02201d55 not found: ID does not exist"
Sep 10 19:50:35.906863 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:35.906835    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7d4dc54f4c-wvlr7"]
Sep 10 19:50:35.915141 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:35.915119    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-7d4dc54f4c-wvlr7"]
Sep 10 19:50:37.239826 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:37.239790    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d2ff726-2d16-4771-b26e-4a11e90db6f5" path="/var/lib/kubelet/pods/3d2ff726-2d16-4771-b26e-4a11e90db6f5/volumes"
Sep 10 19:50:45.366178 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:45.366144    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bmdg9_9817270c-8629-44c4-ab8f-f3982e1f0e2f/console-operator/1.log"
Sep 10 19:50:45.370806 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:45.370784    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8r66s_2826f8ec-78f6-4cab-879e-47a21854ed5a/ovn-acl-logging/0.log"
Sep 10 19:50:45.372721 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:45.372706    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bmdg9_9817270c-8629-44c4-ab8f-f3982e1f0e2f/console-operator/1.log"
Sep 10 19:50:45.377135 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:45.377118    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8r66s_2826f8ec-78f6-4cab-879e-47a21854ed5a/ovn-acl-logging/0.log"
Sep 10 19:50:54.355966 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:54.355934    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-h2rcd_87c2c3c1-8abb-447e-a1c2-dc779a3cb86a/manager/0.log"
Sep 10 19:50:54.597159 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:54.597130    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-77c474c76b-jhn5d_3aeab5a1-59dd-4975-811f-8c9eb50a3605/manager/0.log"
Sep 10 19:50:54.723301 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:54.723278    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-w2wms_c472bc15-0bbd-43bb-8cd3-277fb3d01ffd/server/0.log"
Sep 10 19:50:54.845117 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:54.845089    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-2gt8j_4cc26bdf-0a6c-42a8-a000-a97fdd40b51d/manager/0.log"
Sep 10 19:50:55.070436 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:55.070361    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d477566c-8rcd6_8c10ecbe-e921-4f4b-b077-0bf836adf8b5/copy-manifests/0.log"
Sep 10 19:50:55.077086 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:55.077059    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d477566c-8rcd6_8c10ecbe-e921-4f4b-b077-0bf836adf8b5/manager/0.log"
Sep 10 19:50:55.567203 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:55.567174    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-8cb5f7c95-jvf59_6b9348d4-d92a-4a6e-944d-1611a38bc965/maas-api/0.log"
Sep 10 19:50:56.006031 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:56.005998    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-2vq2x_79c2ba5f-2254-4d6e-8f90-697212b36ff1/postgres/0.log"
Sep 10 19:50:56.887527 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:56.887496    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-hjhj4_3a2b3262-dec1-4ede-9891-b202c0ee6574/manager/0.log"
Sep 10 19:50:57.000165 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:57.000138    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-zw56j_0c31700a-7f6f-464d-940a-77e756cb8f9b/manager/0.log"
Sep 10 19:50:57.102960 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:57.102928    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-97xw4_6151b239-2930-4ba6-b5c0-1d12b366a700/kuadrant-console-plugin/0.log"
Sep 10 19:50:57.436925 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:57.436888    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-5svd7_37dfe835-023f-4221-aa07-f8025479ee52/manager/0.log"
Sep 10 19:50:57.777763 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:57.777673    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k_e52d0ec4-e194-4be2-9690-c2c541a41f7f/istio-proxy/0.log"
Sep 10 19:50:58.005931 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:58.005901    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dd7bdc797-jljr6_896dee6e-5770-4d1a-9637-e2573b9ebc16/kube-auth-proxy/0.log"
Sep 10 19:50:58.231674 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:58.231642    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-ltgk2_4fd6f983-387d-48bd-9942-0157de3ce6de/istio-proxy/1.log"
Sep 10 19:50:58.952961 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:58.952931    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-klfdk_9eb974ab-1b17-4f69-9829-397b61fd5aae/main/0.log"
Sep 10 19:50:59.065024 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:50:59.064997    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-z8q2v_8a97f317-2ff5-4001-bb6c-f3a09d29ca52/main/0.log"
Sep 10 19:51:05.068872 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:05.068828    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5hbck/must-gather-lf7s6"]
Sep 10 19:51:05.069269 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:05.069256    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d2ff726-2d16-4771-b26e-4a11e90db6f5" containerName="maas-api"
Sep 10 19:51:05.069319 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:05.069271    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d2ff726-2d16-4771-b26e-4a11e90db6f5" containerName="maas-api"
Sep 10 19:51:05.069354 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:05.069332    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d2ff726-2d16-4771-b26e-4a11e90db6f5" containerName="maas-api"
Sep 10 19:51:05.072775 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:05.072737    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5hbck/must-gather-lf7s6"
Sep 10 19:51:05.076978 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:05.076952    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5hbck\"/\"kube-root-ca.crt\""
Sep 10 19:51:05.077128 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:05.077088    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5hbck\"/\"default-dockercfg-r6dbn\""
Sep 10 19:51:05.077206 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:05.077130    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5hbck\"/\"openshift-service-ca.crt\""
Sep 10 19:51:05.094837 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:05.094816    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5hbck/must-gather-lf7s6"]
Sep 10 19:51:05.193849 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:05.193817    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqkbh\" (UniqueName: \"kubernetes.io/projected/e6e69cce-32fd-46f0-81e6-330a4bf9ada3-kube-api-access-hqkbh\") pod \"must-gather-lf7s6\" (UID: \"e6e69cce-32fd-46f0-81e6-330a4bf9ada3\") " pod="openshift-must-gather-5hbck/must-gather-lf7s6"
Sep 10 19:51:05.194026 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:05.193871    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e6e69cce-32fd-46f0-81e6-330a4bf9ada3-must-gather-output\") pod \"must-gather-lf7s6\" (UID: \"e6e69cce-32fd-46f0-81e6-330a4bf9ada3\") " pod="openshift-must-gather-5hbck/must-gather-lf7s6"
Sep 10 19:51:05.295061 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:05.295013    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqkbh\" (UniqueName: \"kubernetes.io/projected/e6e69cce-32fd-46f0-81e6-330a4bf9ada3-kube-api-access-hqkbh\") pod \"must-gather-lf7s6\" (UID: \"e6e69cce-32fd-46f0-81e6-330a4bf9ada3\") " pod="openshift-must-gather-5hbck/must-gather-lf7s6"
Sep 10 19:51:05.295245 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:05.295085    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e6e69cce-32fd-46f0-81e6-330a4bf9ada3-must-gather-output\") pod \"must-gather-lf7s6\" (UID: \"e6e69cce-32fd-46f0-81e6-330a4bf9ada3\") " pod="openshift-must-gather-5hbck/must-gather-lf7s6"
Sep 10 19:51:05.295440 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:05.295419    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e6e69cce-32fd-46f0-81e6-330a4bf9ada3-must-gather-output\") pod \"must-gather-lf7s6\" (UID: \"e6e69cce-32fd-46f0-81e6-330a4bf9ada3\") " pod="openshift-must-gather-5hbck/must-gather-lf7s6"
Sep 10 19:51:05.304025 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:05.303999    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqkbh\" (UniqueName: \"kubernetes.io/projected/e6e69cce-32fd-46f0-81e6-330a4bf9ada3-kube-api-access-hqkbh\") pod \"must-gather-lf7s6\" (UID: \"e6e69cce-32fd-46f0-81e6-330a4bf9ada3\") " pod="openshift-must-gather-5hbck/must-gather-lf7s6"
Sep 10 19:51:05.382648 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:05.382617    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5hbck/must-gather-lf7s6"
Sep 10 19:51:05.658695 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:05.658668    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5hbck/must-gather-lf7s6"]
Sep 10 19:51:05.660567 ip-10-0-142-20 kubenswrapper[2579]: W0910 19:51:05.660535    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode6e69cce_32fd_46f0_81e6_330a4bf9ada3.slice/crio-9b34f6990a3e0ac50c7718709c2341a5faee2472ff73f134dc702afe4b56b341 WatchSource:0}: Error finding container 9b34f6990a3e0ac50c7718709c2341a5faee2472ff73f134dc702afe4b56b341: Status 404 returned error can't find the container with id 9b34f6990a3e0ac50c7718709c2341a5faee2472ff73f134dc702afe4b56b341
Sep 10 19:51:06.056968 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:06.056872    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5hbck/must-gather-lf7s6" event={"ID":"e6e69cce-32fd-46f0-81e6-330a4bf9ada3","Type":"ContainerStarted","Data":"9b34f6990a3e0ac50c7718709c2341a5faee2472ff73f134dc702afe4b56b341"}
Sep 10 19:51:08.074278 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:08.073346    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5hbck/must-gather-lf7s6" event={"ID":"e6e69cce-32fd-46f0-81e6-330a4bf9ada3","Type":"ContainerStarted","Data":"87d06b6c202e0525ddada0ba018091b9239b8e0ed8280462a2feb566e931de7c"}
Sep 10 19:51:08.074278 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:08.073409    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5hbck/must-gather-lf7s6" event={"ID":"e6e69cce-32fd-46f0-81e6-330a4bf9ada3","Type":"ContainerStarted","Data":"327da437ed33d2848c3d025d64e482dfe6cfdf00dd34bbb8a54c60524f3154c9"}
Sep 10 19:51:08.093555 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:08.091284    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5hbck/must-gather-lf7s6" podStartSLOduration=1.7713971659999999 podStartE2EDuration="3.091263004s" totalImagesPullingTime="1.319865838s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:51:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:51:08.090258795 +0000 UTC m=+2123.310107276" watchObservedRunningTime="2026-09-10 19:51:08.091263004 +0000 UTC m=+2123.311111488"
Sep 10 19:51:08.715680 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:08.715652    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-mkkzj_acd262af-057c-4bbf-93e3-faa020c1825c/global-pull-secret-syncer/0.log"
Sep 10 19:51:08.936161 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:08.936115    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-cllmw_b47634b2-779d-426a-8a6b-c74a246654d4/konnectivity-agent/0.log"
Sep 10 19:51:09.006435 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:09.006342    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-20.ec2.internal_e0bff5a7795653f306d187693d0b9c36/haproxy/0.log"
Sep 10 19:51:13.883006 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:13.882962    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-hjhj4_3a2b3262-dec1-4ede-9891-b202c0ee6574/manager/0.log"
Sep 10 19:51:13.910832 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:13.910723    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-zw56j_0c31700a-7f6f-464d-940a-77e756cb8f9b/manager/0.log"
Sep 10 19:51:13.937152 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:13.937121    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-97xw4_6151b239-2930-4ba6-b5c0-1d12b366a700/kuadrant-console-plugin/0.log"
Sep 10 19:51:14.082535 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:14.082493    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-5svd7_37dfe835-023f-4221-aa07-f8025479ee52/manager/0.log"
Sep 10 19:51:15.880505 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:15.880155    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-d679v_acc7de42-5736-472c-958d-63e300039ca9/kube-state-metrics/0.log"
Sep 10 19:51:15.907827 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:15.907791    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-d679v_acc7de42-5736-472c-958d-63e300039ca9/kube-rbac-proxy-main/0.log"
Sep 10 19:51:15.933049 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:15.933023    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-d679v_acc7de42-5736-472c-958d-63e300039ca9/kube-rbac-proxy-self/0.log"
Sep 10 19:51:15.963619 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:15.963541    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-55f6cbdc4b-6s7ds_6071f2a7-2810-4cd3-93e2-f18ac96d1950/metrics-server/0.log"
Sep 10 19:51:15.996675 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:15.996643    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-t9rm6_3fe9e5b0-7ea7-4b64-b60b-b098a1a2aec2/monitoring-plugin/0.log"
Sep 10 19:51:16.193793 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:16.193710    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tphzt_8b85fa77-814d-412c-a98e-e424d077a16c/node-exporter/0.log"
Sep 10 19:51:16.213834 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:16.213724    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tphzt_8b85fa77-814d-412c-a98e-e424d077a16c/kube-rbac-proxy/0.log"
Sep 10 19:51:16.234712 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:16.234669    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tphzt_8b85fa77-814d-412c-a98e-e424d077a16c/init-textfile/0.log"
Sep 10 19:51:16.678820 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:16.678794    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7967dd7f95-fzkhg_f1d1a41d-eb53-4873-852d-50ceb50156e2/thanos-query/0.log"
Sep 10 19:51:16.701726 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:16.701700    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7967dd7f95-fzkhg_f1d1a41d-eb53-4873-852d-50ceb50156e2/kube-rbac-proxy-web/0.log"
Sep 10 19:51:16.725077 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:16.725048    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7967dd7f95-fzkhg_f1d1a41d-eb53-4873-852d-50ceb50156e2/kube-rbac-proxy/0.log"
Sep 10 19:51:16.746154 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:16.746125    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7967dd7f95-fzkhg_f1d1a41d-eb53-4873-852d-50ceb50156e2/prom-label-proxy/0.log"
Sep 10 19:51:16.767124 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:16.767081    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7967dd7f95-fzkhg_f1d1a41d-eb53-4873-852d-50ceb50156e2/kube-rbac-proxy-rules/0.log"
Sep 10 19:51:16.789870 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:16.789842    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7967dd7f95-fzkhg_f1d1a41d-eb53-4873-852d-50ceb50156e2/kube-rbac-proxy-metrics/0.log"
Sep 10 19:51:18.575566 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:18.575528    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5hbck/perf-node-gather-daemonset-rv9pk"]
Sep 10 19:51:18.584681 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:18.584652    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-rv9pk"
Sep 10 19:51:18.652677 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:18.652645    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5hbck/perf-node-gather-daemonset-rv9pk"]
Sep 10 19:51:18.745333 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:18.745291    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/62b5ff3b-aa2a-405d-837e-04978bf4076b-proc\") pod \"perf-node-gather-daemonset-rv9pk\" (UID: \"62b5ff3b-aa2a-405d-837e-04978bf4076b\") " pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-rv9pk"
Sep 10 19:51:18.745540 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:18.745340    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/62b5ff3b-aa2a-405d-837e-04978bf4076b-podres\") pod \"perf-node-gather-daemonset-rv9pk\" (UID: \"62b5ff3b-aa2a-405d-837e-04978bf4076b\") " pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-rv9pk"
Sep 10 19:51:18.745540 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:18.745368    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/62b5ff3b-aa2a-405d-837e-04978bf4076b-lib-modules\") pod \"perf-node-gather-daemonset-rv9pk\" (UID: \"62b5ff3b-aa2a-405d-837e-04978bf4076b\") " pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-rv9pk"
Sep 10 19:51:18.745540 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:18.745428    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26sd4\" (UniqueName: \"kubernetes.io/projected/62b5ff3b-aa2a-405d-837e-04978bf4076b-kube-api-access-26sd4\") pod \"perf-node-gather-daemonset-rv9pk\" (UID: \"62b5ff3b-aa2a-405d-837e-04978bf4076b\") " pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-rv9pk"
Sep 10 19:51:18.745540 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:18.745458    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/62b5ff3b-aa2a-405d-837e-04978bf4076b-sys\") pod \"perf-node-gather-daemonset-rv9pk\" (UID: \"62b5ff3b-aa2a-405d-837e-04978bf4076b\") " pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-rv9pk"
Sep 10 19:51:18.785004 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:18.784949    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bmdg9_9817270c-8629-44c4-ab8f-f3982e1f0e2f/console-operator/1.log"
Sep 10 19:51:18.791343 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:18.791293    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-bmdg9_9817270c-8629-44c4-ab8f-f3982e1f0e2f/console-operator/2.log"
Sep 10 19:51:18.846401 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:18.846309    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/62b5ff3b-aa2a-405d-837e-04978bf4076b-proc\") pod \"perf-node-gather-daemonset-rv9pk\" (UID: \"62b5ff3b-aa2a-405d-837e-04978bf4076b\") " pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-rv9pk"
Sep 10 19:51:18.846635 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:18.846615    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/62b5ff3b-aa2a-405d-837e-04978bf4076b-podres\") pod \"perf-node-gather-daemonset-rv9pk\" (UID: \"62b5ff3b-aa2a-405d-837e-04978bf4076b\") " pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-rv9pk"
Sep 10 19:51:18.846774 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:18.846738    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/62b5ff3b-aa2a-405d-837e-04978bf4076b-lib-modules\") pod \"perf-node-gather-daemonset-rv9pk\" (UID: \"62b5ff3b-aa2a-405d-837e-04978bf4076b\") " pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-rv9pk"
Sep 10 19:51:18.846931 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:18.846913    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26sd4\" (UniqueName: \"kubernetes.io/projected/62b5ff3b-aa2a-405d-837e-04978bf4076b-kube-api-access-26sd4\") pod \"perf-node-gather-daemonset-rv9pk\" (UID: \"62b5ff3b-aa2a-405d-837e-04978bf4076b\") " pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-rv9pk"
Sep 10 19:51:18.847059 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:18.847031    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/62b5ff3b-aa2a-405d-837e-04978bf4076b-podres\") pod \"perf-node-gather-daemonset-rv9pk\" (UID: \"62b5ff3b-aa2a-405d-837e-04978bf4076b\") " pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-rv9pk"
Sep 10 19:51:18.847059 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:18.847050    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/62b5ff3b-aa2a-405d-837e-04978bf4076b-lib-modules\") pod \"perf-node-gather-daemonset-rv9pk\" (UID: \"62b5ff3b-aa2a-405d-837e-04978bf4076b\") " pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-rv9pk"
Sep 10 19:51:18.847243 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:18.847041    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/62b5ff3b-aa2a-405d-837e-04978bf4076b-sys\") pod \"perf-node-gather-daemonset-rv9pk\" (UID: \"62b5ff3b-aa2a-405d-837e-04978bf4076b\") " pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-rv9pk"
Sep 10 19:51:18.847243 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:18.846513    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/62b5ff3b-aa2a-405d-837e-04978bf4076b-proc\") pod \"perf-node-gather-daemonset-rv9pk\" (UID: \"62b5ff3b-aa2a-405d-837e-04978bf4076b\") " pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-rv9pk"
Sep 10 19:51:18.847243 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:18.847092    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/62b5ff3b-aa2a-405d-837e-04978bf4076b-sys\") pod \"perf-node-gather-daemonset-rv9pk\" (UID: \"62b5ff3b-aa2a-405d-837e-04978bf4076b\") " pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-rv9pk"
Sep 10 19:51:18.856255 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:18.856222    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26sd4\" (UniqueName: \"kubernetes.io/projected/62b5ff3b-aa2a-405d-837e-04978bf4076b-kube-api-access-26sd4\") pod \"perf-node-gather-daemonset-rv9pk\" (UID: \"62b5ff3b-aa2a-405d-837e-04978bf4076b\") " pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-rv9pk"
Sep 10 19:51:18.897114 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:18.897078    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-rv9pk"
Sep 10 19:51:19.203633 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:19.203586    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5hbck/perf-node-gather-daemonset-rv9pk"]
Sep 10 19:51:19.228061 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:19.227973    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-rv9pk" event={"ID":"62b5ff3b-aa2a-405d-837e-04978bf4076b","Type":"ContainerStarted","Data":"9af77c22c6a850771011e7ccf84fa0238f824732396b2b18e5adc998a49040ff"}
Sep 10 19:51:19.396875 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:19.396827    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-f87fb7bb8-rzrd4_303f0788-2826-4563-9f31-b5f843038645/console/0.log"
Sep 10 19:51:20.239225 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:20.239194    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-rv9pk" event={"ID":"62b5ff3b-aa2a-405d-837e-04978bf4076b","Type":"ContainerStarted","Data":"247ea2782ee4f2a7756095a245a250a3f9a56be1c327016966a92008c91b29c9"}
Sep 10 19:51:20.239670 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:20.239352    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-rv9pk"
Sep 10 19:51:21.389834 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:21.389802    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vv7rr_feafd719-ab13-485e-b195-ecbd0900e222/dns/0.log"
Sep 10 19:51:21.410873 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:21.410846    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vv7rr_feafd719-ab13-485e-b195-ecbd0900e222/kube-rbac-proxy/0.log"
Sep 10 19:51:21.435158 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:21.435132    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-k8tfc_c357f393-afff-4968-aba1-3ef6f02620eb/dns-node-resolver/0.log"
Sep 10 19:51:22.058016 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:22.057958    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-757bf78f57-kszf7_db963baa-0057-405d-b1b1-2307a554d4c3/registry/0.log"
Sep 10 19:51:22.133557 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:22.133526    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zdnmx_98e70b32-791a-4a50-b558-5337fc78d24c/node-ca/0.log"
Sep 10 19:51:23.084873 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:23.084817    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k_e52d0ec4-e194-4be2-9690-c2c541a41f7f/istio-proxy/0.log"
Sep 10 19:51:23.334313 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:23.334284    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dd7bdc797-jljr6_896dee6e-5770-4d1a-9637-e2573b9ebc16/kube-auth-proxy/0.log"
Sep 10 19:51:23.388408 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:23.388369    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-ltgk2_4fd6f983-387d-48bd-9942-0157de3ce6de/istio-proxy/0.log"
Sep 10 19:51:23.399784 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:23.399733    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-ltgk2_4fd6f983-387d-48bd-9942-0157de3ce6de/istio-proxy/1.log"
Sep 10 19:51:24.423580 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:24.423556    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gdvkn_27b0fcb8-4e92-4753-8562-6e1e8bfa9e97/serve-healthcheck-canary/0.log"
Sep 10 19:51:25.088259 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:25.088234    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ssqgh_d1058bfb-6fa3-4f17-bc23-87860236c43a/kube-rbac-proxy/0.log"
Sep 10 19:51:25.109081 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:25.109051    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ssqgh_d1058bfb-6fa3-4f17-bc23-87860236c43a/exporter/0.log"
Sep 10 19:51:25.130582 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:25.130554    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ssqgh_d1058bfb-6fa3-4f17-bc23-87860236c43a/extractor/0.log"
Sep 10 19:51:26.261513 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:26.261482    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-rv9pk"
Sep 10 19:51:26.282430 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:26.282370    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5hbck/perf-node-gather-daemonset-rv9pk" podStartSLOduration=8.282348841 podStartE2EDuration="8.282348841s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:51:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:51:20.261676362 +0000 UTC m=+2135.481524844" watchObservedRunningTime="2026-09-10 19:51:26.282348841 +0000 UTC m=+2141.502197316"
Sep 10 19:51:27.448970 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:27.448882    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-h2rcd_87c2c3c1-8abb-447e-a1c2-dc779a3cb86a/manager/0.log"
Sep 10 19:51:27.830022 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:27.829991    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-77c474c76b-jhn5d_3aeab5a1-59dd-4975-811f-8c9eb50a3605/manager/0.log"
Sep 10 19:51:27.857037 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:27.857003    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-w2wms_c472bc15-0bbd-43bb-8cd3-277fb3d01ffd/server/0.log"
Sep 10 19:51:27.963225 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:27.963154    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-2gt8j_4cc26bdf-0a6c-42a8-a000-a97fdd40b51d/manager/0.log"
Sep 10 19:51:28.066362 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:28.065999    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d477566c-8rcd6_8c10ecbe-e921-4f4b-b077-0bf836adf8b5/manager/0.log"
Sep 10 19:51:28.095344 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:28.095315    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d477566c-8rcd6_8c10ecbe-e921-4f4b-b077-0bf836adf8b5/copy-manifests/0.log"
Sep 10 19:51:29.230037 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:29.230010    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k_e52d0ec4-e194-4be2-9690-c2c541a41f7f/istio-proxy/0.log"
Sep 10 19:51:29.457202 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:29.457176    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dd7bdc797-jljr6_896dee6e-5770-4d1a-9637-e2573b9ebc16/kube-auth-proxy/0.log"
Sep 10 19:51:29.501625 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:29.501540    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-ltgk2_4fd6f983-387d-48bd-9942-0157de3ce6de/istio-proxy/0.log"
Sep 10 19:51:29.513858 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:29.513827    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-ltgk2_4fd6f983-387d-48bd-9942-0157de3ce6de/istio-proxy/1.log"
Sep 10 19:51:31.207265 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:31.207231    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k_e52d0ec4-e194-4be2-9690-c2c541a41f7f/istio-proxy/0.log"
Sep 10 19:51:31.473337 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:31.473222    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dd7bdc797-jljr6_896dee6e-5770-4d1a-9637-e2573b9ebc16/kube-auth-proxy/0.log"
Sep 10 19:51:31.521854 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:31.521808    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-ltgk2_4fd6f983-387d-48bd-9942-0157de3ce6de/istio-proxy/0.log"
Sep 10 19:51:31.530066 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:31.530038    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-ltgk2_4fd6f983-387d-48bd-9942-0157de3ce6de/istio-proxy/1.log"
Sep 10 19:51:34.761722 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:34.761698    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-87jkz_9a81694c-77e4-4fe2-9c3a-739d14ebfdbe/migrator/0.log"
Sep 10 19:51:34.783440 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:34.783414    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-87jkz_9a81694c-77e4-4fe2-9c3a-739d14ebfdbe/graceful-termination/0.log"
Sep 10 19:51:35.158231 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:35.158199    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-5psdd_77f7916d-5334-4e57-848f-afedd9e63d29/kube-storage-version-migrator-operator/1.log"
Sep 10 19:51:35.159151 ip-10-0-142-20 kubenswrapper[2579]: I0910 19:51:35.159130    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-5psdd_77f7916d-5334-4e57-848f-afedd9e63d29/kube-storage-version-migrator-operator/0.log"